This is a big demand I completed independently, feeding the price to the project contract. The original intention of writing this article is to record his growth and help Hou Lang sum up his experience at the same time.

background

Conflux doesn't have a useful Blockchain Oracle service, only a Witnet waiting to go online. We are worried that TriangleDao is online, but Witnet is not online yet. After discussion, the team decided not to use other people's services, but to write their own Oracle pricing service. Thanks to the super strict guidance given to me by my brother in stafi protocol, I am still very confident in the preparation of this script service. Many problems (such as http reconnection) have been considered by me.

framework

After discussing with brother Xiu Hongge of smartbch and XD of TriangleDao, if only the contract self feeding price is required, the whole architecture can be designed very simply: only two processes are required, one is responsible for obtaining the latest quotation of cfx from binance/okex, and the other is responsible for writing the latest quotation to the chain. Of course, you also need to write a simple sdk to let colleagues call the contract and obtain the quotation.

difficulty

The overall architecture is divided into two parts, one is read and the other is write. A robust Oracle service cannot make mistakes in both aspects.

read

1. [endpoint fault tolerance] I don't think I can guarantee that the API of biannce or okex will not make mistakes, or the endpoint will crash. I need to change an endpoint. Therefore, an endpoint fault tolerance is required here.

2. [database insert error] I think it is still necessary to insert the data read records into the database to facilitate future debugging and even data recovery. I set a status field. Used to indicate the status of a record.

3. [network congestion, request timeout] the network environment may be unstable sometimes. Fault tolerance must be done here. At present, if the network environment of the server is unstable, I have no way for the time being. Solution: in fact, the best is distributed deployment and multi node disaster recovery.

At present, two key functions have been written, getAvgPrice and getAvgPrice.

    def getRemotePrice(self, symbol, price_dimansion):
        
        binance_res, binance_avg_price, binance_avg_time = self.binanceHolder.getAvgPrice(symbol, price_dimansion)
        
        print("binance finish")

        okex_res, okex_avg_price, okex_avg_time = self.okexHolder.getAvgPrice(symbol, price_dimansion)

Binance has two methods to obtain price: synchronous and asynchronous. According to the requirements, I need a synchronous blocking method here.

class BinanceHolder():

    def __init__(self) -> None:
        self.client = Client(api_key, api_secret)

    # async def init(self):
    #     self.client = await AsyncClient.create(api_key, api_secret)

    def getAvgPrice(self, symbol, price_dimansion):
        try:
            avg_price = self.client.ßget_avg_price(symbol='CFXUSDT')
            
            print("biancne getavg price: ", avg_price)

            binance_avg_time = int(avg_price['mins'])
            binance_avg_price = int( float(avg_price['price']) * price_dimansion)


            #  {'mins': 5, 'price': '0.32856984'}
            # binance_res, binance_avg_price, binance_avg_timse

            print("binance_avg_price, binance_avg_time : ", binance_avg_price, binance_avg_time)
            return True, binance_avg_price, binance_avg_time

The same is true of Okex, which adopts the method of synchronous blocking

class OkexHolder():

    def __init__(self) -> None:
        self.spotAPI = spot.SpotAPI(api_key, secret_key, passphrase, False)
    
    def getAvgPrice(self, symbol, price_dimansion):
        
        try:    
            result = self.spotAPI.get_deal(instrument_id="CFX-USDT", limit='')

            # {"time": "2021-10-21T18:59:19.640Z", "timestamp": "2021-10-21T18:59:19.640Z", 
            # "trade_id": "6977672", "price": "0.33506", "size": "32.531486", "side": "sell"}
            firstResult = result[0] 
            print(firstResult["price"])

            # okex_res, okex_avg_price, okex_avg_time
            okex_avg_price = int( float(firstResult["price"]) * price_dimansion )
            okex_avg_time = 5   

            print(okex_avg_price, okex_avg_time)    
            return True, okex_avg_price, okex_avg_time
        except:
            traceback.print_exc()
            return False, 0, 0

Both parts need an error and retry function to detect error restart. Keep trying again.

write

Look at writing, I need to write data to the chain. Use the contract to record the status of price. I think that the status is only related to four factors: "price, price_dimension, symbol and source". Because solid has no way to store floating-point numbers, I have to multiply price by a power of 10 to become a large number and store it in the contract. For function, there are only two functions for the contract: putPrice and getPrice. Therefore, the contract in the first version is as follows:

pragma solidity >=0.6.11;
import "./Ownable.sol";

contract triangleOracle is Ownable {    

    // 16 + 16 + 16 + 16 = 64 bytes
    struct PriceOracle {
        uint128 price_dimension;    // 16 bytes
        uint128 price;              // 16 bytes
        bytes16 symbol;           // 16 bytes
        string source;           // 16 bytes
    }
    PriceOracle latestPrice;

    event PutLatestTokenPrice(uint128 price, string source, bytes16 symbol, uint128 price_dimension);


    function putPrice(uint128 price, string memory source, bytes16 symbol, uint128 price_dimension) public onlyOwner {
        latestPrice = PriceOracle ({
            price: price,
            source: source,
            symbol: symbol,
            price_dimension: price_dimension
        });

        emit PutLatestTokenPrice(price, source, symbol, price_dimension);

    }

    function getPrice() public returns (uint128 price, string memory source, bytes16 symbol, uint128 price_dimension) {
        return (latestPrice.price, latestPrice.source, latestPrice.symbol, latestPrice.price_dimension);
    }

}

To write data on the chain, the most important thing to consider is that if the miner packs the transaction at the lowest cost quickly. If the miner does not pack, the update of data on the chain will be delayed. First, we need to know that gas can be estimated through cfx_estimategasandcollaborative. Gas refers to the maximum number of calculations that can be performed by the miner. This is to prevent malicious execution of logic or On conflux, the final miner fee is equal to gasUsed * gasPrice. Therefore, set the gas and gas price parameters.

There are also several parameters to be noted, such as storageLimit, epochHeight and nonce. These are also very key parameters and the key to whether they can be successfully packaged.

First of all, the gas price of conflux is very low, usually set to 0x5~0x1. I set it to 0x5.
Secondly, gas needs to request the gas on the chain to estimate the gas we need. The function is estimategasandcollaborative.

def gasEstimated(parameters):
    r = c.cfx.estimateGasAndCollateral(parameters, "latest_state")
    return r

# // Result
# {
#   "jsonrpc": "2.0",
#   "result": {
#     "gasLimit": "0x6d60",
#     "gasUsed": "0x5208",
#     "storageCollateralized": "0x80"
#   },
#   "id": 1
# }

I use gas = gasUsed + 0x100 to set the value of gas. Similarly, the parameter storageLimit is also set by storageCollateralized + 0x20.

parameters["storageLimit"] = result["storageCollateralized"] + 0x20
parameters["gas"] = result["gasUsed"] + 0x100

Finally, write the contract.

def gasEstimated(parameters):
    r = c.cfx.estimateGasAndCollateral(parameters, "latest_state")
    return r

def send_contract_call(contract_address, user_testnet_address,  contract_abi, private_key, arguments):
    try:
        # initiate an contract instance with abi, bytecode, or address
        contract = c.contract(contract_address, contract_abi)
        data = contract.encodeABI(fn_name="putPrice", args=arguments)
        
        # get Nonce

        currentConfluxStatus = c.cfx.getStatus()   
        CurrentNonce =  c.cfx.getNextNonce(user_testnet_address)

        parameters = {
            'from': user_testnet_address,
            'to': contract_address,
            'data': data,
            'nonce': CurrentNonce,
            'gasPrice': 0x5
        }

        result = gasEstimated(parameters)
        
        parameters["storageLimit"] = result["storageCollateralized"] + 0x20
        parameters["gas"] = result["gasUsed"] + 0x100
        parameters["chainId"] = 1
        parameters["epochHeight"] = currentConfluxStatus["epochNumber"]

        # populate tx with other parameters for example: chainId, epochHeight, storageLimit
        # then sign it with account
        signed_tx = Account.sign_transaction(parameters, private_key)

        print(signed_tx.hash.hex())
        print(signed_tx.rawTransaction.hex())
        c.cfx.sendRawTransaction(signed_tx.rawTransaction.hex())

    except:
        traceback.print_exc()

summary

I feel that there are still a lot of things that haven't been summarized in place, including architecture, engineering and details. In fact, I feel that I stepped on a lot of holes when I did it, but there's not much to sum up. The code hasn't been completely sorted out. I'll summarize it first.