Recently, a module on the project line needs to obtain the K-line data of fire coin. In the early stage, I used the Workerman timing task to get the data through URL request every second. After the task was done, the boss felt that the data was not real-time, so I couldn't optimize it. Fortunately, in GitHub, I saw an old brother who wrote to get the fire coin data through WebSocket. I didn't say much, so I started the code directly below. If you don't write well in your first blog, please forgive me.
URL request method
It's found that this method is also flawed in implementation. Most of the Internet only Posts interface documents and codes, but the actual operation will find that it's impossible to request the fire coin server. Why? Because people in foreign countries occasionally have a low probability of request, so the code can only be executed in the external network server, so it's very troublesome to develop and debug. Later, I figured out a way to find an external network server and arrange a script proxy request (external network request of non fire currency is also OK), so that the fire currency interface can also be requested in China, which is more convenient for debugging.
Here is the proxy script code:
$url = urldecode($_GET['url']);
if ($url) {
echo curl_get($url);
die();
}else{
echo "How are you";
}
function curl_get($url, $timeout = 5)
{
$ssl = substr($url, 0, 8) == "https://" ? TRUE : FALSE;
$ch = curl_init();
$headers = array(
"Content-Type: application/json charset=utf-8",
'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36',
);
$opt = array(
CURLOPT_URL => $url,
CURLOPT_HEADER => 0,
CURLOPT_CUSTOMREQUEST => strtoupper('GET'),
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_TIMEOUT => $timeout,
CURLOPT_HTTPHEADER => $headers,
);
if ($ssl) {
$opt[CURLOPT_SSL_VERIFYHOST] = false;
$opt[CURLOPT_SSL_VERIFYPEER] = FALSE;
}
curl_setopt_array($ch, $opt);
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
Code to get data:
$now = time();
$diff = intval((strtotime(date('Y-m-d H:i:00'), $now) - $find['add_time']) / self::$time_list[$period]);
$size = $diff + 1 > 2000 ? 2000 : $diff + 1;
$url = "https://api.huobipro.com/market/history/kline?period={$period}&size={$size}&symbol={$symbol}";
$log .= ",url:{$url}";
//If the server is in China, you need to set the post.php File deployment to Internet server proxy request fire currency api
$post_url = 'http://xxx.com/post.php?url='.urlencode($url);
$log .= ",post_url:{$post_url}";
$res = self::curl_get($post_url, 5);
if (!$res) throw new Exception(lang('Fire coin request failed'));
$res = json_decode($res, true);
if ($res['status'] != 'ok') throw new Exception("Fire coin net returned error,err-code:{$res['err-code']},err-msg:{$res['err-msg']}");
if (empty($res['data'])) throw new Exception("Fire coin net return data is empty");
$huobi = $res['data'];
$ids = array_column($huobi,'id');
array_multisort($ids,SORT_ASC,$huobi);
$add_list = [];
$update_list = [];
foreach ($huobi as $key1 => $value1) {
$where1 = $where;
$where1['add_time'] = $value1['id'];
$find1 = (new self)->where($where1)->order('id', 'desc')->find();
if ($find1) {//Record already exists, update existing record
$update_list[] = [
'id'=>$find1['id'],
'open_price'=>number_format($value1['open'],6,".",""),
'close_price'=>number_format($value1['close'],6,".",""),
'high_price'=>number_format($value1['high'],6,".",""),
'low_price'=>number_format($value1['low'],6,".",""),
'amount'=>number_format($value1['amount'],6,".",""),
'count'=>number_format($value1['count'],6,".",""),
'vol'=>number_format($value1['vol'],6,".",""),
'ch'=>$res['ch'],
//'add_time'=>$value1['id'],
'update_time'=>time(),
];
}
else {
$add_list[] = [
'period'=>$period,
'symbol'=>$symbol,
'open_price'=>number_format($value1['open'],6,".",""),
'close_price'=>number_format($value1['close'],6,".",""),
'high_price'=>number_format($value1['high'],6,".",""),
'low_price'=>number_format($value1['low'],6,".",""),
'amount'=>number_format($value1['amount'],6,".",""),
'count'=>number_format($value1['count'],6,".",""),
'vol'=>number_format($value1['vol'],6,".",""),
'ch'=>$res['ch'],
'add_time'=>$value1['id'],
'update_time'=>time(),
];
}
}
static function curl_get($url, $timeout = 30)
{
$ssl = substr($url, 0, 8) == "https://" ? TRUE : FALSE;
$ch = curl_init();
$headers = array(
"Content-Type: application/json charset=utf-8",
'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36',
);
$opt = array(
CURLOPT_URL => $url,
CURLOPT_HEADER => 0,
CURLOPT_CUSTOMREQUEST => strtoupper('GET'),
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_TIMEOUT => $timeout,
CURLOPT_HTTPHEADER => $headers,
);
if ($ssl) {
$opt[CURLOPT_SSL_VERIFYHOST] = false;
$opt[CURLOPT_SSL_VERIFYPEER] = FALSE;
$opt[CURLOPT_SSLVERSION] = 3;
}
curl_setopt_array($ch, $opt);
$result = curl_exec($ch);
if (!$result) {
$error = curl_error($ch);
$errno = curl_errno($ch);
Log::write("curl_get,url:{$url},error:{$error},error:{$errno}", 'INFO');
}
curl_close($ch);
return $result;
}
WebSocket method acquisition
This method mainly treats the server as a WebSocket The client connects to the WebSocket server of fire coin, subscribes to the K-line data of different time granularity of the transaction pair. After the success, the fire coin server will actively push the message to the server when the K-line data changes. After receiving the push, it can store and other operations. At the same time, the server also pushes the received push message to the WebSocket Client connected to the server.
Because the fire coin server is in the external network, the WebSocket method also requires the deployment of a foreign server, which is used to subscribe data to fire coin and receive fire coin push. Other domestic servers can obtain fire coin K-line data by connecting to this server. This should also be distributed deployment, you can try. The structure diagram is as follows:
Connect to fire coin server code:
$info = "Connect to server:{$this->host}";
echo "\r\n".$info;
$this->saveLog("huobi", $info);
// Asynchronously establish a connection to the fire coin server
$con = new AsyncTcpConnection($this->host);
if ($this->flag) {//Formal environment
$con->transport = 'ssl';
}
$con->onConnect = function($con)
{
$this->onAsyncConnect($con);
};
// When data is sent from the server connection, it is forwarded to the corresponding client connection
$con->onMessage = function($con, $message) use($worker)
{
$this->onAsyncMessage($con, $message, $worker);
};
$con->onError = function($con, $err_code, $err_msg)
{
echo "$err_code, $err_msg";
$info = "Async onError err_code:{$err_code},err_msg:{$err_msg}";
echo "\r\n ".$info;
$this->saveLog("huobi", $info);
};
$con->onClose = function($con)
{
$info = "Async onClose";
echo "\r\n ".$info;
$this->saveLog("huobi", $info);
$this->reconnect_num++;//Number of reconnections + 1
//Update K-line data before reconnection
$info = "to update K Line data-Before reconnection-start:".date('Y-m-d H:i:s');
echo "\r\n ".$info;
$this->saveLog("huobi", $info);
foreach ($this->trade_list as $value) {
$symbol = $value;
foreach ($this->time_list as $k => $v) {
$info = "create_kline:{$symbol}-{$k}";
echo "\r\n ".$info;
$this->saveLog("huobi", $info);
$r = \app\common\model\TradeKlineKline::create_kline($symbol, $k);
if ($r['code'] == SUCCESS) {
}
}
};
$info = "to update K Line data-Before reconnection-end:".date('Y-m-d H:i:s');
echo "\r\n ".$info;
$this->saveLog("huobi", $info);
// If disconnected, reconnect after 1 second
$con->reConnect(1);
};
// Perform asynchronous connection
$con->connect();
//Callback method for connecting fire currency successfully
function onAsyncConnect($con)
{
$this->async_message_time = time();
$info = "Connect to server:{$this->host},success";
echo "\r\n".$info;
$this->saveLog("huobi", $info);
$info = "Start subscription K Line data";
echo "\r\n".$info;
$this->saveLog("huobi", $info);
//$this->saveLog("huobi", 'onAsyncConnect:'.print_r($con, true));
$this->saveLog("huobi", 'onAsyncConnect,cid:'.$con->id.',reconnect_num:'.$this->reconnect_num);
$make = explode(',', TradeConfig::get_value('trade_kline_symbols', 'btcusdt,ethusdt,eosusdt,ltcusdt,etcusdt'));
$this->huobi_id = $con->id;
foreach ($make as $key => $value) {
$symbol = $value;
foreach ($this->time_list as $k => $v) {
$info = "sub:{$symbol}-{$k}";
echo "\r\n".$info;
$this->saveLog("huobi", $info);
$data = json_encode([ //quotation
'sub' => "market." . $symbol . ".kline." . $k,
'id' => "id" . time(),
'freq-ms' => 5000
]);
$con->send($data);
}
}
/*foreach ($this->trade_list as $key => $value) {
$symbol = $key;
foreach ($this->time_list as $k => $v) {
echo "sub:{$symbol}-{$k}\r\n";
$data = json_encode([ //quotation
'sub' => "market." . $symbol . ".kline." . $k,
'id' => "id" . time(),
'freq-ms' => 5000
]);
$con->send($data);
}
};*/
}
//Receive fire coin push callback method
function onAsyncMessage($con, $message, $worker)
{
$data = json_decode($message, true);
if (!$data) {//GZIP compression is adopted
$data = gzdecode($message);
$this->saveLog("huobi", $data);
$data = json_decode($data, true);
}
else {
$this->saveLog("huobi", $message);
}
if(isset($data['ping'])) {
$this->async_message_time = time();
$con->send(json_encode([
"pong" => $data['ping']
]));
// Heartbeat client
foreach($this->all_cons as $kk=>$vv){
if (array_key_exists($vv["sid"], $worker->connections)) {
$info = "\r\n sid ".$vv["sid"]." send ping";
echo $info;
$this->saveLog("all", $info);
$worker->connections[$vv["sid"]]->send(json_encode($data));
}
else {
unset($this->all_cons[$kk]);
}
}
} else if (isset($data['ch'])) {
$this->async_message_time = time();
$info = "Push received,ch:{$data['ch']}";
echo "\r\n".$info;
$this->saveLog("huobi", $info);
//Log::write(print_r($data, true), 'INFO');
$symbol = $data["ch"];
$info = "\r\n on mess size:".sizeof($this->all_cons)." conn-size: ".sizeof($worker->connections)." symbol:".$symbol;
echo $info;
$this->saveLog("all", $info);
$pieces = explode(".", $data['ch']);
switch ($pieces[2]) {
case "kline": //Market chart
$market = $pieces[1]; //Fire coin pair
if (in_array($market, $this->symbol_list)) {
$period = $pieces[3];
$tick = $data['tick'];
//tick description
//"tick": {
// "id": K line id,
// "amount": volume,
// "count": number of transactions,
// "open": opening price,
// "close": closing price, when K line is the latest one, it is the latest transaction price
// "low": lowest price,
// "high": the highest price,
// "vol": transaction amount, i.e. sum (each transaction price * the transaction volume)
//}
$id = $tick['id'];
$where = [
'period'=>$period,
'symbol'=>$market,
'add_time'=>$id,
];
$find1 = \app\common\model\TradeKline::where($where)->order('id', 'desc')->find();
if ($find1) {//Record already exists, update existing record
if ($find1['open_price'] != $tick['open'] ||
$find1['close_price'] != $tick['close'] ||
$find1['high_price'] != $tick['high'] ||
$find1['low_price'] != $tick['low'] ||
$find1['amount'] != $tick['amount'] ||
$find1['count'] != $tick['count'] ||
$find1['vol'] != $tick['vol']) {//No data change, no update
$update_list[] = [
'id'=>$find1['id'],
'open_price'=>number_format($tick['open'],6,".",""),
'close_price'=>number_format($tick['close'],6,".",""),
'high_price'=>number_format($tick['high'],6,".",""),
'low_price'=>number_format($tick['low'],6,".",""),
'amount'=>number_format($tick['amount'],6,".",""),
'count'=>number_format($tick['count'],6,".",""),
'vol'=>number_format($tick['vol'],6,".",""),
'update_time'=>time(),
];
$kline = new \app\common\model\TradeKline;
$res2 = $kline->isUpdate()->saveAll($update_list);
if (empty($res2)) {
var_dump(lang('failure updating record-2').'-in line:'.__LINE__);
//throw new Exception(lang('update record failed - 2 ').' - in line: '__ LINE__ );
}
}
}
else {
$add_list[] = [
'period'=>$period,
'symbol'=>$market,
'open_price'=>number_format($tick['open'],6,".",""),
'close_price'=>number_format($tick['close'],6,".",""),
'high_price'=>number_format($tick['high'],6,".",""),
'low_price'=>number_format($tick['low'],6,".",""),
'amount'=>number_format($tick['amount'],6,".",""),
'count'=>number_format($tick['count'],6,".",""),
'vol'=>number_format($tick['vol'],6,".",""),
'ch'=>$data['ch'],
'add_time'=>$id,
'update_time'=>time(),
];
$kline = new \app\common\model\TradeKline;
$res1 = $kline->saveAll($add_list);
if (empty($res1)) {
var_dump(lang('Insert record failed').'-in line:'.__LINE__);
//throw new Exception(lang('Insert record failed ').' - in line: '__ LINE__ );
}
}
}
break;
}
$time_1 = microtime(true);
if (array_key_exists($symbol, $this->all_symbols)) {
foreach ($this->all_symbols[$symbol] as $key => $val) {
$info = " symbol ".$symbol." | ch ".$data["ch"]." sid ".$val." send \r\n";
echo $info;
$this->saveLog("all", $info);
$worker->connections[$val]->send(json_encode($data));
}
}
$time_2 = microtime(true);
$cost = $time_2 - $time_1;
if ($cost > 1) {
$info = " symbol ".$symbol." | ch ".$data["ch"]." cost {$cost} \r\n";
echo $info;
$this->saveLog("all", $info);
}
}
else {
echo "undefind message\r\n";
var_dump($data);
}
}
It's just part of the code. All the codes are in the https://github.com/Weper-zhu/huobiwebsocket