[Linux] take the matching system as an example to get started with the Thrift framework

[Linux] take the matching system as an example to get started with the Thrift framework

Reference blogs / articles:
Detailed explanation of Apache Thrift series (I) - overview and introduction
acwing Linux basic course (mainly refer to the tutorial here)
You'll know what RPC is after reading it
thrift official website
c + + parallel programming

What is Thrift

Thrift is a lightweight, cross language remote service invocation framework. It generates RPC server / client template code of various mainstream languages through its own IDL intermediate language and with the help of code generation engine.
RPC (Remote Procedure Call) is a protocol that requests services from remote computers through the network without understanding the underlying network technology.

In short, it is convenient for you to call the program on the remote server

Learning Thrift framework through game matching cases

The game matching service is realized with the help of the server provided by acwing

Task list:

  • Describe the interface
  • Implement the game node (written in Python, match_client side)
  • Realize matching system nodes (written in C + +, match_server side, save_client port)

match matching service and save data saving service are the client and server of the two services

acwing in the data storage center (save_server port) has been implemented with the help of the previous myserver server

Step.1 create corresponding project files

Create a project on gitlab, and then create three file directories under the project folder
match_system: match system
game: game
thrift: interface description

Step.2 create an interface description file

Two interface files

Provide add_user interface and remove_user interface for matching clients

Provide a save_data interface for the matching server to send data to the myserver server (data storage server) and save it

Enter thrift directory
Enter the following into match.thrift

namespace cpp match_service

struct User{
    1:i32 id,
    2:string name,
    3:i32 score

service Match{
    i32 add_user(1:User user,2:string info),
    i32 remove_user(1:User user,2:string info),

The first line of namespace describes the language used. We choose c + +, so the syntax of thrift file is basically the same as that of c + +. There are several differences: int should be written as i32, and variable parameters should be labeled before
We write a structure User and describe the two interfaces add_user and remove_user as parameters

Enter the following into save.thrift

namespace cpp save_service

service Save{
    #username:myserver Name: the server used for data saving
    #Password: the md5sum first 8 digits of the password of Myserver
    #1 will be returned if the verification is successful, otherwise 0 will be returned
    i32 save_data(1:string username,2:string password,3:i32 player1_id,4:i32 player2_id)

username and password are used for authentication
md5sum processing server password is used to facilitate code disclosure and ensure security

Step.3 Create matching client files

We use python to write the add_user and remove_user functions of the game matching client


  1. We should judge whether to add or remove according to the input, and convey relevant user parameters.
  2. We need to use the framework to make the client call the server program

Use the following command to generate a frame under the current folder

thrift -r --gen py tutorial.thrift

tutorial.thrift should be changed to its own match.thrift position

After Gen py is generated, rename it, and then find the PY file of the client. I need to modify it (header file and related functions)
Because we are writing to the client, we should pay attention to deleting the files related to the server (python can not be deleted, but the following c + + can not. See the thrift official website for details)

client.py file example

from match_client.match import Match
from match_client.match.ttypes import User
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from sys import stdin

def operate(op,user_id,username,score):
    # Make socket
    transport = TSocket.TSocket('localhost', 9090)

    # Buffering is critical. Raw sockets are very slow
    transport = TTransport.TBufferedTransport(transport)

    # Wrap in a protocol
    protocol = TBinaryProtocol.TBinaryProtocol(transport)

    # Create a client to use the protocol encoder
    client = Match.Client(protocol)

    # Connect!
    if op=="add":
    elif op=="remove":
    # Close!

def main():
    for line in stdin:
        op,user_id,username,score=line.split(" ")

if __name__=="__main__":

Step.4 create matching server files

We use c + + to write the game matching server


  1. Handle add_user and remove_user calls
  2. Match for players

Part.1 framework generation

Similarly, use similar operations to generate the cpp framework of thrift and modify its contents (header file and function implementation)

thrift -r --gen cpp tutorial.thrift

Part.2 handling interface calls and matching

We found that the processing needs to process two characters at the same time: interface call and player matching. We can consider that there is a parallel requirement here.
Next, we need to understand several concepts

Matching pool

We create a Pool class to describe a matching Pool
Take the user as mathematics and define methods such as save_result, add_user, remove_user and match

Producer consumer

Concurrent design pattern: producer consumer
We regard the two tasks as producers and consumers respectively
Producer: interface calls for add_user and remove_user
Consumer: matching work

We add the interface call to a Message_Queue
And match according to the message queue

Parallel programming

Because the two tasks are carried out at the same time, parallel programming is needed.
Multithreading is realized by calling thread related library, and then mutex (lock) is used to solve the problem of resource contention in parallel programming.
Since I don't know much about parallel programming, I don't talk about it here

Part.3 improvement of matching method

At the beginning, we wrote the most original matching method: the matching pool matches when there are more than two players. However, the matching mechanism of the games we usually play is not so easy. We can consider adding the parameter of player waiting time and changing the matching range according to the waiting time to achieve more realistic matching.

Part.4 single threaded framework To multi-threaded framework

The initial framework for instruction generation is a single threaded framework. We refer to the case on the official website and modify it to a multi-threaded framework to accelerate the matching service at one time.

The following is the finished product code matching the server

// This autogenerated skeleton file illustrates how to build a server.
// You should copy it to another filename to avoid overwriting it.

#include "match_server/Match.h"
#include "save_client/Save.h"
#include <thrift/protocol/TBinaryProtocol.h>
#include <thrift/server/TSimpleServer.h>
#include <thrift/transport/TServerSocket.h>
#include <thrift/transport/TBufferTransports.h>
#include <thrift/transport/TTransportUtils.h>
#include <thrift/transport/TSocket.h>
#include <thrift/concurrency/ThreadManager.h>
#include <thrift/concurrency/ThreadFactory.h>
#include <thrift/server/TThreadedServer.h>
#include <thrift/TToString.h>


using namespace ::apache::thrift;
using namespace ::apache::thrift::protocol;
using namespace ::apache::thrift::transport;
using namespace ::apache::thrift::server;

using namespace  ::match_service;
using namespace  ::save_service;
using namespace std;

struct Task{
    User user;
    string type;

struct MessageQueue{
    queue<Task> q;
    mutex m;
    condition_variable cv;

class Pool{
        void save_result(int a,int b){
            printf("Match Result:%d vs %d !!!\n",a,b);
            std::shared_ptr<TTransport> socket(new TSocket("", 9090));
            std::shared_ptr<TTransport> transport(new TBufferedTransport(socket));
            std::shared_ptr<TProtocol> protocol(new TBinaryProtocol(transport));
            SaveClient client(protocol);

            try {


            } catch (TException& tx) {
                cout << "ERROR: " << tx.what() << endl;
        bool check_match(uint32_t i, uint32_t j)
            auto a = users[i], b = users[j];

            int dt = abs(a.score - b.score);
            int a_max_dif = wt[i] * 50;
            int b_max_dif = wt[j] * 50;

            return dt <= a_max_dif && dt <= b_max_dif;

        void match(){
            for(uint32_t i = 0 ; i < wt.size() ; i ++ ){
                sort(users.begin(),users.end(),[&](User& a,User b){
                        return a.score<b.score;
                bool flag=true;
                for(uint32_t i = 0; i < users.size(); i ++ ){
                    for(uint32_t j = i+1; j < users.size() ; j ++ ){
                            auto a=users[i],b=users[j];


        void add(User user){
        void remove(User user){
            for (uint32_t i=0;i<users.size();i++)

        vector<int> wt;//Waiting time in seconds
class MatchHandler : virtual public MatchIf {
        MatchHandler() {
            // Your initialization goes here

        int32_t add_user(const User& user, const std::string& info) {
            // Your implementation goes here
            return 0;

        int32_t remove_user(const User& user, const std::string& info) {
            // Your implementation goes here

            return 0;


class MatchCloneFactory : virtual public MatchIfFactory {
        ~MatchCloneFactory() override = default;
        MatchIf* getHandler(const ::apache::thrift::TConnectionInfo& connInfo) override
            std::shared_ptr<TSocket> sock = std::dynamic_pointer_cast<TSocket>(connInfo.transport);
            /*cout << "Incoming connection\n";
            cout << "\tSocketInfo: "  << sock->getSocketInfo() << "\n";
            cout << "\tPeerHost: "    << sock->getPeerHost() << "\n";
            cout << "\tPeerAddress: " << sock->getPeerAddress() << "\n";
            cout << "\tPeerPort: "    << sock->getPeerPort() << "\n";*/
            return new MatchHandler;
        void releaseHandler(MatchIf* handler) override {
            delete handler;

void consume_task(){
            auto task = message_queue.q.front();

            //do task
            else if(task.type=="remove")pool.remove(task.user);



int main(int argc, char **argv) {
    TThreadedServer server(
            std::make_shared<TServerSocket>(9090), //port

    cout<<"Start Match Server"<<endl;

    thread matching_thread(consume_task);

    return 0;

Step.5 create data save client file

Since acwing on the data storage server has been implemented, you only need to generate the client as needed
Create a save_client in the match_system/src directory

thrift -r --gen cpp tutorial.thrift

Here, tutorial.thrift is written to the save.thrift location under the thrift folder
Delete the cpp file on the server side at the same time (because c + + can't have two main files at the same time, it may be ok if Python doesn't delete them)

Test whether the matching result data can be saved to the myserver server. Of course, before that, you should write the save_result method in the matching server file


The files of the project are placed on gitlab. You can have a look if you are interested, Project address
Through this study, many knowledge points have been added. Later, I will write some blogs to summarize them

  • RPC framework taking thrift as an example
  • c + + parallel programming
  • Basic concepts of threaded process
  • Proficient in git operation and related specifications
  • c + + conditional variables, basic use of classes
  • Concept of lock

Tags: Python Linux Back-end Project

Posted on Fri, 01 Oct 2021 15:02:47 -0400 by victor78