Search interface optimization scheme -- basic use of elastic search distributed search engine

preface:

In general, there is a search function in development projects. If it is a search function for the C end, it often tests the performance. For example, the commodity search in the general mall system or the website search of some resources.

Perhaps the previous approach is to do a fuzzy query by name or product description on the product table. What's better is to segment the search keywords and build a special search thesaurus. However, in the early stage, we need to disassemble the search words and then combine them with power sets and associate them with the commodity ID. the search words and the words in the thesaurus are queried and found in a completely matching way.

Although building a thesaurus is a good solution, it is relatively troublesome to disassemble the repository and build an index. So I learned about elastic search on the Internet. I plan to use ES in the future. Here is a brief introduction to its basic usage.

 

ES introduction:

Elasticsearch is a Lucene based search server. It provides a distributed multi-user full-text search engine based on RESTful web interface. Elasticsearch is developed in Java and released as open source under Apache license. It is a popular enterprise search engine. Designed for cloud computing In, it can achieve real-time search, stability, reliability, fast, easy to install and use.

 

Download and install:

1. Find the official website and download the corresponding version according to your own system. At present, there are more than 200 M.

    

2. After decompression, open "elasticsearch.bat" in bin in the directory to start.

3. The start-up time is relatively slow. The following only demonstrates the use of single machine and single node, and the distributed can be configured later.

 

Coding use:

The following is a demonstration using ThinkPHP6.0.2, so after the ES service is enabled, open the browser to access 127.0.0.1:9200. If you return to the following, it indicates that it is successfully enabled.

1. Download ES dependency package through composer install elasticsearch/elasticsearch in thinkphp6.0. If composer is slow, try downloading method and then install.

 

And each time you want to use ES locally, you must first open the elasticsearch.bat file, otherwise the following prompt will appear.

 

2. Before use, we need to understand several terms of es: index, Type, document, field and map. And ES queries in restful style, so you can associate the previous terms with MySQL database, table, column and SQL. For example, an index is equivalent to a MySQL database, and a document is equivalent to a record in a table. Restful requests are similar to SQL statements. For example, GET index name / Type/ID can GET document data, POST, PUT, DELETE can create indexes, create / modify / DELETE documents can query official documents, and then use POSTMAN to use different requests 127.0.0.1:9200.

 

3. Now go back to ThinkPHP6, after installing es dependency. It can encapsulate an ES basic method tool class.

<?php


namespace app\common\helper;

use Elasticsearch\ClientBuilder as ESClientBuilder;
class Elasticsearch
{
    protected $config = [
        "127.0.0.1:9200",
    ];
    protected $es;

    /*
     * @Title:  Constructor
     * @Author: Beiqiaosu
     * @Times:  2020/5/14 17:35
     * */
    public function __construct()
    {
        $this->es = $this->init();
    }

    /*
     * @Title ElasticSearch Service initialization
     * @Return object ES example
     * */
    private function init()
    {
        // Read Elasticsearch server list from configuration file
        $builder = ESClientBuilder::create()->setHosts($this->config);

        // Application debugging mode
//        if (env('APP_DEBUG')) {
//            //Configure the log, and the request and return data of Elasticsearch will be printed to the log file
//            $builder->setLogger(app('log')->error());
//        }
        return $builder->build();
    }

    /*
     * @Title: Get ES service information
     * @Return: ES Basic information of service
     * */
    public function info()
    {
        return $this->es->info();
    }

    /*
     * @Titles: Create index
     * @Param: string $name Index name
     * @Param: array $mappings Field mapping
     * @Return: boolean Create results
     * */
    public function index($name, $mappings=[])
    {
        $params = [
            'index' => $name,
            'body' => [
                'settings' => [
                    'number_of_shards' => 1,                // Index sharding (an index can be divided into multiple shards on different nodes)
                    'number_of_replicas' => 0,              // Index sharded copy
                ]
            ]
        ];
        // Create index setting field and field type word breaker, etc
        if($mappings) {
            $params['body']['mappings']['properties'] = $mappings;
        }

        return $this->es->indices()->create($params);
    }

    /*
     * @Title: Get the index configuration. You can get a single index. The parameter not passed is all indexes on the node
     * @Param: array $name Index name array
     * @Return:array Index name, number of fragmented copies and other data
     * */
    public function getIndexConfig($name=[])
    {
        $params = [];
        if(!empty($name)) {
            $params = [
                "index" => $name
            ];
        }
        return $this->es->indices()->getSettings($params);
    }

    /*
     * @Title: Create / change mappings (you can also specify mappings when creating indexes - in fact, the type of fields that store data)
     * @Param: string $name Index name
     * @Param: array $data Field settings
     * @Return boolean Modify results
     * */
    public function putMapping($name, $data)
    {
        $params = [
            'index' => $name,
            'body' => [
                '_source' => [
                    'enabled' => true
                ],
                'properties' => $data
            ]
        ];

        return $this->es->indices()->putMapping($params);
    }

    /*
     * @Tile: Get field mapping for index settings
     * @Param: string $name
     * @Return: array
     * */
    public function getMapping($name)
    {
        $params = ['index' => $name];
        return $this->es->indices()->getMapping($params);
    }

    /**
     * @Notes: Get documents in index
     * @param $name Index name
     * @param $id number
     * @Return: Document data
     */
    public function getIndex($name, $id)
    {
        return $this->es->get(['index' => $name, 'id' => $id]);
    }

    /**
     * @Notes: Delete index
     * @param $name Index name
     * @return array
     */
    public function deleteIndex($name)
    {
        return $this->es->indices()->delete(['index' => $name]);
    }


    /***********************Document section********************************************************/

    /*
     * @Tiele: Create a document in an index (if the index is not mapped, adding documents will automatically create field types, etc.)
     * @Param: string $name Index name
     * @Param: array $data Document data
     * @Return array
     * */
    public function createDoc($name, $data)
    {
        $params = [
            'index' => $name,
            'id'    => $data['id'],
            'body'  => $data
        ];
        $response = $this->es->index($params);
        return $response;
    }

    /**
     * @Notes: Update document
     * @param $index Indexes
     * @param $id Document id
     * @param $data field
     * @author: Beiqiaosu
     * @Time: 2020/4/21 9:51
     */
    public function updateDoc($index, $id, array $data)
    {
        $params = [
            'index' => $index,
            'id' => $id,
            //'type' => '_doc',
            'body' => [
                'doc' => $data
            ]
        ];

        $this->es->update($params);
    }

    // Delete a document
    /*
     * @Tile: Delete a document
     * @Param: array $param Delete index name, ID
     * @Return array
     * */
    public function delDoc($param)
    {
        $this->es->delete($param);
    }

    // Create documents in bulk
    public function bulk($params)
    {
        return $this->es->bulk($params);
    }

    // search
    public function queryData($index,$kye, $from = 0, $size = 10)
    {
        $params = [
            'index' => $index,
            'body'  => [
                'from'  => $from,
                'size'  => $size,
                'query' => [
                    'match' => [
                        'class_name'  => 'edition',
                    ],
                ]
            ],
        ];

        return $this->es->search($params);
    }

    /*
     * @Tile: Multifunctional search
     * @Param string $index Index name
     * @Param array $where Condition array
     * @Param array $sort Sort array
     * @Param int $from Where to start the query
     * @Param int $size Total number of queries
     * @Return array Query results
     * */
    public function query($index,$where,$sort='',$from=0,$size=1500)
    {
        $params = [
            "index" => $index,
            "body" => [
                "from" => $from,
                "size" => $size,
                "query" => $where
            ]
        ];

        if($sort) {
            $params['body']['sort'] = $sort;
        }

        return $this->es->search($params);
    }



}

4. Business use code block.

The following app('es') is registered in the provider.php container

<?php
namespace app\admin\controller;

use app\BaseController;

class Test extends BaseController
{
	
    public function index()
    {
    	var_dump(2323);die;
    }

    // Test ElasticSearch
    public function esTest()
    {
        $res = app('es')->info();
        dump($res);die;
    }

    // Create an index
    public function createIndex()
    {
        $res = app('es')->index("goods");
        var_dump($res);die;
    }

    // create documents
    public function createDoc()
    {

        $goodsList = \app\common\model\Goods::page(0,1500)->select()->toArray();

        foreach ($goodsList as $val) {
            $res = app('es')->createDoc('goods',$val);
        }

        dump($res);die;

        $goods = \app\common\model\Goods::find(1);
        var_dump($goods);die;
    }

    // search
    public function queryDoc()
    {
        // Get all
        $where = [
            'match_all' => new \stdClass(),                 // Empty object, PHP5.3 or (object) [] / / is also the way to give an empty array
        ];

        // Single field exact match
//        $where = [
//            'match_phrase' => [
//                'class Ou name' = > class',
//            ]
//        ];

        // Single field fuzzy matching
//        $where = [
//            'match' => [
//                'goods Gu name' = > Apple '
//            ]
//        ];

        // Multi field matching
//        $where = [
//            'multi_match' => [
//                'query' = > Apple ',
//                'fields' = >
//                'type' = >
//            ]
//        ];

        // Combined search must be combined, should be combined, must not be combined. Combined search is similar to SQL multi field query (just use must)
//        $where = [
//            'bool' => [
//                'must' => [
//                    'match' => [
//                        'goods Gu name' = > Apple '
//                    ]
//                ],
//                'should' => [
//                    'match' => [
//                        'level' => '3'
//                    ]
//                ]
//            ]
//        ];

        // Sort not required
        $sort = [
            "sort" => "asc"
        ];

        $res = app('es')->query('goods',$where,$sort,0,3000);
        dump($res);die;

//        $res = app('es')->queryData('class','class_name',0,20);
//        dump($res);die;
    }

    // Get index (a document in the index)
    public function getDoc()
    {
        $res = app('es')->getIndex('goods',1500);
        dump($res);die;
    }

    // Index configuration obtained
    public function getIndexConfig()
    {
        $res = app('es')->getIndexConfig('class,user,goods');
        dump($res);die;
    }


    
}

Sample source code, official account reply "es" to get.

Tags: Programming ElasticSearch SQL PHP MySQL

Posted on Sun, 17 May 2020 02:26:17 -0400 by Dasndan