See four different ways to implement pre input search using Elasticsearch for a better end-user experience. Pepper shell - wish you peace and health
Nowadays, search is an important function in enterprise applications. End users are addicted to the experience of Google search and expect application search to provide a similar experience. Pepper shell - wish you peace and health
This requires us to design and implement a search engine and your golden source (RDBMS/NOSQL). At present, there are many search engines in the market, such as Elasticsearch, Apache Solr, Azure Cognitive Search, etc. They provide better search experience and functions, such as pre input, fuzzy search, correlation based search result promotion, similarity search and so on. Pepper shell - wish you peace and health
What is pre input search?
Pre input search, also known as auto suggestion or auto completion, is a method of filtering data by checking whether the user input data is a subset of data. If so, all text that partially matches the user is a way to provide a prompt when typing text. This feature undoubtedly contributes to the end user's search experience.
How to use Elasticsearch to realize pre input search
Elastic search provides four different ways to implement pre input search. Let's look at all four methods to see which is the best and better implemented:
- Match phrase prefix
- Edge Ngram
- Complete recommendations
- Search on input
Match phrase prefix query
In this approach, we need to use prefix queries on the search fields. The query returns documents containing the words of the prefix text provided in the same order as provided.
For example:
JSONGET /_search { "query": { "match_phrase_prefix": { "message": { "query": "quick brown f" } } } }
The above search returns the document quick brown fox or two quick brown ferrets, but not fox is quick and brown, because the query will match the phrase beginning with quick brown f -
Edge Ngram
Using this approach, we need to configure the custom analyzer using the edge-n-gram filter.
JSONPUT myIndex { "settings": { "analysis": { "analyzer": { "autocomplete": { "tokenizer": "autocomplete", "filter": [ "lowercase" ] }, "autocomplete_search": { "tokenizer": "lowercase" } }, "tokenizer": { "autocomplete": { "type": "edge_ngram", "min_gram": 2, "max_gram": 5, "token_chars": [ "letter" ] } } } }, "mappings": { "properties": { "title": { "type": "text", "analyzer": "autocomplete", "search_analyzer": "autocomplete_search" } } } }
Several points to consider when implementing this method:
- It is best to use the same parser for index and search
- Indexing time may increase because the marker breaks text into words on custom characters
- As mentioned above, index storage will be increased
Complete recommendations
Elasticsearch provides a Completion Suggester as a native solution to the auto complete / search as input function. Autocomplete should be as fast as user input to provide immediate feedback on what the user has entered. Therefore, the completion recommender uses data structures that support fast lookup but are expensive to build and stored in memory.
How do I set the completion advisor?
To use Completion Suggester, we need to make a special mapping for this field:
JSONPUT movie { "mappings": { "_doc" : { "properties" : { "suggest" : { "type" : "completion" }, "title" : { "type": "keyword" } } } } }
Index data:
JSONPUT movie/_doc/1?refresh { "suggest" : [ "Elevate Me Later", "Covert affairs" ] }
Ask:
JSONPOST movie/_search?pretty { "suggest": { "movie-suggest" : { "prefix" : "ele", "completion" : { "field" : "suggest", "skip_duplicates": true } } } }
Search on input
Search as you type is an optimized field type that provides out of the box support for pre entering search or as you type completion of use cases. The search as you type map creates many subfields and indexes the data by analyzing terms, which helps to partially match the indexed text values. It supports prefix completion and infix completion.
To configure search as you type, add the following mappings to your index fields:
JSONPUT search-index { "mappings": { "properties": { "title": { "type": "search_as_you_type" } } } }
You can view the search as you type or complete the suggestion program to use Elasticsearch for pre input. Although using matching phrase prefix or edge-n-gram, we can achieve the same effect, that is, search by type, wrap this implementation internally, and provide efficient and optimized functions as field types.