Setting up auto-complete with Redisearch

The problem

Search is one of the key features on any website. Through search functionality, users can easily explore the content of your website.

There are many ways to provide search capabilities to users. For example, categorizing options for them to choose from, or using tags to filter content, or simply providing a search box for them to freely enter their own search queries.

Nowadays, user experience is becoming increasingly important, and it is the responsibility of website administrators to reduce any hesitation or confusion users may have when using certain features on a website, as this can increase the bounce rate. If you have implemented search functionality, what can you do to make it more useful for users?

Let's take the example of a website that sells various fashion accessories, and one of the trending products is "fashion watches" or "superhero watches". How can you enable the user to simply enter "watch" and immediately see suggested keywords that you have set up in advance?

Auto-completion

Auto-completion is a feature provided by Redisearch for search suggestion. The way auto-completion works is quite simple. You create a separate index specifically for suggesting relevant phrases based on the search keyword.

Redisearch also supports fuzzy suggestions, meaning you can still receive results for a keyword even if the user makes a spelling mistake. This is achieved by using Levenshtein Automaton, which enables efficient search within the Levenshtein Distance. The suggestions are then weighted based on both their score and their Levenshtein distance from the user-entered keyword.

However, fuzzy search (especially for short prefixes) can result in a large number of suggestions. In fact, fuzzy search for any single letter will iterate through the entire dictionary, so this feature should be used carefully as it can impact search speed and server resources.

Setup

All we need to do is create indexes specifically for the suggestion feature and define scores to prioritize the display of search results.

For example, in my blog I have articles related to the topic of Node.js, such as "What is Node.js", "Node.js Event Loop", "Learn Node.js", and I want to suggest these phrases when the user enters "node.js".

127.0.0.1:6379> FT.SUGADD article "What is Node.js" 100
(integer) 1

127.0.0.1:6379> FT.SUGADD article "Node.js Event Loop" 200
(integer) 2

127.0.0.1:6379> FT.SUGADD article "Learn Node.js" 300
(integer) 3

Then let's try searching for suggestions:

127.0.0.1:6379> FT.SUGGET article "node.js" MAX 5 WITHSCORES
Learn Node.js
106.06601715087891
Node.js Event Loop
57.735027313232422
What is Node.js
37.79644775390625

MAX 5 is to fetch the first 5 results, and WITHSCORES is used to display scores. The higher the score, the higher the priority. The order of the results above is determined by setting the score of "Learn Node.js" to the highest (300), then decreasing scores for the remaining phrases.

Suggestions also support searching for misspelled characters. For example, "nodejs", "nopejs", "nope.js", thanks to the Levenshtein distance calculation algorithm. However, along with that comes a decrease in performance. To apply this, simply add the FUZZY keyword in the query.

127.0.0.1:6379> FT.SUGGET article "nope.js" FUZZY MAX 5 WITHSCORES
Learn Node.js
106.06601715087891
Node.js Event Loop
57.735027313232422
What is Node.js
37.79644775390625

Currently, auto-completion only supports prefix keywords, which means it can suggest keywords only if the input keyword matches the beginnings of phrases. In the example above, Redis can only suggest when searching for "no", "node", "node.js", but it cannot suggest for words like "event", "is". Hopefully, future updates to auto-completion will support searching at any position within phrases.

To further explore, you can refer to Redis Auto-completion.

Conclusion

Search suggestion is a useful feature that many websites are using. By utilizing this feature, you can enhance the user experience by suggesting keywords that users are interested in, leading to more focused and relevant searches, such as "hot trends" or the main content of your website, which you want users to discover and pay attention to.

Redisearch's auto-completion provides even more benefits based on your creativity. For example, automating the search index to continuously update suggested phrases based on the search data collected from users to create "hot search trends".