Setting up auto-complete with Redisearch

Setting up auto-complete with Redisearch

Daily short news for you
  • For a long time, I have been thinking about how to increase brand presence, as well as users for the blog. After much contemplation, it seems the only way is to share on social media or hope they seek it out, until...

    Wearing this shirt means no more worries about traffic jams, the more crowded it gets, the more fun it is because hundreds of eyes are watching 🤓

    (It really works, you know 🤭)

    » Read more
  • A cycle of developing many projects is quite interesting. Summarized in 3 steps: See something complex -> Simplify it -> Add features until it becomes complex again... -> Back to a new loop.

    Why is that? Let me give you 2 examples to illustrate.

    Markdown was created with the aim of producing a plain text format that is "easy to write, easy to read, and easy to convert into something like HTML." At that time, no one had the patience to sit and write while also adding formatting for how the text displayed on the web. Yet now, people are "stuffing" or creating variations based on markdown to add so many new formats that… they can’t even remember all the syntax.

    React is also an example. Since the time of PHP, there has been a desire to create something that clearly separates the user interface from the core logic processing of applications into two distinct parts for better readability and writing. The result is that UI/UX libraries have developed very robustly, providing excellent user interaction, while the application logic resides on a separate server. The duo of Front-end and Back-end emerged from this, with the indispensable REST API waiter. Yet now, React doesn’t look much different from PHP, leading to Vue, Svelte... all converging back to a single point.

    However, the loop is not bad; on the contrary, this loop is more about evolution than "regression." Sometimes, it creates something good from something old, and people rely on that goodness to continue the loop. In other words, it’s about distilling the essence little by little 😁

    » Read more
  • Alongside the official projects, I occasionally see "side" projects aimed at optimizing or improving the language in some aspects. For example, nature-lang/nature is a project focused on enhancing Go, introducing some changes to make using Go more user-friendly.

    Looking back, it resembles JavaScript quite a bit 😆

    » Read more

The problem

Search is one of the key features on any website. Through search functionality, users can easily explore the content of your website.

There are many ways to provide search capabilities to users. For example, categorizing options for them to choose from, or using tags to filter content, or simply providing a search box for them to freely enter their own search queries.

Nowadays, user experience is becoming increasingly important, and it is the responsibility of website administrators to reduce any hesitation or confusion users may have when using certain features on a website, as this can increase the bounce rate. If you have implemented search functionality, what can you do to make it more useful for users?

Let's take the example of a website that sells various fashion accessories, and one of the trending products is "fashion watches" or "superhero watches". How can you enable the user to simply enter "watch" and immediately see suggested keywords that you have set up in advance?

Auto-completion

Auto-completion

Auto-completion is a feature provided by Redisearch for search suggestion. The way auto-completion works is quite simple. You create a separate index specifically for suggesting relevant phrases based on the search keyword.

Redisearch also supports fuzzy suggestions, meaning you can still receive results for a keyword even if the user makes a spelling mistake. This is achieved by using Levenshtein Automaton, which enables efficient search within the Levenshtein Distance. The suggestions are then weighted based on both their score and their Levenshtein distance from the user-entered keyword.

However, fuzzy search (especially for short prefixes) can result in a large number of suggestions. In fact, fuzzy search for any single letter will iterate through the entire dictionary, so this feature should be used carefully as it can impact search speed and server resources.

Setup

All we need to do is create indexes specifically for the suggestion feature and define scores to prioritize the display of search results.

For example, in my blog I have articles related to the topic of Node.js, such as "What is Node.js", "Node.js Event Loop", "Learn Node.js", and I want to suggest these phrases when the user enters "node.js".

127.0.0.1:6379> FT.SUGADD article "What is Node.js" 100
(integer) 1

127.0.0.1:6379> FT.SUGADD article "Node.js Event Loop" 200
(integer) 2

127.0.0.1:6379> FT.SUGADD article "Learn Node.js" 300
(integer) 3

Then let's try searching for suggestions:

127.0.0.1:6379> FT.SUGGET article "node.js" MAX 5 WITHSCORES
Learn Node.js
106.06601715087891
Node.js Event Loop
57.735027313232422
What is Node.js
37.79644775390625

MAX 5 is to fetch the first 5 results, and WITHSCORES is used to display scores. The higher the score, the higher the priority. The order of the results above is determined by setting the score of "Learn Node.js" to the highest (300), then decreasing scores for the remaining phrases.

Suggestions also support searching for misspelled characters. For example, "nodejs", "nopejs", "nope.js", thanks to the Levenshtein distance calculation algorithm. However, along with that comes a decrease in performance. To apply this, simply add the FUZZY keyword in the query.

127.0.0.1:6379> FT.SUGGET article "nope.js" FUZZY MAX 5 WITHSCORES
Learn Node.js
106.06601715087891
Node.js Event Loop
57.735027313232422
What is Node.js
37.79644775390625

Currently, auto-completion only supports prefix keywords, which means it can suggest keywords only if the input keyword matches the beginnings of phrases. In the example above, Redis can only suggest when searching for "no", "node", "node.js", but it cannot suggest for words like "event", "is". Hopefully, future updates to auto-completion will support searching at any position within phrases.

To further explore, you can refer to Redis Auto-completion.

Conclusion

Search suggestion is a useful feature that many websites are using. By utilizing this feature, you can enhance the user experience by suggesting keywords that users are interested in, leading to more focused and relevant searches, such as "hot trends" or the main content of your website, which you want users to discover and pay attention to.

Redisearch's auto-completion provides even more benefits based on your creativity. For example, automating the search index to continuously update suggested phrases based on the search data collected from users to create "hot search trends".

Premium
Hello

Me & the desire to "play with words"

Have you tried writing? And then failed or not satisfied? At 2coffee.dev we have had a hard time with writing. Don't be discouraged, because now we have a way to help you. Click to become a member now!

Have you tried writing? And then failed or not satisfied? At 2coffee.dev we have had a hard time with writing. Don't be discouraged, because now we have a way to help you. Click to become a member now!

View all

Subscribe to receive new article notifications

or
* The summary newsletter is sent every 1-2 weeks, cancel anytime.

Comments (1)

Leave a comment...
Avatar
Thành Đỗ2 years ago

Nghĩa là mình vẫn phải tạo một index rồi thêm data tìm kiếm vào chứ nó không dựa theo data sẵn có được à bạn

Reply
Avatar
Xuân Hoài Tống2 years ago

Đúng rồi bạn ơi, tính năng này chỉ hỗ trợ tạo data trong index để hỗ trợ suggest thôi