Contextual chatbot design with decision tree

Hey guys, contextual chatbots are very much essential for talking to the users and to get better user experiences which are already used by some of the tech giants like Zomato, Swiggy, etc to maintain the best of the marketplace and user interests. Today we will see how to make a robust and very simple way of implementing a contextual chatbot with basic decision/category/priority tree architecture using MERN stack and to render them to the final user.

For the illustration, we have used the Groww website to demonstrate how it works. The project used in this article is contributed by myself Arnab Ghosh (arnabghosh31031998@gmail.com) and Sonal Prabhu (sonalsprabhu16@gmail.com) and is implemented as part of Crio Winter Of Doing (CWoD) 2021 Externship Programme Stage 3.

Prerequisites required for this article includes basic to medium level knowledge on NodeJS,Mongoose/MongoDB, ReactJS, Data structures and algorithms.

Github Source:- https://github.com/sonalprabhu/Groww-chatbot

Application Demo:-

Github source also contains the Deployed URLs of the application under README.md which you can try out (Desktop preferred).

Are you excited about this? Let's get started :).

What is a contextual chatbot?

A chatbot is a widget found on the websites of many companies that provides a chat interface for the users to prompt them to perform certain actions and aid them in navigating through the website to find what they want.

A contextual chatbot is an upgraded version of a traditional chatbot that takes user context as well as page context and displays a specific set of FAQs based on which route the user is currently in or some FAQs based on user details like some sort of pending activity like updating email or mobile number, etc.

Note before starting, since we are using the Groww website as a reference it is recommended to get a quick walkthrough of the site.

Backend Setup

Step 1: Install NodeJS LTS

Go to the website https://nodejs.org/en/download/ and proceed with the installation process as per your operating system and whether it is 32-bit or 64-bit supported.

Open up your command prompt or terminal and type the below command to ensure your installation worked properly

node --version

Check your npm version as:

npm --version

For the current illustration, we are using node v14.16.x and npm v7.9.x

Step2: Initialize a NodeJS project and install dependencies and devDependencies for the backend

To initialize a NodeJS project do:

npm init

Accept the defaults and an empty NodeJS project will be created which contains at least two files by default:- a)package.json b)package-lock.json.

The dependencies and devDependencies used in the project in the package.json file:

package.json containing dependencies and devDependencies

Copy the above contents into your package.json and do “npm install” or you can manually install each of the packages.You can leave out bcrypt,swagger-ui-express,swagger-jsdoc,@hapi/iron .These dependencies are used for encryption/decryption of passwords or keys and swagger libraries for API documentation.

Note: We will be using MongoDB Atlas that’s why MongoDB is not installed.

As per Mozilla docs:

Installing Mongoose adds all its dependencies, including the MongoDB database driver, but it does not install MongoDB itself. If you want to install a MongoDB server then you can download installers from here for various operating systems and install it locally. You can also use cloud-based MongoDB instances

At this stage, our backend project is ready now let's understand the design architecture of the chatbot

Chatbot Design Architecture

This is where things get interesting. As a primary step, let's dive into a basic hypothetical overview of the decision tree.

Approach

We will go with a basic tree-like architecture and use that tree to reach a specific set of FAQs as they are more optimally suited for this problem. You are free to use your data structure or other designs too :).

The idea is to make each of the nodes as part of the page context or route appearing in the frontend starting from a root node and attaching FAQs at the leaf node.Each of the complete path from root to a specific leaf node defines a unique category of FAQs and can be send to the frontend as a final set of requested FAQs thereby making use of page context. For the user context we can send user id or any other user details to identify the user and can fetch additional FAQs from other path consistently along with page context FAQs.

Note: Here context means we mean page context plus user information. In more complex scenarios, context may contain much more information.

To see the above idea visually, look at the diagram below that will give a basic skeleton of a decision/priority tree design.

Decision/Priority/Category tree hypothetical view

Each of the nodes may or may not contain routing information or may contain related information to link the path with the URL.

Keeping this design architecture in mind let's build a basic tree design for the Groww website as well. The tree somewhat looks like this (assume the array of FAQs is attached with each leaf nodes)

Category tree used in making Groww chatbot backend

For example:- If the user is in the stocks category the frontend will parse the URL and detect it to be on the stocks page, and accordingly, our API will hit the Stocks node and will fetch all the FAQs under the leaf nodes coming under Stocks.

Another example might be supposed if a user is in the orders completed section then parsed URL will link to the path root -> Orders -> Completed and the array of FAQs under the “Completed” node will be sent as an API response to the frontend.

Implementation of the Tree

Let's implement the above design in our backend code. To do this create a file categoryGraph.js either under the root directory of the project or any other place you prefer to. To create the graph we will go level by level and create nodes and append them in our tree.

Now we will see how do we create the Category Schema and seed the database with the category graph created previously.

To know about FAQ schema and other schema details move over to the “models” folder in chatbot-backend in the mentioned GitHub repository. Alternatively, you can look at the swagger-ui API documentation page under the schema section as already mentioned in the beginning.

Now let's create another file populateBackend.js in which addCategories will be called along with other seeding procedures. To look at all the seeding procedures you can navigate to the complete fill_data_db.js in the repository under the “mock_data” folder.

MongoDB connection and API structure with illustration

Since we are using Mongoose therefore to connect with our MongoDB Atlas we need Mongo DB URI from Mongo DB Atlas and we put it in our .env file as MONGODB_URI. We also need to seed our database upon connection therefore we will use populateBackend call only when we need to put fresh data in our DB and we will comment it for further times. Note that seeding may have other ways too and the way we are doing here is just for illustration.

If you look at lines 112 to 114 we put a DB call to Category collection using the category name that we get from the frontend which gets the value initially from the page URL. Also, we are checking for KYC details in user collection to see whether it is in completed status or not and if it is not in completed status we are appending KYC FAQs along with category-specific FAQs in the response object.

In the same way, we can go on with writing other API endpoints for sending an initial set of FAQs corresponding to the input context information.

Frontend Illustration

Finally, let's take a look at parsing page URL and sending context information on the frontend side after performing the following steps to install create-react-app. Here the name of our frontend application is chatbot-frontend.

> npm install -g create-react-app
> create-react-app chatbot-frontend
> cd chatbot-frontend

The package.json used is:-

Dependencies used in chatbot-frontend

Redux/Redux persist has been used to cache chat history with the user. For making the chatbot UI/UX design react-chatbot-kit is used. Feel free to use any other library or a custom UI/UX from scratch of your choice.

Let's now write code for parsing the page URL and send user/guest information currently logged into the application.

Now we can use the FAQ component created above to configure the widget for the chatbot.

Note: We have used cookies for putting our user information. The props object being passed contains the user information.

Let's see how our complete web application works. We are running our frontend locally at http://localhost:3000 and our backend at http://localhost:8081.

Groww chatbot showing Stocks specific questions when no user is logged in or if the user is a guest

Groww chatbot showing Mutual Funds specific questions when a user is logged in whose KYC is completed.

Groww chatbot showing FIxed Deposits specific questions when a user is logged in whose KYC is not completed (See the first four questions).

There are lots of improvements we still need to make to make it a more sophisticated or dynamic model, but all thanks to the Crio team for giving us this wonderful opportunity to learn new stuff. With this, we have come to the end of this article. We hope that at least you got a bunch of ideas to make your own pluggable and an extensible chatbot decision tree for your web development.

Happy coding :).