python
Build a ChatBot Using Python, Django
A ChatBot has become one of the must-have features of modern-day web applications. It plays a huge role in customer service which was once traditionally held by human beings. Through ChatBot, you can automatically generate a response to a user’s input by making use of different machine-learning techniques.
Note:- If you encounter any issues throughout the tutorial, you can check out the code in the GitHub repository.
Table of Contents
- Prerequisite
- Project Configuration
- WebSockets Using Django Channels
- Tailwind CSS Configuration
- Frontend - Create a WebSocket Connection
- Enable a Channel Layer
- Backend - WebSocket Consumers
- ChatterBot - Generate Automated Response
- ChatterBot Training
- Offload Automated Response Generation to Celery
- Conclusion
Prerequisite
This guide assumes that you have intermediate-level Django knowledge.
Project Configuration
To implement the ChatBot, the following technologies will be used:
- ChatterBot Library - To generate automated responses to a user’s input.
- Celery - Getting the automated response from a machine-learning dialog engine is going to take a while. So Celery will be used to perform this task in the background.
- WebSockets using Django Channels - To send to the client the automated response generated by the machine learning model immediately when it’s available.
- Redis - Will be used as a message broker and result backend for Celery. In addition, it will be used as a channel layer for WebSocket communication.
- TailwindCSS - To create the user interface.
We will dive into the details of each of them, but first, let’s set up the project. I have created a starter Docker template with all the configurations so that we can move forward with the development without wasting time.
Create a folder on your local machine > cd
into it > clone the template:
mkdir chatbot
cd chatbot
git clone https://github.com/earthcomfy/blog-template .
This will generate the following folders and files inside your project:
There isn’t much going on except a basic Django setup in Docker with web
, db
, redis
, and celery
services. Refer to this article for more information on how to set up a Django project inside Docker. If you don’t want to use Docker, you can also follow along by setting up a virtual environment and running the necessary services on your machine.
In the root directory of your project, there is a file named .env.template
, rename it to .env
and update the environment variables accordingly.
Great, now build the image and spin up the containers by running:
docker-compose up --build
Go to http://0.0.0.0:8000/ to check if everything worked correctly.
Great, everything is working. Now close Docker by pressing ctrl + c
and create a Django app named chat
docker-compose run --rm web python manage.py startapp chat
Add it to the list of installed apps inside settings/base.py
# settings/base.py
INSTALLED_APPS = [
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
# Local apps
"chat"
]
Hurray! Let’s move on to the next section where the fun begins.
WebSockets Using Django Channels
It goes without saying that in a typical Django project the client makes an HTTP request > Django calls the view that is responsible for managing this request and returns a response back to the client.
This communication is pretty standard. However, ever since Django introduced ASGI and started supporting it natively, writing Django applications with asynchronous code has become possible. ASGI not only allows you to run asynchronous apps, but it also makes it possible to work with more advanced protocols like WebSockets.
Django Channels, which is built on ASGI, goes beyond HTTP and support other protocols such as WebSockets. The underlying implementation of Django Channels is very similar to regular HTTP views.
That being said, the following packages are needed to set up channels in a project:
channels
daphne
Both of them were added to the requirements.txt
file so they must have already been installed when you built your Docker image.
Daphne is an HTTP, HTTP2, and WebSocket protocol server for ASGI, and was developed to power Django Channels. This server needs to be added to the list of installed apps so that it can take over the runserver
management command.
# settings/base.py
INSTALLED_APPS = [
"daphne", # here
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
# Local apps
"chat"
]
Note:- Please be wary of any other third-party apps that require an overloaded or replacementrunserver
command. Daphne provides a separaterunserver
command and may conflict with it. In order to solve such issues, make suredaphne
is at the top of yourINSTALLED_APPS
Next, go to your project's asgi.py
file and adjust it as follows to wrap the Django ASGI application:
# asgi.py
import os
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
from decouple import config
from django.core.asgi import get_asgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))
# Initialize Django ASGI application early to ensure the AppRegistry
# is populated before importing code that may import ORM models.
django_asgi_app = get_asgi_application()
application = ProtocolTypeRouter(
{
"http": django_asgi_app,
# We will add WebSocket protocol later. For now, it's just HTTP.
}
)
Finally, set your ASGI_APPLICATION
setting to point to that routing object as your root application:
# settings/base.py
ASGI_APPLICATION = "config.asgi.application"
Perfect! From this point onward, the daphne
development server will be run. To ensure this, fire up the containers by running:
docker-compose up
Notice the ASGI/Daphne version 4.0.0 development server
Alright, this application is going to require a single view where the user and the ChatBot can interact. For this, let’s first set up Tailwind CSS in the project.
Tailwind CSS Configuration
There are many ways to set up Tailwind CSS in Django. One way to do so is by using a package called django-tailwind. This package provides an easy way to use Tailwind CSS in a Django project.
This package was included in requirements.txt
so it must have already been installed when you created the Docker image.
After installation, add tailwind
to the list of installed apps in the settings:
# settings/base.py
INSTALLED_APPS = [
"daphne",
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
# Third-party apps
"tailwind",
# Local apps
"chat",
]
Now, Create a TailwindCSS-compatible Django app. When prompted for the app name you can give it any name or proceed with the default theme
docker-compose run --rm web python manage.py tailwind init
Add the app - theme
to the list of installed apps in the settings:
# settings/base.py
INSTALLED_APPS = [
# ...
# Local apps
"chat",
"theme",
]
Register the generated app theme
by adding it to the settings as follows:
# settings/base.py
# Tailwind
TAILWIND_APP_NAME = "theme"
Install Tailwind CSS dependencies by running the following command:
docker-compose run --rm web python manage.py tailwind install
Perfect! Now, head over to theme/templates/base.html
This file will be used as a base template to include Tailwind’s style sheet in all the other templates. Let’s modify it as follows:
{% comment %} theme/templates/base.html {% endcomment %}
{% load static tailwind_tags %}
<!DOCTYPE html>
<html lang="en">
<head>
<title>{% block title %}Django Chatbot{% endblock %}</title>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
{% tailwind_css %}
</head>
<body>
{% block body %} {% endblock %}
</body>
{% block scripts%}{% endblock %}
</html>
Then, head over to the chat
app created earlier and create a templates
directory. Within the templates
directory, create another directory named chat
, and within that create a file named chat.html
Put the following snippet inside the file:
{% comment %} chat/templates/chat/chat.html {% endcomment %}
{% extends 'base.html' %} {% block body %}
<div class="p-6 w-[800px]">
<h1 class="text-3xl tracking-tight font-light" id="chat-header"></h1>
<div
id="chat-log"
class="mt-4 w-full relative p-6 overflow-y-auto h-[30rem] bg-gray-50 border border-gray-200"
></div>
<div class="mt-4">
<input
id="chat-message-input"
class="py-2 outline-none bg-gray-50 border border-gray-300 text-gray-900 text-sm focus:border-blue-500"
type="text"
placeholder="Write your message here."
/>
<button
id="chat-message-submit"
class="py-2 px-4 ml-2 border border-transparent text-sm font-medium rounded-md text-white bg-blue-800 hover:bg-blue-900"
type="submit"
>
Send
</button>
</div>
</div>
{% endblock %}
The above is a basic Django HTML file with a div
where chat history will be displayed and an input
box for the user’s input with the submit
button.
Next, create a view that will render the above template:
# chat/views.py
from django.views.generic import TemplateView
class ChatView(TemplateView):
template_name: str = "chat/chat.html"
Then, Create urls.py
module inside the chat app and map the ChatView
to the URL patterns.
# chat/urls.py
from django.urls import path
from .views import ChatView
app_name = "chat"
urlpatterns = [path("", ChatView.as_view(), name="chat_view")]
Finally, go to the project's urls.py
file and include the chat app's URL:
# config/urls.py
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
path("admin/", admin.site.urls),
path("", include("chat.urls", namespace="chat")),
]
Done! Now, make sure the styles are applied properly and that what we have so far is working.
Start tailwind:
docker-compose run web python manage.py tailwind start
In another terminal spin up the containers:
docker-compose run web python manage.py tailwind start
Go to http://localhost:8000/ and you will see the following 🎉
Note:- If the styles aren’t applied, make sure you have started tailwind and restart the containers.
Frontend - Create a WebSocket Connection
Inside chat.html
, you need to write a script that will open a WebSocket connection to a Django Server and listen to the established connection to send and receive data.
Open the chat.html
file and add the following script just under the body
block i.e. {% endblock %}
{% block scripts %}
<script>
var wss_protocol = window.location.protocol == "https:" ? "wss://" : "ws://";
var chatSocket = new WebSocket(
wss_protocol + window.location.host + "/ws/chat/"
);
var messages = [];
chatSocket.onopen = function (e) {
document.querySelector("#chat-header").innerHTML =
"Welcome to Django Chatbot";
};
chatSocket.onmessage = function (e) {
var data = JSON.parse(e.data);
var message = data["text"];
messages.push(message);
var str = '<ul class="space-y-2">';
messages.forEach(function (msg) {
str += `<li class="flex ${
msg.source == "bot" ? "justify-start" : "justify-end"
}">
<div class="relative max-w-xl px-4 py-2 rounded-lg shadow-md
${
msg.source == "bot"
? "text-gray-700 bg-white border border-gray-200"
: "bg-blue-600 text-white"
}">
<span className="block font-normal">${msg.msg}</span></div></li>`;
});
str += "</ul>";
document.querySelector("#chat-log").innerHTML = str;
};
chatSocket.onclose = function (e) {
alert("Socket closed unexpectedly, please reload the page.");
};
document.querySelector("#chat-message-input").focus();
document.querySelector("#chat-message-input").onkeyup = function (e) {
if (e.keyCode === 13) {
// enter, return
document.querySelector("#chat-message-submit").click();
}
};
document.querySelector("#chat-message-submit").onclick = function (e) {
var messageInputDom = document.querySelector("#chat-message-input");
var message = messageInputDom.value;
chatSocket.send(
JSON.stringify({
text: message,
})
);
messageInputDom.value = "";
};
</script>
{% endblock %}
- The above script creates a WebSocket connection to the
/ws/chat/
path > listens to different events and manipulates the DOM. - Depending on the source of the message (user or bot), different alignments and styles are applied (see under the
chatsocket.onmessage
)
If you run the server at this point, type in any message, and hit submit, the WebSocket connection will be opened but you will see an error in the JavaScript console because we don’t have a consumer that accepts WebSocket connections yet.
Enable a Channel Layer
A Channel layer provides an abstraction for multiple consumers to talk with each other and with other parts of Django. It is the middleman that passes messages from senders to receivers. The Channel layer that will be used in this guide is Redis. Aside from serving as a WebSocket communication, Redis can also be used as a:
- Message Broker and Result backend for Celery.
- Caching.
It is an all-in-one tool!
We have already configured a Redis service inside Docker. Django Channels also needs to know how to interface with Redis. This is provided by a package called channels_redis
channels_redis
is an officially maintained channel layer that uses Redis as its backing store. It is already included in requirements.txt
so it must have already been installed when you built your Docker image.
Next, add the Channel layer settings:
# settings/base.py
# Django Channels
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [config("REDIS_BACKEND")],
},
},
}
A channel layer has the following:
1) Channel - a mailbox where messages can be sent to. Each channel has a name. Anyone who has the name of a channel can send a message to the channel.
2) Group - a group of related channels. A group has a name. Anyone who has the name of a group can add/remove a channel to the group by name and send a message to all channels in the group.
Backend - WebSocket Consumers
When a WebSocket API opens a connection, Channels need to be informed where to look to accept and handle the connection. This is where Consumers come in. Consumers correspond to Views in normal Django HTTP requests.
When a message is sent from the client, it is received by the consumers listening to the group or channel on the other end.
Create a file named consumers.py
inside the chat app and add the following:
# chat/consumers.py
import json
from asgiref.sync import async_to_sync
from channels.generic.websocket import WebsocketConsumer
class ChatConsumer(WebsocketConsumer):
def receive(self, text_data):
text_data_json = json.loads(text_data)
# The consumer ChatConsumer is synchronous while the channel layer
# methods are asynchronous. Therefore wrap the methods in async-to-sync
async_to_sync(self.channel_layer.send)(
self.channel_name,
{
"type": "chat_message",
"text": {"msg": text_data_json["text"], "source": "user"},
},
)
# We will later replace this call with a celery task that will
# use a Python library called ChatterBot to generate an automated
# response to a user's input.
async_to_sync(self.channel_layer.send)(
self.channel_name,
{
"type": "chat.message",
"text": {"msg": "Bot says hello", "source": "bot"},
},
)
# Handles the chat.mesage event i.e. receives messages from the channel layer
# and sends it back to the client.
def chat_message(self, event):
text = event["text"]
self.send(text_data=json.dumps({"text": text}))
- The above consumer accepts a WebSocket connection on the path
/ws/chat/
, takes a message, and sends a response to the client (for now the response is just a simple message “Bot says hello”). - Any consumer has a
self.channel_layer
andself.channel_name
attribute, which contains a pointer to the channel layer instance (Redis in our case) and the channel name respectively.
Similar to urls.py
Channels uses a routing configuration to route to the consumer. Inside the chat
app, create a file named routing.py
and add the following:
# chat/routing.py
from django.urls import re_path
from . import consumers
websocket_urlpatterns = [
re_path(r"ws/chat/$", consumers.ChatConsumer.as_asgi()),
]
Next, head over to asgi.py
and modify it as follows:
# config/asgi.py
"""
ASGI config for config project.
It exposes the ASGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/4.1/howto/deployment/asgi/
"""
import os
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
from django.core.asgi import get_asgi_application
from decouple import config
os.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))
# Initialize Django ASGI application early to ensure the AppRegistry
# is populated before importing code that may import ORM models.
django_asgi_app = get_asgi_application()
import chat.routing
application = ProtocolTypeRouter(
{
"http": django_asgi_app,
"websocket": AllowedHostsOriginValidator(
AuthMiddlewareStack(URLRouter(chat.routing.websocket_urlpatterns))
),
}
)
- The above code points the root routing configuration to the
chat.routing
module. This means when a connection is made to the development server (channel's development server),ProtocolTypeRouter
checks whether it's a normal HTTP request or a WebSocket. - If it's a WebSocket,
AuthMiddlewareStack
will take it from there and will populate the connection’s scope with a reference to the currently authenticated user, similar to how Django’sAuthenticationMiddleware
populates therequest
object of aview
function with the currently authenticated user. - Next,
URLRouter
will route the connection to a particular consumer based on the provided URL patterns.
Now, if you head over to http://localhost:8000/ and type in any message, you will get a response that says “Bot says hello”
Great! Now, the response to a user’s input needs to be automated. Let’s do that in the next section using ChatterBot.
ChatterBot - Generate Automated Response
ChatterBot is a Python library that makes it easy to generate automated responses to a user’s input. To use this in your project, add it to the list of installed apps in the settings:
# settings/base.py
INSTALLED_APPS = [
# ...
# Third-party apps
"tailwind",
"chatterbot.ext.django_chatterbot",
# Local apps
"chat",
"theme",
]
Under the hood, ChatterBot uses logic adapters to determine how to respond to a given input statement. You can have multiple logic adapters for your bot. Most of the logic adapters use the Naive Bayesian classification algorithm to determine if an input statement meets a particular set of criteria and to generate a response accordingly. The Naive Bayesian classification algorithm is based on conditional probability. Learn more about it here.
For a list of logic adapters, you can check this out. For the purpose of this tutorial, let’s use a single adapter called Best Match Adapter. As the name implies, the BestMatch
logic adapter selects a response based on the best-known match to a given statement. Let’s add it to the settings:
# settings/base.py
# Chatterbot
CHATTERBOT = {
"name": "User Support Bot",
"logic_adapters": [
"chatterbot.logic.BestMatch",
],
}
If you use multiple adapters, the response with the highest calculated confidence score will be considered a valid response to the input. The confidence score returned by each logic adapter determines how valid or close the response is to the output.
The following diagram from ChatterBot documentation provides a very nice flow of what goes behind the scenes.
ChatterBot Training
In the beginning, a ChatterBot instance is like a blank slate. This means that it starts off with an empty database. You fill it with statements (a single string of text representing something that can be said) and gradually it learns to reply to your input with higher accuracy. This training process is going to take some time. In order to simplify this process, ChatterBot provides multiple ways to initially train your bot. In this tutorial, you will learn how to train your bot using sample list data.
Let’s create a simple custom management command where we will pass in a list of statements. Head over to the chat
app and create a management
directory. Within the management
directory, create another directory named commands
, and within that create 2 files __init__.py
and train.py
Put the following script inside train.py
# chat/management/commands/train.py
from django.core.management.base import BaseCommand
from chatterbot import ChatBot
from chatterbot.ext.django_chatterbot import settings
from chatterbot.trainers import ListTrainer
class Command(BaseCommand):
help = "Training the chatbot"
def handle(self, *args, **options):
chatterbot = ChatBot(**settings.CHATTERBOT)
trainer = ListTrainer(chatterbot)
trainer.train(
[
"Hello",
"Hi there!",
"How are you doing?",
"I'm doing great.",
"That is good to hear",
"Thank you.",
"You're welcome.",
]
)
self.stdout.write(self.style.SUCCESS("Successfull!"))
Next, run the following command to load the example dialog into your project’s database.
docker-compose run --rm web python manage.py train
Now, create a superuser:
docker-compose exec web python manage.py createsuperuser
and go to the admin panel http://localhost:8000/admin/ Go to the statements
tab and voila, you will see a list of texts with their responses.
Offload Automated Response Generation to Celery
Needless to say, generating a response from ChatterBot is time-intensive. This is where Celery comes to the rescue. Celery will pick up queued tasks and manages a separate server to run them in the background. As mentioned before, we will use Redis as a message broker. It is already set up in Docker so let’s proceed to defining the task.
Create a file named tasks.py
inside the chat
app and put the following Celery task:
# chat/tasks.py
from asgiref.sync import async_to_sync
from celery import shared_task
from channels.layers import get_channel_layer
from chatterbot import ChatBot
from chatterbot.ext.django_chatterbot import settings
channel_layer = get_channel_layer()
@shared_task
def get_response(channel_name, input_data):
chatterbot = ChatBot(**settings.CHATTERBOT)
response = chatterbot.get_response(input_data)
response_data = response.serialize()
async_to_sync(channel_layer.send)(
channel_name,
{
"type": "chat.message",
"text": {"msg": response_data["text"], "source": "bot"},
},
)
Finally, update consumers.py
with the following content:
# chat/consumers.py
import json
from asgiref.sync import async_to_sync
from channels.generic.websocket import WebsocketConsumer
from .tasks import get_response
class ChatConsumer(WebsocketConsumer):
def receive(self, text_data):
text_data_json = json.loads(text_data)
get_response.delay(self.channel_name, text_data_json)
async_to_sync(self.channel_layer.send)(
self.channel_name,
{
"type": "chat_message",
"text": {"msg": text_data_json["text"], "source": "user"},
},
)
def chat_message(self, event):
text = event["text"]
self.send(text_data=json.dumps({"text": text}))
You have to restart the container so that Celery can pick up the task and add it to its queue. Press ctrl + c
and run:
docker-compose up
Go to http://localhost:8000/ and type in hello and you will get the response back 🚀
Conclusion
This tutorial is intended to be a starting place. The ChatterBot documentation is very concise so make sure to check it out to implement more features. For instance, you can train your data with corpus data. You can use multiple logic adapters or even create your own. In addition, check out Django Channel’s documentation to understand more about consumers, channel layers, and so on.
If you got lost somewhere throughout the guide, check out the project on GitHub
Happy coding! 🖤
Explore other topics
python
Elevate Your Django Projects with GraphQL Integration
If you are a Django developer looking for something beyond the traditional REST APIs to add some extra power and flexibility to your projects, integrating GraphQL might be a good place to start. Grap...
March 23, 2023
python
Build a Blog API With JWT Authentication Using Django Rest Framework
Django REST framework is a powerful and flexible toolkit for building Web APIs. You can easily build a REST API using DRF and consume the endpoints from a React, Angular, or other Frontend application...
December 18, 2022
javascript
Why use NextJS?
Introduction Next.js is an open-source, lightweight and JavaScript framework, which allows you to develop fast and user-friendly web applications and static websites using React. It is a ReactJS fram...
December 15, 2022