How to build chatbots with Contentful and Azure functions

Specific context and tech stack for chatbot

One of our enterprise customers wanted to build an FAQ chatbot with Contentful after they used the content platform to successfully power their web products. They were already using Microsoft cloud services, so we decided to build a quick proof-of-concept using Microsoft Azure functions and knowledge base.

Contentful works with these tools, but can also be used with other cloud and bot services like Amazon Web Services, Google Cloud Platform, or IBM Watson thanks to its APIs. After some initial investigation, we developed the architecture presented here, which uses Contentful webhooks, Azure cloud functions and Microsoft’s knowledge base service.

About the architecture

Publishing flow

• Editors will publish the FAQ in Contentful

• Publishing of the FAQ will trigger a webhook in Contentful

• The webhook will call our publish cloud function

• The function will push the content to our knowledge base

Unpublishing flow

• Editors will unpublish the FAQ in Contentful

• Unpublishing of the FAQ will trigger a webhook in Contentful

• The webhook will call our unpublish cloud function

• The function will remove the FAQ from our knowledge base

How it was implemented

First, Contentful

We used Contentful to store questions and answers for the chatbot. In the Contentful web app, editors can easily write, edit and publish that content. 

Next, the content model

We’re keeping the content model very simple for this demo — just a single content type holds our question and answer pairs, as well as an internal title for content editors to have as a reference. 

Then, webhooks

We use webhooks for publish and unpublish events, triggering our cloud functions.

When creating the publish event webhook, make sure to set the trigger of the webhook for the entry publish event as shown below. For the URL, we will need to add the endpoint of our cloud function, as we have not configured any cloud function yet. We can use the DummyWebhook website or ngrok to create dummy endpoints and add them.

When creating the unpublish event webhook, make sure to set the trigger of our webhook for the entry unpublish event as shown below. Just like last time, we’ll populate the URL with a dummy value from DummyWebhook or ngrok.

Once done, you can create an entry and test our workflow by publishing and unpublishing it and checking the webhook payload being sent. You can then view the payloads in our webhook logs.

Publish entry webhook payload:

Unpublish entry webhook payload:

For now, that's all we have to do within Contentful. 

Creating the FAQ chatbot in Microsoft Knowledge Base

1. Go to https://www.qnamaker.ai/ and create a free account.

2. Create a knowledge base.

3. Create a QnA service.

4. Go back to https://www.qnamaker.ai/ and refresh to see your subscription name and Azure QnA service.

5. Complete the rest of the steps to create a knowledge base. Once done, you will have a knowledge base similar to the one shown below.

APIs to integrate

Now that we’re done with creating our knowledge base, we need to populate it with questions and answers coming from Contentful for the chatbot to use. Let's explore which APIs we can use to add, update or delete information from our knowledge base.

We will be using the update endpoint to add and remove content. We will also use the download endpoint to get all questions and answers and then filter them.

Azure functions

Azure functions are a powerful way to develop and deploy serverless applications. For this use case we chose Azure functions as we can quickly develop and test our proof of concept.

Setup

This video tutorial from Microsoft gives you a guide to set up an Azure function project using Visual Studio Code.

We used the following settings for this chatbot project.

• Language: you can select multiple languages, but we used JavaScript  

• Function trigger: http

• Name: the function we used publishWebhookHandler for the first function and unpublishWebhookHandler for the second function

• Access Rights: set it to anonymous for now. For production, make sure to implement proper security mechanisms 

We created a new Azure function project with two functions: publishWebhookHandler and unpublishWebhookHandler.

publishWebhookHandler

This function will be used to push content into our knowledge base using the webhook coming from Contentful whenever the content is published.

Algorithm

1. Retrieve the following from the webhook: entry ID, question and answer

2. Make an API call to our knowledge base to get all the questions and the answers. Once you have all questions, filter the question using metadata and check for the entry ID. We’re doing this to check if we have duplicate questions in our knowledge base, so we update them instead of adding a duplicate entry.

3. If the question is present we will update that question using the update endpoint. Use the same endpoint to add new questions. The only difference is the body of the payload. You can see in the metadata that we are adding key contentfulid and its value. This helps us to map entries in Contentful with entries in the knowledge base and will help us for deleting and updating entries.

Adding a new question:

Updating a question:

Sample function for adding content to our knowledge base:

unpublishWebhookHandler

This function removes content from our knowledge base using the unpublish webhook, whenever content is unpublished.

Algorithm

1. Retrieve the entry id from the webhook

2. Make an API call to our knowledge base to get all the questions, and filter the question using the entry id

3. Make an API call to remove the question from our knowledge base. The payload will look similar to the following:

Deploy

Once both functions are ready, they can be deployed to Azure Cloud.

When the deployment is complete, get the publishWebhookHandler and unpublishWebhookHandler URLs from Microsoft Azure portal. Add them to the webhooks, which we created in the first step.

Testing our chatbot end to end

Populate content

We published three Q&As for testing in Contentful. Each of the publish events triggered a webhook and they were pushed into our knowledge base. To check, we logged into https://www.qnamaker.ai to see if the questions are published.

On the screenshot below you can see the three published Q&As on Contentful as well as added on our knowledge base.

Now, our knowledge base will train the content. Inside QnA Service, clicking on Test you can test your chatbot.

Results

Now you have an intelligent chatbot customers can talk to. This way customers do not need to go through the extensive list on a website searching for a simple answer. You can also measure the sentiments of the customers and their frequent questions. With this additional information, we are empowered to improve our FAQs. 

Future enhancements for our chatbot

1. Our content model can be extended to add multiple questions for a single answer, and for every entry, additional metadata could be added. For this we have to update the sync functionality within the publishWebhookHandler function.

2. Contentful also provides different localization patterns, which can be used to create content in multiple languages, allowing us to power multilingual bots. 

3. In this article, we explored using the Microsoft tech stack, but similar things can be achieved using AWS Lambda and AWS Lex. Any chatbot platform that provides API access to their knowledge base can be used with Contentful. Using webhooks and cloud functions allow us to push content into a knowledge base and train it afterward.

If you're looking to implement your own conversational chatbot, you can sign up and try it for yourself. Feel free to contact us. Our solution services team works with enterprise customers to accelerate their digital journey.