In a previous post, we see how to use OpenAI Embeddings to add contextual information to OpenAI chat methods.
Now, we'll see how to use the TsgcOpenAIEmbeddings component from sgcWebSockets package to build AI applications using our own data.
In the post Delphi Voice ChatBot, we used the component TsgcAIOpenAIChatBot to build a ChatBot handled by voice, now we'll expand the ChatBot using not only the openAI data trained till 2021, and we will add our custom data building a Questions & Answers with the data obtained from the help file and the interfaces of the package.
Convert Own Data to Vectors
The first step is take the sgcWebSockets pdf manual and convert to text. We'll use this file to create embeddings with useful information about how to configure the package, use the components... and this information will be used when asking to OpenAI.
Once we've the pdf manual in text format we can use the component TsgcAIOpenAIEmbeddings to obtain the Vectors from OpenAI for every section of the file and in this case we'll store into a text file (for production, you can use a Vector Database like Pinecone).
ChatBot & Embeddings
Once we've converted all our data to vectors, we can start to build our own model, the idea behind is very simple, every time we ask the bot, first we convert the question to a vector, then we search into our database which vector is more similar to the question, and finally we use the most similar data to the question and add it as a context.
Delphi Windows Demo
Find below a sample for Delphi using the sgcWebSockets AI Package which shows how to build a Questions & Answers application using customized data. The demo contains 2 projects:
1. sgcCreateEmbeddings: this projects loads a text file, split in chunks and convert every chunk into a vector and stores it into a database file. The demo already contains the vector database in text file format for easy of use.
2. sgcQuestionsAnswers: using the previously database records, every time the user ask a question to the chatbot, first the application searches into the database which is the context more similar to the question and adds as a contextual information to help the ChatBot to provide more accurate responses.