How to Integrate GPT Models into Your Existing App

In the vast sea of information surrounding GPT models and ChatGPT, the emphasis has largely been on mastering prompts and maximizing their potential. However, there's a lack of practical guidance on integrating these language models into existing software applications, accompanied by relatable real-world examples, so let's fill that gap.

To ensure clarity and transparency, all sections containing technical content have been clearly labeled as "Technical". This labeling helps readers navigate the article more effectively and focus on the information that is most relevant to their interests.

The "What"

Let's start by understanding what ChatGPT is. It's the most popular web-based chat application that uses GPT models* (versions 3, 3.5, and now 4) to have conversations with users.

Now, what exactly is a GPT model? While there are different definitions, at its core, a GPT model is, as Sam S. said on MS Build, a function that operates on text. It takes input text and provides different text as output, without any side effects. It's important to note that it's not even idempotent, meaning the same input may not consistently yield the same results.

Most people use this function as a standalone tool, treating it as a time-saving virtual assistant. There are numerous tutorials, guidelines, posters, and infographics available to help users become proficient in crafting input to obtain desired results. While this is useful, it doesn't fully showcase the model's potential.

The "Why"

Now, why should we integrate a GPT model into our software system? The answer is simple. By integrating it into a larger system, the GPT model gets a chance to shine. Suddenly, your app gains an interface that not only understands the natural language of user requests but also comprehends their intentions.

What's even more intriguing is that when you describe your system's capabilities, functions, and APIs in natural language, the GPT model can autonomously decide how to best utilize those resources to serve the customer.

It's like having a human in the back office with multiple buttons, each triggering a different action. This human listens to what the user wants and then makes a conscious decision about which buttons to press and in what order to deliver the results.

The "How"

Now, let's explore different levels of integration, considering cost and result quality, using an example.

Our example system is FindMyHome, a real estate marketplace that allows users to find listings matching their criteria using map search, filtering, sorting, and more.

Technical: Our imaginary app has a mobile and web frontend with backend services that expose a RESTful API with automatically generated OpanAPI specs.

Sample swagger generated by gpt-4

Let's gradually spice up our system with GPT capabilities.

Level 0: ChatGPT with Your Plugin

I've labeled this as Level 0 because it doesn't involve directly integrating your app with the GPT model. Instead, it works the other way around. We create a plugin wrapper that encompasses a portion of our software, API, or endpoint, and make it accessible to ChatGPT.

Fortunately, it requires minimal effort. It's important to note that each user must manually register our plugin before ChatGPT becomes aware of its existence and can utilize it. Once registered, queries such as "Find me two-bedroom apartments in Poughkeepsie" would trigger the plugin to retrieve results from our system.

Technical: We already have a RESTful API exposed with an OpenAPI spec. Here's what else we need to do:

• Make sure all the elements of the specification are well described in a way that are easy to understand for a human.

• Create a manifest file that includes a machine-readable description of the plugin's capabilities and invocation instructions, along with user-facing documentation.

Since the endpoint doesn't require authorization and is publicly accessible, we're good to go.

Now our request: Find me two-bedroom apartments in Poughkeepsie will be translated into API call:

Sample API call, generated by GPT-4

Level 1: Leveraging the Power of GPT Model Using Your Existing RESTful API

The second option is relatively straightforward. We aim to leverage our current functionality and introduce a new chatbot to our app allowing users to communicate their needs in natural language. With the assistance of the GPT model, our system will interpret their input, apply the appropriate filters, and generate the desired results.

Sample frontend for a chat search experience

By leveraging suitable frameworks and SDKs, integrating the GPT model into our existing system becomes a straightforward process. It doesn't require extensive modifications to handle both regular form search and the enhanced "smart" or "semantic" search powered by the GPT model.

Level 2: Leveraging the Power of GPT Model with Embeddings

The most advanced use case involves making full use of the GPT model in three ways:

1. Understanding the user's question and determining their intent, just like in Level 1.

2. Converting our unstructured data, such as text descriptions and feature lists of real estate properties in our database, into embeddings. These embeddings are a condensed representation of our database items and they help us find similar items or items that closely match the user's input, resulting in smarter and more accurate search results.

3. Using the results from the smart search, we ask the GPT model to generate a relevant and meaningful response that takes into account both the obtained results and the original query.

Technical: Embeddings are vector representations of unstructured data generated using the GPT embedding endpoint. While the GPT model is involved in their creation, the embeddings themselves are stored in specialized vector databases (like Pinecone or Qdrant). These databases offer a built-in similarity function and often the ability to combine vector similarity search with payload filtering. This allows for the utilization of both semantic search and rigid filtering within a single request.

This advanced integration allows us to leverage the GPT model not only as an "intent interpreter" but also as a powerful "search enhancer." The outcome is remarkably accurate search results that seem almost magical.

What else?

There are other interesting and often useful use cases, depending on the nature of your app:

• Contextual meta-prompts: Imagine you have an ecommerce website, and you want to give users the ability to ask questions about specific products they are viewing. You can easily add a meta-prompt with all the product details available on the page and ask GPT to respond accordingly, taking into account the provided information.

• Fine-tuning: This involves making ChatGPT your database querying engine. You can manually feed your data to ChatGPT, and with the knowledge it gains, it can answer questions your users might have. This approach yields even more accurate responses compared to our Level 2 integration. However, it requires careful handling of changes in the database. For frequently changing knowledge (like a marketplace) and especially with a large volume (like a real estate marketplace), embeddings seem like a better approach.

If you need help leveraging any of these integrations, we are here to assist you.

Stay tuned and happy coding!