Originally posted on Skanda Vivek's blog on Medium
You’ve probably tried ChatGPT or GPT-4 at least a few times over the past months and seen real exciting results. Maybe you use it every day or week as part of your creative process to help generate ideas and other content. Or maybe you want an answer to a question, not easily found on Google. Or want a chat like answer from an expert, instead of combing through multiple resources.
ChatGPT and GPT-4 fall under the umbrella of “Generative AI” — AI that can be used to generate new content from existing content. This is the very foundation of how GPT models were trained. They were trained to predict subsequent words texts. For example, given the part of the phrase:
| “The quick brown fox jumped over the lazy”
the right answer for the following word would be:
| “dog”
After training on multiple such cases and large text corpuses, the model basically develops a good understanding of language in general. Then the model can be fine-tuned on tasks like question answering.
You can use ChatGPT and maybe GPT-4 now, by logging into the OpenAI chat portal. But what if you wanted to do the same for your customers? Big companies are starting to incorporate OpenAI and GPT in their products. Snapchat released a Chatbot powered by ChatGPT. Spotify just released an “AI DJ” that generates useful insights based on your song selections — combined with text to voice technology, that brings this DJ to life. Slack has also incorporated ChatGPT and there are tons of use cases — like summarizing threads, drafting messages, sending actionable suggestions, etc.
Generative AI tech for products is such a new area, and while there are many big companies incorporating GPT in their products, this is just the beginning of what Generative AI can do to empower companies by empowering interactions with customers. How about if you were the one to incorporate or lead discussions incorporating Generative AI at your company?
First, how do you make GPT-4 accessible to your customers? You can’t have them go into the open AI chat portal each time, copy content from your website, and interact with that right? That defeats the whole purpose of streamlining and improving customer conversations!
This is where the OpenAI API comes in. What is an API? Well it is basically a software interface where you can make calls to a service, without dealing with all the stuff under the hood. Each time you interact with an API, you make “API calls.” You use APIs every day — whether it is opening up your email, or searching for movies on Netflix. Each time you search for a movie, it is basically making an API call to Netflix, that information is transmitted to their systems, and they return a list of movies or the specific movie you chose based on comparing your query with their Databases.
Here are the less than 3 lines of code you need to call the GPT-4 API!
import openai
openai.api_key = "BLAH"
#if you don't have access to gpt-4; you can call ChatGPT using:
#model="gpt-3.5-turbo"
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[
{"role": "user", "content": "Please give me ideas for a technical blog on the GPT 4 API capabilities"},
],
temperature=0,
)
Now that you have the API, it is fairly easy to wrap this in a way so that you don’t import python packages like OpenAI every time, but just as a URL to which you make requests. Once you have that, you just embed in your website, and that’s it — you have the power of GPT-4 at your (or your user’s) fingertips!
Let’s look at what is possible!
Summarizing Information
Let’s say you sell a product where tens of hundreds or more users have given detailed reviews. Now when someone wants to buy something, they might want to consolidate all that information. I know that when I go on Amazon and compare products, I don’t just look at star ratings, I also look at what users say — not all 5 star ratings are the same. As an example, I was shopping for hearing aids for my 85 year old grandma.
For example, both are 5-star reviews of 2 different products:
I got these hearing aids for my mom who has difficulty hearing and they are amazing for her. Before, I would have to speak very loudly for my mom to hear me but with these hearing aids this problem was quickly solved. Great product that is worth the purchase, definitely recommend!
I’m a 30 year old who got diagnosed with sudden sensorineural hearing loss about 5 months ago. My hearing loss is pretty minor. Everything in my right ear is muffled and it gets hard to hear in loud settings. I ordered these because it was in my budget. As soon as I put them in I noticed a difference. I’m hearing a lot better than before. It increases all volume, so sudden loud noises are a bit unsettling but for the price I’d say they are excellent quality.
But I personally prefer to buy the first product (even though it is 20$ more expensive) because the second one says Excellent — given the price but the first one says Excellent, period. Summarizing all reviews could be a good way to quickly get a sense of what people are saying.
However, you might run into issues if you pass in text that is too long. Here for example, I pass in the entire text of the GPT-4 paper as the context.
prompt=f"Summarize the Context\n\nContext: {context}:"
MODEL = "gpt-4"
response = openai.ChatCompletion.create( model=MODEL,
messages=[
{"role": "user", "content": prompt},
],
temperature=0,
)
response
What do you do in such cases? Well, you could summarize chunk by chunk — first summarize the first 8192 tokens, then the next, so on and so forth. FYI, tokens relate to the way OpenAI embeds text. Roughly, each token is ~3/4th a word.
Then once you have an entire list of summaries, you can concatenate them, pass the summaries to the GPT-4 model, and summarize the summaries in turn! Not only that, you can summarize only certain aspects — like “summarize all positive reviews” or “summarize all negative reviews”.
The other thing you could do — is to first mark reviews as positive or negative by passing them through GPT-4 using the prompt:
| “Is the context a positive review?”
or
| “Is the context a negative review?”
Based on this, you can only summarize positive or negative reviews.
Question Answering
Many companies have truckloads of unstructured data that is tremendously useful for customers. Take Slack for example. Company Slack channels can be really overwhelming. Let’s say you wanted to find some particular information that details when person X is going to do task Y. If you just do a plain word search as is commonplace in Search algorithms, you could spend a lot of time searching for this nugget of information. You can ask GPT-4 to search over your documents, in the manner as below:
prompt=f"Answer the question based on the context below,
and if the question can’t be answered based on the context,
say \"I don’t know\"\n\nContext: {context}\n\n — -\n\nQuestion: {question}\nAnswer:"
An issue quickly arises however, if your document length exceeds the same limit of 8192 tokens in GPT-4. What do you do in this case? Of course, you could use similar logic as above and ask your question on a summarized version of the contents. But here is the thing — you are searching for needles in haystacks. What if the summarization bulldozes over the very content you are interested in?
There’s is one trick you can do to solve this. Basically, when transformer models are fed in text, the first step is to convert text into tokens. This way, each chunk of text is represented by a series of numbers. By comparing the tokens in the text, and the question you ask, you can seriously narrow down the size of the article text, and find the region most likely to contain the answer to your question. This is sort of “pruning the haystack” to find the needle. Once you have a small enough text, you can feed this into the model, ask your question, and get your answer!
There are a few tricks that are associated with this method of computing similarities between question and context that I go through in this technical tutorial.
Customer Dialogues
Customers also might want to have dialogues with a service agent. Wouldn’t that be nice if a customer visited your website, and their questions were answered by a nice person-like AI assistant?
But here’s the question — how does this AI assistant remember your previous chats? It would be quite awkward to interact with a near-perfect AI language model, but that has the attention span of a toddler! This is where the OpenAI chat APIs offer the option to have dialogues through message prompts as below. By chaining prompts, you effectively introduce memory into the agent!
# Example OpenAI Python library request
MODEL = "gpt-3.5-turbo"
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Knock knock."},
{"role": "assistant", "content": "Who's there?"},
{"role": "user", "content": "Orange."},
],
temperature=0,
)
Where the response from the model is
| Orange who?
Perfecting The Art To ChatGPT
There are many other areas to implement OpenAI chat for your customers. These include multilingual support, personalized recommendations, educative tools, etc. Maybe there’s some creative component involved such as users wanting to write their own content. There’s a potentially great business case for customers to easily create their own E-books and sell this on Amazon Kindle, with content and images generated by AI. However, in order to unlock all the above potentials of AI — there needs to be great thought, experimentation, and creative mindsets.
Take the issue of content creation: GPT-4 is great at generating small booklets a few pages long. But how do you make this content read more like an interesting story? And how do you make interesting characters with complex backgrounds? Can you somehow link various prompts together?
Or take customer dialogues over specific contexts. Is there a way to feed in key aspects of previous conversations with customers through prompts so that the user experience is similar to having long, multiple conversations with a particular customer representative that you can form a relationship with?
Or in tutoring and education. Can interactions with the chat assistant be used to gauge student domain knowledge? And can that information be used to guide seamless real-time conversations, similar to those you would have with your personal tutor?
I believe that the future holds a lot of promise for generative AI. Google recently announced its waitlist for BARD. Generative AI is not going anywhere anytime soon, and be ready to make use of this, or be a bystander while this is being used to empower customers across all industries.