OpenAI has finally launched the ChatGPT API
For some time, companies have claimed to use ChatGPT in their technology stack, something we've highlighted in the past. In reality, they were using GPT-3.5, also known as the davince-003 model.
While this model was an improvement from GPT-3, it lacked the fine-tuning conversational capabilities of ChatGPT that many people have been looking for or at least thought was available.
Emphasis on shopping
A number of big vendors have actually already had the ChatGPT API; Snapchat’s My AI feature, Shopify, Instacart, and Quizlet are some examples.
OpenAI's blog release post shows an obvious emphasis on ecommerce with short demos in the OpenAI blog post for Snapchat, Snapchat, Instacart, Quizlet, Shopify, and Speak.
It's now called gpt-3.5-turbo
The ChatGPT model is now called gpt-3.5-turbo.
OpenAI will be releasing updated models regularly which may affect the format of API calls but it's good to know developers will have access to the latest models.
It's much cheaper
The cost has reduced by 90%. Yes, it's really 10 times cheaper than GPT-3! This is really important, as I've lately had some misgivings about the rising cost of the most expensive Davinci model when creating chat applications and crunching through tokens.
It's the new go-to model
It looks like OpenAI is pushing gpt-3.5-turbo as the model to use for many of the tasks people are using Davinci for, not just chat applications:
- Compose emails or other written pieces
- Code in Python
- Provide answers to document-based queries
- Design conversational AI agents
- Give your software a human-like interface
- Offer tutelage in various subjects
- Translate seamlessly between languages
- Bring video game characters to life and more!"
It looks like Davinci prompts could well be transferable after a little rewriting.
There is an important change in the API. ChatGPT Consumes messages with metadata, not just unstructured text. The input, rendered as "token" sequences, is presented in a new format called chat markup language (ChatML).
So the main input is an array of message objects. You can see a request/response here:
The main security vulnerability for calls to the OpenAI LLMs has always been prompt injection attacks.
The prompt itself is of course just text. Messages to the chat completion are now segregated into 3 types:
So the creation of ChatML is significant as it will help shore up these security vulnerabilities and allow for future development of the language e.g different datatypes to be sent as the payload.
One of the challenges of working with LLMs to accomplish more complex tasks is managing successive prompts to send to the model which is all sent as unstructured text.
Frameworks like Langchain help with this but having a structured standard to sequence prompt nodes in a chain is a huge step forward.
Instructions & token management
The docs encourage experimentation to get the required results.
It looks like best practices may change depending on model versions...which might be interesting down the line for app integrations. There are still plenty of options to fine-tune responses e.g temperature
There is an emphasis on token management. Let's face it, setting a token limit and watching how many tokens have been used in an API call will be important... even though it's a lot cheaper.
Yes, as far as the API docs state...no fine-tuning of the model is available as of Mar 1st, 2023.
It remains to be seen whether this will be coming.
API usage data is no longer used for model training unless you opt-in.
Whisper, the speech-to-text model that was open-sourced in September 2022 has also just been released as an API.
Whisper is a multi-language speech recognition model so being able to provide a voice front-end interface with ChatGPT responses is now a nice option.
Tips. Insight. Offers. Are You In?
In conclusion, OpenAI has finally launched the much-awaited ChatGPT API, which offers a step up from the GPT-3.5 model.
Companies like Snapchat, Shopify, Instacart, and Quizlet have already embraced the API with an emphasis on e-commerce.
The new gpt-3.5-turbo model is not only cheaper but also offers more versatility beyond just chat applications and looks to be the model of choice now for most apps.
The API now consumes messages with metadata, presented in a new format called Chat Markup Language (ChatML), offering improved security and the ability for future development.
Despite the reduced cost, token management remains important and there is currently no option for fine-tuning the model.
OpenAI has also released its speech-to-text model, Whisper, as an API, allowing for a voice front-end interface with ChatGPT responses.
About The Bot Forge
Consistently named as one of the top-ranked AI companies in the UK, The Bot Forge is a UK-based agency that specialises in chatbot & voice assistant design, development and optimisation.
If you'd like a no-obligation chat to discuss your project with one of our team, please book a free consultation.