As competition in the generative AI space grows fiercer, OpenAI is upgrading its text generation models and lowering prices.
OpenAI announced the release of new versions of GPT-3.5-turbo and GPT-4, the latter being its latest text-generating AI, with an ability called call function. As OpenAI explains in a entry in your blog function call allows developers to describe programming functions for GPT-3.5-turbo and GPT-4 and have models create code to execute those functions.
For example, function calls can help build chatbots that answer questions by calling external tools, convert natural language into database queries, and extract structured data from text. “These models have been tuned to detect when a function should be called…and to respond with JSON which sticks to the signature of the function,” says OpenAI. “Function calls allow developers to get structured data from the model more reliably.”
Beyond function calls, OpenAI presents a version of GPT-3.5-turbo with a greatly enlarged context window. Context window, measured in Tokens or raw text fragments, refers to the text that the model considers before generating any additional text. Models with small context windows tend to "forget" the content of even very recent conversations, leading them to go off-topic, often in problematic ways.
The new GPT-3.5-turbo offers four times the context length (16 tokens) of vanilla GPT-000-turbo at twice the price: $3.5 per 0,003 input tokens (i.e., tokens entered into the model) and $1000 per 0,004 output tokens (tokens generated by the model). OpenAI says it can ingest around 1000 pages of text at a time, short of the hundreds of pages AI startup Anthropic's flagship model can process, for comparison. OpenAI is testing a version of GPT-20 with a context window of 4 tokens, but only in a limited version.
On the plus side, OpenAI says that it is reducing the price of GPT-3.5-turbo, the original, not the version with the expanded context window, by 25%. Developers can now use the model for $0.0015 per 1000 input tokens and $0.002 per 1000 output tokens, which equates to roughly 700 pages per dollar.
The price is also being reduced for text-embedding-ada-002, one of the most popular OpenAI text embedding models. Text embeds measure the relationship of text strings and are commonly used for search (where results are ranked by relevance to a query string) and recommendations (where items with related text strings are recommended).
text-embedding-ada-002 it now costs $0.0001 per 1,000 tokens, a 75% reduction from the previous price. OpenAI says the reduction was made possible by greater efficiencies in its systems, a key area of focus for the startup, to be sure, as it invests hundreds of millions dollars in R&D and infrastructure.
OpenAI has noted that incremental updates to existing models, not massive new models from scratch, are their modus operandi following the release of GPT-4 in early March. At a recent conference hosted by the Economic Times, CEO Sam Altman reaffirmed that OpenAI has not started training the successor to GPT-4, indicating that the company "has a lot of work to do" before starting that model.
