Deleted transcript of Sam Altman's talk: Open AI also lacks GPUs, and cost reduction is the primary goal

1.2 billion US dollars, almost all the computing power, after Microsoft handed over "half life" to OpenAI.

Author | Lingzijun

Editor | Wei Shijie

SamAltman’s European tour is still underway. Not long ago, he had a private meeting with the CEO of the artificial intelligence company HumanLoop in London. HumanLoop is a company that provides services for building applications on large language models, and its goal is to help developers achieve this goal.

HumanLoop CEO Raza Habib recorded the key points of the conversation and made them available on the company’s official website. But then the minutes were removed at the request of OpenAI. This actually increased the curiosity of the outside world about this conversation. Some people speculate that some of the ideas involved in OpenAI have changed.

After browsing the deleted conversation minutes, Geek Park discovered that it not only involved Sam’s short-term plan for OpenAI, but also hidden informationAfter receiving strong support from Microsoft’s cloud computing resources , the pressure on OpenAI. After all, Model fine-tuning and inference still consume a lot of computing resources. According to The Information, the Open AI model has cost Microsoft Azure $1.2 billion. Focusing computing resources on supporting OpenAI also limits the servers available to other Microsoft departments.

In this regard, Sam said thatReducing costs is the current primary goal.

In addition, Sam also revealed: Currently, services such as opening longer context windows and providing fine-tuning APIs are limited by GPU resources;

In this dialogue, Sam Altman responded to many issues of concern to the outside world, such as competition and commercialization:

1. Despite having just hired a world-class product manager, Peter Deng, OpenAI will not consider releasing any more products;
2. The future application trend is to embed the functions of large models into more APPs, rather than growing more plug-ins on ChatGPT, because in reality most plug-ins do not show PMF (Product/Market Fit) ;
3. In the past few years, OpenAI has expanded model scale at a rate of millions of times, but this speed is unsustainable. Next, OpenAI will continue to increase the model size at a rate of 1 to 3 times to improve model performance.

The minutes of the conversation were made public on May 29 and were deleted around June 3 according to netizens’ records. The following is what is obtained from the backup:

01

OpenAI is currently under

Serious limitations of GPU

As the conversation expands,

Required computing resources grow exponentially

Currently OpenAI’s GPUs are very limited, which delays many of their short-term plans. The biggest complaints from customers are the reliability and speed of the API. Sam acknowledged their concerns and explained that most of the issues were due to GPU shortages.

The longer 32k context can』t yet be rolled out to more people. OpenAI haven』t overcome the O(n^2) scaling of attention and so whilst it seemed plausible they would have 100k - 1M token context windows soon (this year) anything bigger would require a research breakthrough.

The longer 32K context is not yet available to more people. OpenAI has not yet overcome the O (n ^ 2) scaling problem of the attention mechanism, although it looks like they will have a context window of 100k-1M Tokens soon (this year). Any larger window would require research breakthroughs.

Note: O (n^2) means that as the sequence length increases, the computing resources required to perform Attention calculations increase exponentially. O is used to describe the upper limit or worst case scenario of the growth rate of algorithm time or space complexity; (n^2) means that the complexity is proportional to the square of the input size.

The fine-tuning API is also currently limited by GPU availability. They haven't used efficient fine-tuning methods like Adapters or LoRa, so running and managing (the model) through fine-tuning is very computationally intensive. Better support for fine-tuning will be provided in the future. They might even host a community-based marketplace for model contributions.

Dedicated capacity provision is subject to GPU availability. OpenAI provides dedicated capacity to provide customers with private copies of models. To get the service, clients must be willing to commit $100,000 upfront.

02

OpenAI’s near-term roadmap

2023, reduce smart costs;

2024, Limited Demonstration of Multimodality

Sam also shared what he sees as an interim near-term roadmap for the OpenAI API.

2023:

• Cheaper and faster GPT-4 ーーThis is their top priority. Overall, OpenAI's goal is to reduce the "cost of intelligence" as much as possible, so they will work hard to continue to reduce the cost of the API over time.
• Longer context window ーーIn the near future, the context window may be as high as 1 million Tokens.
• Fine-tuning API ー The fine-tuning API will be extended to the latest models, but the exact form will depend on developers indicating what they really want.
• A stateful API - When calling the chat API today, you have to pay the same tokens over and over again through the same conversation history. There will be a future version of the API that remembers session history.

2024：

Multimodality - This is being demonstrated as part of the GPT-4 release, but won't scale to everyone until more GPUs come online.

03

Commercial prediction and thinking:

Plug-in "No PMF",

May not appear in the API soon

Many developers are interested in ChatGPT plugins with API access, but Sam said he doesn't think those plugins will be released anytime soon. In addition to the Brosing plugin, the usage of other plugins indicates that there is no PMF (Product/Market Fit) yet. He pointed out that many people think they want their applications to be within ChatGPT, but what they really want is for ChatGPT to exist within the application.

04

In addition to ChatGPT,

OpenAI will avoid competing with its customers

All great companies have them

A killer app

Many developers say they are nervous about developing using the OpenAI API because OpenAI may eventually release products that are competitive with them. Sam said that OpenAI will not release any more products outside of ChatGPT. He said that historically, great platform companies have a killer application. ChatGPT will allow developers to improve the API by becoming customers of their own products. The vision of ChatGPT is to become a super-intelligent work assistant, but there are many other GPT use cases that OpenAI will not cover.

05

Need supervision,

but not now

「How many people and companies have I respected

Doubtful about the ability to hold large models」

While Sam calls for regulation of future models, he does not believe existing models are dangerous and believes regulating or banning them would be a big mistake. He once again emphasized the importance of open source and said that OpenAI is considering making GPT-3 open source. They haven't open sourced it yet, in part because he's skeptical about how many individuals and companies are capable of holding and serving large language models (LLMs).

06

The law of scale still applies

Expansion speed of millions of times in a few years,

It can’t go on forever

There have been many articles recently claiming that "the era of giant AI models is over." This is not accurate. (Note: At an event at MIT in April, Sam Altman said: We are now nearing the end of the era of giant models.)

OpenAI's internal data shows that The law of scale for model performance still applies, increasing model size will continue to improve performance.

Since OpenAI has scaled up its models millions of times in just a few years, this rate of expansion cannot be sustained. This doesn’t mean that OpenAI won’t continue to try to make models bigger, but it does mean that instead of increasing by many orders of magnitude, they may double or triple in size each year.

Having the law of scale in effect has important implications for the AGI development timeline. The law of scale assumes that we probably already have most of the elements needed to build AGI, and the remaining work is mainly about scaling existing methods to larger models and larger data sets. If the age of scale is behind us, then we may be even further away from AGI. The fact that the law of scale continues to apply strongly hints at a shorter timeline.

Geek One Question

What do you think of OpenAI

What is the recent technical route?

Hot video

Full version of Apple Vison Pro product display revealed! No hardware required, see each other. Seamlessly blend the digital and real world!

Like and followGeek Park Video Number,

Watch more exciting videos

Read more

The above is the detailed content of Deleted transcript of Sam Altman's talk: Open AI also lacks GPUs, and cost reduction is the primary goal. For more information, please follow other related articles on the PHP Chinese website!