Large AI models are very expensive and only big companies and the super rich can play them successfully

ChatGPT fire has led to another wave of AI craze, but the industry generally believes that when AI enters the era of large models, only large companies and super-rich companies can afford AI, because AI Large models are very expensive to build.

First of all, it is computationally expensive. Avi Goldfarb, a marketing professor at the University of Toronto, said: "If you want to start a company, develop a large language model yourself and calculate it yourself, the cost is too high. OpenAI is very expensive, costing billions of dollars." Of course, leasing computing will be much cheaper, but enterprises will still have to pay expensive fees to companies such as AWS.

Secondly, data is expensive. Training models requires massive amounts of data, sometimes the data is readily available, sometimes not. Data such as Common Crawl and LAION are free to use. For this type of data, the cost mainly comes from data cleaning and processing. The cost can vary widely, ranging from a few hundred dollars to millions of dollars.

Debarghya Das, founding engineer of Glean, said that in the United States, based on some rough mathematical calculations based on large language model papers, if Facebook LLaMA is used, the training cost (not considering iterations or errors) is about US$4 million. , if it is Google PaLM, about $27 million.

Even if you use free data, the cost is not low. "When you download terabytes of data, if you want to filter or use the data in some special way, such as using a text-image model, researchers will focus on certain subsets of the data," said Sasha Luccioni, a researcher at Hugging Face. Only in this way will the model get better), the whole process is quite tricky." It requires powerful computing power and a large number of professionals.

Thirdly, the cost of hiring professionals is also very high. Debarghya Das did not consider labor costs when making the above cost estimate. Sasha Luccioni pointed out: "Machine learning professionals are paid very well because they compete with Google and other technology giants for talent, and sometimes a professional talent can cost millions of dollars." In 2016, the salary of the top researchers at OpenAI was about 190 Ten thousand U.S. dollars.

Moreover, the costs of training models and hiring professionals are not one-time but ongoing. For example, if you are developing a customer service chatbot, you need to optimize it every week or every few weeks. The model is also subjected to stress testing to ensure that the answers it generates are correct. As Sasha Luccioni explains: "The most expensive cost comes from the ongoing work, having to continuously test the model, having to make sure that the AI ​​is doing what it is supposed to do."

Finally, ongoing operating expenses are not cheap either. When everything is ready and the model is open to the public, it will receive thousands of inquiries every day. At this time, it is necessary to ensure that the model is scalable and highly stable. The maintenance cost is also high and requires professionals to handle it.

The above is the detailed content of Large AI models are very expensive and only big companies and the super rich can play them successfully. For more information, please follow other related articles on the PHP Chinese website!