Llama becomes the top model among big models, Zuckerberg starts a debate: Playing open source, times have changed

The dispute between open source and closed source has been going on for a long time, and now it may have reached a new climax. When it comes to open source large models, the Llama series has been a typical representative since its birth. Its excellent performance and open source features have greatly improved the applicability and accessibility of artificial intelligence technology. Any researcher and developer can benefit from it, making research and applications more widespread. Now, Meta Llama 3.1 405B is officially released. In the official blog, Meta said: "Until today, open source large language models have mostly lagged behind closed models in terms of functionality and performance. Now, we are ushering in a new era led by open source."

Meta founder Zach Berg explains the significance of open source to AI

1. Open source is a necessary condition for the development of AI

Meta founder and CEO Zuckerberg said that open source is crucial to the positive development of AI. Citing the development of Unix and Linux as examples, he believes that open source AI will promote innovation, data protection and cost-effectiveness.

2. Open source Llama model to build a complete ecosystem

Zuckerberg believes that the open source Llama model can build a complete ecosystem to ensure technological progress and avoid losing advantages due to competition.

3. Meta’s Open Source History and Vision

Meta has a successful open source history. Zuckerberg hopes to promote equal and safe application of global technology through open source AI models.

Original link: https://about.fb.com/news/2024/07/open-source-ai-is-the-path-forward/
The following is the original content:
Zuckerberg injects open source genes into Meta
In the early days of high-performance computing, major mainstream technology companies invested heavily in developing their own closed-source versions of Unix. At the time it was difficult to imagine any other way to develop such advanced software.
However, open source Linux gradually became popular: initially because it allowed developers to freely modify the code and was more affordable, but later it became more advanced, more secure, and had a wider ecosystem of support than any closed source Unix More features. Today, Linux is the industry-standard foundation for cloud computing and the operating system that runs most mobile devices, and everyone benefits from its superior products.
I believe that the development of artificial intelligence will follow a similar trajectory. Today, some tech companies are developing leading closed-source models, but open source is quickly closing the gap.
Last year, the Llama 2 was only comparable to a generation older model. And this year, the Llama 3 is already competing with or even ahead of the leading models in the industry in some areas. Starting next year, we expect future Llama models to be the most advanced large models in the industry. Prior to this, Llama also already led the way in openness, modifiability and cost-effectiveness.
Today, we’re taking the next step – making open source AI the industry standard. We released the first cutting-edge open source AI model, Llama 3.1 405B, as well as improved versions of the Llama 3.1 70B and 8B models. These open source models are significantly more cost-effective than closed source models, especially the open source nature of the 405B model, making it the best choice for fine-tuning and distilling small models.
In addition to releasing these models, we are also working with multiple companies to expand the broader ecosystem. Amazon, Databricks, and Nvidia are launching full suites of services to enable developers to fine-tune and distill their own models. Innovators like Groq have built low-latency, low-cost inference services for all new models.
These models will be available on all major cloud platforms including AWS, Azure, Google, Oracle, etc. Scale.AI, Dell, Deloitte and others are already ready to help enterprises adopt Llama and train custom models using their own data. As the community grows and more companies develop new services, together we can make Llama an industry standard and bring the benefits of AI to everyone.
Meta is committed to open source AI, here are the reasons why I think open source is the best development platform, why open source Llama is good for Meta, and why open source AI is good for the world and will be here for a long time.
Open Source AI for Developers
When I talk to developers, CEOs, and government officials around the world, I typically hear a few themes:
We need to train, fine-tune, and distill our own models. Every organization has different needs that are best served by using models that are trained or fine-tuned at different scales and with specific data. On-device tasks and classification tasks require small models, while more complex tasks require large models. Now you can take state-of-the-art Llama models, continue to train them on your own data, and then distill them to the model size that best suits your needs—without us or anyone else seeing your data.
We need to control our own destiny instead of being "locked" by closed source suppliers. Many organizations do not want to rely on a model that they cannot run and control themselves. They don't want closed source model vendors to be able to change the model, change the terms of use, or even stop the service entirely. They also don’t want to be locked into a single cloud platform with exclusive rights to their models. Open source enables a broad ecosystem of compatible toolchains that you can easily switch between.
We need to protect our data. Many organizations handle sensitive data that needs to be protected and cannot be sent to a closed source model via cloud APIs. Some organizations simply don't trust closed-source model vendors with their data. Open source solves these problems because it allows you to run the model anywhere you want. It is well known that open source software is more secure because the development process is more transparent.
We need an efficient and economical model. Developers can run Llama 3.1 405B on their own infrastructure for inference at approximately 50% the cost of using closed-source models such as GPT-4, suitable for both client-side and offline inference tasks.
We want to invest in an ecosystem that will become the long-term standard. Many see open source evolving faster than closed models, and they want to build their systems on an architecture that provides the greatest advantages over the long term.
Open Source AI to Meta
Meta’s business model is to build the best experiences and services for people. To achieve this, we must ensure that we always have access to the best technology and not be locked into a closed ecosystem of competitors so that they cannot limit what we develop.
I want to share an important experience: although Apple allows us to build content on its platform, we are still restricted when building services.Whether it's the taxes they impose on developers, the arbitrary rules they impose, or all the product innovation they prevent, it's clear that if we can build the best version of our product and competitors can't limit what we build, Meta and many other companies will be able to provide better services to people. On a philosophical level, this is a big reason why I believe so strongly in building an open ecosystem for the next generation of computers in AI and AR/VR.
People often ask me if I'm worried about losing technical advantage by open sourcing Llama, but I think this misses the big picture for a few reasons:
First, to ensure that we can maintain technology leadership in the long term and not be locked into a closed source ecosystem , Llama needs to evolve into a complete ecosystem including tools, efficiency improvements, hardware optimizations, and other integrations. If our company was the only one using Llama, the ecosystem would not grow and we would be no better off than with closed-source variants of Unix.
Second, I expect AI development to continue to be highly competitive, meaning that at any given moment, open sourcing a model won’t cause us to lose a huge advantage in the competition against the next best model. Llama's path to becoming an industry standard is by remaining competitive, efficient and open generation after generation.
Third, a key difference between Meta and closed source model providers is that selling access to AI models is not our business model. This means releasing Llama publicly will not impair our revenue, sustainability or ability to invest in research, whereas closed source providers would. (This is one reason why some closed-source providers have been lobbying public administrators against open source.)
Finally, Meta has a long history of success with open source projects. We've saved billions of dollars by sharing our server, network and data center designs with the Open Compute Project and standardizing the supply chain. By open-sourcing leading tools like PyTorch, React, and more, we benefit greatly from innovation in the ecosystem. This approach has been extremely effective for a long time.
Open source AI to the world
I believe open source is necessary for the future of AI. AI has the potential to increase human productivity, creativity, and quality of life more than any other modern technology, and to advance medical and scientific research while accelerating economic growth. Open source will ensure that more people around the world can reap the benefits and opportunities from the development of AI, that power is not concentrated in the hands of a few companies, and that technology can be deployed more evenly and securely across society.
There is an ongoing debate about the safety of open source AI models. My point is that open source AI will be safer than the alternatives. I think governments will eventually come to the conclusion that they support open source because it will make the world more prosperous and secure.
In the security framework I understand, we need to guard against two types of harm: unintentional and intentional.
Unintentional harm refers to the possibility that an AI system may unintentionally cause harm while running. For example, modern AI models may inadvertently give incorrect health advice. Or, in future scenarios, there is concern that models may inadvertently replicate themselves or over-optimize goals, to the detriment of humans.
Intentional harm is when bad actors use AI models with the intention of causing harm.
It’s worth noting that unintentional harm covers most of the concerns people have about AI – from the impact of AI systems on billions of users to most truly catastrophic science fiction scenarios. In this regard, the security offered by open source is even more significant because the system is more transparent and can be widely scrutinized.
Historically, open source software has been more secure for this reason. Likewise, using Llama and its security systems like Llama Guard will likely be safer and more reliable than a closed-source model. As a result, most discussions about open source AI safety focus on intentional harm.
Our security process includes rigorous testing and red team evaluation to verify whether our models have the potential to cause substantial harm, with this goal

The above is the detailed content of Llama becomes the top model among big models, Zuckerberg starts a debate: Playing open source, times have changed. For more information, please follow other related articles on the PHP Chinese website!