WizardLM-2, which is 'very close to GPT-4', was urgently withdrawn by Microsoft. What's the inside story?

Some time ago, Microsoft made an own mistake: it grandly open sourced WizardLM-2, and then withdrew it cleanly soon after.

Currently queryable WizardLM-2 release information, this is an open source large model "truly comparable to GPT-4", with performance in complex chat, multi-language, reasoning and agency has been improved.

The series includes three models: WizardLM-2 8x22B, WizardLM-2 70B and WizardLM-2 7B. Among them:

• WizardLM-2 8x22B is the most advanced model and the best open source LLM based on internal evaluation for highly complex tasks.
• WizardLM-2 70B has top inference capabilities and is the first choice of the same scale;
• WizardLM-2 7B is the fastest, among which Performance is comparable to existing open source leading models that are 10 times larger.

Additionally, based on human preference evaluation, WizardLM-28x22B’s capabilities “were only slightly behind the GPT-4-1106 preview, but Significantly stronger than CommandRPlus and GPT4-0314."

##It will become another popular version like LLaMa 3. Open source milestone?

While everyone was busy downloading the model, the team suddenly withdrew everything: blog, GitHub, HuggingFace all got 404.

Picture source: https://wizardlm.github.io/WizardLM2/

Team’s The explanation is:

Hello to all Huggingface friends! Sorry, we removed the model. It's been a while since we released a model from a few months ago, so we're not familiar with the new release process now: we accidentally left out a necessary item in the model release process - toxicity testing. This is a step that all new models currently need to complete.

We are currently completing this test quickly and will re-release our model as soon as possible. Don't worry, thank you for your concern and understanding.

However, the AI ​​community’s attention and discussion on WizardLM-2 has not stopped. There are several doubts:

First, the deleted open source projects are not just WizardLM-2, all the Wizard series work of the team is gone, including the previous WizardMath and WizardCoder.

Secondly, some people question why the blog is also deleted when the model weights are deleted? If it is just missing the testing part, there is no need to withdraw it completely.

#The team’s explanation is: “According to relevant regulations.” What are the specific regulations? No one knows yet.

Third, there are also speculations that the team behind WizardLM has been fired, and that the withdrawal of the Wizard series project was also forced.

However, this speculation was denied by the team:

Picture source: https://x.com/_Mira___Mira_/status/1783716276944486751

##Picture source: https://x.com/ DavidFSWD/status/1783682898786152470

And when we search for the author’s name now, it has not completely disappeared from Microsoft’s official website:

Image source: https://www.microsoft.com/en-us/research/people/qins/

Fourth, some people speculate that Microsoft has withdrawn this The open source model is, firstly, because its performance is too close to GPT-4, and secondly, because it “collides” with OpenAI’s technical route.

What is the specific route? We can take a look at the technical details of the original blog page.

The team stated that through LLM training, human-generated data in nature is increasingly exhausted, and AI-carefully created data and AI Step-by-Step supervised models will be the gateway to more powerful The only way to go with AI.

Over the past year, the Microsoft team has built a synthetic training system fully powered by artificial intelligence, as shown in the figure below.

is roughly divided into several sections:

Data preprocessing:

1. Data analysis: Use this pipeline to obtain the distribution of different attributes of new source data, which helps to have a preliminary understanding of the data understanding.
2. Weighted sampling: The distribution of the best training data is often inconsistent with the natural distribution of human chat corpus, and the weight of each attribute in the training data needs to be adjusted based on experimental experience.

Evol Lab:

1. Evol-Instruct: A lot of effort has been invested in re-evaluating the various problems existing in the original Evol-Instruct method and making preliminary modifications to it. The new method can allow various agents to automatically Generate high-quality instructions.
2. Evol-Answer: Guide the model to generate and rewrite responses multiple times, which can improve its logic, correctness and affinity.

AI Align AI (AAA):

1. Co-teaching: Collect WizardLM and each Licensing state-of-the-art models, both open source and proprietary, and then letting them work together to teach and improve each other, including mock chats, quality critiques, suggestions for improvements, and closing skill gaps.
2. Self-Teaching: WizardLM can generate new evolutionary training data for supervised learning and preference data for reinforcement learning through activation learning.

Learning:

1. Supervised learning.
2. Stage - DPO: In order to perform offline reinforcement learning more effectively, the preferred data is divided into different fragments and the model is improved step by step.
3. RLEIF: Using a method that combines the instruction quality reward model (IRM) and the process supervision reward model (PRM) to achieve more precise correctness in online reinforcement learning.

The last thing to say is that any speculation is in vain, let us look forward to the comeback of WizardLM-2.