Matryoshka dolls are not advisable: Researchers confirm that training AI with AI-generated results will lead to model degradation

IT House News on June 14th, IT House friends may have imagined what kind of results can be obtained if the results generated by AI are used to train AI and perform "matryoshka-style training"? There is currently a research team that has observed and recorded this, and detailed papers and results have been published on arXiv.

Summary in one sentence - "Using content generated by the model in training will lead to irreversible defects in the subsequently generated model." In human terms, the researchers found that "training AI with results generated by AI will only make the The models are getting worse."

▲ Picture source arXiv

It is reported that the researchers specifically studied the probability distribution of AI generation models, mainly focusing on "text to text" and "image to image", and finally concluded: "Because the results generated by each model have certain characteristics , so if you train AI with a model generated by AI, over time, the latter will forget the real underlying data distribution."

▲ Picture source arXiv

Ilia Shumailov, one of the main authors of the paper, also said that “over time, errors in the generated data (IT Home Note: such as false examples) will force AI to further misperceive reality. We were surprised to observe that the model Crashes happen quite quickly, and models can quickly forget much of the original data they originally learned from."

But friends may have questions. If the results generated by AI are manually polished and then put into model training, can the model be "degraded"?

The answer is no. The researchers found that "the model degradation process is inevitable", so even for "polished and idealized AI output content", the model will experience certain degradation after long-term learning.

For any large model, due to their excessive learning data, they will inevitably come into contact with data generated by other AIs, so the researchers said that "AI identification should be introduced to pick out learning data that may contain errors." To improve the learning ability and accuracy of the model.

The above is the detailed content of Matryoshka dolls are not advisable: Researchers confirm that training AI with AI-generated results will lead to model degradation. For more information, please follow other related articles on the PHP Chinese website!