Model collapse refers to a situation where a machine learning model, particularly one trained on synthetic data, deteriorates over time. This decline is characterized by:

• Limited output: The model gets stuck producing a narrow range of outputs, repetitive and lacking diversity.
• Reduced creativity: The model struggles to generate original or surprising content.
• Playing it safe: The model tends to favor generic or bland outputs, avoiding any risks.

Here's a simplified analogy to understand model collapse: Imagine a model trained on a dataset with mostly yellow objects and a few blue ones. Over time, the model might prioritize yellow so much that it forgets about blue objects altogether.

There are two main reasons why model collapse happens:

1. Uncurated training data: Machine learning models are trained on data, and the quality of that data significantly impacts the model's performance. If the training data is synthetic or AI-generated and not carefully reviewed, it can contain biases, inaccuracies, or lack real-world context. This can lead the model to learn faulty patterns and ultimately collapse.
2. AI cannibalism: This refers to a scenario where AI models are trained on data that includes outputs from other AI models. This creates a situation where models are essentially re-learning the same biases and limitations from each other, amplifying the problem and accelerating collapse.

Model collapse is a growing concern in AI research as the use of synthetic data becomes more widespread. Researchers are actively looking for solutions to prevent model collapse, such as developing methods to better curate training data and detect biases.