Loading…
Glossary term
Degradation that can occur when models are repeatedly trained on data generated by other models.
If models are trained repeatedly on data produced by earlier models, rare details can vanish and the output distribution narrows. Over generations quality and diversity can decline.
Mixing in fresh human data and filtering carefully reduces the risk.
As AI-written text fills the web, researchers worry about training on it unknowingly.