Synthetic Data and AI Model Inbreeding
Training repeatedly on generated outputs can distort a model's view of the world.
The Mechanism: Synthetic data is produced by algorithms rather than collected directly from the world. Research has shown that recursive training on model-generated material can degrade distributions under some conditions, though outcomes depend on data quality, filtering, and mixing with original sources.
The Philosophical Pivot: A model trained on its own echoes can make repetition look like consensus. The danger is not simply technical error; it is a culture where convenient replicas displace the messy evidence of human experience.
The Horizon: As generated material expands, provenance becomes a public good. Future knowledge systems may depend on preserving links to observed events, diverse communities, and the limits of what a dataset can honestly claim.
The issue reaches beyond model quality to cultural memory. If generated summaries replace the sources they summarize, future systems may inherit a narrower picture while appearing to draw from an abundance of material.
A healthy information ecosystem needs records of origin, not merely more content. Preserving disagreement, minority experience, and source context may matter as much as increasing computational scale.
A mirror can multiply an image while quietly losing the world that stood before it.
Where am I mistaking repeated information for independent evidence?