What happens when you teach a parrot to teach a parrot?

We’ve built LLMs by training them on the sum total of humanity’s written knowledge. That’s honestly an incredible achievement. These models can write code, explain legal concepts, design mechanical parts, summarize research papers, and somehow know the answer to that obscure Linux error from a no name tool.

They’re impressively capable. But, at the end of the day, they’re still fundamentally predicting the next word based on what humanity has already written. They aren’t discovering new laws of physics in the background. They’re regurgitating what already exists.

---

I suspect a surprising amount of an LLM’s usefulness comes from something innocuous. It has read things that you havent and cant. Sure, some of that information is available with a google search if you are willing to spend a couple of hours digging through pages. But, a huge amount of a valuable knowledge is scattered across academic papers, conferences, videos, textbooks, paywalled journals, etc… You only have access to wikipedia and forum posts. Wikipedia is incredible, dont get me wrong. But, that is not a level playing field.

The model has access to everything, and I mean everything. I wonder, what would happen if you took the original google search algorithms and pointed them at the entirety of human knowledge instead of the paid public web? Would we still think AI is magical? Maybe. Probably not. But I digress…

---

One thing that’s been true in machine learning since the very beginning is that your training data matters. Ask anyone who has trained production models and they’ll all tell you the same thing: cleaning your data buys you more performance than pretty much anything else. It doesn’t matter if you are training a perceptron model, a CNN, a RNN, an LLM. Garbage in, garbage out. Its as simple as that.

That leads to a question I do not hear discussed nearly enough. What happens when the amount of text produced by LLMs surpasses the amount of text produced by humans? How far away are we from that reality?

---

An increasing fraction of everything written online is generated or heavily assisted by a model. My storefront certainly is. My emails are proofread by one. Hell, parts of this post are.

What happens when we train that next generation of the model? Well, we are going to train it on its own output, or at the very least previous generations outputs. So, we are training a model to spit out text which sounds like an LLM spitting out text which sounds like a human. How do you separate the original human knowledge from the synthetic version?

---

I suspect theres a tipping point where this stops making models better and starts making them worse. Maybe its a form of overfitting. Maybe it deserves a new name. Whatever you want to call it, it deserves a name. The result seems inevitable. I don’t know if it will happen in one generation, or two, or ten, or a hundred. But, language will slowly drift towards AI sounding because thats what future models are trained on.

Having an output which sounds AI sounding wont be so bad, will it? But, what about the hallucinations? Training models on hallucinations? Every copy gets worse and worse.

Remember when you were a kid stealing music? You would hold a microphone up to a speaker and burn it onto a cassette or disk? Every copy got worse and worse. Not that I ever did that of course!

LLMs start producing text to sound like LLMs producing text to sound like LLMs producing text to …