Lately my feeds have been full of death-of-truth video essays and dead internet threads. It keeps coming up with friends too: someone shares a thing, someone asks "wait, is that even real?", and we can't tell. I went looking for research on what happens when checking becomes too much work and people stop trying.
A lot of the AI risk discussion I read concerns deliberate deception: models concealing their behaviour or gaming evaluations. I'm also worried about the amount of content being produced. Even when nobody intends to deceive, someone still has to check the claims.
One January 2026 study showed participants AI-generated videos of crimes and told them in advance that the videos were fake. Participants who accepted that warning were still influenced by what they'd seen. That's the part I find worrying: recognising a fake didn't fully remove its effect in this experiment.
(I'm starting to hate this fucking timeline more and more gng)
By mid-2025 there were over 1,200 AI-generated "news" sites publishing under plausible mastheads in sixteen languages. The EU counted AI involvement in 27% of foreign information-manipulation attempts, nearly triple the year before. AI-generated misleading posts were also going disproportionately viral despite mostly coming from small accounts.
I remember reading reports about bots making up most internet traffic during COVID, though I can't place the study now. My feeds have since filled with mass-produced engagement bait, including shrimp Jesus, my beloved. I don't think traffic figures alone tell us how much of a conversation is fake, but the posts themselves are hard to miss.
Like bruh there's literal "authors" and "musicians" and "photographers" that now exist that are just a fucking prompt loop with a payout account attached.
What bothers me is how tired people seem of checking. Hank talks about this in a video I liked:
I've been reading about several problems under the heading of "epistemic collapse". The terms differ across fields, and I think some of the problems are connected.
The ML people mean model collapse where you train a model on the outputs of a previous model, repeat a few times, and the tails of the original distribution disappear while each generation drifts further from whatever ground truth it was anchored to, and this has been proven mathematically btw. Model collapse can't be avoided when training solely on synthetic data. The ecosystem version is worse: when researchers tested 27 LLMs across 155 topics every single one was less epistemically diverse than a basic web search and the larger models produced less diverse claims.
Another concern is validation overload. Peer reviewers, fact-checkers, editors, courts, and teachers have limited time. They can receive more material than they can check, especially when producing it takes very little human work.
Then there's 'misrecognition': treating generated text as established knowledge without checking what supports it. A reader can repeat a model's output as a fact, and the next person may trust the reader without knowing where the claim originated.
The social problems include groups relying on incompatible accounts of events and losing common ground. I suspect the reliability of their information sources and the burden of checking them contribute to that, alongside the usual discussion of social media and polarisation.
I think these problems could reinforce one another. Less reliable tools produce more material that needs checking, while the people doing that checking already have more than they can handle. If readers treat the unchecked material as established knowledge, later discussions start from claims nobody verified.
Misinformation is old, and journals, courts, editorial desks, and peer review already deal with it. What worries me is how cheaply someone can now produce a large amount of plausible material. Checking a claim can still take a person hours of reading, tracing sources, or repeating an experiment.
I'm worried that people will start discounting digital evidence generally if distinguishing authentic material from synthetic material becomes too difficult. That would make genuine records harder to use too.
The flip side is the "liar's dividend", once everyone knows convincing fakes exist, anyone caught doing something real on camera can just shrug and go "that's AI", and the more slop is out there the more plausible that defense gets. Let's take some of the AI slop text that's getting generated these days or patterns like: "It's not X it's Y". It's valid english, true, but since we see so much AI slop we tend to start associating that pattern with "AI generated content" and then before long, NOBODY can fucking use that cause even if it's hand written they'd be assumed to have used AI.
Oh and this doesn't include the fact that we humans write similarly to the type of language we use/hear/read and see more of. Which means that over time more PEOPLE will start sounding like AI. (LinkedIn endgame fr)
AI Futures Project's "Plan A" gives much more detail on compute verification, optical network taps, supply chain audits, and deterrence than on epistemics, which gets a two-page appendix. I like its idea of a "basin of sanity": truth-seeking AI tools helping society become better at finding out what's true. I wanted more detail on how to reach that state, measure progress toward it, and keep it stable when some actors benefit from misinformation.
I don't think controlling compute and improving model alignment is enough to handle this. Content produced in good faith can still be wrong and still take time to check. Reducing deliberate deception would help, but it wouldn't remove that workload.
Responsibility is harder to assign here, too. A lab can put a team in charge of its model. Verification work is spread across publishers, researchers, courts, readers, and other institutions with different budgets and incentives. I suspect that makes it harder to organise and fund, though I don't have a funding comparison to establish that.
There are proposals for ecosystem health metrics, diversity requirements for training data, and better-funded verification tools. I don't know which would help enough. I'd want to see them tested against the work people actually do: tracing a claim to its source, checking whether the source supports it, and correcting it when it doesn't.
That's the work I'd like to see get more attention and funding. If we keep making publication cheaper, we also need to make those checks easier.
Anyways, that's the rant. imma fuck of to sleep, gn people.
~ A.