In the Ruins of AI: Alchemy Meets the Luddite Resistance

In this paper I describe two contrasting trends that provide insight into the
development and reception of AI systems as epistemic tools. Destabilising patterns are
identified that cast doubt on the long-term sustainability of AI as flawed systems
infiltrate and disrupt knowledge production and dissemination networks at scale,
triggering an epistemological crisis. Finally, I develop a speculative scenario of total AI
collapse that points to new modes of cultural resistance and the potential for an
emancipatory AI.

We begin with the persistence of Artificial General Intelligence (AGI) narratives,
both in public communications and as driver of infrastructural investment. AGI drives
the incentive to scale large language models (LLM) at all costs with devastating
environmental consequences, despite the overwhelming consensus amongst
researchers (AAAI, 2025) that AGI cannot be achieved with current methods. These
‘Sparks of AGI’ (Bubeck at al., 2023) arguments, which have a long history (Dreyfus,
1966), maintain AGI as Silicon Valley’s philosopher’s stone, while the operationalisation
and monetisation of actually-existing AI is more prosaic. These speculative narratives
distort discourse around AI ethics by introducing existential risk scenarios, whereby AGI
inexorably leads to a Terminator-type malevolent superintelligence. A fabulation
manifested by the popularisation of P(doom) numbers amongst Silicon Valley insiders.
(Roose, 2023)

This is contrasted with the rise of what might be called neo-luddism. This
revisionist reading of the 19th century Luddite movement seeks to conceptually frame
disparate movements resistant to the widespread implementation of AI first policies,
from algorithmic sabotage to limiting data centre growth. (Merchant, 2023) This trend
draws parallels between the luddite struggle to preserve workers’ rights and resist the
deskilling and proletarianisation of industrial capitalism and AI’s extraction of
knowledge to automate and displace workers; with the products of this automation
lower quality, more environmentally damaging, socially regressive, and contributing to
the generalised degradation of everyday life of ‘enshittification’.

This paper proposes that these trends signal the growing epistemic crisis
triggered by LLM, which is intensifying as they infiltrate all areas of knowledge
production. As LLM exhaust available training data the distinction between authentic, or
original, knowledge and generated knowledge for training begins to blur which as recent
research shows destabilises models resulting in model collapse. (Shumailov, 2024) In
parallel, as generated material floods the network the ability of humans to distinguish is
reduced with uncertain epistemic consequences.

The paper concludes with a speculative scenario of AI collapse brought about by
model collapse. This fabulation maps such a model in a near future scenario where AI first policies result in widespread integration of AI across society, supported by weakened ethics standards, energy-hungry infrastructural growth, and erosion of privacy and intellectual property rights. The far-reaching consequences are explicated with this analysis indicating future directions, new modes of cultural resistance, and the potential for an emancipatory AI.

References
“AAAI 2025 Presidential Report on the Future of AI Research.” Washington DC:
Association of the Advancement of Artificial Intelligence, March 2025.
https://aaai.org/wp-content/uploads/2025/03/AAAI-2025-PresPanel-Report-FINAL.pdf.
Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric
Horvitz, Ece Kamar, Peter Lee et al. "Sparks of Artificial General Intelligence: Early
experiments with GPT-4." arXiv e-prints (2023): arXiv-2303.
Dreyfus, Hubert L. Alchemy and artificial intelligence. 1965.
Merchant, Brian. Blood in the machine: The origins of the rebellion against big
tech. Hachette UK, 2023.
Roose, Kevin. “Silicon Valley Confronts a Grim New Metric.” New York Times,
December 6, 2023. https://www.nytimes.com/2023/12/06/business/dealbook/siliconvalley-artificial-intelligence.html.
Shumailov, Ilia, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross
Anderson, and Yarin Gal. “AI Models Collapse When Trained on Recursively Generated
Data.” Nature 631, no. 8022 (July 25, 2024