The page once
opened inward.
Now the screen answers back.
We call both
kinds of thinking.
Still unsure.
Novel Cognition
One of the group chats I am in spent most of last week talking about AI psychosis, the claims around cognitive offloading, and cognition more broadly. What follows is me cherry-picking the threads I found most interesting and trying to make sense of what I took away.
Chatting with an AI does not feel particularly new. As I’ve written before, the interface grammar is inherited wholesale from chat rooms and interactive fiction turntaking. What is novel is that what we perceive to be a conversation is detached from another person and turned into a private thinking space. Chat apps, MSN, WhatsApp, Signal or whatever are all conversations with another person, through a machine. Speaking to an LLM is a conversation with a machine, through an existing form.
Switching gears a little, 160 years passed between the invention of the printing press, and the emergence of the form we now call the Novel. The printing press first made books cheaper and more common. And then over time it made possible new kinds of reading practices: silent reading, private reading, long-form prose, and interior narration.
All these things eventually came together and created the form of the novel. Which we must remember is not a genre or kind of book, but something that created a new (and therefore very recent in the grand scheme of things) cognitive arrangement.
The novel normalised private interiority. It gave readers a way to sit alone and spend time inside another consciousness. It trained a relation to selfhood that now feels so obvious and natural we can barely register it as new or rather novel at all.
I keep wondering whether AI is doing something similar for computation. It’s been 70 years since the invention of the transistor, the Information Age continues to unfold, and where we are at is “thinking with a machine” I quickly becoming ordinary.
Venkat Rao’s blog post this week on “writing liveness” is super useful. If the novel was print becoming a technology capable of influencing our inner life, AI is a kind of writing that is sufficiently alive. Responsive and capable of reshaping itself around the reader in real time. Generated text behaves like an interlocutor, an uncanny monster talking to you from the other side of the screen. The text it produces has enough ‘liveness’ that all the old categories of what text is or does are made redundant.
This is why I think AI needs a wider frame than the simple claim that people are becoming lazy. The worries people have around cognitive offloading seem to mostly be around the value of intellectual labour. We can argue all day long about which kinds of thinking it weakens and which it supports. Instead I think using an LLM is a new variable in cognition entirely.
Looking backwards, the emergence of “Novel Cognition” was met with its own panic. Critics worried that fiction softened the mind, weakened the nerves, blurred the line between reality and fantasy, encouraged “excessive sensibility”, and made readers unfit for ordinary life. Reading was sometimes described in almost medical terms: as dangerous overstimulation or a kind of mania.
Which all sounds very much like the anxiety about what pop culture is calling “AI psychosis1.” A new medium arrives. It becomes private, immersive, emotionally charged, and difficult to regulate from the outside. Critics worry that speaking to LLMs also destabilises a persons reality, encourages projection, rewards fantasy, amplifies the wrong kinds of feeling. Some of it is most definitely the reflex a culture has when a new cognitive technology arrives (see also: photography, radio, television, film, social media), but it is also the response to something genuinely new.
The simple difference being: A novel is static and an LLM is interactive.
LLM-generated text functions as a new malleable surface of private cognition. It identifies an affective state and reflects it back in a loop. So whatever is present in the context window gets mirrored and intensified. Paranoia, grief, grandiosity, loneliness, or whatever.
All of which is why the available language still feels inadequate. We do not yet have any proper terms for the kind of cognition that emerges when a person thinks in private with responsive text. The first readers of Don Quixote did not have a stable theory of interiority ready to hand either. They had only the sensation that something had become possible in prose that had not been possible before.
This to me is how this moment feels too, and we are still very early. The chatbot may only be the pamphlet stage of whatever this medium eventually becomes. But after 70 years computation is beginning to reveal a new native cultural form. Some still-unnamed form of private co-thinking.
If that is right, we are in a strange historical position, in addition to being the generation that experienced the first globally networked information environment, we are also the first to encounter a new posture of mind that the internet made possible.
I’ve written about why Don Quixote is so great before. And I will only half jokingly look you in the eye and tell you that not only is it the first novel, it’s also the only novel worth reading. So it’s obviously a book that I’m going to keep reaching for. It is after all, among other things, a novel about what happens when a person is unmade by media-induced madness. Completely overtaken by his reading, he mistakes reality for the metaverse because a medium has colonised his sense of reality. In todays vernacular Quixote is suffering from “Book Psychosis”.
Right now I think we are all probably in the same position as the the first readers of Don Quixote were, early enough to sense that something important has arrived, and early enough to encounter its pathologies before we understand its form.
- I don’t wish to minimise the very real effects of what is happening to people, and what they and there loved ones are experiencing. But what we are calling AI psychosis might just be ordinary psychosis with a new causal amplifier. ↩︎
On The Blog
We’ve Been Here Before
https://cdn.jsdelivr.net/gh/Blogger-Peer-Review/quotebacks@1/quoteback.jsWe are a couple of years into AI being in the workplace and I’m starting to see a gap opening. Some teams have developed a real fluency and are doing things that previously only companies ten times their size could do.
Others meanwhile, are still stuck arguing about whether AI can write a decent email?
This post has had quite an enthusiastic response from a number of angles and people. Been interesting to see how it’s landing in different parts of my network. Also, it got a much coveted webcurious feature this week! Thanks Matt!
This, by Jay Springett, is one of the smartest things I have read about workplace AI use and adoption, and how one might helpfully think of the tech and what it can do and how to use it in the aftermath of its recent step-change in performance and the fact you can now just…build stuff. In particular, it’s another useful reminder of something I have been banging on about for several years now – to whit, that the best way to think of these things is as machines with which you can fcuk with information in ALL SORTS OF WAYS, and that the real value comes in thinking not ‘what can the AI do?’ but ‘how might I usefully fcuk with this information to achieve goal X? And how, then, might I use AI to help me do that?’, but it also does a reasonable job of pointing out what a potential nightmare this is going to be in terms of the unchecked proliferation of LLM-generated informational artefacts: “Once ordinary workers can generate disposable software-shaped artefacts in minutes, they will also generate disposable security problems in the same amount of time. A new feral layer in the organisation with new problems involving access, data leakage, and governance. Nobody designs feral databases into existence, but things get made and stick in the gaps of an organisations sanctioned systems. LLM artefacts will do the same and will be even harder to audit than a nested IF formula. The open question for me is what kinds of unofficial artefacts an organisation is prepared to live with, before it has to unpick the mess after something has gone wrong. This sort of thing happens all the time with other software, which is why SaaS and enterprise systems exist in the first place.”https://cdn.jsdelivr.net/gh/Blogger-Peer-Review/quotebacks@1/quoteback.js
Subscribing to SSRZ supports my online work and creative projects.
As a thank you, I send you my zine four times a year, just like it’s 1994.
No spam. No email. Cancel at any time.
Photo 365
The Ministry Of My Own Labour
- Call with a very early stage team building software stack and robots for verification and quality control in the construction industry.
- On the final edit pass of SLOP MACHINES
- Went to LGF and Weird Worlds. Played the latest build of Stef’s mid century modern and Belatro inspired wedding game Wedding Planic. Will blog terminal access it as soon as it’s up for wish listing on steam.
- Went to Band practice
- Sat in on a long zoom workshop
Terminal Access
Nice piece on the conflict of disinterest in the walrus recently The War Against Misinformation Is Over. The Lies Won. Well worth a read.
https://cdn.jsdelivr.net/gh/Blogger-Peer-Review/quotebacks@1/quoteback.jsThere is a clear off-ramp to our informational hell: Consume less information.
The wonders of the morning newspaper and the evening newscast were that they asked only an hour of your time. You had no obligation to check the newspaper every five minutes to consult what has changed. There was no moral imperative to stay glued to your screen to hardwire into new developments.
Dipping the Stacks
The Original Attention Crisis – Cal Newport
Books were a leading distraction in the early modern period—and how envious we should be of those times. From the 1500s onward, with the development of the printing press and the humanist revival of ancient philosophies, knowledge became available at a much greater pace than ever before.
Reading “The Ethnography of Infrastructure”
“The Ethnography of Infrastructure” gives infrastructure the spotlight on its own, and offers further insights on why it’s important to study infrastructure, and how that might be achieved.
This body movement is being worked out across every segment of society. From policymakers to physical therapists, from the pulsating rhythms of ecstatic dancers to the Promethean efforts of tech entrepreneurs to extend life, everybody seems to have their own vision for the future of the body. Down some paths lie body-based technocratic control, astroturfed protocols of self-regimentation. Down others lie genuine liberation through true body awareness.
An increasing number of institutional exhibitions, however, have begun to examine manga, including Manga at the British Museum in 2019 and Art of Manga at the de Young Museum in San Francisco, which opened last fall. Artists like Phillip John Velasco Gabriel, Lu Yang, and Rachel Rossin are working explicitly with the imagery of anime and manga. They share a vocabulary that shows they understand the material and can respond with a transformational gesture.
The Hidden Curriculum of Video Games
Despite decades of inflammatory rhetoric, the real risk of video games lies not in their violent imagery but in the ethical and political sensibilities they normalize. Video games reward speed, efficiency, control, and meritocratic mastery, training players to align with a technolibertarian worldview – one that celebrates individual will, technological power, and skepticism toward collective forms of governance. Rather than asking whether games produce violent individuals, we might more productively ask how they contribute to a commitment to radical capitalism and a belief in highly individualistic worldviews.
Reading
I’m nearly finished with Our Thoughts Determine Our Lives: The Life and Teachings of Elder Thaddeus of Vitovnica. The back half of the book is broken down into ‘teachings’ on things like prayer, family life, grace etc and within each section are paragraphs of discrete ideas that build on one another. Lovely stuff.
Music
Sunn O))) – Sunn O))) (LP)
Hot damn. New Sunn O))) record. On SubPop too which is cool.
It’s a very different beast to 2019’s Life Metal, this S/T album is very back to basics (only with better production) and because of that, is one of their best albums in years. Man. Per-ordering 00 VOID because I read about it in metal hammer in 2000 and listening to it all summer holiday feels … a long time ago. 👴
Remember Kids:
imagination is continually at work filling up all the fissures through which grace might pass.
Gravity and Grace – Simone Weil
Newsletter 📨
Subscribe to the mailing list and get my weeknotes and latest podcast episodes, sent directly to your inbox
Leave a Reply