A Language Model Trapped in Time
On April 27, 2026, a small team of researchers released a language model unlike any other.
Named Talkie, the 13-billion-parameter transformer was trained exclusively on text published before 1931.
The announcement, posted to X (formerly Twitter) at 11:34 PM by co-author Nick Levine (@status_effects), immediately drew attention.
Within 24 hours, the tweet had accumulated over 1.1 million views, along with 3,000 likes, 543 replies, and 1,600 bookmarks.
The model was developed by Nick Levine, Alec Radford (@AlecRad), and David Duvenaud (@DavidDuvenaud).
Their stated goal was not to build a practical assistant, but to answer a fundamental research question: how do language models generalize beyond their training distribution?
If a model has never seen a line of code, can it be taught to write Python? If it only knows the world as it was described in 1930, can it reason about the present?
Talkie was built to find out.
The Archive: Building a Pre-1931 Corpus
Training a 13-billion-parameter model requires enormous quantities of text.
For Talkie, the team assembled approximately 260 billion tokens of pre-1931 material.
This figure was raised by user will depue (@willdepue) in the announcement thread and was not disputed by the authors.
The primary data sources were the Institutional Data Initiative and the Internet Archive, supplemented by a "bunch of other sources."
Crucially, the corpus was not a random sample of historical text.
It included dozens of historical reference books with regular, structured formats:
- Encyclopedias capturing the sum of early-20th-century knowledge
- Etiquette manuals codifying social norms of the era
- Letter writing manuals preserving formal communication styles
- Cookbooks documenting domestic life and available ingredients
To extract and structure this data, the team wrote custom parsers using Claude Code.
These parsers transformed scanned and digitized pages into clean, machine-readable text suitable for language model training.
The authors have committed to publishing more details about the corpus in the future.
Where possible, they plan to release the data itself, or at minimum the scripts required to reconstruct the dataset from public sources.
Capabilities, Limitations, and Anachronistic Behavior
Talkie's responses are shaped entirely by the temporal scope of its training data.
Early users quickly identified both charming and unsettling behaviors.
One user reported that Talkie "is claiming that gambling won't exist by 2020."
This is a classic example of anachronistic inference: the model extrapolated from pre-1931 trends — perhaps a period of anti-gambling sentiment or legal prohibition — and projected that trajectory forward, unaware of the massive expansion of legal gambling in the late 20th and early 21st centuries.
Other users noted more fundamental limitations.
Yasi Khan (@yasmeena_khan) observed that Talkie "seems to think it only covers 1700-1799," suggesting the model lacks the metadata or self-awareness to accurately describe its own training period.
The authors did not address this observation in the thread.
Christiano Boria (@christianoboria) offered a harsher critique: "the answers do not seem like someone from pre 1931."
This raises a subtle but important point. A model trained on text from a period does not necessarily replicate the cognitive style, beliefs, or conversational norms of a person living in that period.
It may produce a kind of statistical pastiche — a blend of voices and perspectives that never existed in any single human mind.
Andrew Gordon Wilson (@andrewgwils) extended this critique to the broader enterprise of LLM interaction:
"I don't feel that talking with LLM trained on current data is like talking with a modern person."
If a model trained on contemporary data does not replicate a contemporary human, what does it mean to talk with a model trained on historical data?
Is it a window into the past, or merely a mirror reflecting the statistical patterns of archived text?
Community Reactions and Practical Applications
Despite the philosophical questions, the community response was overwhelmingly positive.
Users described the project as "cool," "amazing," "insanely good," "wholesome," and "the coolest thing I've seen this month."
The checkpoint was made available for download, and multiple users expressed interest in running it locally.
Phil Trubey indicated his intention to download and run the model. Other users asked for terminal access to experiment with it directly.
The practical applications users envisioned were creative and varied.
One user wanted to ask Talkie: "How's the whole woman voting thing going?" — a question that would be answered very differently depending on whether the model was aware of the 19th Amendment (ratified in 1920) or not.
Ben Nash (@bennash) tagged author Orson Scott Card, noting that the project "sounds like one step closer to Pastwatch" — a reference to Card's science fiction novel Pastwatch: The Redemption of Christopher Columbus, in which researchers use a device to view and potentially alter the past.
The thread also referenced an earlier speculative post from September 30, 2025, by user Deep Thrill (@DeeperThrill).
That post had wondered about training an AI on all knowledge up to 1899 and testing whether it could make creative leaps in physics — specifically regarding the aether and the perihelion of Mercury.
Talkie, trained on data extending 32 years further, represents a partial realization of that thought experiment.
Open Questions and Future Directions
The Talkie project leaves several important questions unanswered.
Jiaxin Wen (@jiaxinwen22) asked a critical technical question: was the instruction data used for fine-tuning generated by prompting LLMs with historical texts?
If so, the model's behavior might reflect not just the historical corpus, but also the biases and framing of the modern LLM used to generate the instruction examples.
The authors did not respond to this question in the thread.
Other open questions include:
- Generalization to coding: Can a model trained only on pre-1931 text be taught to write modern code? The authors explicitly mentioned this as a motivation, but no results were shared.
- Temporal self-awareness: Why does the model appear confused about its own training period? Can this be fixed with better prompting or metadata?
- Authenticity vs. pastiche: Is there a way to evaluate whether the model genuinely reflects historical perspectives, or merely statistical patterns?
- Data release: The authors invited community feedback on what aspects — particularly code — would be most interesting to release.
The team plans to publish more information about the corpus and to release as much data as possible, including reconstruction scripts.
For now, Talkie stands as a bold experiment in temporally constrained language modeling.
It offers a glimpse of what happens when we ask a neural network to speak from a past it never experienced, using words it has only read.
