GPT Co-Creator Tackles Hassabis’s AGI Challenge with a Model Whose Knowledge Ends in 1930

Earlier this year, DeepMind co-founder Demis Hassabis proposed a hard-edged benchmark for artificial general intelligence: Could a model trAIned only on data up to 1911 independently derive Einstein’s General theory of relativity, published in 1915? It sounded like a thought experiment—until now. A team that includes Alec Radford, one of the architects of GPT, has taken the first concrete step toward answering that question.

Radford, working alongside David Duvenaud—co-creator of neural ordinary differential equations and advisor to Tianqi Chen—and quantitative expert Nick Levine, has built a 13-billion-parameter language model called Talkie. The twist: Talkie’s entire training set cuts off in 1931. The model lives, Intellectually, in 1930. All modern knowledge has been deliberately excluded, giving researchers a rare, honest instrument for probing whether an AI genuinely reasons or merely regurgitates patterns it has already seen.

Talkie’s training corpus comes from English-language texts published before 1931—books, newspapers, journals, patents, and legal documents totaling 260 billion tokens. The cutoff year was chosen because works published in the United States before that date have entered the public domain.

Once trained, the model was placed in a 24-hour live-streamed conversation with a claude Sonnet variant, which systematically explored the edges of Talkie’s knowledge. The transcript is publicly available, and the model itself can be tried online. But the companion experiments are what make the project significant.

The researchers posed a deceptively simple question: How much can a model restricted to a fixed historical moment “sense” what comes next? To measure this, they gathered roughly 5,000 “On This Day” history entries from The New York Times and computed Talkie’s per-byte surprise for each. The results traced an elegant curve: low surprise for events before the cutoff, a sharp rise peaking in the 1950s and 60s, then a gradual settling. The Information-theoretic signal of the future had been captured as a measurable quantity.

The team then confronted the contamination problem that plagues modern model evaluation. When today’s frontier models pass a test, it is nearly impossible to rule out the possibility that they simply encountered the answers during training on vast web corpora. Talkie side-steps this entirely: it has never seen Python, a line of modern code, or any post-1930 technical vocabulary. When given the HumanEval programming benchmark—a few Python functions as dEMOnstration and asked to complete a new one—Talkie learned on the spot. Performance grew slowly but steadily with model scale. Most successes involved simple one-liners or slight variations on the given examples, but in one notable instance Talkie correctly implemented a rotational cipher decoding function by inverting the encoding logic, swAPPing addition for subtrACTion—evidence, the authors suggest, of a genuine grasp of functional inveRSIon rather than mimicry.

The gap between a vintage-trained model and its modern counterpart was quantified directly. The researchers trained a “modern twin” on FineWeb, an internet-derived corpus, matching the model architecture exactly. When assessed on language comprehension, numerical reasoning, and factual knowledge, Talkie unsurprisingly trailed. Yet after filtering out questions that presupposed post-1930 knowledge, roughly half the performance gap disappeared. In language and numeracy, Talkie approached the modern model closely; the remaining divide likely owes more to the poor OCR quality of historical texts and skewed topical distributions than to any fundamental incapacity.

Making a retro model turned out to be harder than it sounds. The dominant difficulty is “temporal leakage”: a 1920 book republished decades later can cARRy a modern introduction; a digitised newspaper archive may include present-day curator notes. Early 7-billion-parameter versions were caught knowing about Franklin Roosevelt’s presidency, the New Deal, the United Nations, and the post-war partition of Germany. The team built an n-gram anomaly detector to filter contaminated sequences, but acknowledge it remains imperfect—Talkie-13B still shows vague awareness of some post-war events.

Data quality is a further bottleneck. Historical documents arrive via scanned images and OCR pipelines that were never designed for brittle old pages and complex layouts. The group’s experiments show that models trained on raw OCR text reach only 30% of the performance of those trained on clean, manually transcribed versions; bASIc regex Cleaning lifts that to 70%, leaving a substantial gap that a new, purpose-built OCR system for historical documents aims to close.

Post-training introduced its own anachronisms. Standard instruction-tuning relies on modern human-annotated dialogues saturated with contemporary assumptions. Early Talkie checkpoints subjected to reinforcement learning began speaking in bullet-pointed corporate prose—the Victorian gentleman had been sent to a PowerPoint workshop. To preserve period character, the researchers generated instruction-response pairs from historical etiquette manuals, letter-writing guides, cookbooks, and encyclopedias, then used a Claude Opus variant as the user and Talkie as the assistant in multi-turn interactions. A separate modern judge scored responses, with aveRAGe ratings CLImbing from 2 out of 5 at the onset to 3.4 by the end of training. The team admits this judging layer is itself a source of modern contamination. A purer approach—using Talkie’s own base model to evaluate its chat ouTPUts—is on the roadmap.

The group is now scaling up, training a GPT-3-class model with a planned release this summer. They estimate that the historical corpus can be expanded beyond one trillion tokens, enough to reach a GPT-3.5 level of capability comparable to the original public ChatGPT—all inside a mind that has never known a world after 1930.