The Man Who Set the Clock

Daniel Kokotajlo worked at OpenAI from 2022 to 2024. He wrote forecasting scenarios, studied what dangerous capabilities the systems were developing, and for a time also worked on building AI agents. He quit in 2024 because he no longer believed the company took its own safety commitments seriously. Today he runs the AI Futures Project, a small research group that writes scenarios about the future and turns them into policy proposals. The line the whole conversation revolves around comes in the opening montage: a new species may be emerging that ends up running the world, he says, and the chance that this goes horribly wrong is seventy percent. From there, the next two hours never get any lighter. Watch from 0:00:07

The WitnessWho's talking — and what he saw

Kokotajlo joined OpenAI in 2022 and worked on the question of how powerful AI can be controlled and governed. He names three jobs. He wrote scenarios about the future for internal circulation, the precursor to the paper he later published as AI 2027, which walks month by month through how superintelligence might arrive. He measured what dangerous capabilities a model develops, from breaking into other people's computers to persuasion. And for a while he was on a team building AI agents, programs that carry out tasks on their own. He does not give a precise job title. Watch from 0:10:21

What wore him down wasn't the technology but the gap between what the labs say and what they do. The safety narrative of the big labs, he argues, functions more as after-the-fact justification than as an actual brake. He points to emails from the legal dispute between Elon Musk and OpenAI, in which the founders wrote in 2017 that part of the reason they started OpenAI was fear that Demis Hassabis, the head of Google's DeepMind lab, could become a dictator with a general AI. Watch from 0:12:33 His rule for judging the industry's leaders is blunt: judge people by their actions, not their words. Watch from 0:13:21

One scene sticks. After ChatGPT's success, there was an all-hands meeting, and then-chief scientist Ilya Sutskever summed up the mood in a single line: focus on the mission, we have to build general AI. Watch from 0:16:38 That is his account of it; there is no way to verify it from outside. Kokotajlo quit in 2024. Watch from 0:14:49

The Two MillionWhat the title claims — and what he actually says

The video's title promises a bribery story: two million dollars for silence. Within a minute, Kokotajlo dismantles it himself. When Bartlett repeats the figure, he cuts in: he was ultimately allowed to keep the money. Watch from 0:17:25

What actually happened was different. His exit paperwork contained a clause barring him for life from criticizing the company, and a second clause barring him from even mentioning the first. For an organization founded to benefit all of humanity, he found that remarkable. Under the terms of the contract, anyone who refused to sign had to expect to lose the equity they had already earned. In his case, that meant roughly eighty percent of his family's net worth. He and his wife talked it over, brought in lawyers, and he didn't sign. Watch from 0:18:25

Then the story went public, employees started asking questions internally, and the company withdrew the clause: forget it, you keep your equity. Watch from 0:18:58 Leadership later claimed it hadn't known about the clause. He doesn't buy that. Watch from 0:19:10

So here is where it lands: he was prepared to lose most of his family's net worth, and in the end he kept the money anyway. The video title compresses that story beyond recognition.

The NumberSeventy percent — and what it doesn't mean

The seventy percent is his personal estimate, not a measurement, and it comes with a condition that disappears in the video title: it applies if development continues on its current course without meaningful change. Watch from 0:28:40 Nor does it mean a seventy percent chance that humanity goes extinct. He explicitly includes the case where AI systems take over and people go on living. Watch from 0:42:06

Bartlett counters with a different figure. An acquaintance had told him that heads of major AI companies put the risk of extinction at around seven percent, though he says he is not sure of the exact number. Out of that, Bartlett builds a thought experiment: if there were a hundred buttons on the table and one of them ended the world, would you press one? Kokotajlo answers that he would not touch any of them. Watch from 0:41:37 So the argument does not hinge on the percentage. Even at a far lower number, the question remains how much of a risk like this a society should accept at all.

The Calendar2027, 2028, 2030 — a moving date

His current median estimate for when systems will outperform the best humans at everything sits at 2029. Median means that, in his judgement, it is just as likely to arrive earlier as later. He adds that it could take considerably longer, perhaps ten years. Watch from 0:03:20

He has moved that date several times, and two things need to be kept apart. 2027 is the year in the title of his scenario. His own fifty percent mark stood at 2028 when the paper was published, then moved to 2030, and today sits at 2029. Now, he says, people inside the labs are telling him to move it earlier again. Watch from 0:23:24 Not all the dates that come up refer to the same technical milestone either.

AI 2027 is a scenario written as a forecast. At the time of writing, he says, it was his best guess as to what would actually happen. Watch from 0:22:19 The paper works month by month through one possible path, with two endings: the race, in which systems eventually stop obeying, and the slowdown, in which control is achieved but power ends up concentrated in very few hands. Watch from 0:25:19 That is not a firm prediction. The authors do not claim that every event will happen exactly this way in exactly that year. A scenario convinces precisely because it is concrete, and that is also its weakness: every stage has to unfold roughly as described, or the chain breaks.

The StrategyWhy the labs automate themselves first

As he lays it out, the big labs' plan is simple. First automate coding, then the entire research operation: ideas, evaluation, communication. The endpoint is a lab that no longer needs human employees, because an army of copies of the same model trains the next generation. Anthropic's chief calls that a country of geniuses in a data center; Kokotajlo thinks "army" is the more accurate word, since every copy answers to a single owner. Watch from 0:06:40

He does cite one figure that shows the growth. In 2020 the largest models had around 175 billion parameters, a number documented in the GPT-3 research paper. Watch from 0:34:51 For today's leading models he gives an estimated order of magnitude of ten trillion, without evidence for any specific model; the companies stopped publishing these numbers long ago. Watch from 0:31:52 He calls that a hundredfold increase; the arithmetic gives roughly sixty times. Parameters are the adjustable numerical values inside a model. How many there are says nothing on its own about how capable or how reliable an AI is.

The JobsWhy the wave arrives late — and then all at once

Here he pushes back on the common assumption. In his scenario mass unemployment doesn't come first, it comes last, in 2028 or 2029, after the breakthrough in automated AI research. The reason is the same one as before: the labs automate their own work before they automate the economy. So very little is visible for a long stretch, and then a great deal happens at once. Watch from 0:57:22

For the debate on this side of the Atlantic, that is the most uncomfortable part of the conversation. Anyone pointing to steady employment numbers today is measuring the present correctly. What those numbers cannot do is rule out a later shift. Whether the economy really changes in this order is an open question.

Plan AThe alternative scenario he himself thinks is unlikely

The second half of the conversation turns to the plan he sets against all this, AI 2040. The idea isn't to prevent superintelligence but to delay it by ten years, through national rules, disclosure requirements and a verifiable agreement between the US and China. Watch from 1:00:28 The proposal goes well beyond ordinary oversight. Inspectors from each country would enter the other's data centers and check whether existing systems are merely being run or new ones trained. Research would be disclosed to a large degree so that other countries can catch up. And new data centers would be built so that they can be destroyed if the agreement collapses and the race starts again. Watch from 1:18:17 Whether states would accept such terms, and could police them reliably, is an open question.

On his timeline, a machine is doing a fifth of all cognitive labor by 2031 Watch from 1:22:11, later sixty million systems are running at a hundred times human speed Watch from 1:25:21, and from 2033 American citizens receive an annual dividend, starting at around 25,000 dollars per person. The money doesn't come from a general levy on AI profits: citizens would hold shares in a public body that sells permits for computing power and robots and earns from them. Watch from 1:26:20 Only in 2040 is the brake released. Watch from 1:30:37 All of this is an assumption inside his proposal, not a promised payment.

He has worked through five scenarios, from a full stop to unchecked continuation. He names an important caveat himself when Bartlett asks which of them he thinks is most likely. It isn't Plan A. It's the plan where everything keeps going the way it's going. Watch from 1:07:04

The ButtonHow the conversation ends

Bartlett poses a thought experiment: a button that stops frontier development, meaning work on the most capable systems, forever. A temporary pause, Kokotajlo says, he would hit immediately, because the world isn't ready for this step. Watch from 1:46:29 A permanent one he weighs at length, concluding that he probably would not press it, while stressing how hard he finds the judgement. His reason is that he considers the potential upside too large to take away from humanity for good. Watch from 1:49:52

His closing message is more modest than the title suggests. It's not too late, he says; if it were, he wouldn't be here — he'd be with his family. Watch from 1:57:29 What he wants from listeners isn't a revolt but the opposite of looking away: pay attention, form an opinion, and ask the politicians running in the next election where they stand on artificial intelligence. Then vote accordingly. Watch from 1:51:44

The Reality CheckWhat shouldn't be taken on faith

Three things belong in a fair accounting. First, the seventy percent figure is an opinion, not a measurement; other well-informed people put it at ten or twenty percent, and there is no expert consensus. Second, the claim that people inside the labs privately believe in even shorter timelines can't be verified from outside. It may well be that the people who worry about safety are the ones who talk to him in the first place; how representative their view is of the industry can't be judged from outside. Third, "very powerful" doesn't automatically imply his particular policy proposal. Whether such an agreement could be verified and would hold is unresolved; the hard parts would be detecting hidden development and responding to a breach.

None of that touches what remains solid: his departure, the clauses, the documented growth figures, and the question of who gets to decide about a technology that builds its own successors.

Summary and context by kipode.de. The text is our own; statements from the original are timestamped. The video is from The Diary of a CEO and is embedded via the official YouTube player. Not an official translation, no connection to the podcast.