TThe Diary of a CEO
← All episodes
04 September 2023

Google DeepMind Co-founder: AI Could Release A Deadly Virus - It’s Getting More Threatening! Mustafa Suleyman

4Frameworks
11Insights

Frameworks in this episode

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Myth Buster· 1

Myth Buster1:33:30

The Metaverse Is Already Here—It Just Is Not a Separate World

Suleyman rejects the idea that the metaverse must be a distant virtual realm populated by avatars. He defines it as the parallel digital space already created by screens, social platforms, videos, podcasts, and online interactions alongside physical life.

  • Suleyman begins with the large share of daily attention already spent on screens
  • He says the popular avatar-world framing placed the metaverse somewhere else
  • His broader definition includes ordinary digital communication and media
  • The digital layer exists in parallel with and in relation to physical life
  • This is a conceptual reframing rather than a technical definition accepted by everyone

The metaverse is already here.

Mustafa Suleyman · 1:34:30

It's this parallel digital space that is going to live alongside with and in relation to our physical world.

Mustafa Suleyman · 1:35:00
#metaverse#digital-life#screens#media

Hot Take· 3

Hot Take24:30

Suleyman Warns Synthetic Biology Could Magnify Pathogen Risk

Suleyman says falling genome-sequencing costs and growing ability to synthesize and engineer DNA create valuable applications, but he warns that future tools might also enable more transmissible or lethal pathogens. This is his prospective risk claim in the interview, not evidence that an AI system had created such a pathogen or that the hypothetical outcomes had occurred.

  • Suleyman says genome sequencing has become dramatically cheaper since the first human-genome project
  • He describes DNA synthesis and engineering as increasingly accessible capabilities
  • He identifies accidental or intentional pathogen modification as a dark scenario
  • He argues for limiting access to tools, know-how, compute, software, cloud systems, and relevant substances
  • Bartlett's detailed engineered-virus scenario is a hypothetical host claim, not an established event

The darkest scenario there is that people will experiment with pathogens.

Mustafa Suleyman · 25:30

That's where we need containment.

Mustafa Suleyman · 26:00
#synthetic-biology#pathogens#biosecurity#risk
Hot Take1:08:00

Four-Year Politics Cannot Own a Twenty-Year Technology Risk

Bartlett argues that elected leaders have incentives to capture AI's projected economic gains rather than slow development, and Suleyman agrees that short-termism is a major problem. Suleyman proposes a global technology-stability function able to coordinate competitors and introduce precautionary friction, while acknowledging that no such effective mechanism currently exists.

  • Bartlett frames near-term economic growth as a political incentive against restraint
  • Suleyman says election cycles leave little institutional ownership of long-term stability
  • He imagines a global body that could coordinate containment and introduce friction
  • He compares the missing function with international institutions created after the Second World War
  • The proposal is aspirational and the transcript does not demonstrate that it is feasible

Short-termism is everywhere.

Mustafa Suleyman · 1:09:00

We don't have an institutional body whose responsibility is stability.

Mustafa Suleyman · 1:10:00
#short-termism#governance#politics#coordination
Hot Take1:37:30

Suleyman's Best Case: AI Helps Create Radical Abundance

Suleyman predicts that successfully contained AI could make intelligence widely available and help reduce the costs of energy, food, healthcare, transport, and education. These are optimistic forecasts rather than established outcomes; he also acknowledges that abundance creates secondary problems, using obesity in food-rich societies as an analogy.

  • Suleyman links broad access to intelligence with greater creativity and productivity
  • He predicts major cost reductions across essential goods and services
  • He imagines less dependence on income if basic production becomes much cheaper
  • He supports liberation from unwanted work but expects people to seek new forms of purpose
  • He acknowledges that abundance can generate unintended consequences

I think that ends up producing radical abundance over a 30-year period.

Mustafa Suleyman · 1:38:30

That is a better problem to have.

Mustafa Suleyman · 1:38:30
#radical-abundance#energy#future-of-work#forecast

Explainer· 1

Explainer09:00

Why Language Models Surprised an Early DeepMind Founder

Suleyman says image and audio generation felt intuitively predictable because nearby pixels or sounds have local structure. Language seemed more abstract and open-ended, so he was surprised that scaling similar generative methods produced capable conversational models; the computational figures he gives are his own account in the interview.

  • Local relationships between pixels made image generation easier for him to imagine
  • Next-word prediction appeared to have a much larger and more abstract possibility space
  • He attributes much of the improvement to sustained increases in model scale
  • He describes conversational quality as an unexpected result of that scaling
  • The segment is his retrospective explanation, not a complete technical account of language models

What was much more surprising to me was that those same methods for generation applied in the space of language.

Mustafa Suleyman · 10:00
#language-models#scaling#generative-ai#compute

Story· 2

Story05:00

The Atari Experiment That Made AI Feel Thrilling and Frightening

Suleyman recalls DeepMind training an AI on raw Atari pixels, a small action set, and game score as a reward. He says it discovered strategies that surprised people in the office, making the prospect of machine-generated knowledge feel both exciting and alarming to him.

  • The system received screen pixels rather than a hand-coded description of the game
  • Its available actions were limited to controls such as movement and shooting
  • Game score linked actions to reward
  • Suleyman says the system found strategies that surprised observers
  • He saw the same inventive capacity as a source of benefit and risk

This simple system that learns through a set of stimuli plus a reward to take some actions.

Mustafa Suleyman · 06:00

It presents the opportunity to invent new knowledge.

Mustafa Suleyman · 06:30
#deepmind#atari#reinforcement-learning#ai-history
Story40:00

Move 37 Captures the Creativity-Control Paradox

Bartlett recounts AlphaGo making a Go move that expert observers initially struggled to understand, and Suleyman confirms the commentator thought it was a mistake. Suleyman says this kind of unexpected invention is exactly why people build AI, while acknowledging that the same creativity raises concern about behavior humans may not like.

  • AlphaGo learned through play rather than receiving every useful move from a person
  • The interview identifies the famous unexpected action as Move 37
  • An expert commentator initially interpreted the move as an error
  • Suleyman says novel strategies are a desired feature, not merely a defect
  • The ability to surprise creates tension between invention and control

The commentator actually thought it was a mistake.

Mustafa Suleyman · 41:30

When it discovers a new strategy or it invents a new idea, that's why we're building it.

Mustafa Suleyman · 42:00
#alphago#move-37#creativity#control

Q&A· 1

Q&A1:03:30

Why Suleyman Builds AI While Warning About It

Asked why he founded another AI company despite the risks, Suleyman says practical work at the frontier is the best way to understand models and demonstrate safer design. He argues that standing back would surrender influence to actors driven by geopolitical rivalry or profit, while conceding that building does not solve every contradiction.

  • Suleyman treats hands-on development as a way to learn the models' limits
  • He believes builders can demonstrate safer and more ethical approaches
  • He says critics who leave the frontier give up some ability to shape outcomes
  • Experimentation reduced some of his short-term fear by exposing current weaknesses
  • He remains more concerned about horizons of several decades

The best way to demonstrate how to build safe and contained AI is to actually experiment with it in practice.

Mustafa Suleyman · 1:04:00

They're not superhuman yet. They make tons of mistakes.

Mustafa Suleyman · 1:04:30
#inflection-ai#safety#building#responsibility

Tool· 1

Tool1:28:00

The Corporate Structure Suleyman Uses to Temper Profit Incentives

Suleyman says Inflection was organized as a public benefit corporation, giving it a legal obligation to balance profit with the consequences of its actions. He presents the structure as a small, important incentive change rather than a complete safeguard against AI risk.

  • Suleyman says the company must consider effects on users, non-users, people, and the environment
  • The structure is intended to broaden directors' duties beyond profit optimization
  • He connects corporate form to the incentives shaping technology development
  • He explicitly says the structure is incremental and not a panacea
  • The transcript does not independently evaluate how effectively Inflection met those obligations

We are a public benefit corporation.

Mustafa Suleyman · 1:28:30

It doesn't solve everything. It's not a panacea.

Mustafa Suleyman · 1:29:30
#public-benefit-corporation#corporate-governance#incentives#inflection-ai

Takeaway· 2

Takeaway1:02:00

Suleyman Says the Strongest Pattern Is Still Human Plus AI

Discussing defenses against harmful AI, Suleyman says that in games such as chess and Go the combination of a person and AI has often remained powerful. He uses a recent Go example and familiar assistive technologies to argue that humans adapt alongside machines rather than becoming static comparison points.

  • Suleyman rejects a simple picture in which AI improves while humans remain unchanged
  • People can use AI tools to invent new strategies and defenses
  • He says a human recently beat a leading Go program using a newly discovered strategy
  • Glasses serve as his simple example of technology extending human capability
  • The interview does not establish that human-AI teams will dominate every future domain

So far tended to be the AI plus the human that has that is still dominating.

Mustafa Suleyman · 1:02:00

Humans also adapt.

Mustafa Suleyman · 1:02:30
#human-ai#adaptation#augmentation#games
Takeaway1:41:00

What Suleyman Thinks Failure of Containment Would Mean

Suleyman defines failure not only as missing the benefits of abundance but as allowing powerful tools to spread to people intending harm. He warns that a small group could use networked technologies for rapid, large-scale disruption, presenting this as the danger containment is meant to reduce rather than as a certain forecast.

  • Failure includes losing potential social benefits as well as failing to control harmful access
  • Suleyman focuses on the combination of powerful tools and malicious intent
  • Networked systems can expand the reach of a small number of actors
  • Containment aims to prevent access to the means of large-scale destabilization
  • He does not specify one inevitable failure scenario

A tiny group of people who wish to deliberately cause harm are going to have access to tools.

Mustafa Suleyman · 1:41:30

That's what containment is about.

Mustafa Suleyman · 1:42:00
#containment#bad-actors#proliferation#systemic-risk