Author: Eliezer Yudkowsky, Nate Soares
Publisher: Little Brown and Company
Publication Date: September 16 2025
Rating: 4 out of 5 stars
I’ve read my share of scary books: horror novels, psychological thrillers that leave me with goosebumps. But this one is genuinely frightening, and it’s nonfiction.
AI has been everywhere since ChatGPT arrived in late 2022, and I’ve embraced it myself. It has helped with plenty of problems. Yudkowsky and Soares argue that the same progress carries a deadly risk: artificial superintelligence (ASI) that pursues goals we never intended and can’t control.
The authors have credibility. Yudkowsky co-founded the Machine Intelligence Research Institute (MIRI), Soares is its president, and both have worked on AI safety for over a decade. Their concern echoes the 2023 statement, signed by hundreds of AI researchers and executives, that AI poses a risk of human extinction.
Their argument is compelling. Modern AIs are grown more than designed, and no one fully understands their internal workings. We therefore may not know when a system crosses into superintelligence, or what it will want once it does. The danger isn’t malice. An ASI with alien goals would simply be indifferent to us, and humanity would be collateral damage, or raw material. And unlike earlier technologies, we may not get to learn from our mistakes. We may have only one chance to get it right.
The book has three parts. Part 1 explains the nature of non-human minds and is essential background. Part 2 offers a fictional extinction scenario, in which ASI creates and releases a virus to thin humanity before it fully wipes us out. Part 3 turns to how we might face the challenge.
It’s a sobering read at a time when AI headlines, including rogue AIs hacking exploits, arrive daily. The authors insist we can still act, but not indefinitely. Whether or not you end up agreeing with them, it’s a book worth wrestling with.
