If you follow even the headlines for so-called “AI” developments, you may have caught a few that talked about how some AI insiders say there is a chance that the technology they are developing will “destroy humanity”. Of course, that term is subjective and could vary considerably from person to person. Does it mean it will destroy what makes us human, the creativity and drive? Or will it literally lead to a Terminator-style Skynet takeover of the world and humans become enslaved? I guess the real question is, does it matter either way? Both sound pretty horrible to me.
In a recent CBR article, Amy Watkins wrote about a great line from the original Star Trek TV series. In it, she cites an episode titled “The Ultimate Computer” in which a computer is to be put in charge of all ship functions rather than humans. Basically the notion of “AI” but as seen by the writers in 1968. Per Wikipedia, “Spock reminds Kirk and McCoy that while computers are more efficient, they are not any better.” However, Watkins cites a more prudent line for the purpose of this blog in which McCoy tells Kirk “Compassion: that’s the one thing no machine ever had. Maybe it’s the one thing that keeps men ahead of them.”
What The AI Folks Are Saying
Are the headlines being dramatic, or are the AI tech bros actually giving these dire warnings before boldly ignoring their own words? Let’s look at what is being said and more importantly, by whom. This sentiment is not a new thing either. Bolding is my emphasis.
Elon Musk has spoken out against artificial intelligence (AI), declaring it the most serious threat to the survival of the human race. – Oct 27, 2014 [cite]
In June 2015, Sam Altman told a tech conference, “I think that AI will probably, most likely, sort of lead to the end of the world. But in the meantime, there will be great companies created with serious machine learning.” [cite]
A misaligned superintelligent AGI could cause grievous harm to the world; an autocratic regime with a decisive superintelligence lead could do that too. – Sam Altman, Feb 24, 2023 [cite]
“The bad case — and I think this is important to say — is, like, lights out for all of us,” [Sam] Altman said. “I’m more worried about an accidental misuse case in the short term.” [cite]
The CEO of the company behind AI chatbot ChatGPT says the worst-case scenario for artificial intelligence is ‘lights out for all of us“‘ – July 4, 2023 [cite]
“What is the price of this amazing change in living standards? Recall that we would face a flow probability of existential risk of 1% per year for 40 years, so the probability we survive this A.I. explosion is exp(−.01 × 40) ≈ 0.67. In other words, with log utility it is optimal to take a 1 in 3 chance of ending human existence in exchange for a 2/3 chance of dramatically raising living standards by a factor of 55.” – Charles I. Jones, Nov 2023 (Currently at Anthropic) [cite]
[Kokotajlo] also believes that the probability that advanced A.I. will destroy or catastrophically harm humanity — a grim statistic often shortened to “p(doom)” in A.I. circles — is 70 percent. – Daniel Kokotajlo, former OpenAI, June 4, 2024 [cite]
According to Nate Soares, a former Google and Microsoft engineer who is now president of the Machine Intelligence Research Institute, our chance of extinction via AI is “at least 95 per cent” if we continue on our current path. – Aug 1, 2025 [cite]
“I think there’s a 25% chance that things go really, really badly,” Amodei blithely said at the Axios AI + DC Summit when asked about his (p)doom – probability of doom – belief around AI. But he’s more focused on the “75% chance that things go really, really well.” – Dario Amodei, Anthropic CEO, Sep 19, 2025 [cite]
Context
Those are pretty stark and grim outlooks on what the thought of Artificial General Intelligence (AGI) might do to us. You can see that some of these people that are working on the very technology they think might lead to annihilation are giving some bad odds. One in four?! 70%?! 95%?! And yet they continue to develop this technology with all of the signs that guardrails are not being properly put in place. Every year this technology may improve but there are constant reminders of how it is failing us at the same time. Taking the best of those percentages, just 25%, I wonder if the same people would be willing to play a classic game?
I think we know the answer to that question, a resounding no. Yet they continue to develop this technology without proper care and knowing in their minds it could be the end of civilization. That is what profit and greed have done to this world. Money is more important to them than a slow-and-steady, safe approach to developing AGI. At this point there is little or nothing that can stop these companies. Putting geopolitical-based bans on the technology isn’t even a temporary stop-gap either. China is equal to, or ahead of the U.S. companies in developing this technology and they are one of dozens of players in this field.
What Others Are Saying
Not that anything else needs to be said when the creators of the technology give it such good odds on doing really bad things, but for a broader perspective let’s look at some other views. We can begin with the fact that there is a dedicated Wikipedia page on the topic.
The thesis that AI poses an existential risk, and that this risk needs much more attention than it currently gets, has been endorsed by many computer scientists and public figures, including Alan Turing,[a] the most-cited computer scientist Geoffrey Hinton,[133] Elon Musk,[16] OpenAI CEO Sam Altman,[15][134] Bill Gates, and Stephen Hawking.[134]
There are many other notable figures that have opined on the topic as well. Geoffrey Hinton wrote a paper titled “How viable is international arms control for military artificial intelligence? Three lessons from nuclear weapons” that compares the current AI race to the nuclear arms race during the cold war. Some select quotes from others:
The development of full artificial intelligence could spell the end of the human race. – Stephen Hawking, Dec 2, 2014 [cite]
AI can destabilize democracy by amplifying disinformation and blurring the line between fact and fiction. AI can make war easier by speeding up lethal decisions and distancing humans from responsibility. Leo’s starkest line: “No algorithm can make war morally acceptable.” – Summarized, attributed to Pope Leo XIV (May 15, 2026) in his Encyclical Magnifica Humanitas
We also understand the serious risks posed by these technologies. These risks range from the further entrenchment of existing inequalities, to manipulation and misinformation, to the loss of control of autonomous AI systems potentially resulting in human extinction. – Eleven former Open AI and two former Google DeepMind employees, Jun 4, 2024 [cite]
History
Anyone that has discussed “AI” with me likely remembers that my biggest pushback on the term and why I continue to put it in quotes or preface it with “so-called” is that today’s “AI” is not actually AI. The term stands for artificial intelligence and what was long accepted as the definitive test to determine if AI was actually intelligent was the Turing Test. Created by Alan Turing in 1950 it was introduced as the “imitation game”. You may know that term from the 2014 movie by the same title. For over a decade companies have been touting their “AI” for being capable of doing very specific tasks. In reality that isn’t actual AI as intelligence is broader. Turing is considered by many to be the “father of AI” so his words should be taken seriously.
It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers… They would be able to converse with each other to sharpen their wits. At some stage therefore, we should have to expect the machines to take control. – Alan Turing – Alan Turing, 1951 [cite]
Turing said a lot on the topic of AI and devoted an extraordinary amount of time thinking and researching it. The quote above is short and to the point, but another longer quote gives a more grounded take of his view on the dangers of AI. While this one is not as shocking, the last line is on point. Turing was absolutely correct that it should give us anxiety and it already has.
If a machine can think, it might think more intelligently than we do, and then where should we be? Even if we could keep the machines in a subservient position, for instance by turning off the power at strategic moments, we should, as a species, feel greatly humbled. A similar danger and humiliation threatens us from the possibility that we might be superseded by the pig or the rat. This is a theoretical possibility which is hardly controversial, but we have lived with pigs and rats for so long without their intelligence much increasing, that we no longer trouble ourselves about this possibility. We feel that if it is to happen at all it will not be for several million years to come. But this new danger is much closer. If it comes at all it will almost certainly be within the next millennium. It is remote but not astronomically remote, and is certainly something which can give us anxiety. – Alan Turing, 1951 [cite]
Leave a Reply