So, I recently realised that a theory I learned of a while back should be capable of producing testable predictions.
It had to do with psychological projection and the belief of some Silicon Valley technologists that 'there is 10% chance AGI will exterminate humanity in the next decade'*.
A certain Ted Chiang had speculated that such sentiments could be caused by tech sector people reflecting the behaviour and attitudes of Silicon Valley startups into their products:
This summer, Elon Musk spoke to the National Governors Association and told them that “AI is a fundamental risk to the existence of human civilization.” Doomsayers have been issuing similar warnings for some time, but never before have they commanded so much visibility. Musk isn’t necessarily worried about the rise of a malicious computer like Skynet from The Terminator. Speaking to Maureen Dowd for a Vanity Fair article published in April, Musk gave an example of an artificial intelligence that’s given the task of picking strawberries. It seems harmless enough, but as the AI redesigns itself to be more effective, it might decide that the best way to maximize its output would be to destroy civilization and convert the entire surface of the Earth into strawberry fields. Thus, in its pursuit of a seemingly innocuous goal, an AI could bring about the extinction of humanity purely as an unintended side effect.
This scenario sounds absurd to most people, yet there are a surprising number of technologists who think it illustrates a real danger. Why? Perhaps it’s because they’re already accustomed to entities that operate this way: Silicon Valley tech companies.
Consider: Who pursues their goals with monomaniacal focus, oblivious to the possibility of negative consequences? Who adopts a scorched-earth approach to increasing market share? This hypothetical strawberry-picking AI does what every tech startup wishes it could do — grows at an exponential rate and destroys its competitors until it achieved an absolute monopoly. The idea of superintelligence is such a poorly defined notion that one could envision it taking almost any form with equal justification: a benevolent genie that solves all the world’s problems, or a mathematician that spends all its time proving theorems so abstract that humans can’t even understand them. But when Silicon Valley tries to imagine superintelligence, what it comes up with is no-holds-barred capitalism.
From: Silicon Valley Is Turning Into Its Own Worst Fear
I am not very convinced but it does appear more plausible to me than the 'they have watched too much Terminator'-theory.
Recently I realised that this theory implies testable predictions.
Such as, people working on AI in environments very unlike to 'libertarian' Silicon Valley techno-capitalism where those hypothesised psychological factors are absent should be much less inclined to think in that way. I know such groups exists, for example, the LLM Apertus is the product of a collaboration of Swiss public universities and the Swiss National Supercomputing Centre. Though, I don't have any knowledge of whether such '10% chance AGI will exterminate humanity in the next decade'-sentiment is less prevalent amongst public and/or university AI developers.
Does anybody here knows anything of how much truth there would be to any statement that such fears are less common the greater the socio-cultural distance from Silicon Valley is?
* Weirdly enough, in none of the, admittedly few, examples of such fearmongering I encountered was provided a plausible mechanism for 'AI to exterminate humanity in the next decade'. An LLM hacking the US' nuclear arsenal and Terminator factories might work in Hollywood movies, but I doubt it would work in real life. And I don't think we in this decade would see enough androids for a robot revolution to make more sense than LLMs going Skynet on us.
Lastly, to avoid being misunderstood, I am not saying there is nothing to worry about when it comes to the development of 'AI'; I am worried about various current developments for diverse reasons and think not enough is done to make those opaque Black Boxes which display all kinds of unforeseen and unintended behaviour transparent and ensure they behave. However, those worries are, I think, a bit more grounded than such '10% chance AGI will exterminate humanity in the next decade'-sentiment.