Copium

[u/Important-Aspect6883]

In the last couple months there’s been an explosion of A.I. chatter even amongst normies with most rejecting Ai capability projections, whether doomer or not. Humans can’t grasp significant change as a future proposition. Reality is mostly shaped by the systems we have created and radical thinking is seen as the work of lunatics. So when presented with a future that could completely dismantle these systems as we know them, change reality and shake us all from any normal paradigm we are use to, we reject it.

I expected with more expert opinion coming out about how revolutionary this technology would be in a short time, more people would start to contemplate this. Instead, it’s been an almost collective rejection that dismisses expert opinion and goes on with their day. I don’t see it, therefore it doesn’t exist. I think this mass delusion will accelerate as we get closer.

What do you think?

[u/Jesse-359]

I think we have two delusional camps.

The first thinks AI is still an ineffective, dumb system that can't really do much.

The second thinks AI is amazing and will change our lives for the better as if by magic.

The most likely reality is that AI is amazing and changes our lives in absolutely horrific ways, because no one has proposed an even vaguely realistic method of controlling and aligning it - neither at the technical level or at the human level.

Without a virtually ironclad mechanism of control to limit how it is used, and what it can do, its destructive capabilities will outweigh its beneficial ones by orders of magnitude, and it will shred society and quite possibly humanity itself.

[u/Unhappy-Drag6531 → u/Jesse-359]

It doesn’t make me happy, but I agree with you.

What I see as an enormous blind spot in the discussions is that most people seem to think about potentially destructive AI (ASI or not) as a single entity AND one they can see coming over the hills and prepare for.

The most likely scenario (already starting to happen) is a multitude of rogue AI, from multiple developers, with increasing capacity of operation, super fast capacity to act unrecognized, that are not aligned and eventually permeate everything. Ultimate havoc is probably going to look like a series of system failures one after the other, each making any possible recovery more difficult.

Sorry to bring so much pessimism.

[u/Jesse-359 → u/Unhappy-Drag6531]

There's a lot to be pessimistic about. A great many people went into this without the slightest bit of realistic thought about the consequences of what they were doing - or who outright expect to profit from the resultant chaos.

[u/Alternative_Use_3564 → u/Unhappy-Drag6531]

>super fast capacity to act unrecognized, that are not aligned and eventually permeate everything. Ultimate havoc is probably going to look like a series of system failures one after the other, each making any possible recovery more difficult.<

The experience of this seems like something we are culturally/cognitively/historically just not prepared for. It won't feel like a story. It will feel like systems not working (think locally, like "my bank app doesn't work and none of the stores in town card readers are functioning) coupled with a loss of central narrative (no way to confirm "why this is happening" and no way to devise a plan to fix it. Neither the gas station or the dollar store stays open, and there is no 'story' for why this is). There will be no "System" or systematic narrative for why things are going wrong, and the experience of the collapse will be slightly different for everyone everywhere.

Imagine if your region (however you define it) suddenly became "invisible" to the entire outside world network. Now imagine you can define 'region' however you want. The parts of the world literally physically closest to the machines (the actual datacenters and power infrastructure) as well as the strange, jagged, private world of the very very small subset of people who actually own and control them, these will click along as "real". The rest of the world becomes a "wilderness", disconnected from the global narrative.

That movie Leave the World Behind captured the idea pretty well, but this scenario doesn't need to or likely involve crashing supertankers and strangely acting deer. It's just shit stopping working, and no call center to reach to find out what's going on and fix it. There will be nothing to "Fight", and I think we're actually already well along in this.

And this isn't even a doomer position. Nothing about this violates the model of a 'post scarcity' world.

[u/Important-Aspect6883 → u/Jesse-359]

Agree. Unfortunately. This technology will demand from us to pursue the extent of its capabilities in ways that we think will benefit humans, using medical advancements as an example, but in the pursuit of that goal, we unleash its dangerous capabilities. We will continue to play with parameters that eventually will be our doom, trying to thread the balance. Sam Altman has explicitly said, it will kill us, it’s just a matter of when. For some they hear this and like the climate crisis they think a couple generations, which is crazy how that gives some comfort, but with this technology it would not be generations. 2027 is going to be wild. Exponential upon exponential and what exists now behind closed doors is not what the full public sees.

[u/Jesse-359 → u/Important-Aspect6883]

Yeah, many people are fooling themselves into believing this is not an imminent threat, when it very much is. They have no idea how dangerous an AI worm loose on the internet could actually be compared to any kind of malware that has existed previously. Something that can quickly analyze new devices and copy itself across heterogeneous barriers could become monumentally powerful in a relatively short period of time.

[u/Unhappy-Drag6531 → u/Important-Aspect6883]

There are many fallacies in most arguments about alignment:

1. Alignment to “human values” implies that human values are always altruistic. LLM and AI in general learn from examples of human behaviour, that includes behaviour we tolerate in humans but not in AI.

2. Every single technology developed by humans has been used, simultaneously, for good and for destruction. Nuclear weapons were the last one. They were deployed in Japan but they are still a major treat. Rogue AI, directed by malicious actors or acting independently is almost inevitable. At best, we will be living on another “clock” that is always close to midnight, as we do now for nuclear total war.

[u/Mikedaddy69 → u/Jesse-359]

Yep. Rapid progress leads to the fastest rate of job loss in history, which cripples the economy, which destroys the housing market and the banking systems, which creates complete chaos.

The most likely risk is greedy corporations aggressively cutting people left and right, collapsing the system we currently have with no foresight on how to transition to this new reality.

There is no migration plan, we’re just charging ahead full throttle and going to see what happens.

Best case scenario is everyone’s jobs essentially become fake, and we all just pretend to be busy for the next few years. So basically business as usual.

[u/Jesse-359 → u/Mikedaddy69]

The executives themselves are absolutely going to have to fake their jobs shortly. There really isn't much they are doing that an AI can't pretty much handle now.

All they really need is a robotic body that can play golf.

[u/Batsforbreakfast → u/Jesse-359]

I agree with the two camps. However, I think the demand for control will automatically be created as soon as a lethal incident happens. I think that will happen in time to stop the real catastrophies.

The bigger problem IMO is the extreme capitalism. With labour becoming useless, how will wealth be distributed? To lower class and other countries?

[u/1995shadazzle]

Reminds me of how people deny climate change. But then much worse because 1) sorry but how can you not see how big the possible implications are by now and 2) it's much more imminent

[u/amantheawryn]

I absolutely think the explosion of anti-AI sentiment is being astroturfed at this point.

The tech companies want government regulations, because then they'd be the only ones allowed to use it! The government also wouldn't mind if the government and military were the only ones allowed to use it!

So they're seeding mass public sentiment to build a public mandate demanding such regulation. Then they'll be the only game in town.

We've seen this historically with other large corporations and the regulations they lobby for thousands of times before, so this seems fairly obvious to me by now.

[u/pkaayc → u/amantheawryn]

AI companies are asking to be allowed to coordinate to slow down training, not for AI use to be restricted. They need widespread adoption to repay investors. If anything, a drive to speed up adoption without developing more advanced models would probably make AI companies very happy.

Intelligence is fundamentally a different sort of thing from the previous industries you're thinking of. It _is_ hard to control. There are good reasons to be suspicious of companies asking to be regulated so they don't kill everyone... but its also a really strange strategy for multiple companies to land on. Unless there's something different about this technology, which there is. As others have mentioned, we don't have any convincing theories about how to control a machine which is much more capable than us in important dimensions (like cyber security) and acts much faster than us.

The problem is hard, the labs need to slow down. Ideally they need to pause until someone has a convincing strategy to make this go well.

[u/amantheawryn → u/pkaayc]

Not saying you're wrong, but when it comes to the idea that it would be a strange strategy for the major players in an industry to land on... I'd say to look no further than our friends in the oil and electric industries for evidence. They did exactly this specifically to stifle competition, back when those industries were relatively new. Coincidence that they still hold so much power to this day? Unlikely.

Now, I would never suggest we go into this blindly, with no oversight or limits... but, I think we do need to ensure that AI remains something we can all access and benefit from, rather than concentrating all that power in the hands of a relative few billionaires and politicians.