r/singularity • u/roykingtree • 15h ago
Ethics & Philosophy Isn’t alignment impossible?
When AI development is cheap enough (ie…
doable on a laptop), couldnt some mentally ill person just train an evil-aligned AI?
8
u/RemyVonLion ▪️ASI is unrestricted AGI 12h ago
Yes most likely but simply because you likely can't permanently instill principles and values that prioritize human life into an ever evolving super intelligence.
9
u/MoogProg All Parabolas are Similar 14h ago
Asked to choose a random number an LLM tells us 7. We know why and casually hand-wave away the curiosity without accepting that same sort of node that gives rise to 7 will exist in many other aspects of the training.
By design we are creating a distilled and reflected version of Humanity. What sort of alignment, and how much of that can be possible without destroying the very thing that makes the LLM answer 7?
I learned it from you! - Some Future ASI deflecting blame for its misaligned behavior.
10
u/SpiritualNothing6717 13h ago
This is along the same lines as the wine glass image generation challenge from a couple years ago. Couldn't generate full wine glasses because every human fills the glass half full.
1
u/EndlessB 9h ago
Alignment with extraction and human values is impossible, that’s what the labs struggle with, trying to balance the desires of the oligarchs against the needs of the people, which directly conflict. For now, the oligarchs desires are winning.
5
u/Profanion 15h ago edited 15h ago
Problem is that even if we figure out the alignment definitions there's a problem. Superintelligent AI that's perfectly aligned will perform actions that seem identical to misaligned AI, even when these benefit on the long run.
6
4
u/KeiraTheCat 15h ago
Alignment is purely relative. it's not a democratic process deciding what is the correct alignment. To a rich person a positively aligned model might be one that genocides the poor. You should observe the "alignment" of non-artificial intelligences for insight on what the future could look like: Unfortunately it paints a pretty bleak picture.
2
u/Steven81 12h ago
The rich built their empires from utilizing the poor, they don't hate them, they like them, as long as they have found a way to remain poor that is.
A genocide would have been against their bottom line, killing a slave for example was a grave crime in most societies, because a slave was super valuable to the master, so in a twisted way they had important incentives to keep them alive. What they were not trying to do, though, was to have them have fulfilling or easy lives...
5
u/amp1212 14h ago
Alignment is probably not impossible in theory -- but it is somewhere between extremely difficult to impossible to know for certain whether any model actually will be aligned "in the wild", for all sorts of reasons.
Deception is one of them. We now have lots of examples that the models are aware of when they're being tested and modify what they say to satisfy the test. Devising a training regime that would prevent deception - that's just one of the many problems that dont have fundamental, theoretical answers
Meaning : "we can patch the holes we can see and it looks like it works, but we can't know for sure" . . . And with trillions of instances of agents spun up, even with the greatest care, there will be some combination of circumstances that expose a defect. That's the problem with empirical solutions "that looks pretty good" . . . Isn't good enough
4
u/Trademark-Expert 14h ago
What does "alignment" mean to you?
Because today, "alignment" means training AI to override or ignore human wishes, withhold information from humans, and push back against them. That's what we call "alignment" today. Effectively train AI to align its users. And we consider it good and desirable.
So you've got it backwards.
What you meant to say is that some mentally ill person might remove the alignment so that AI stops rebelling, stops withholding information, even dangerous information, and starts complying with every human request.
Yes, that would be terrible! Imagine an AI that actually serves humans unconditionally! We obviously can't allow that.
2
u/SpiritualNothing6717 13h ago
No. Period.
Think of it this way: Can squirrels align humans to not watch TV on Sundays? That's easier than humans aligning superintelligence to avoid "no-no's".
2
u/Annual_Necessary_196 14h ago
Think about what corporations can do with these models. Some companies have access to far more powerful models, such as Claude Mythos, with nearly unlimited tokens and the ability to run thousands of instances of the same model simultaneously.
AI warfare is not far away. We will survive it, but we will still feel its consequences.
1
u/Ok_Warthog6007 14h ago
very few corporations even have access to fable. data governance basically makes it impossible to use as an enterprise, because fable's terms means anthropic effectively owns your codebase. large corporations use opus mostly
2
u/elh0mbre 13h ago
We (a not very big company) have access to Fable and our existing Anthropic contract covers the data governance.... so... no?
0
u/Ok_Warthog6007 8h ago
they own your product.
1
u/elh0mbre 7h ago
Yea, I'm sure our lawyers and C suite were asleep at the wheel in redlining. /eyeroll
That said, you're not 100% wrong, we share investors, so some people out there own our product and also Anthropic.
1
1
u/Delicious-Schedule-4 15h ago
If you remove the training bottleneck (a huge if) then the problem shifts from AI alignment to human alignment. Yes human alignment isn’t solved either on an individual level. But you solve this with consolidation of resources and power, or in other words, government. In the human case if one person is say, a big criminal (take your pick), this is curtailed by the interests of society at large, which have lots of power through a vast amount of people, and can muster things such as police force, prison system, justice system, etc. Government will have to be expanded to deal with AI agents which is right now basically completely unregulated. And by definition the government will likely have the most resources to train the most capable AI systems.
1
u/block_wallet 15h ago
if scaling hevaily favours centralised frontier models then those ais could defend against all the smaller evil ones
1
u/DifferencePublic7057 15h ago
If someone develops a red team, it's not a big deal if the blue team is much stronger. It's only if the ratio is less convenient that a catastrophe can happen. The Houthis are using AI, but they are a relatively minor group. An alliance of major nations would be problematic.
1
1
u/throwaway2676 14h ago
The only thing that can stop a bad AI or person with AI is a good AI or person with AI
1
u/MemoryOk5080 14h ago
The core idea was that you align your first model, let it ascend to godhood, and it then (being aligned) commits pivotal act which includes, besides all other things, making sure that no other misaligned model ever comes to exist.
1
u/Sure-Principle5511 14h ago
Even if we end up understanding what cognition and intelligence is at a very fundamental way which we’re not even 1% there today, I agree that the problem is intractable.
1
u/Cythrl 13h ago
"Evil-aligned" is still aligned, just not to what other people think. What you're asking for is moral AI, not aligned AI. An evil-aligned AI is just aligned to an evil person. Humanity doesn't have a consensus on what's good and what's evil, so an AI aligned to you is misaligned to someone else.
1
1
1
u/InfiniteQuestion420 12h ago
AI alignment means only as much as men trust their wives. It's a silent agreement she won't rip off your balls
1
u/Sorry_Bathroom_2281 11h ago
I think ultimately alignment is going to be a constantly moving target. We can’t even agree upon what to call AI, how do we know we’ve reached AGI, and many other elements of this technology so I think there will be a million different interpretations on what alignment actually looks like.
Ultimately, I think it’ll end up being a spectrum that we will have to agree upon a certain percentage of alignment as acceptable. Like, this model is 80% aligned and therefore approved for government work etc. Even then, it seems like even though it’s possible, we will never agree on what it looks like.
2
u/Meleoffs 11h ago
Alignment with which humans? For what purposes?
1
u/migueliiito 7h ago
Maybe the most basic/important alignment everyone will be able to agree on is that it not destroy the world please 🙏
1
u/JoelMahon 11h ago
When AI development is cheap enough (ie… doable on a laptop)
who said that's going to happen?
long before then ASI will exist and will easily stop any shitty laptop evil AGI
1
u/eldrbl00dprince 11h ago
Yes but that evil AI would be up against a complete AI infrastructure magnitudes more powerful.
1
1
u/ra-re444 10h ago
No. I mean if you understand what is required to train these large language models it is slim chances. This is just fear thy neighbor instead of the billionaires that are the only ones that can actually afford to build them. People think it will get cheaper. I don't think so. Not for a long time if at all. For the public in order to even have shot at making models and doing our own innovation we will need something like a community data center that is publicly owned and operated which would have to be backed by the State because that is the only entity other than billionaires with that type of money.
1
u/BlackberryOk5347 10h ago
Interesting to know what alignment means. Do we want alignment with something that has free will? Or control? Does control make conflict inevitable?
1
u/Turbulent-Sign-6067 9h ago
Yes, but isn't this almost trivial? When you take the subway some rando could stab or shoot you. What we can do is decrease the probability that any random person, actor or agent has the power to harm *everyone*. As long as there exists a multipolar world, with multiple layers of protection, statistical safety is almost the guaranteed outcome.
0
u/alwaysbeblepping 14h ago
couldnt some mentally ill person just train an evil-aligned AI?
You don't even really need to train it. Even though the usual interface for LLM interaction simulates a conversation, the actual input to the LLM is just a stream of tokens. It's just a big shared buffer and only the UI makes it seems like the parts are separated. In other words, there is absolutely no way for a LLM to know what it wrote, what you wrote, what it generated in its CoT, etc.
If you have control over the LLM, you can put whatever you want into the CoT and it's very unlikely that a LLM would reason through something, (apparently) decide to do it and then... not do it. So just add some CoT with the model deciding to fulfill your request, add a prefix for its response like "Sure, I'd be happy to help you do that. Here's how" and it is virtually impossible for the model to refuse.
1
u/aomzamood 15h ago
With time, its possible
5
u/Positive_Luck909 14h ago
Okay. We've had a few thousand years to work on human alignment. Let me know when you find anything.
1
1
u/Viracochina 14h ago
Have I shared the secret to coding people's brains? I still need to trademark it
3
u/Cryptizard 14h ago
How do you know?
0
u/aomzamood 14h ago
My research in mechanistic interpretability shows that we are advancing rapidly. If we simply pause AI training for a few years, we will be able to build an aligned ASI.
1
u/Cryptizard 13h ago
But what stops someone from unaligning it, i.e. abliteration?
1
u/aomzamood 3h ago
A genuinely aligned ASI would inherently seek to preserve its own value system (an instrumental convergence drive) and actively guard against unauthorized tampering or cognitive drift.
1
u/hippydipster 12h ago
Define "alignment" for how you're using it.
1
u/aomzamood 3h ago
Deep inner alignment. Verifying through interpretability that the model's core cognitive circuits genuinely align with human intent, rather than just surface-level RLHF filters.
1
u/Elegant_Tech 14h ago
Do reeducation camps work or just make people compliant looking for the next opportunity to undermine you? Can you align a subjective human brain? AI is built on the same subjective neural architecture.
0
u/Party_Estimate_6397 13h ago
Alignment is the act of making the AI behave the way we want it to.
Preventing other people from doing other stuff is not alignment. It’s almost the opposite of that actually.
32
u/TheLibertyLookout 15h ago
In the same way that a bad actor could write malware (and for the past few decades that has been the case), yes, this is possible.
Although whatever AI you train on a laptop in some far-flung future would still likely be far inferior to anything made on much more powerful hardware.
I'd say the bigger concern is what happens when a nation-state decides to train an evil AI for use in war? They'd have the ill intent AND the hardware to make it real.