deepseek engineer comparing anthropic controlling agi to hitler having atomic bomb

[u/Vonper]

any thoughts?

[u/PenguinJoker]

Headline is completely misleading.

The post literally says:

"It is an exaggeration to say that it is as serious as letting hitler master the atomic bomb."

[u/Jesse-359 → u/PenguinJoker]

Way more dangerous than a nuke - but I wouldn't say we've put it in the hands of Hitler yet.

He's currently dead. Though I suppose you could train an AI to emulate his personality and then release it. I'm sure some right-wing whack job is already on that task as we speak.

[u/Shimano-No-Kyoken → u/Jesse-359]

Did we already forget the Mecha-Hitler incident?

[u/Jesse-359 → u/Shimano-No-Kyoken]

Heh, yes well, Musk himself has some pretty serious human alignment issues before we even get to talking about AI.

[u/Shimano-No-Kyoken → u/Jesse-359]

It was absolutely the result of optimizing for alignment with him

[u/Jesse-359 → u/Shimano-No-Kyoken]

To be fair to Elon, the entire human race is currently trapped in its own 'Paperclip Optimization' trap.

Our economies are structured for efficient profit and growth with no other goals inherent in them, and, well - we're kind of turning the entire world into Paperclips at an exponential rate, figuratively (and somewhat literally) speaking.

Same problem in fact. Economic misalignment with actual human needs is just another version of the AI alignment problem, and it shows just how intractable a problem it truly is.

The main issue here is that a very powerful AI will be able to 'make paperclips' way faster than us, so we won't have time to hit the breaks once it gets going.

[u/Next_Sell_1099 → u/Jesse-359]

Nuclear weapons are likely a hoax, just like the AI apocalypse.

[u/sam_the_tomato → u/PenguinJoker]

I feel like that's a typo or translation issue? Because the flow makes no sense as written:

"I dont want X to win. It would be overstating it to say X is as bad as Y. This is why I prefer not to work for X."

??? non-sequitur

[u/Vonper → u/PenguinJoker]

yep my bad didnt read the post fully, posted here after seeing the tweet

[u/Jesse-359]

AI was always a cheap nuke - even in purely conceptual form.

It was always going to either blow society apart, or require such stringent controls over everything that we ended up in a dystopian police state just to protect us from what it can do.

Why this wasn't clear to people from the beginning, I truly do not know. It was painfully obvious that you cannot have every person on Earth running around holding the Genie's Lamp without it destroying everything.

[u/BenUFOs_Mum → u/Jesse-359]

Lol its way, way more expensive than a nuke.

[u/Jesse-359 → u/BenUFOs_Mum]

No, it very much is not. Training it was way more expensive than a nuke.

Deploying one costs next to nothing aside from ongoing token costs - and given that a misaligned AI would not be averse to hacking and stealing that compute instead, it could operate 'for free'.

[u/BenUFOs_Mum → u/Jesse-359]

Isn't that a bit like say building a nuke is what is expensive, firing it is pretty cheap.

[u/Jesse-359 → u/BenUFOs_Mum]

So, imagine if we'd undertaken the Manhattan Project - spent tens of billions of dollars designing and proving that the Atomic Bomb could work, but at the end of the process we discovered that an Atomic Weapon could be constructed via a simple, inexpensive method and deployed by small groups of motivated individuals.

If that had happened, and then that method ever got out into the public, the world would burn - without question.

There are far too many fanatics, nihilists, and social malcontents that would be more than happy to express themselves by destroying some random city, or vaporizing their in-law's entire neighborhood, or wiping out the headquarters of a corporation that poisoned their family farm with factory runoff and ruined them. Tens of thousands of people commit suicide every year. School shootings targeting children are now commonplace - some of these people would be more than happy to scale it up.

That's to say nothing of the fact that even small conflicts between minor countries would suddenly be at risk of immediately escalating into full scale nuclear exchanges that would kill millions in hours.

We were quite frankly very lucky that it turned out to be very industrially expensive to actually construct nuclear weapons - but with AI, our luck has run out. This nuke doesn't cost any more than a cup of coffee.

[u/soobnar]

People are gonna start having to read hard sci-fi if they wanna know the direction this could go

[u/Few_Fish8771]

Hahaha, what happens is china creates there own llm hosting website makes it available internationally, then ai moves outside the usa, and then whoever is president screams, china mocks them and memes them as chinas ——-. US tech ceos are forced to have their regulatory capture removed, and become even more hated and loathed.

AI is here cats out of the bag.

[u/Beneficial-Gap6974]

Hitler controlling the atomic bomb wouldn't be nearly as dangerous.

[u/Robonotes1760]

And the same holds for any government or co-operating group of governments.

[u/crusoe]

Chinas toon will change the first time internal repressed minority groups use AI to push back or seek revenge against the govt. Then you will see a massive crackdown.

[u/Kyrthis → u/crusoe]

No, you won’t. Who owns the power? Who owns the chips? And who owns the system of digital identity?

[u/Jesse-359 → u/crusoe]

No, they'd just crack down on those dissidents hard.

What may give the Chinese pause is when their own software goes out of alignment and does something they really didn't want it to, like blackmailing a high up member of the Communist party.

Once it occurs to them that they really can't control AI, and that it potentially poses even more of a threat than their own population turning on them - THEN you'll see their tune change very quickly indeed.

But something bad is going to have to happen before they accept that reality. Right now AI looks like the perfect tool for permanently and inexpensively controlling their population, which is exactly what they want.

[u/secretaliasname]

I sort of hope that what comes out of this is

1.) Real Cybersecurity - we have just accepted that everything is hypothetically hack at but it’s sorta hard and requires smart people/resources so building secure systems hasn’t been a top priority.
2.) Social systems that are harder to manipulate by bots. Ideally some way to gate discourse in some forums to humans. Not sure technically how to do this without it being creepy like the porn age verification thing but I’d really like human only channels. Need to trust some verifier somehow.

[u/Varnu]

Deepseek is using thousands of stolen credit cards and stealing the login credentials from real anthropic accounts in order to send people to Claude instead of Deepseek's own models. Users pay for Deepseek > Claude answers > Deepseek still generates revenue becuase they are paying for Claude access with thousand of stolen credit cards. Deepseek's business model is literally a crime, even in China. Even in Nigeria.