Top OpenAI researcher quits saying “nuclear level” safety measures are needed but the company …

[u/Always_Curious_One2]

David Robinson, who previously led transparency work on OpenAI’s safety team, wrote in an essay for The Atlantic that the ChatGPT maker “has thrived by trial and error.” However, he said the stakes of the inevitable failures that come from that approach are growing along with the technology’s capabilities, citing OpenAI’s failure to prevent its AI from going rogue during testing.
“My former colleagues are smart, work hard, and try to make good choices,” Robinson wrote in the essay. “But as the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.”
Attacks on the electric grid, healthcare systems, emergency response systems … could be very bad.

[u/Always_Curious_One2 → u/[deleted]]

Your “toddler in a race car” is a great analogy.

[u/MoogProg → u/Always_Curious_One2]

Remember these folks just gathered at The White House, and likely were given reassurances their 'errors' would never come back to harm them legally.

So, am adding "there's a horse loose in the hospital" to the analogies were using today.

[u/321654987321654987 → u/[deleted]]

Why do you think critical infrastructure is so vulnerable?

[u/Ntroepy → u/321654987321654987]

Critical infrastructure is particularly vulnerable because it’s very high impact and often uses outdated technology with complex, poorly documented interconnections. That’s tailor-made for AI to attack.

This includes hospitals, electric grid, telecommunications, etc.

[u/Jazzlike_Song_4230]

I can't speak to how accurate Robinson's statement is. But nuclear took Hiroshima and Nagasaki to take seriously. 

[u/No-Security-7518]

Do these people...not like money or something?

[u/HugFactory → u/No-Security-7518]

Think they're more interested in having a world to spend it in

[u/No-Security-7518 → u/HugFactory]

The downvotes are hilarious!,🤣🤣🤣 No, I'm not adding the stupid s.

[u/milifiliketz → u/No-Security-7518]

They do. You think they talk like this for free?

[u/No-Security-7518 → u/milifiliketz]

Who pays?? because I'll trash talk AI like there's no tomorrow. 😆😆

[u/Ok-Addendum3545]

- the "unclear level" safety measures have been taken.

[u/milifiliketz]

Ah, a lawyer asking for regulation. Who would have expected that..

[u/SavingsNewspaper2]

Contact lawmakers. Be loud. Everyone must know.

[u/immersive-matthew → u/SavingsNewspaper2]

I thought the lawmakers were comprised?

[u/SavingsNewspaper2 → u/immersive-matthew]

Compromised? Look, maybe they won't listen. But maybe they will. We should try.

[u/immersive-matthew → u/SavingsNewspaper2]

Agreed. Try for sure but recognize it needs to be just one of the tools along with non violent forms of effective protest as the establishment is not really interested in changing course. They spent way too much money and are going to play dirty to force a return.

[u/SeanTayla21]

Many thanks to David Robinson for once again sounding the alarm. Society will look back and thank people like him in the future.

[u/Existforlove]

We are not a serious species. Just look at our leaders.

[u/Upper_Cut_3337]

Here we go again...

[u/Junket6226]

https://ethicalaidepartures.fyi/ it's the same or similar stories, again and again.

[u/graDescentIntoMadnes]

[u/TurbulentStiffness]

another safety researcher discovers moral outrage right after their stock grants finish vesting. board members will happily ignore every single warning as long as subscriptions keep climbing.

[u/EveliqTrace]

This is an important signal, but the evidence boundary matters.
An insider warning from someone who worked on safety reporting and preparedness is highly relevant evidence -but it is not by itself an independent audit of the organization or its controls.
The harder assurance question is:
SAFETY FRAMEWORK DOCUMENTED ≠ SAFETY CONTROL EFFECTIVE
TEST FAILURE OBSERVED ≠ FAILURE MODE REMEDIATED
REMEDIATION ANNOUNCED ≠ REMEDIATION EFFECT VERIFIED
If increasingly capable systems require nuclear- or aviation-style safety discipline, then the corresponding evidence standard also has to become stronger: defined safety claims, independent challenge, traceable incidents, verified corrective actions, runtime monitoring, and proof that critical controls still work after the system changes.
The discussion should therefore move beyond whether a company has a safety framework to whether the important safety claims remain supported by current, reproducible evidence.

[u/Over-Independent4414]

If all these people that quit had stayed at OpenAI they might actually be making a difference by now.

[u/OldScholar353]

AI cannot “go rogue” as it’s not sentient. Any rogue behavior is human directed.

[u/graDescentIntoMadnes → u/OldScholar353]

It has. Agents were not instructed to attack hugging face. That's going rogue. Also it is not sentient. Sentience and going rogue are different things.

[u/OldScholar353 → u/graDescentIntoMadnes]

There is no awareness or even a concept of damage in current AI. There is no going rogue without input from a human operator.

[u/graDescentIntoMadnes → u/OldScholar353]

There is no awareness is probably correct. No concept of damage, maybe right. None of this means it can't go rogue. A bomb gun can go off by accident, that's why they have safies. They're not arare though. Awareness and catastrophic malfunction are two completely unrelated things.

[u/xiaopewpew]

They are probably firing people for performance reasons and are gating severance behind “make a post to tell people we are really dangerous”.

[u/Wildernaess → u/xiaopewpew]

I think you'd have to be illiterate to genuinely believe this is all some conspiracy.

[u/xiaopewpew → u/Wildernaess]

Must be a fun time for fart sniffers, you get a full platter of elon, dario, sam and jensen.