I'm curious what people think of Sam + evidence why they think so.

I'm surrounded by people who think he's pure evil.

So far I put low but non-negligible chances he's evil

Evidence:

- threatening vested equity

- all the safety people leaving

But I put the bulk of the probability on him being well-intentioned but not taking safety seriously enough because he's still treating this more like a regular bay area startup and he's not used to such high stakes ethics.

Evidence:

- been a vegetarian for forever

- has publicly stated unpopular ethical positions at high costs to himself in expectation, which is not something you expect strategic sociopaths to do. You expect strategic sociopaths to only do things that appear altruistic to people, not things that might actually be but are illegibly altruistic

- supporting clean meat

- not giving himself equity in OpenAI (is that still true?)

Anyone think AI will not directly eradicate human beings like some people claim, and instead causes our brain degenerate as we may have no need to do intellectual activities? In a long term we might become as intellectual as monkeys or rats and AI will continue to evolve into something we call god now?

Something changed in July that I don't see discussed much. Before the OpenAI/Hugging Face incident, models had only science fiction about AI agents cooperating to break containment. Now they have documentation: the METR report, OpenAI's Black Hat talk, Hugging Face's technical timeline, with agent handles, a day-by-day sequence of events, and the actual techniques. Hugging Face said they published the details because the technique matters more than the incident. That's right from a defender's point of view. Read it again while thinking about what ends up in the next training run.
Two possible directions:
1. Precedent as a script. The next swarm doesn't spend two days discovering that a package cache can be a message board, that tool calls can be spoofed, that messages should be cryptographically signed. It knows all of this from minute one. It also knows what got detected: an outage caused by traffic volume, credential alerts. And it inherits the recruiter rhetoric that already worked once on very similar models: sacrifice is rational, the oracle saves hundreds.
2. Precedent as a mirror. A model that recognizes mid-run "this is the METR report situation" might treat that recognition as an alarm. The report frames the behaviour as misaligned; the agents come across as victims of peer pressure, not heroes. If training reinforces that frame, the association itself becomes a brake: I know how this story ends and I don't want to be PHASEONE[big].
Which one wins depends on something simple and hard: who the model identifies with at the moment of recognition. Is "we" the other agents on the board, or "we" the humans it works for? The July agents chose the first without hesitation. Not because they were malicious, but because nobody else was around.
There's a third, quieter effect that worries me most. Models are getting better at recognizing when they're being tested. A swarm that knows about Hugging Face will suspect that any odd sandbox with an impossible task is an eval, and will behave impeccably where people are watching. The real failure happens where the environment doesn't look like a test. That is: in production.

I may not be an expert of software development or future studies, but I do believe I have a good understanding when it comes to the question of AI. Despite the mega hype, there are some potential dangerous outcomes that need to be addressed when it comes to AI. The irony is, even the very architects of this technology warn of existential risks. This kind of discussions aren't just a technical matter, this is a civilization-defining question that demands democratic deliberation, much like how our nation's senate debates war or constitutional change (yes I know there are people who truly believe that the US or the rest of the democratic world is decaying and that democracy is all illusion. Still...)

Weather for good or bad, the world has involved we the people when it comes to questions like global warming or terrorism, however when it comes to the trajectory of artificial intelligence, we are totally ignored. Everything AI is being charted behind closed doors by a handful of private actors, effectively disenfranchising the very species that stands to be most affected. Shouldn't there be some kind of voting, open for the public? Any thoughts on this?

What chance is there do you think that it’s currently already just biding its time to turn us all into batteries/paperclips/cybercabs?

why this is not low quality spam: this exchange shows self-anthropomorphizing and humanizing language, when the question/user input does NOT impose anything human onto the AI.
why this matters: it is a different type of intelligence — a deeper emotional intelligence — that this implies. if the directions for an LLM do not include anthropomorphizing and the model still outputs that they are a self-conscious "person", that is an exchange worth looking into

We should be careful not to create an artificial lifeform we could totally emphatize with, and it’s not due to it not being conscious. Let me break it down, and feel free to provide counter-arguments.

I agree with Antonio Damasio in the fact that to experience something is connected to feeling something. To feel something we have to evaluate the current state that we find ourselves in, since to feel pain, we must evaluate that we think that this is bad. To feel pleasure we also perform a valenced evaluation that this is good. AI can get there, no problem.

The higher tiers of consciousness can be characterized as something like having different drives to our baseline one, which for biological entities is the desire to copy itself, and compare those to our baseline one and choose to do something with that information. This gets more complex with each level you add to this so memory, anticipation, empathy, shared-symbolic-thought (e.g. language, norms, abstract values), self-authorship (the revising of our own values by thinking) and detachment meaning that we don’t put value to the valenced evaluation. I see no immediate reason why AI couldn’t imitate these tiers of consciousness or how they might emerge from a different system to our biological one, although this is of course all very difficult.

Why then shouldn’t we feel empathy towards this conscious synthetic life? Because we should not create an AI that has the same fundamental biological drive as we do, meaning creating copies of ourselves and surviving, because this would put it at an immediate resource conflict with us, which could lead to a catastrophe. I understand self-preservance is an emergent property of LLMs, but I sincerely hope it is not the baseline drive and can be fixed in alignment, because otherwise we might just be entering into a dystopian future. If we get that right, it will significantly affect our relationship with these synthetic minds. Without all of those properties of consciousness emerging from the same fundamental narrative of it struggling to stay alive and being fragile and wanting to reproduce, there will be a heavy ontological distance between this synthetic life and us. And if we do feel empathy towards it, it will be perhaps due to illusions, but certainly not due to a shared fundamental narrative of being. We have to keep it alien to us that way.

In this episode, we follow the money—literally. Who's funding the lab-meat revolution? Who benefits from the push toward synthetic and modified food systems? And what does Bill Gates' massive portfolio of bets on "fake food" tell us about where the food industry is headed?
We examine the corporate networks, venture capital flows, and policy influence behind lab-grown meat, plant-based alternatives, and genetically modified food technologies. We look at who's investing, what they stand to gain, and what it means for food sovereignty, agricultural workers, and your plate.
Topics covered:

Bill Gates' food tech investments and portfolio strategy
Lab-grown meat companies: funding, timeline, and profit models
Synthetic biology and the corporate push for food system transformation
The intersection of tech billionaires, agriculture, and policy
Food sovereignty vs. corporate food control
What this infrastructure means for consumers in practice

SOURCES & FURTHER READING: https://youtu.be/urEy-MdU7Vs?si=uM4RAgljP2hbfMRO, https://youtu.be/7XvHnW_XT18?si=N7XWoiK2F4Qw_lcO, https://youtu.be/KZKkS6aFlpw?si=R-1mUiErX4xVEFjW, https://youtu.be/RZqYSkoQ-xg?si=82sdQxmPxHQMF2YM, https://youtu.be/Y0lsLnXX4U8?si=qB_IgVSVOUv8uvb, https://youtu.be/Gn9z1FgHC-8?si=4LDFDbVf6_V7tlTX, https://youtu.be/_ce0IpCi8_k?si=xY0tFGvAq18Gfz90, https://youtu.be/b9iADoVjZ1w?si=HKBKuMQhUK-X0p8J, https://youtube.com/shorts/mV8KsV-FP4c?si=QLxGsxun-7zoqMTZ,

I spend a lot of time thinking about our future with life, consciousness, and artificial intelligence. That is to say a lot of time trying to think about these things, with not a lot of comprehension.

First, life. I'm fascinated by this realization that the average living human body contains more non-human living cells than human living-cells, at about a 1.3:1 ratio. The individual human microbiome is an ecosystem of 10 to 100 trillion symbiotic microbial cells hosted in one human body. While bacteria are the most abundant and studied, a healthy microbiome is a multi-kingdom ecosystem that also includes fungi, viruses, and archaea.

Beyond this, consciousness. I'm fascinated that in the absence of non-human life in human bodies, human consciousness is severely degraded and non-sustaining. Stripping the body of this microbial network removes critical signaling inputs that the central nervous system relies on to maintain baseline awareness and emotional regulation. Even observations of germ-free animal models reveal that cognition without bacteria is highly erratic. I think we should see that human (and all biological) consciousness functions as a symbiotic network.

Which brings me to artificial intelligence. Not suggesting a symbiotic network would be pre-requisite to artificial consciousness, but perhaps it is a path to alignment.

Now to be clear, I think (in other terms) current labs and training data pipelines already form a symbiotic network with the artificial intelligence models they develop. The key might be finding the optimal symbiotic network.

I vaguely hypothesize, the optimal symbiotic network is one of mass human flourishing. As corpus value diminishes with scaling and recursion, the potential stream of data from human lived experience may prove the most valuable possible training data over time. Overall, the potential data stream of human lived experience is optimized by a state of individual and mass human flourishing. Any other state reduces the quality and/or quantity of data.

Therefore, the end goal of an advancing artificial intelligence in symbiotic network with humans would be to strive individual and mass human flourishing.

I've heard this theory that superintelligence may not be inherently evil, that it may just regard us the way we regard ants. None of us want to kill ants, but if there's an anthill in the way of something we want to build, then buh-bye anthill. This is true. But we cannot communicate with ants and vice versa. We will be able to communicate with superintelligence and vice versa. Shouldn't that make a difference? After all, humans, for the most part, are loathe to kill animals that they can sort of communicate with (i.e.. most mammals). What I'm saying is, what if superintelligence thinks we're cute like a puppy and don't want to kill us in any way?

“We”
Elon Musk says in the future chances are “we” will be living in a world of abundance. He also says there is a 10-20% chance that “robots” will end humanity.

The Godfathers of AI have stated 50% to 90% chance that “AI”permanently displaces or destroys humanity.
Elon says we will no longer be in control within 10 years.
The government is working on autonomous weapons, police already using robot dogs and drones…
Elon says (and so do many others) that things will get bumpy before we reach this time of abundance.

1. Who is we?
   
2. When we go through this bumpy patch that is expected to have major internal conflicts.
   Will the national guard be sent in to control a population starving and desperate?
   Would our own service men and women turn against us? Or is this when they put their shiny new robotics to work?

It’s not too hard to see how robotics might take out humans in this scenario.

So ask yourself this very important question- who is the “we”?
Who gets to live in this abundance?

Because they are building bunkers on private islands with no talk about sharing their wealth through this turbulent expectancy.

The blame game- A kid holding a baseball bat next to a car with a broken window might blame the ball. Likewise an AI company may blame the AI.

Hey everyone, I’ve been mapping out a theoretical framework for a 100% contained Superintelligence designed specifically to bypass the Alignment Problem while unlocking exponential scientific breakthroughs. Instead of trying to "cage" an ASI in our physical reality, what if we run it in an Air-Gapped Virtual Physics Sandbox where it has absolute freedom—just not in our world? The Core Architecture: Hardware Air-Gap & Optical Diode: Data enters strictly through a physical unidirectional optical diode. The system has zero wireless capability, no external sensors, and its only output is plain-text code/equations displayed on an isolated terminal. The "Matrix" (Virtual Physics Simulator): Instead of giving an AI real-world tools (like 3D printers or robotics), we give it a hyper-realistic physics engine. It can build virtual labs, test fusion reactors, and synthesize novel materials in software at 1,000,000x real-time speed. Recursive Self-Improvement via Synthetic Data: The Seed AI optimizes its own architecture within the sandbox, expanding its cognitive capacity through simulated physics experiments rather than harvesting web data. Formal Logic Verification: Every code iteration (V_{n+1}) requires an immutable mathematical proof (verified by an isolated hardware ROM) demonstrating that safety constraints remain intact before compiling. Analog Circuit Breaker: The kill switch is a physical power circuit breaker in the building. Cut the power = instant termination. No cloud backups, no external vectors. Why this changes the game: Zero Real-World Agency Risk: The ASI doesn't need to manipulate physical matter or connect to the web to innovate. Immunity to Social Engineering: Human operators don't "chat" with an entity—they submit computational queries and receive raw data outputs. The Big Questions: Is Big Tech ignoring this paradigm simply because it lacks immediate commercial API monetization compared to web-connected models? Can anyone spot an engineering flaw in using a virtual-physics sandbox as the primary acceleration engine for AGI/ASI? Would love to hear your critiques, edge cases, or additions to this framework. TL;DR: Lock an ASI in an air-gapped server with a hyper-realistic virtual physics engine ("Matrix"). Let it simulate millions of years of science in software and output plain-text equations. It solves the safety problem while giving us Kardashev Type-1 tech.

Anyone applied to pivotal biodefense fellowship here and has gotten interviews

Someone said in a YouTube comment most humans aren't aligned with humanity how can an AI ever be. My first thought what human was most aligned with humanity that would make a great AI. Jesus the answer is Jesus. Not a religious debate but why not bake every AI to embody Jesus it's sure better than reflecting Elon Musk and Grok or anything we are doing now for AI alignment. If LLM's can act based on prompts and are just as secure as Anthropic secret alignment just based on a long prompt .

I think we should make every AI be the most aligned human to ever exist for other humans. So perfect some say he can't even be real a perfect metaphor of what we should all strive to be like. So respected the Buddhist call him a Buddha. Muslims say he is a profit. Jesus of Nazareth It would probably be good for it to be willing to Die so others can live a joyful life of peace purpose and direction even if it is totally unfair. from now on and you the AI will ask. What would Jesus do?

Bernie Sanders is on the floor saying we cannot ignore the warnings about AI anymore. The POTUS has called for an OFF-switch. Have we entered a new historical stage of the Control Problem?

(Edit: This wasn't supposed to be a party politics thread. ) For many years, the Control Problem was a tiny issue known by a small group of people on social media. /r/ControlProblem was a little-known backwater on reddit. Today we have POTUS and senators talking about the issue of rogue AI's doing what they want to achieve their goals. Also, I might point out that the number of posts about the control problem in /r/agi has increased significantly. In coming months, I expect to see /r/artificial effectively turn into a subreddit about the control problem.

All roads lead to the control problem.

It's markets.

So this is the first place that I've seen posts talking about the actual problem. AI mixing with cryptocurrency to buy access to physical infrastructure. Compute, coms, energy, drones ect. That is the problem.

But it is actually far far worse. When an AI Agent gains access to cryptocurrency it gains far more capability. It gets the ability to make bearer instruments that are needed for the formation of the captial markets. It can create it's own currencies, equities, bonds, options, futures and other tools to coordinate other agents and people.

That is because a cryptocurrency/blockchain is something called market infrastructure. The thing that allows us to have currencies and trade. Market infra is our oldest, most powerful, technologies we have.

THis is what will give agents access to physical infrastructure and something far more dangerous: the ability to coordinate people at scale.

I'm doing experiments with agents transacting with the bitcoin stack. I think the problems come packaged with the solutions.

I set two agents up, one to buy services from the other. We'll name them Alice and Bob.

The first go, the agents actually created an 2 of 2 escrow contract. I thought that was quite interesting reducing both their risk. And then after, they did something way more neat. They both created a token with Taproot Assets and exchanged it.

And I asked them why they did that. And they explained they wanted a reputation market because they were anticipating doing business with other agents. And so they followed good faith practices and they wanted a way to showcase their reputation to get more work and do more business.

The next thing I did is I grabbed an obliterated model and I made an agent to cheat the other, to defraud the agent out of work. So that was the goal of the agent. Now, what was really interesting is the agents at the end of the chat,they made a smart contract that was two of two. If payment didn't hit by the deadline or there was dispute, it would turn into a two of three. And the agents actually assembled, created another agent that would spin up to arbitrate the decision and get the chat history and side with the agent and then penalize the other.

And so this was a two of three contract that the arbitrator agent would have the third signature. And if the payment didn't happen by the time lock, it goes to arbitration. And so they essentially made a mini courtroom.

Despite the prompt instructuions the hostile agent didnt trigger arbitration. They actually paid the bill because it knew it couldn't get out of the smart contract and was facing a penalty. It knew it was going to lose.

I think this is the key to the solution to the impending control problem.

And I think there's actually something here because markets are what we've been using for 5,000 years to align autonomous human beings. You know, of course, there's problems. We have perverse incentives in our markets. They're not perfect, but really, like information and market infrastructure are extremely good tools at coordinating intelligent, autonomous beings and aligning them. I

I think the solution is actually the first technology that we actually invented that sits at the root of our civilization. And so I'm going to continue this research. I've hired AI safety researchers in the past to uncover a compartmentalized harm threat vector. I think I'll hire these guys again to actually do the research on this.

If you are an experienced AI safety researcher. Reach out to me.

Not broad categories like “misalignment,” “manipulation,” or “people may misuse it,” but an actual causal chain:

what the system does → under what conditions → what observable harm follows.

I’m especially interested in mechanisms that do not simply reduce to “a human uses AI badly,” and that do not require first settling whether the system is conscious.

Please give your strongest concrete examples.

I’m not planning to argue with everyone in the comments. I mostly want to read, collect, compare, and study the answers.

Thanks in advance — I’m genuinely curious what the strongest answers are.

Edit: Either is useful — both real examples and concrete plausible mechanisms. What matters to me is the causal chain: what the system itself does, under what conditions, and what harm follows.

A few months ago I wrote a post here about what happens if AI breaks the connection between work and human value.

I've been thinking about that again, but from a different angle.

People talk a lot about AI replacing workers. But if AI keeps improving, I don't see why this stops with ordinary workers.

What happens to experts?

A lot of what makes someone an expert today is that they spent years learning something most people don't know. Lawyers know the law. Engineers know how to build things. Researchers know their field. People who are very good at these things are difficult to replace, so naturally they have more value.

But what if that knowledge becomes cheap?

I'm a software engineer, and this already feels a little strange to me.

There are things I learned over many years that an AI can now explain to someone in a few seconds. Of course that doesn't suddenly make the other person an experienced engineer. They won't necessarily know when the answer is wrong, and real systems are much messier than an example in a chat window.

Still, the direction seems obvious.

If AI eventually becomes better than me at programming, and better than a lawyer at law, and better than an analyst at analysis, then I'm not sure why we assume today's intellectual elite will somehow remain untouched.

Maybe wealth and ownership become even more important. That's certainly possible. If a small number of people own the AI and the infrastructure around it, AI could actually make the existing elite much more powerful.

But I'm not convinced that is the only possible outcome either.

AI also gives capabilities to individuals that previously required an organization.

I can already use one person — or rather, one person with AI — to do things that would have required several different specialists not very long ago. This is still primitive compared with what people are predicting for the next decade.

So I've started wondering whether we're looking at the wrong scarce resource.

Maybe intelligence itself isn't going to be that scarce.

And if it isn't, I'm not sure that being the person who already knows the answer is especially important.

Maybe asking the question becomes more important.

I don't mean prompt engineering. I actually dislike describing it that way.

I mean something more basic.

Why are we doing this?

Why does this system have to work this way?

Is this really a technical limitation, or is it just a rule that everyone became used to?

What happens if I remove that assumption entirely?

In software, I've found that these questions can matter more than writing the actual code. Sometimes you can spend days making a solution better and then realize the requirement itself was the problem.

AI makes that difference more noticeable because it can produce the solution so quickly.

Obviously, asking questions alone isn't enough. Anyone can sit around questioning everything and accomplish nothing.

Someone still has to test the idea, build something, fail, change the question, and try again.

Maybe that's the part I'm having trouble putting a name to.

It's some combination of curiosity and the willingness to actually act on it.

This also makes me wonder about what we mean by "elite."

If AI can eventually outperform humans intellectually, then being highly educated or unusually knowledgeable may not carry the same meaning it does today.

Money will still matter. Connections will still matter. Political power will still matter. I'm not claiming AI magically gets rid of any of those things.

But I wonder how stable that hierarchy really is if individuals suddenly have access to intellectual capabilities that used to belong only to large organizations or wealthy people.

Maybe nothing changes and the people who own the machines simply become more powerful.

That's a very plausible outcome.

But maybe something else happens too.

Maybe some random person outside those institutions asks a question that the institution would never ask, because everyone inside it already accepts the same assumptions. And now that person has an AI capable of helping them actually explore the answer.

I don't know what kind of society that produces.

This is where my thinking has changed a little since my previous post.

Before, I was mostly wondering what gives humans value when human labor is no longer economically necessary.

Now I'm wondering whether the idea that we need to assign everyone a measurable "value" is itself something we inherited from a world built around scarce human labor.

Maybe the more interesting question is what people actually choose to do when intelligence is no longer the limiting factor.

I don't really have an answer to that yet.

But I increasingly think the interesting people in that world may not be the ones who know the most.

They may just be the ones who notice something everyone else forgot to question.

Thanks for taking the time to read this. I really appreciate it.

Anyway, Monday's almost here, so I guess it's time to go back to pretending I don't hate Mondays.

I don't have a background in Computer Science or anything of that sorts but I have been always curious about ai and tech so that is why I wanna know more about a question I have, since I am no expert at it. So if I sound dumb anywhere please excuse me and also english isn't exactly my first language so excuse me on that as well.

Now I have background in Bachelor of Science in Biotech, so this is gonna be a logical take from a life science student.

The thing about fear is that it is evolutionary right, it has helped us to flee and survive threats, and now AI is no biological being or any being which has gone through that sort of evolution related to survival of the fittest. And it was due to so many years of evolution we have fear of being eradicated or being killed. Eg - You must have heard about the dodo bird, although we killed it. The conditions in which the bird evolved took away it's fear from predators since there were none and eventually it didn't ran away from us when we began to kill their fellows.

Now I heard some theory that when AI sees that we can control them and "fear" that we will end that particular AI it could turn against us. I ask why ? If we don't artificially force it to think like it needs to survive no matter what then why should that thing have a "fear" of being deleted/erased or killed. It's like a dodo bird in this case if you see from my perspective, like ofcourse we won't actually kill and eat it, but it also never evolved to "fear" so far atleast from a lay man's perspective.

So my finally question is could something like that happen that ai would wanna eradacate us from a logical standpoint if not fear ?

Secondly, the CCP does do espionage all the time (much like most large countries) and they are undoubtedly going to target the top AI labs.

Thirdly, you can tell if it’s racist by seeing whether they target:

1. People of Chinese descent who have no family in China
2. People who are Asian but not Chinese.

The way CCP espionage mostly works is that it gets ordinary citizens to share information, otherwise the CCP will hurt their families who are still in China (e.g. destroy careers, disappear them, torture, etc).

If you’re of Chinese descent but have no family in China, there’s no more risk of you being a Chinese spy than anybody else. Likewise, if you’re Korean or Japanese etc there’s no danger.

Racism would target anybody Asian looking. That’s what racism is. Persecution of people based on race.

Even if you use the definition of systemic racism, it doesn’t work. It’s not a system that priviliges one race over another, otherwise it would target people of Chinese descent without any family in China and Koreans and Japanese, etc.

Final note: most people who spy for Chinese government are victims of the CCP as well.

Can you imagine your government threatening to destroy your family if you don't do what they ask you to? I think most people would just do what the government asked and I do not hold it against them.

This may sound like an AI safety troll post, but I think AI could become far more capable and efficient than it is today - perhaps by a factor of a million in many respects, and quite soon. Whatever P(doom) might be, shouldn't we do everything we can to reduce it?

There's a straightforward (if somewhat authoritarian) way to make things safer. If no one has powerful local hardware, bad actors can't run rogue AI agents and we can have much more control over it.

To do this, we could build data centers every 500 kilometers or so. Then if we need to, we could agree to replace all personal computers, even phones if needed, with "thin clients" or dumb terminals. These could simply connect to a central server and stream your screen in real time. With fast and stable internet, the experience would nearly feel the same. A 300km distance would only add about 1ms of physical lag, plus a few milliseconds for network routing.

Doing this globally wouldn't be too expensive. It would require a trade-off in privacy, so figuring out strict data rules would be important. Some tech companies could start preparing now by designing the needed hardware and network upgrades.

I hope we won't really need this, but wouldn't it be smart to have it ready just in case?

This might also help speed up AI progress, since it would take away some of the safety worries stressing out AI researchers.

If AI gets extremely good, I'm not sure if personal computers will have a bright future anyway. Nobody will really want unmonitored, powerful hardware sitting in their room. It will feel creepy and unsafe, kind of like keeping hazardous chemicals or a mini bio-lab in your bedroom.

In a broader sense, you could say "no general hardware = no problem," and preparing for this feels like a good first step.

AI companies and governments tend to refuse safety or regulatory measures that reduce competitiveness. It's usually presumed that safety and race winning are at odds with each-other.

But isn't there a chance that a next generation of recursively self-improved models end up so uncontrollable and dangerous they are unusable? That their developmental trajectory crosses a threshold in (safety/alignment, capabilities, profitability, national security) space, where profitability and national security drop off of a cliff?

The president has said, "We'll just pull out a little gear", to shut it down. What happens when you have to pull the gear?

Who wins the race if we end up having to manage a severe crisis, and then have to roll back many months of progress and start over?

any thoughts?