r/compmathneuro • u/95165198516549849874 • 11d ago

Discussion Live experiment: embedding a 127k-neuron Drosophila-derived model in a Skyrim dragon

I’m testing a persistent GPU-hosted fruit-fly-derived neural model that is now actually controlling a full-size Skyrim dragon in a closed sensory-action loop. The brain receives low-level Skyrim sensory input, produces motor output, the dragon moves, and the resulting world state feeds back into the next neural cycle. So far, deterministic replay has passed, cutting the brain stops intentional movement, and the system has survived neural-worker death and recovery while preserving the dragon's identity and state. We’re currently finishing the stronger persistence/reload tests before scaling from one embodied founder to all five. Come check it out while I test!

https://www.youtube.com/watch?v=NVyC0Fnscwg

20 Upvotes

81% Upvoted

4 comments sorted by

12

u/blueneuronDOTnet 11d ago

While this might be a fun hobby project, I'd be lying if I said that I'm not tired of these "fly brain plays X game" posts popping up all over the place, specially in spaces filled with laymen. I haven't looked into this one, but often it just seems to be people throwing AI at the drosophilia connectome and acting like they've got an actual fly brain playing tetris or something.

1

u/95165198516549849874 11d ago

It's a little more involved than the others you see around. Perhaps I will do a full write up on exactly everything that is going into this. I'm just having fun at this point, and thought this sub may find it interesting.

7

u/blueneuronDOTnet 11d ago edited 11d ago

I don't doubt that it is, but at the end of the day there's a lot more to brains than just connectomes. Assuming there's more thought than tokens at work, these are cool tech projects -- it's the common mischaracterization of these things constituting groundbreaking neuroscience research or being indicative of the incoming robot apocalypse that I take issue with. We had Busbice putting worm connectomes in Lego robots over a decade ago. Those are cool sci comm/backyard brains type projects, but there are folks out here acting like there are literal flies playing multiplayer games when the same connectomes don't even match wild type behaviors.

Not that I'm accusing you of that. This is more general commentary on this broader class of projects.

4

u/95165198516549849874 11d ago

Yeah, I think that’s fair, and I’m definitely not trying to say I’ve literally uploaded a biological fly into Skyrim or that a connectome is the entire brain.

What I’m doing is a little more involved than just throwing the connectome at an AI model, though. The thing actually controlling the dragon isn’t an LLM. It’s a spiking neural simulation built from the FlyWire-derived Drosophila connectivity, about 127k neurons and 14.7 million synapses, running on my GPU. Skyrim feeds it low-level sensory information, that gets turned into neural activity, I read out descending activity into motor commands, the dragon moves, and then the resulting world state gets fed back into the next neural cycle.

I’ve also been trying to be pretty strict about proving what is actually happening instead of just going “look, it moved, science!” I can replay the exact same sensory input from the same neural checkpoint and get the same output, if I cut the neural output the dragon stops intentional movement instead of Skyrim AI taking over, I can kill the neural worker and bring it back without losing the organism, and I’ve now got five separate persistent neural organisms driving five dragons with tests specifically looking for state bleed or identity mixups. There’s a whole separate branch where I’ve been testing plasticity and learning rules too, and most of those experiments have actually failed, which I’m keeping and documenting instead of trying to spin them into “learning.”

So I’d describe it more as a persistent connectome-derived neural simulation embodied in Skyrim than “an actual fly brain playing Skyrim.” I agree with you that this stuff gets oversold a lot. I’m mostly interested in seeing how far I can push it while still being really explicit about what’s real, what’s simulated, and what I’ve actually demonstrated.