DOOMFLY
a fly, a brain map,
and DOOM + chess.
What happens when you connect a mapped fly nervous system to a game—and try to give a living fly the same controls?
Part desk experiment, part software rabbit hole. It started with Doom, reached a chess board, and taught me as much about measurement as it did about interfaces.
Not a simulated fly trajectory.
An actual arena on my desk.
166,700
retained neurons
25.58M
retained graph edges
Doom + chess
two game interfaces
Experimental
not biological validation
use the map. don’t invent the fly.
There are already mapped fly connectomes. We don’t need to invent a fake fly trajectory—or build our own game.
The idea was to put two very different signal sources behind one game-control contract: a simulated network built from MaleCNS v1.0, and motion measured from a living fly in an overhead camera feed.
Underneath it was a bigger curiosity. Where AI goes next probably isn’t only bigger models — people are already exploring biological substrates, neurons and cells as compute. This was my much smaller, desk-scale version of that idea: a reason to get my hands on physical, sophisticated hardware instead of only training models like I usually do.
The connectome is the measured wiring map. Retinal projection, neuron dynamics, and the translation from descending-neuron activity into buttons are chosen modeling and engineering assumptions. A full retained graph is not a literal living brain.
MODELED PATH
Game pixels → sensory input → full retained neural network → fixed descending-neuron decoder → controls
PHYSICAL PATH / ATTEMPTED
Enclosed fly → overhead phone camera → position and movement tracking → the same control fields
The intended comparison was shared controls, not identical cognition. Reliable measurement was a prerequisite—and the physical arm did not meet it.
a small arena. a lot of cables.
Transparent plates, a circular enclosure, binder clips, overhead illumination, my phone as the overhead camera (DroidCam), and my laptop generating the feedback signal shown to the fly. A small budget — the desk, some clips, and the principle that if something looks fun to try, I build it. These are my three physical-setup clips and still photographs, supplied on 15 September 2026. Web copies are silent and metadata-stripped; originals are preserved.
The whole rig
35 secInside the arena
38 secBehind the scenes
15 sec
Camera → controller
The phone watched the arena; the laptop turned tracked movement into the same control fields the connectome drove — position, heading, move and attack — and rendered the feedback signal.
Game → feedback
A FlyVision page mapped the current game/control signals onto a nearby display. It was an engineered stimulus interface, not a claim about the fly’s subjective vision.
Nothing aversive, nothing implanted
No electrodes, no shocks, no sound or electrical punishment anywhere in the loop. The fly’s own movement was the only input; light on a screen was the only feedback; sucrose was the only reward ever used.
Borrowed, not bought
The flies came from teaching practicals — lab stock already on hand, not animals ordered for the project. Whatever signal the camera picked up was the whole experiment: movement in, controls out.
Picking them up, putting them back
A length of tubing doubled as a handheld aspirator — a gentle draw pulls a fly into the chamber, and it rides the airstream into the arena. When a session ended, the flies went back the same way.
They kept to themselves after
Back in the vial with its mates after a session, a fly didn’t go where the others were going — it stayed apart from the group. Inside the arena, stillness had to be logged as stillness too, never scored as a finished turn.
field notes — what the fly showed.
Kept apart from its mates
Returned to the vial after sessions, the fly didn’t go where its mates were going — it stayed apart from the group. And inside the arena, a still fly produced no motion signal: stillness was logged as stillness, never as a finished turn. Quiet was data, not an ending.
One substrate kept improving
Across later sessions the fly’s game-relevant signals got better — it arrived at useful behavior on its own. The connectome, once set up, was static. The shame of the small rig: the fly clearly carried far more usable signal than I could harness.
Exploration before tailoring
Before the feedback signal was tuned toward the game, the fly roamed more of the arena. My own thought, not a result: once an agent knows enough, a constrained channel may reveal intent better than an open world — worth remembering for how we constrain models, even in RL, and note what they do.
first stop: Doom.
ViZDoom supplied the actual game. The modeled controller consumed game frames; the physical channel accepted measured-motion controls. The phone was the camera; my laptop decoded the fly’s movement into the shared control fields and generated the visual signal back to the fly.
The envelope arm constrained the connectome’s signal magnitudes and timing to recorded fly activity — that pairing, not the unrestricted run, is the fair comparison inside this setup.
| Channel | Observed episodes | Observed kills* | Reading |
|---|---|---|---|
| Living-fly channel | 9,098 | 3,758 | Watched live; periodic drift |
| Connectome + fly envelope | 8,832 | 3,170 | The fair comparison |
| Connectome | 563 | 6,527 | Unconstrained run |
| Random baseline | 8,468 | 6,820 | Game telemetry |
*Sum of available per-(run, episode) maximum kill counters, deduplicated. The fair pairing is the living fly versus the connectome under the fly’s own recorded envelope — and the living channel ran ahead (3,758 vs 3,170). Under matching signal constraints, the fly did more with the same budget; the unconstrained connectome outscored both only with a far richer signal path than any fly could ever drive.
then, a completely different board.
After the physical session, I ran a separate connectome-only chess experiment. This time, chess pixels—not recycled Doom actions—fed the network. The opponent was a seeded random legal-move sampler, not the living fly.
The living fly never got a board: the rig captured enough signal to steer Doom, not enough for 186 plies of menu navigation. That gap is itself part of the finding — the fly’s channel was real but thin.
A rendered board and sorted legal-move menu fed the existing retinal sampler and native neural model. Fixed decoded controls moved a menu cursor; neural attack activity selected an entry. The menu is an explicit interface aid, not chess strategy hidden inside the brain.
What stayed fixed?
All 166,700 retained neurons and 25,582,938 graph edges. The original baseline dynamics and BCI decoder. No fly-envelope handicap, no strategic engine, no reward current, no plasticity, and no learning during this chess run.
The RGB image was sampled as luminance at the existing receptor coordinates. Cursor movement used the fixed decoded turn/forward channels; selection had a 1,000-neural-ms UI interval. These choices are engineering, not measured fly chess behavior.
The final result was independently replay-checked for legal moves, PGN agreement, and matching saved input-frame hashes. One game against random moves does not establish chess skill.
The page it actually ran on.
No dashboard, no styling pass — a monospace page serving the board and the raw match state while the connectome worked through the menu. What you see above is a reconstruction; this is the original interface as it ran.
the interesting part wasn’t winning.
Measurement comes first.
A plausible trail can be a watermark. A fresh action can still be wrong. Verify the identity of the signal before interpreting the score.
The interface is part of the experiment.
A legal-move menu makes a chess session possible, but changes what the task measures. State that choice instead of attributing it to the connectome.
Keep the failures in the record.
The physical setup happened. The tracking limitations happened too. Retain the footage and telemetry without filling in missing results.
A fun project, not a claim that I taught a fly to understand Doom or chess. The useful outcome is a traceable game interface, a completed neural-control chess record, a living channel that outperformed the envelope-matched connectome inside this rig, and a much clearer sense of what needs to be measured next.