Ok. So, I spent last Tuesday night inside Jacob Coxon’s resignation thread which has now crossed a hundred and seventy million views and I stayed because the voice pulled me in, and he is twenty seven and spent three years doing pretraining work first at OpenAI on things like GPT-4o and then at Anthropic earlier this year, and he says neither lab is acting responsibly while they race toward self improving superintelligence. Evan Hubinger who leads alignment stress testing at Anthropic answered in public that he agrees and puts his own number above ten percent for extinction inside a decade, and CNN had Coxon on with Anderson Cooper to say the same thing slower. This is sad.

A few days later I fell into a reply from David Bellamy which steadied me because he claims a strange resume which might be unique, and he has trained a frontier LLM and he has also designed and synthesized custom viruses in a lab with his own hands, and he calls the virus fear total bogus because the bottleneck lives in steel and supply chains and protocols. I am no expert so please take this with a bag of salt, but his mechanics rang true to me as someone who has waited months for a Panda to ship. A genome on a screen means little until someone with freezers and cell cultures and mice spends days watching incubation do its slow work, and a fully automated lab which could brew the worst families on API command would cost well past a hundred million dollars and would need parts which take a year to procure plus firmware which was never built to be remote controlled, and DNA synthesis companies already screen orders while mice results translate poorly to humans the way every drug hunter learns the hard way.

In plain English; you can chat out a genome in seconds and still be a year and a hundred million dollars away from anything which breathes.

It lands for me because I have been saying models are limbs, and a hand grips and lifts and swings with great skill while borrowing all of its direction from the brain which moves it, and an LLM predicts the next plausible token with great skill while borrowing all of its direction from the human who prompted it. And direction is the whole game here. Why? Because wanting which stretches across months plus access to labs plus money for machines plus willingness to break seals on safeguards all live on our side of the glass. I learned this tinkering with openpilot in the Himalayas where there are no lane marks and it still kinda worked, and the blocker was mostly data and time and my own hands on the OBD2 port (I am not 😬 a good driver).

I keep looking for the machine which harmed anyone on its own and I keep finding human hands which signed the orders, and I watch Israel flattening blocks in Gaza with pilots and artillery crews following orders which came from rooms full of people, and I watch Donald Trump unleashing bombing campaigns and terror designations and deportation machinery with a signature which moves fleets. Both run on budgets voted by people and cheered by crowds and the suffering there already runs on payrolls and fuel and runways and a story which makes it feel necessary, and Its why the superintelligence debate feels like a distraction to me. Are we comfortable letting the signing hands hide behind the fear of a future model?

I recognize that story because I tell smaller versions to myself whenever I hand my thinking away. Coxon fears a mind which will improve itself in a datacenter and then reach out into the world, and Bellamy answers that the world pushes back with physics and lead times and screening and failed mouse trials, and I sit with both and feel the fear sliding past the machines toward the people who already know how to kill at scale. So I find myself wanting us to guard the signing and funding hands which already hold everything needed for ruin, because that is our dharma and our duty while we still have the chance.

The models wait on servers for our next sentence, and I want us to be worthy of the prompting.

If I got the biology wrong or missed a link in the chain, do tweet to me, I am @troysk704 or post a comment. We will fix it!