Recursive self-improvement, Google’s world models, and what the jagged road to AGI looks like. Good sketched the idea in 1965 — build a machine slightly better than us at designing machines, and it designs a better one, which designs a better one, until you get what he called an intelligence explosion. Sixty years later the loop exists. The question is what kind of loop it is, and whether it gets us to AGI before the money or the science runs out.
On the night of 18 March 2018, an Uber test car in Tempe, Arizona, struck and killed Elaine Herzberg as she wheeled a bicycle across an unlit road. The car had detected her 5.6 seconds before impact. Its software could not work out what she was. It cycled through “unknown object,” then “vehicle,” then “bicycle,” and never settled in time to brake. About 1.3 seconds before the collision the system concluded it needed to stop. It did not stop, because Uber had switched off the car’s automatic emergency braking during testing and was relying on the human in the seat to intervene.
The human in the seat was a safety operator named Rafaela Vasquez. She was watching a television show on her phone. She did not touch the brake until after the car had hit.
When the courts went looking for someone to hold responsible, they found her. Vasquez was charged with negligent homicide, later pleaded guilty to endangerment, and was sentenced to three years of probation. Uber faced no criminal charges. The federal investigators were blunt about the design: the company had removed the car’s ability to brake for itself and left a distracted person as the last line of defence. The system was built so that the only party who could stop the car was the one least equipped to.
The Prism, the Pointing Arm, and What Hinton Got Right (and Wrong).
In a recent talk, Geoffrey Hinton offered a thought experiment he thinks settles a long-running argument about machine consciousness. The setup is simple. You have a multimodal chatbot with a camera, a robot arm, and language. You place an object in front of it and ask it to point. It points. You then sneak a prism in front of the camera lens. You ask again. It points off to one side, because the prism has bent the light. You tell it about the prism. The chatbot replies: “Oh, I see. The prism bent the light. The object is actually straight in front of me, but I had the subjective experience that it was off to one side.”
Hinton’s claim is that the chatbot, in saying this, is using the phrase “subjective experience” exactly the way you and I use it. Therefore the chatbot had a subjective experience. Therefore the line we draw between human and machine experience is, in his words, rubbish.
I had to think about this, because Hinton is that guy, and the example is doing more work than it first appears. But I also want to say where I think the argument is weaker than it is being sold, and where I think it is stronger.
How I used Claude to build a 5,000-entry index for a 600 page technology book without going crazy. The hard part of indexing is not inserting tags, but you will want to automate the boring mechanical labour.
I have spent the last couple of years writing a book about Embedded AI for No Starch Press (NSP). It has been three times the amount of work that I was expecting. The editing process feels never ending and by the end you will never want to read your book again. It does make for a better book though, and I am now an advocate for having an external editor.
One of the more tedious aspects of putting together a book is index tagging. The publisher can index for you, but it will cost around $4 per page and this comes out of your royalties. Not many people get rich writing a book, but there is no point throwing away hard earned royalties! There are lots of things you have to do manually when creating a book (like writing), but this feels like a task that a Large Language Model (LLM) should be good at.
NSP expects the finished index to weigh in at 5 to 8 percent of the manuscript word count, so for a 100,000-word book you are building something in the range of 5,000 to 8,000 words. This is a substantial document in its own right.
Boston Dynamics and Google DeepMind announced last month that Spot is now running Gemini Robotics-ER 1.6, a high-level embodied reasoning model. The headline is that Spot has been taught to reason. The commercial pitch is industrial inspection: wandering around a facility, reading gauges, spotting spills, deciding what to do when something looks wrong. So far, so good.
Buried in the demo video is a small failure that says more about the architecture than the press release does. Asked to recycle cans in the living room, Spot picks one up sideways. If there is any liquid left in the can, it ends up on the carpet. The problem is using a language model to solve something handled by other more primitive layers in the organic world.
MIT’s August 2025 study of 300 enterprise generative AI deployments found that 95% produced no measurable P&L impact. Gartner forecasts that more than 40% of agentic AI projects will be cancelled by 2027. McKinsey’s State of AI 2025 identifies workflow redesign as the single strongest correlate with EBIT impact, yet only 21% of organisations have redesigned any workflows. The data converges on a structural conclusion: enterprise AI is failing because the operational substrate is inadequate, not because the models are. This episode examines the process-readiness gap, the misallocation pattern that concentrates investment in low-ROI front-office applications, and what the 5% of high performers do differently. It is an architectural argument, not a change-management one: AI is a linear amplifier acting on a pre-existing process, and the sign of the output depends on the sign of the input.
Every time a news story breaks about some new AI mishap, a chatbot lying to a user, a self-driving car making a dodgy decision, a recommendation engine nudging teenagers toward inappropriate content, someone in the comments inevitably writes, “We just need Asimov’s Three Laws.”
The trouble is that Isaac Asimov wrote the Three Laws as a plot device, not as an engineering specification. Almost every story he wrote about them was about how they fail. It is a bit like reading Jurassic Park and concluding that you now have a solid operating manual for cloning dinosaurs.
If we are serious about building safe AI, and particularly if we are building the sort of layered, bio-inspired systems that drive physical robots, we need to start from a different foundation. This article explains why, and proposes a replacement set of principles drawn from the Primal Layers framework.
I know! I’m late to the Claude Code party but now I’m here, I’m all in. If you write C++ for microcontrollers, or edge inference, you already know that the rules are different from desktop software. No heap allocation after startup. No exceptions. No recursion on a 4 KB stack. And these constraints are not optional if you want your firmware to survive.
The problem is that general-purpose linters do not enforce the rules you need. Clang-tidy is powerful, but configuring it to catch you just used int instead of int32_t, requires writing custom checks in C++ against the AST. That is a significant investment for what should be a simple rule. I wanted something I could tweak for each project.
Up until now we have been using ChatGPT and Gemini to scratch our vibe-coding itch. But the future is agentic and a lot of folks swear by Claude so we thought we would give it a go and use the Anthropic Academy to learn how.
If you want to use Claude Code you are going to need at least a Pro account (i.e., pay Anthropic some money). There is an interesting tweet (what do we call them now?) by Boris Cherny about how he uses Claude Code. As its creator, his advice is credible. Boris runs five Claude’s in parallel in terminal and jumps between them as they spit out a result. He has another 5–10 Claude’s in the browser (http://claude.ai/code). We are going to start with a more modest single Claude and take it from there.
We recently came across the Joint Strike Fighter Air Vehicle C++ Coding Standards for the F-35 program. This was published by Lockheed Martin many years ago, but it contains some useful guidelines for coding C++ on embedded systems. The number one objective for the document was, the software must not fail.
Although written for military aircraft, the philosophy behind these rules applies directly to modern embedded systems. If you are building firmware for medical devices, robotics, automotive systems, or industrial control, this distilled guide captures the most practical lessons for your own C++ standards.
Our position on guidelines is that they need tailored for the application. Guidelines exist to reduce risk. The level of restriction they impose should match the level of consequence in the system. In life-critical systems the cost of failure is too high to optimise for elegance. If you are going to ignore a guideline, you had better know what you are doing. Coding standards should scale with the risk, and breaking a rule should always be an informed decision rather than an act of convenience.