Update as of Jul 19, 2026: As of now, GPT 5.6 Sol can do this. Will post some results soon!
I started working on adding raytracing (RTX) to The Legend of Zelda: Ocarina of Time. As in my other posts, I’m generally trying to drive the project via autonomous LLM agents.
I was hoping the project could be as simple as running Claude Opus 4.6 in a loop, saying “implement RTX into this engine, make no mistakes.” I got surprisingly far with this technique. However, I started noticing that it hit a wall, and that it seemed to be bottlenecked by its ability to understand the engine’s screenshot output, identify issues, and figure out what to work on next.
