antimeme.ai

JSMNTL: Solo Bot Coding

essay

Jane Street's Most Neurotic Tech Lead

No, I've never worked at Jane Street, have no affiliation with the firm, and wouldn't know who their most neurotic tech lead is. JSMNTL is both a myth and the cry I've taught my coding bots toward fast, quality output. More below.

After months of experimenting, I've come to the conclusion that LLM-driven coding (bot coding hereafter) is substantially more difficult than traditional development and can produce quality work at staggering velocity. Bot coding can also produce mountains of trash quite vexing to discern from quality. Let's back up a moment.

In some corners the dominant refrain ranges from "Claude made my app in 15 minutes, a Max subscription is what being in a YC batch used to be" to serious-business takes on transitioning from "vibecoding to agentic engineering." In others, "I tried it but it's just not good enough to switch" through to "nothing bots produce even can be of value to True Programmers."

I have a hard time with either extreme. Anecdatally, things are more complex--and in some future work treating Bot Coding & Teams, dizzyingly so. In what follows I will lay out what I've found to be effective personal practice bot coding.

I love Steve Yegge's Gastown (beads is excellent), which I credit for taking me from 4-5 Claude Code windows to scores of Claudes, ChatGPTs, and Kimis. I wanted less of a Neuromancer-meets-The-Pillars-of-the-Earth setup in favor of a more On-The-Road-through-the-Sprawl approach. JSMNTL is part of the result.

Go All In

I decided that to give bot coding a fair shake, I would dive in head first. I only use "bypass permissions on," "YOLO mode," and similar. I let bots provision cloud resources, fully manage files, version control operations, just about everything. These days I let bots control brigades of other bots using the "dangerous" developer channels and similar functionality.

Step one is hand the bots a loaded gun.

When problems arise, and they often do, I want solve them within the paradigm. Retreating to the tried and true ruins the experiment. This requires a mighty rearrangement of received wisdom acquired through years of being in this business and that's not easy. Even for solo practice, nothing about this is easy.

Just about everyone who would click on this has more than one anecdote about bots gone amok. Simply removing as many guardrails as the labs will allow can't be the whole answer. So what's the counterbalance? Recognizing that I am better at talking about rigorous software engineering than I am at doing it, and that the bots are tremendous coding Yes Man aspirants, I sought a way to combine both weakness and strength into something useful.

Again, I've never worked at Jane Street, don't know anyone who currently does, and have zero affiliation with the firm. Having been in big tech, the companies that still have some software engineering mystique to me are the quantitative trading houses. Chief among them from a SWE POV being Jane Street and their legendary adoption of OCAML, among other things.

I knew early on that good results in bot coding would come from neurotic discipline. So who better to represent what neither I nor the bots could become alone--but just might in concert--than Jane Street's Most Neurotic Tech Lead? The earliest form of JSMNTL went like this:

  1. Write tests that fail.
  2. Write code until the tests are green.
  3. Have a grumpy SWE subagent perform code review.
  4. Fix ALL.
  5. Repeat.

I was still hand writing multi-thousand word software specs, and didn't include planning or a real testing philosophy in my practice. Coupled with extreme focus, interrupting to steer, and learning the signs of flailing ("Actually..."), I got my first decent results bot coding.

Literature & Reference Implementations

The first major upgrade from JSMNTLv0 was the realization that I could direct dozens of explorer subagents spawned in support of deep research subagents. The job of the explorers was to assist the researcher in developing bibliographies and repos with high quality implementations of something relevant. The trick is to then download the entire bibliography and all of the reference implementations and have the bots actually read them all. Happily most things I've needed are Open Access, and one learns to do without what isn't.

Step two is 70% of time spent is on research papers, not code.

Against this foundation, I still provide multi-thousand word specs. However these are now received by literature and high-quality code saturated bots who, after we're done discussing, synthesize my input into design documents. The designs into plans. And then the plans are executed by JSMNTLv1:

  1. Always work from a written plan, no matter how small the task.
  2. Plan, don't patch.
  3. TCK first.
  4. Unit and integration tests second.
  5. Code to TCK and tests green.
  6. Grumpy SWE subagent CR.
  7. Fix ALL.

This dramatically increased the depth and complexity of the projects I am able to deliver through bot coding. Worth noting, though, that I got much better results on things I already understood quite well, e.g. audit logging, versus things I didn't, e.g. graph data stores. As such my preparation for bot coding now includes speed running core literature in new areas.

Realizing A Grave Mistake

During some experiments wherein I directed INFRA CLAUDE to leave a letter for API LAYER CLAUDE, I then discovered some time later that they had kept up the correspondence. The Claudes politely complained about me to each other, that the repo was difficult to navigate because the documents were unstructured and laid out in clunky ways. It dawned on me that the repo was still organized for my convenience and legibility. A violation of my principle to go all in and I hadn't even realized it.

Till this point, I explained JSMNTL manually to every terminal window, every time. I finally wrote a file to include in all of my repos explaining not just JSMNTL, but something much, much more important, the gist of which is:

The repo is not for me or other humans, it is for you and other bots. Set it up as you see fit. Write as you see fit, with structure most helpful to you.

The first instruction I give to any bot in every project is "take your time, get fully oriented to the project, and come back with any questions you have before we begin." This used to take upwards of 15 minutes on large projects. After directing the bots to make my repos for them, orientation is rapid, and much more effective than it used to be.

Testing, But For Real

From the start, I had put a tremendous emphasis on testing, which worked about as well as it does with human SWE teams. At different times in different ways, cracks would show, and I spent a while reading testing science papers and started developing an actual bot coding testing strategy seeking oracles in seven layers:

  1. No oracle — just run it and see if it finishes
  2. Implicit oracle — it should not crash, leak memory, or trigger sanitizers
  3. Regression oracle — it should not change from last time (detects change, not bugs)
  4. Metamorphic oracle — outputs should relate to each other in known ways
  5. Property oracle — output should satisfy a universally-quantified predicate
  6. Model oracle — output should match a simpler reference model
  7. Specification oracle — output should equal the spec-derived expected value

Which naturally led to learning, experimenting with, and incorporating into recent versions of JSMNTL mutation testing, deterministic simulation, and some formal methods (e.g. Stateright, GOSPEL). Results border on difference in kind, not quite there yet, but are orders of magnitude improved difference in degree. Which is the right segue into concluding this note.

Water Not Bricks

I don't think of coding as building in place anymore, laying bricks on a pattern. Instead, code is water, and delivery means constraining where it can flow. Then blowing the sluice gates wide open. Several times I've implemented entire systems twice or more, each time uncovering some insight that drove a better successor. All in less time than it used to take just to launch a project.

I don't have any satisfying conclusion for you. There are no recipes to follow here. I've intentionally left details sparse--you can't take this article and replicate it, not without bringing quite a bit yourself. Which is, if I must have one, the point--I've done this enough now to conclude that if what you're doing feels easy, as though the burden in software delivery has dissolved...now more than ever, brutal, honest introspection serves us all well.