Skip to content
Linkdrop
Links

S04E01 · Mechanics Behind Autonomous Systems: MEGA lesson video by John Lindquist

John Lindquist explains how to build a software factory by centering work on one focused conversation and surrounding it with supporting agents. He introduces five principles for agentic tooling and demonstrates branching conversations with Herdr and Pi.

videoJohn Lindquist · hq.mega.dev · Saved Nov 19, 2026

mega course, software factories, agent orchestration, agent infrastructure, agent evaluation

Transcript of the lesson video for S04E01 of the MEGA.dev course, by John Lindquist. Agenda: Autonomy · Runtime · State · Control · Observation and Recovery. The video plays on the lesson page in MEGA Student HQ (needs a login).

John Lindquist (MEGA.dev) S04E01

What is a software factory?

00:00:02,080 --> 00:02:58,000 [John Lindquist]

The term software factory has a million different definitions. None of them are right, but none of them are wrong. So, what is a software factory? Let's frame it from your perspective. You've been using agents for a while, so you know this feeling. You open one conversation, you dump everything into it. The plan, the code, the questions, the fixes, and it works. But you end up with a big mess. And cleaning up is just too hard. The plan is up here, then the fix down here. Then a side question you just had to ask. Every tangent lands in the same context, so the task gets harder to follow, and the real job gets buried. So how do we get out of this mess? In our workspace, let's start with our main conversation as usual. We'll call this Main, because Main is your focus. It's where you work on your current task. Now let's trigger a shortcut to open a second supporting critic agent next to Main. Back in Main, we enter a prompt. Add a feature where guests must book at least a few days at the house. Our second supporting pane notices your prompt. And magically begins to criticize it. Two critical issues. One: please be more specific with days. Two: this might break existing bookings, how should we roll this out? You read this feedback and decide it's actionable, so you interrupt Main before it breaks anything. Now let's open a notes agent under the critic agent. While Main is busy, you have a place to dictate random thoughts and ideas. Just drop it to the notes agent. Notes is fully aware of the context of the main conversation, so you can just chat about what's on the screen without worrying about explaining anything. Main stays locked in on its task with a lean, focused context. This makes Main smarter and faster. This also eliminates your mental load that comes with a messy conversation. And a super bonus: the conversation is a high-quality history of your decisions related to that task, which makes it an absolute gem of an artifact. So here's how to think about it. Main is the backbone of your task. One idea, one conversation, whether it's an issue or some feature or an exploration. You talk with Main directly as you work. The critic and notes agents are your support. They run on cheap, fast models, feeding off the context of Main for all that extra stuff to help you stay locked in. That's our first big idea this week, and the heart of a software factory. You lock in on a single conversation and build your agentic workspace around it. A critic and a notes pane are just the start. So let's see how this idea evolves. A software factory replaces the frictions of a software development lifecycle with agents. You start on a software factory by noticing that friction and erasing it with agents. Every development team runs the same loop: design, develop, test, deploy, maintain, and back around. But every team struggles in different places. Those sore spots are exactly where your agents go.

The five pillars

00:02:59,380 --> 00:04:50,000 [John Lindquist]

This week we'll focus on the five pillars of agentic tooling. Pillar one, composable. You keep one conversation at the center and add, swap, and grow agents around it, piece by piece, into the workspace optimized for your product. The critic you just saw is just one simple example. Pillar two, intuitive. You build so many agents and tools across so many projects, there's no way you can remember what they all do. So each agent surfaces its own examples, hints, autocomplete for you and your agents just in time. This way you never feel lost because everything is intuitive. In fact, anyone could jump into your project and start contributing. Pillar three, proven. Agentic tooling is built against evals. You write down what you expect, then use a big, smart model to force a small, fast model to fit those expectations. Simply put, your expectations are the ceiling. The more thought and effort you put into what you expect, the more your agents can align with you. Pillar four, glanceable. Agentic tooling is known for producing huge walls of indecipherable text. But you don't have to live that way. Agents can show a grade, a status, a comparison, examples. They can even write a song for you, although I don't recommend it. But putting work into making the output glanceable helps you stay informed without having to ask what's going on. And pillar five, homegrown. Your factory is already buried in your own product. You find it naturally as you uncover and fill in the gaps. So it fits you and your team. It is yours. Or, as I like to put it, the real software factory is the agents we made along the way. You grow agents in your personal lab to solve your problems, then they evolve to help your team, and then finally your customers.

The tools: a workspace and a harness

00:04:50,480 --> 00:06:21,000 [John Lindquist]

Everything we're talking about runs in two tools, a workspace and a harness. For my examples, I'm using Herdr as the workspace, to host all of the agents, and Pi as the harness, running in each pane. I chose those two projects because they're open source, made to be customized by agents, and wildly popular in the community. This makes them a great combo for growing your custom team tooling. Pi's landing page says it best. There are many agent harnesses, but this one is yours. Because they're both open source, you can ask an agent to customize your workspace while you're using it, and it appears almost magically. Ideally, you can build your product the same way with the flexibility for features and fixes to appear like magic. There are many alternatives to Herdr and Pi, and you're welcome to use whatever you want. These tools change and improve every single day, and they're all converging on the same ideas anyway. So you can use Claude Code, Codex, or whatever you want as the harness because all harnesses I'm aware of support things like stop hooks and message passing. But I'd personally recommend something open source, built for agents, with a big enough community to find a ton of examples. All of those references will just make it so much easier for you and your agents to build out those features. And all those same concepts of customizing what you see in a pane from your harness carry over to what you see from an agent, to your phone, your watch, your glasses, and whatever devices the future holds.

Meet RentAPlace

00:06:21,580 --> 00:07:08,000 [John Lindquist]

And here's the example product we'll use as a playground for our agents. RentAPlace is a vacation rental app like Vrbo or Airbnb. Hosts put up places to stay and guests book their trips. We'll grow our factory around it, one agent at a time. You can clone the RentAPlace repo from here, have your agents follow the README to get it all set up. And just note that the RentAPlace repo already has real issues for agents to work against, so you can keep this as real world as possible. Everything I show you is based on real sessions and real research. I'm presenting it through generic windows and components and condensed sessions to avoid getting distracted by unimportant details and to help us move quickly throughout the week. I'll provide as much evidence and as many references as possible in the supplementary materials and in our group chats.

Branch off Main with /split

00:07:08,960 --> 00:10:17,000 [John Lindquist]

So main is locked in on your task with a lean, focused context. Then you have a bright idea, but it's completely off topic. If you drop that into main, you spoil the context and end up doing a weird mental juggling of two ideas in a single conversation. It's like trying to have a conversation with a human about two topics at the same time. It's disorienting for both of you. So, how do you get more agents involved who have the same context as the main conversation? And what if the idea came from something brought up a few messages back? Let's build a tool called /split. The goal of Split is to branch off your main conversation into a new pane. This requires that your harness is aware of your workspace, and vice versa. So when you type a split command into main, your harness can tell your workspace to create a new pane. Then your workspace can spin up a new harness with the context of the main conversation. Invoking split initially shows a UI in the main conversation that allows you to pick where you want to branch from, all the while the main conversation remains completely untouched. Then the harness sends the necessary information to the workspace. Herdr, like many other modern workspaces, has a CLI tool that agents can easily use called hc. Then hc can create the pane, invoke the harness CLI with the required session information, and spin up a session split off the main conversation. The harness does the conversation and the forking, the workspace does panes and processes. Now before you build this, clone the repos for your harness and your workspace onto your machine. For example, I use a dev folder in my home directory, and I clone Pi there, and I clone the Herdr app there. Then start an agent in the harness repo and describe the extension you want to build. You can type a prompt like I want to build an extension with a /split command that allows me to branch off the current conversation and pick which message to branch from, then open a new pane in my Herdr workspace that starts Pi from that branched session. Inject a message at the end of the conversation explaining to the agent that this conversation is split from another running conversation. So it should avoid acting on any previous messages and be aware that another conversation is probably making changes. So a quick pro tip: if this is your first time building an extension, ask an agent, Please help me get oriented around the process of building extensions. And spend a few minutes learning enough that you're generally aware of the correct terminology. Pro tip number two: make sure you have the GitHub CLI installed, and then ask your agent: use the gh CLI to find Pi extensions similar to what we're building. The references and examples you get back will be incredibly helpful for your agent to one-shot and even suggest ideas you didn't think of. Your agent will also naturally crawl your harness and workspace repos as well. So, thanks to the magic of open source, your agent won't just understand the externals and APIs, it will also understand the true capabilities and internals. Open source for the win. If at any point you run into issues, ask your agent to use the gh CLI to search the issues of the harness and workspace for similar reported issues. You'll usually find some hidden gems that will save you a ton of debugging time.

Composable: tools that explain themselves

00:10:17,760 --> 00:11:37,000 [John Lindquist]

Any good workspace and harness will support extensions, plugins, mods, or whatever word they want to use to describe composability. Remember, composable is our first pillar of agentic tooling. You don't want to be hacking the internals of projects. That just starts a tangled mess of fighting with updates. So choose projects that promote extensibility through agents. I personally found Herdr app to be a shining example of this concept, and it's been an absolute delight to customize. After you prompt to get your working version, make sure to tackle pillar number two: intuitive. Ask your agent to have the split command document itself. In Pi, after you type /split, it can automatically show examples and ghost text of how to use it. To start typing to reveal easy menus. As you fill out the arguments, it can continue to show you hints for the remaining arguments. Honestly, it feels magical. I strongly recommend spending time with your agent exploring intuitive self-documentation patterns for both these extensions as well as anything else you're building. Finally, ask your agent to make it so that --help displays an entire guide right in the main conversation. The help flag is something I recommend adopting for all of your commands, agents, and anything you build. It is a universal pattern for humans and agents who are lost and need more information.

Use /split for real

00:11:38,260 --> 00:13:10,000 [John Lindquist]

Now let's use /split for real. Once your agent has finished building the extension in Main, run /reload to reload all of your extensions, and then run /split and the conversation search opens up, you can search and filter to the message you want to fork from. Personally, I usually filter by user and assistant messages, and then I search from there to find the one I want. Once you press enter, a new pane automatically opens up in your workspace with your forked conversation. You can literally ask this conversation anything without disrupting the main conversation. I often use it to explore alternate solutions, such as comparing to other projects on GitHub, or researching an article I just read to see if any of the principles might apply. And you let Main stay locked in on the task at hand and mostly avoid doing any real work in the split conversation to avoid any conflicts. But I'll freely admit I am very guilty of slipping in a fix or two in a split conversation. Once I've got what I want from my split session, I often ask it to take a snapshot as a web page or create an artifact of what I'm talking about. I personally keep a .notes directory, which I globally git ignore for situations like this, or I use a service called here.now for hosting static sites. There are many fantastic alternatives to here.now, and it's fairly trivial to build your own for you or your team. But I've been content with the free tier, so I'm happy to give them a shout-out. So our new /split command introduces core pieces of building a supporting agent, but it is still missing two key pieces. A supporting agent needs the ability to observe the main agent and send messages back.

Build a lab

00:13:10,940 --> 00:15:26,000 [John Lindquist]

Unlike the /split command, which is just a parallel conversation, a supporting agent will have a specific role with a set of user expectations. But before we even start building those supporting agents, how do we know they'll actually be any good at their jobs? Time to talk about pillar number three, proven. We need to build a lab. The lab is your most important tool, and if you only take away one concept from this week, let it be the lab. We don't trust supporting agents until they've graduated from the lab. The key idea is that we put a supporting agent inside of a black box, where no one can see how they were made. We can only see how they respond to user or agent interaction with them. That way we can just have agents hammer them with different expectations, until the supporting agent meets all of those expectations. That's when you have proof that the supporting agent is capable of doing what you need it to do. So a black box means that an agent should never grade its own code. Instead, you ask a lab agent with zero knowledge of the supporting agent to just kind of pepper it with tasks and questions. So that you can surface its behaviors in a bunch of scenarios that you probably never even plan for. In our scenario of building harness extensions, our lab agent will type into prompts and press keyboard shortcuts and it'll read the screen, and use all of the available information through the workspace. And harness CLIs, to verify all these real world scenarios and user stories. That you want it to take care of. Then the lab agent reports back to you the results. Some results can come back in plain English like I tried X and I expected Y, but I got Z. Or they could come back in benchmarks, or other numbers that you might be trying to hit. Then as you read through these reports, you keep on adjusting the supporting agent's code and configuration until your expectations and evaluations align and the lab agent's report lets you know that everything was successful. As with any AI-based project, it's really useful to establish a glossary and use techniques like domain-driven design. Because there is no perfect method for all of this. And so much of it is based on the agents and you using the same definitions for the same words, the outcome relies on you being aligned with the agent as much as the agent being aligned with you.

Isolate, drive, report

00:15:26,440 --> 00:18:59,000 [John Lindquist]

Sometimes the most difficult part is just knowing what you really want. Because at the end of the day, who really knows what they want in life? Other than free, unlimited, ultra-fast Fable. That sounds nice. So your lab works in three steps. Step one, you isolate. You create a completely separate copy of your workspace. Depending on what you want, you can have it share your workspace and harness configurations, or have it use vanilla fresh configurations, if you plan on sharing it with the world. I'll provide the best instructions I can for setting up an isolated workspace for the Herdr app, but just be aware that this might be the trickiest step of getting set up depending on a variety of workspace factors and the environment you're working in. Critically, it just needs to be isolated enough to not ruin any of your configurations or projects. So using some type of sandbox is best. A lot of people use virtual machines, but I'm personally comfortable with opening a second instance, simply working in lab specific directories, because so much of the stuff I do is UI focused and I like to be able to hop in there and interrupt and interact and just get a better feel for it. And you can even configure your harness that your workspace is using to share your config directory so that it shares your provider authentication because it needs to access the models so that the supporting agents can actually run, and then if your supporting agent is specific to a project, you'll also want to copy your project. So your workspace can open the harness in the copy of your project and you can run true user scenarios from your project based on your goals. So after you've isolated it, step two is to start driving it. The lab uses the workspace's CLI, to interact with the workspace just like a user would. Then the lab can read all the visible text and anything the user would be able to see. Just a reminder, it cannot see the internals of the agent, so it's never influenced in any way by the actual implementation. Many workspaces even enable you to take screenshots and videos. Or you can set up separate tooling for that. And those make wonderful artifacts for when you let a lab run overnight on a bunch of scenarios. You can wake up and visually inspect the behavior, to see if that video matches the experience that you're looking for. And there's just something about watching a video when you wake up in the morning. That allows you to feel how the tool is being used. That shows you so much more than text ever can. And also, we're entering an era of models that are really, really good at watching videos and analyzing what comes out of the videos, such as the latest Gemini models. You can set up these loops to feed the resulting video into those models, have it critique the video itself, like the actual user experience. From the video, and then get these amazing results, using these video driven hill climbing techniques. Essentially, the conclusions and the things that really matter come from that real user experience or even the agentic experience, not from the implementation. Then step three after you isolate and you drive is to report and clean up. The lab provides you its report on expectations versus reality and then stops. Stopping can shut down the lab window and anything it was running, remove any of the scratch files or directories. And again, your main workspace window is never touched. Then as a reminder. Please lean into the intuitive concept. Make sure in all of your expectations that your lab agent is looking for helpful examples, guides, and hints. If the lab agent gets lost, then you'll definitely get lost. So always, always, always, whenever building an agent, have it provide the guides and examples and hints, whether it be through the user interface that the agents are using or through an MCP or just whatever. Method you're using to access your agents.

One command: /lab

00:18:59,540 --> 00:20:12,000 [John Lindquist]

So with our lab and our lab extension built out, we can now type /lab and it automatically shows examples and ghost text and what comes next. And then /lab with --help should print out an entire guide for you. Then also if everything ran correctly you'll have a /lab doctor. Which should give you all the information you need to help walk through any issues you might be facing with your lab workspace and lab harness. Let's just run a simple test against the lab just to make sure the simple stuff works, and verify the lab can open a new pane in its workspace, and in that pane run like an ls command, take a screenshot and close the pane. If you got that far, you're ready and have the foundational pieces and you can start testing your agents. Now let's have the lab test the split extension just to make sure that extensions can carry over. So use your lab to run the scenario where you ask it to run a slash split. And watch what actually happens. Does the picker open? Does a new pane show up? And simply verifying that all of your pieces are working together. That's the core loop we'll use for everything we build. You ask for a tool, the agent builds it, you run the tool through the lab. We'll build out our agents inside of a lab and once the reports read that they're ready. Start using it in our own workspace. If something misses, we continue iterating inside of the lab.

Ask for /agent: the critic

00:20:13,240 --> 00:22:55,000 [John Lindquist]

So now let's move on to building supporting agents. Remember the two missing pieces. We need to observe the main conversation and be able to send messages back. So we're gonna ask our agent to build a /agent command. It takes a name and a job and opens a pane beside main and watches main's conversation. And always remember to please make it intuitive. We're also going to allow our agent to have its own model picker. For my examples, I'll use Haiku. It's a nice, fast, cheap default. Even though our lab built the supporting agent against a specific model, it's fully understandable that with the pace of progress and how quickly models release, that you'll be anxious to switch between them. And that the tool you built isn't mission critical, so you're not concerned if the expectations slip just a little bit. So, where do those expectations live? Each agent gets an expectations file with three headings. Do, expect, and never. The Do section is the exact steps the lab agent should take. When testing it. The expect section is what must show up on the screen and the never section is the things that should never happen. And that file sets the baseline for the lab to know what you want. So now the lab command, if you followed our references, should have two arguments, new and test. Slash lab new will interview you and help you write the expectations that will grow your agent in the lab. And /lab test will run the agent against its expectations again. So you can run it every time you change those expectations. So, time to grow a real supporting agent, the critic agent. And we'll write its expectations first. I want the critic agent checking three things on every prompt. One, does the prompt give enough to finish the entire task? Two, does the user understand the blast radius of their prompt? And three, should this prompt be broken down into smaller steps? In your main conversation, you'll run /lab new critic. This will ask you a few questions, each with a suggested answer. Then it writes the expectations and waits for your okay. Once you approve, it writes the agent and runs it in the lab. A few arguments worth knowing. --agent-model picks the model the agent runs on. And it should default to a small fast model available to you. --tester-model picks the judge. And --behind opens the lab behind your window for long runs. Then you wait for a while and you get the report. Two jobs pass, but the blast radius fails because the critic stated the blast radius as fact without checking. That's exactly what the lab is for, and you never spot that by reading the agent's prompt. So you change one thing: the agent's job, not the expectations. Never loosen your expectations just to make an agent pass. And then we can run /lab test critic again, and this time the critic agent passes every run, all three jobs, blast radius included.

Proven: bring the critic home

00:22:55,140 --> 00:24:19,120 [John Lindquist]

And that's pillar three, proven. The lab is the embodiment of proven. A big smart model forced a small, fast model to fit what you wrote down. So your expectations are the ceiling, and the lab is where you prove an agent meets them. Time to bring the critic agent home. In your daily window. We'll install the critic, reload the session, type /agent critic, and it opens beside your main pane. Then in Main, we're in our RentAPlace project, so you can just type a prompt like: add a feature where guests must book at least a few days at the house, hit send, then the critic should already be watching. The critic answers in its own pane with two critical issues. One, please be more specific with days, and two, this might break existing bookings, how should we roll this out? This isn't some huge wall of text you have to wade through. It's a place in your workspace that you know will be critiquing a prompt. And that's pillar four, glanceable. Just one look tells you the information that you're looking for. When a critic's finding is worth keeping, you don't even have to copy and paste it. In the critic's pane, you can hit the up arrow, hit enter on one item, and it can send that back into the main conversation with a note. If you notice the critic call out something dangerous, like breaking bookings, go ahead and interrupt Main yourself before it breaks anything. You can also let the critic agent interrupt Main on its own when an issue is critical. Add that to its job and its expectations and prove it in the lab. But be careful, and the interruption can disrupt a long task you left running.