MEGA.dev live Q&A with Kent C. Dodds
Kent C. Dodds answers audience questions hosted by Wojtek, discussing architectural decision records, coding-agent context, software teams, and software factories.
videoKent C. Dodds · hq.mega.dev · Saved Nov 19, 2026
mega course, context engineering, software factories, engineering teams, spec-driven development
Transcript of the recorded MEGA.dev live Q&A with Kent C. Dodds (Wednesday, Nov 11 · 119 minutes), Kent C. Dodds. The recording plays in MEGA Student HQ (needs a login).
0:00 Kent C. Dodds: Welcome everybody. Let's make sure the chat works.
0:05 Wojtek: Ah, yes. Now I can see everyone is joining. There's always this weird delay.
0:11 Kent C. Dodds: Put where you're coming from in the chat. Let's just make sure chat works. Hey, hey. Happy Independence Day, Poland.
0:23 Wojtek: Chat definitely works. Yeah, we should have had some sort of, uh, festive, uh, uh, graphics ready.
0:34 Kent C. Dodds: Yeah, yeah. Here, here you go. Here's some festive...
0:38 Wojtek: Oh, yeah.
0:39 Kent C. Dodds: [laughs]
0:41 Wojtek: We were just talking, guys, that, uh, in Poland it's Independence Day today, and we were wondering how much of the Polish crowd will, uh, come to the live session. But it seems like you, you're showing up. Croissant Day in Poznań. Exactly, Marcin. Yes.
0:58 Kent C. Dodds: Is that how you celebrate, with the croissant?
1:01 Wojtek: You know what? In Poznań, it actually, it's a city where I'm from, uh, we have this special local type of croissant, uh, that it's made, um, from different batter and with white poppy seeds and with nuts and with honey and different spices. Uh, and it's our local delicacy, sort of, that is served only on this day, and you can get really addicted to it. It's delicious.
1:25 Kent C. Dodds: [laughs] Maybe it's, it's good that it's only served on Independence Day. [laughs]
1:29 Wojtek: [laughs] Yeah, but, uh, people here locally, that's our whole diet for, for a couple days.
1:34 Kent C. Dodds: [laughs] Nice.
1:37 Wojtek: Uh, guys, uh, there's more and more, uh, people coming. We'll be starting, uh, soon. So as you all are well aware, uh, today is, uh, uh, Author's Day, and Kent C. Dodds is our today's author. So say, say hi again, Kent.
1:54 Kent C. Dodds: Hey, everybody. Thank you so much for, uh, joining. I'm excited to answer the questions that you've got, and, um, love to hear your feedback on how things are and, and how you're enjoying this, uh, uh, the whole thing, uh, really. So yeah, I'm excited to go through the questions. I, I've looked through some of them, but some of them will be, um, uh, will be surprises for me. [laughs]
2:17 Wojtek: Well, that's why this, we're go- we're doing this live, uh, for the surprises. [laughs]
2:21 Kent C. Dodds: Yeah. [laughs]
2:21 Wojtek: Uh, it's a, it's a 90 minutes format, guys, so, uh, have, keep this in mind. But Kent, I heard you say that you are willing to stay up to the, uh, last question. So, uh, we'll see, uh, how many questions you guys have. And, uh, the forum on Discord is still open, so you can keep posting questions. We've listened to what you said last time on chat, so we have a special landing page so that we can see the questions which are answered, which are not, and they will be tagged automatically on Discord. So hopefully this time it will be more, um, clear and, uh, transparent for you all. And Kent, I guess we can, uh, uh, we can go.
So, uh, I'll mute myself and share the screen with the first question that got the most upvotes.
3:07 Kent C. Dodds: Sounds good. I, I forgot to mention, I'm obviously not in my regular office recording studio. I'm at a, a conference, um, right now. It's a AI Engineer Code Summit. Uh, I gave a, a workshop yesterday. So apologies that it's not the, the fancy thing that you've, uh, grown accustomed to in, uh, in the videos, but, uh, at least, like, the microphone still works. [laughs] Okay, yeah. Let's just jump into it. So how do you... 17 upvotes on this one. Great. How do you make sure ADRs, Architectural Decision Records, uh, principles, et cetera, are not bloating your context with every new thread? Oh, good question. Uh, the idea with ADR principles looks interesting, but are you sure only what is needed, uh, is loaded and you're not constantly paying for extra tokens?
So, um, the, um, the, the way that this actually works is the ADR sits there in the, the repository. Actually, first of all, I need to say, um, that I, I'm planning on gardening my ADRs. So I'm at over 100 ADRs now, and there are some that have been superseded, and, um, and it would probably be good to trim them down to the things that describe things as they are, um, and, and only, like, references to things as they used to be. Uh, and then we can kind of clean some things up. It is actually good to, uh, to include even the decisions that you decided or, or the things you decided not to do so that the agent doesn't say, "Oh, this would be a good idea.
Why don't we do this?" And then it just goes off and does that. But, uh, but yeah, the way that it typically works is the coding agent, um, does not actually open up, um, any irrelevant ADRs. Um, normally the way that this happens is I have a conversation with one agent, my Grok Bot, and that agent will look through ADRs, look through the principles, look through the primitives, and, um, and that will be context in our conversation as we're making decisions on how to implement a feature or, or whatever. And then, um, that agent will spawn a separate agent with references, um, necessary for that agent work. And so, um, so the context window is separate.
The, the implementer gets only the stuff that they actually need, um, and they don't have to worry about, uh, loading in the unnecessary ADRs. Um, and the results are good. Uh, so that has been working out for me. So hopefully that's, uh, uh, answers that question. Let, we can move on to the next one. And I've got the chat open here too, by the way. So if you've got like a quick follow-up or something, I'm, I'm watching that. Um, uh, yeah. So Conrad, Conrad actually asked a quick follow-up. Is documentation the key? Human agents, uh, also have limited context. Uh, yes, I would say that it's progressive disclosure. And so, uh, being able to quickly find what you're looking for without, um, having to be loaded with everything, uh, upfront, uh, is, um, is the important thing there.
Uh, and yes, uh, Marcin, the agents really do love ADRs. Um, at least the results have been really good [laughs] for me. So, uh, how should we restructure the workflow of a traditional 8 to 10 person software team? Okay, so team is focused on one domain of product and is operating with 500-- uh, a 500 plus tech people department in a fintech company. What fundamental processes do we need to establish first, and what tools actually bring tangible value to this new way of working? Should we move towards spec-specification-driven development? And if so, how should we restructure the team roles, responsibilities around this approach? Um, so I, I always want to make sure that, that there's sufficient context for the answer to my question.
So it's been a long time since I've worked, uh, at a company on a team. Uh, even when I was at Remix, it was a, uh, startup, small, small company. I was co-founder, and so, uh, it, uh, like, that doesn't really count, uh, for this. It's been, uh, since I was at PayPal that I was in this situation. Um, but I don't think that, uh, um, that my answer is meaningless. Otherwise, I would, uh, just tell you to skip this question. So I, I do think that I have some intuition on, uh, on this question or interesting things to say. So, um, things have changed drastically. I'm maintaining a code base of 1.5 million lines, uh, solo.
Um, and so, um, there's, uh, actually a lot to be said for what an individual can build. And when you start adding people to the mix, it, um, like, even with the first person, it starts, um, complicating things. It, it's worth it. I'm not saying, like, everybody should be solo. Definitely not. Um, uh, but it, it does add, uh, overhead because now you have context that's in more than just your brain. It's in everybody else's brain. Uh, and then that, it kind of explodes, um, further as you add more and more people to the team. Um, and so, uh, in the conversations that I've had with people at events like the one I'm at right now, um, it sounds like most people are coming around to the, the idea that your team should be smaller, uh, and that a smaller team can cover more of the, uh, product surface.
Um, and so if you have an 8 to 10 person software team, I would probably, uh, as far as restructuring, I would restructure the, the teams so that you can do more. Um, uh, that, uh, individual teams can, uh, can do more, and you need to have, uh, bigger ambitions. Um, uh, yeah, um, I think it's, uh, Mikhail is saying they broke their, uh, bigger teams and moved them to two to three engineering squads. I think that that is the right direction to go. Um, as far as, like, the, the specific workflows, um, I, I would say one of the most challenging things with adopting these, uh, the workflows that I'm showing you in, um, in my week, as well as those that you've seen so far and what you'll see from John next week, the biggest challenge is actually the people who don't want to adopt those workflows, um, because you, you probably have members of your team who are a lot more, um, reserved in their adoption.
And so, um, that is going to be, um, a challenge that you'll have to, uh, to deal with, have, um, hard conversations with people. But once everybody's on board, um, with, uh, automating the, uh, development life cycle, then it beco-- uh, it, it seems to me, based on conversations that I'm having, that you have one person who's kind of responsible for the software factory, um, maybe, maybe two, and then, um, everybody else, uh, is, um, taking the product work. Uh, and so one or two-- and, and honestly, this doesn't change, change from a, like, a whole lot from the past. When I was at PayPal years ago, um, I was, uh, web infra, um, and so I, I took care of the tooling and, and, like, this was, like, the webpack stuff and the ESLint stuff and, um, and testing infra and, like, all of that.
That was my job that I gave myself. Um, and, uh, and then everybody else did the product work. Uh, and I think that that is kind of what we need now, uh, especially at larger companies. Uh, there, there are so many companies-- I'm, I'm here at this conference, and, like, half of the companies I'm talking to are building software factories and trying to sell it to you. Um, in fact, even, like, in Kody, I'm building a package for managing a software factory. Like, you, you've seen, um, I referenced a couple times some packages that I have in Kody that kind of helped with, help with my software factory, and I'm just kind of putting them all together in one.
Um, and so, yeah, software factory is, like, really hot right now. But, uh, I was talking with the CEO of Warp yesterday, warp.dev. Uh, they, uh, they were the terminal, and now they're doing the software factory thing. And, um, he said that he's going after small and medium-sized businesses and, like, um, maybe, maybe some larger, uh, larger firms as well, but definitely not tech companies and not, uh, large enterprises, because all of them are going to build their own software factory. So if you're at a company, like you, you said, you're at a fintech company, 500 plus tech people department, you're probably in the realm of, like, you're going to build your own software factory.
And so you need, uh, a, a team, maybe a small team of people who are focused on building the software factory, which is what we're doing in, in this week, uh, largely. And then the, the rest of the teams are focused on product work. Um, so-- and then I would also recommend breaking those teams down, uh, a little bit. Um, let me make sure. Fundamental processes, um, that, that's kind of what I'm talking about this week. Um, the, like, it's going to be a lot of, uh, figuring out how to get it so agents can verify their work well, um, having good agentic review, um, and, like, um, we haven't gotten to this part yet, but, uh, limiting the blast radius.
So you've got, um, feature flags and disaster recovery, um, so that you can get that software factory to a point where people just send input- Um, and outcomes, the, the outcomes that you're looking for. Um, yeah, let's see. As far as spec-driven development, um, you can formalize that, uh, and this would probably be good, uh, on my one-person team, uh, solo. Uh, I literally just pull, like, put my AirPods in and I, um, have-- turn on voice mode with Grok Bot, and I talk to Grok Bot for an hour. That is my planning meetings. Um, and so I, I go on a walk, uh, and I just talk and I say, "What's our highest priority thing?"
Or, "I've been thinking about this feature. What do you think about it?" And it will go look at ADRs, it will look at the primitives, and it will, um, say, "Based on the way that things are now, this is what I think you're gonna wanna do," whatever. And, um, and so as, like, as far as spec, uh, spec-driven development, like, the agent is managing the spec, and you can have templates for that. And so the agent, um, kind of fills that out for you, asks you questions. Um, I don't use the grill-me skill or anything like that, um, because I, I don't, I just don't see it as necessary. You have, um, a template for the spec, and it just, um, goes through that.
And, uh, I don't look at the spec. Um, once the agent says that it looks good, and, and if it's like a big thing, I will have it review it with another agent, and they'll iterate together and then come back to me with, uh, um, decisions and recommendations and whatever. And, and then I'll just make sure that those decisions align with what I want. Uh, and then I say, "Great, go and do it," and it ships it to production for me. So you can do spec-driven development, um, but I see that as an implementation detail of the agents. Um, and so this, like, we're just constantly leveling up, um, to the next level of abstraction.
Um, hopefully that answers your question. Uh, I'm seeing a bunch of chats coming in here, uh, but let's, let's focus on the questions that, uh, people are asking. I'll, I'll, like, peek at some of these things, but, um... Yeah, okay. Uh, I love that you're chatting amongst yourselves too. This is, this is super. Um, somebody asked, this, this is relevant here. I'll, I'll jump into this one. Uh, so Nick asked, uh, what a software factory is. Um, so the software factory, um, is like this idea that, um, the implementation is handled by autonomously and with, like, little to no, uh, human in the loop necessary. Um, there are, uh, uh, I, I wouldn't actually say that, um, having no human in the loop is, is absolutely necessary to be considered a software factory.
In fact, like, we've been trying to build software factories for decades now. Um, it's just that the, the workers in the factory were human, and now, uh, the idea is that we elevate to get ourselves out of the factory. Um, and, um, and that, that is a difficult process, uh, that, uh, everybody's at a, a different level. But this week, um, I'm largely talking about how I build out my software factory, uh, with the three pillars of, um, making sure agents build the right thing in the first place with the ADRs and primitives and all of that, uh, verification and, uh, making sure that if, if an agent builds the wrong thing, that we catch that.
And then if it, if we don't catch it, that it, we limit the blast radius, and that's what, um, my week is all about. And so a software factory is, is effectively that. Um, uh, okay, great. Let's see here. Um, let's get to, yeah, back to this question. Is Scrum still relevant, um, in an agentic software development life cycle? Uh, so Scrums, sprint planning, story points, traditional backlogs were designed around human implementation capacity. Now all these [chuckles] fail to be useful, useless. Uh, what should replace that? Um, yeah, so like, I never really loved stand-ups myself, um, but I, I do love being made aware of, um, of where the, um, friction points are.
And I actually, what I, I find is interesting, as I, um, delegate more and more of the work that I used to do to agents, I find myself thinking, "Where are we with, like, progress on this thing?" Like, I, I know I decided this pro- I made this product decision yesterday, and I told it to build it. Is that deployed yet? Like, I haven't seen anything that says it's deployed. And so now I'm going to my agent asking, like, "What's the status?" And I'm like, "This is what my PM used to do." [laughs] The PM used to, like, come to me and bug me about the, what's the status on the work that, uh, he gave me.
And, um, it feels weird, um, and I kind of hate it a little bit. Um, but also, like, I, uh, I, I, I do feel like I've elevated to a, a new level of abstraction on the SDLC, and, uh, and it's also very, uh, very fun. So I've been thinking about, okay, well, let's, I guess we need dashboards and different things like that, and, um, uh, just so that I can get a, a very quick glance at, like, where things are, um, in the, the process. Um, which, of course, agents are going to, to manage, um, for me, and I can just get a, like, "Oh, this thing is stuck." Like, "Let me get that unstuck."
But even as I think about that, I'm thinking, "Well, why do I need to look at that? Why don't I just have another agent that's responsible for making sure that things get unstuck?" Uh, and so, uh, I'm still sort of figuring out what that looks like. But, um, uh, the, the way that I'm thinking about this is, um, we are replacing humans with agents in the context of the SDLC. And so why do humans need to have Scrum or sprint planning or story points or any of that? In fact, like, story points, who cares? Like, this, yes, this, this task is going to cost, um, two story points, which is about a half an hour.
[chuckles] And this task is gonna cost 20 story points, and that's, you know, or whatever, you know. Um, uh, like, it, it just feels so irrelevant. Um, but, uh, I do think that there is, um, uh, there is something about coordination, uh, that we don't want to necessarily lose. Um, that, uh, agents, like, you might have s- if especially you've got a team of people who are all kind of feeding in, into this software factory, there could be overlap on the things that they're working on, and so having some coordination there, um, feels right. Again, I'm not on a team, so I can't speak to that specifically, and you'll have to adapt this to your team environment.
Um, but, uh, yeah, like, some of these things, I agree, do not feel like, um, they're necessary with agents. Um, uh, so what, what should replace that? Uh, I think probably right now, I think some sort of visual into the software factory, like a window into the factory to show you, uh, the progre- progress of things, the cost of things, um, the, like, if the agent had to try and improvise, you want to know that the agent is improvising something that wasn't in the original plan and maybe stop it, uh, from auto-merging if it, if it did, um, so that, like, that can get reviewed and everything. Um, but really, in, in everything that, uh, you're doing, you just think, "What am I doing right now?
Can that be done by an agent?" And if it can, it probably can, um, then you try to get the agent to do it for you. Um, hopefully that answers the question. Let's look at the next one. Uh, no human code review on legacy enterprise codebase. Uh, what you shared in your lessons is probably a modern codebase starting from scratch not so long ago. Correct. My project is back in March, uh, is when I started. With legacy codebase, 20-plus years old. Uh, I've not even been in the industry for 20, uh, years, so there, that's, [laughs] that's a long time. [laughs] Um, uh, uh, every not-so-great decision in the code, uh, we have plenty, becomes an example for the agent to pick up, which produces slop in the PR.
Yep, yep, got that. Uh, how to, uh, come to a point where we can trust the agent to make a good change from first run and can eventually move away from code reviewing every change manually if we cannot start from scratch? Where to start? Great question. Um, so my intuition tells me that, um, everybody is going to be in a different place on, um, whether it's responsible to allow agents to, uh, to auto-merge and, and, like, do the full autopilot thing that I'm talking about in my week. Uh, and a lot of that is going to have to do with the, the way your codebase is set up now. And so this is why I made that, um, that autopilot CLI, um, was so that people can evaluate, uh, where their codebase is currently on that, uh, path.
Um, what maybe another, uh, and maybe one of you could do this. Another, um, useful skill or prompt or something that, um, that I could put together is, uh, one that maps the current, um, the, the current primitives that exist in the codebase already, um, and, uh, and maps where their documentation is, and it'll probably be a bit of a messy map. There will be a lot of overlapping things and, and things. This is the sort of thing that that, uh, team that's responsible for the software factory, um, should be focused on. And in fact, if I were at an organization, if, if it were my business, um, then I would, um, make a team that's responsible for greasing the wheels of our, uh, software yesterday.
Like, this, this would be a really high priority for me, um, so that we can move faster. Uh, this is not a new thing. Like, it, most, uh, mature codebases should have a set of gardeners who are in there just constantly making improvements to the way that, uh, the codebase is developed. Uh, and so, uh, like, this was the same thing with testing. Like, I, when I was way into testing, uh, and teaching people testing, they'd say, like, "Where do we start?" And I think, like, the answer is you, you start anywhere. Like, you just have to get people who are working on it and, and make it easier for, um, other developers to add tests to the new things that they're doing.
Um, and so, yeah, that, that's what I would do is get, um, somebody or a couple somebodies to, um, to, like, map out where the codebase is currently and, um, develop a plan for getting the codebase into, um, a place that where it's easier for agents to contribute. Uh, even if you decide, no, we're not going to go full autopilot. We're not going to let the agents just merge their stuff. Uh, I think that you're wrong. I, I think eventually, um, you will do that. Uh, everybody... Like, listen, you've got, uh, uh, we've got cars that drive themselves now. Um, like, I'm in San Francisco. I see them everywhere. Um, it, it's kind of a novelty, and it's really interesting.
Like, wow, that's so cool. But in, in five years, we're going to look back on this and be like, "I cannot believe that we ever let a human behind a steering wheel." Like, how insane is that? Like, you're driving past hundreds of people every day who you just hope they don't have a death wish or hope they're not being distracted. Like, and so it will be so, it, it will be so foreign to our grandkids that you would ever allow a human to control, uh, like, a bullet that, um, is 2,000 pounds or whatever. And I think that it will be the same for code in the future. Uh, you mean that, like, you let a human review that code?
But what if they find something and they think that it's a bad idea, and so they tell the agent to fix it, and it breaks? Like, the, that's, uh, it's going to completely change our mentality for this. Um, but it's going to take a lot to, to get there, and we're not there yet. Uh, the agents are not necessarily that good. I would say that the agents are probably better than most, um, most of us in this call, uh, at developing software. Um, but, uh, but with, as, as you say in this question, um, they are going to pick up, uh, old, um, patterns. uh, that are no longer recommended because the agents don't question the premise.
So they see this, um, like, "This is the way that things look now. Uh, I guess this is the way we do things here. I, I wouldn't do it this way, but this is the way it is, so I'm gonna keep doing it this way." And so this is why you need to be intentional about cleaning things up. And this is why also in my-- even in my code base, I've got legacy stuff. And so, um, I have a weekly, um, job that kicks off an agent called the Legacy Reaper, and it will just go through and try to delete as much as possible without changing, uh, behavior. And if it's not sure it can safely delete something, then it will instrument that code to measure it, uh, to see whether it's actually ever used, and then, um, it will, it will add a cleanup issue that says, uh, how to check whether, uh, it's safe to delete that code.
And then another agent, uh, every, uh, week will come through and check those cleanup issues and see whether it's safe to delete. And so, um, even in a code base that's not even a year old, um, I already have legacy, um, and, uh, and I'm, I'm thinking about, like, cleaning that stuff up. So, um, yeah, that's a whole bunch of words to, uh, answer that question. Hopefully that answers, um, the question. I, I, I want-- I don't want to be dismissive. Hopefully it didn't sound like I was being dismissive. Um, 20-year code bases, um, could have, like, ten- tens of millions of lines of code, uh, built by, um, uh, like, every skill level under the sun.
Uh, contractors who don't really care and, like, hired guns and whatever. Um, and so it is a massive problem. Um, but I think it's a, a, uh, a problem that we can deal with, uh, if you have enough money for the tokens. [chuckles] And, and, uh, the models are getting really, really good. Uh, and so being, um, uh, intentional about the decisions that you make there, uh, and the, the plans that you come up with to modernize the code base, I think will pay majorly, um, in, uh, like, long term. So, um, that's what I would invest in, is greasing the, the wheels of the, the code base. Uh, okay. Great. Uh, let's go, yeah, to the next one.
Uh, "Should one start with ADRs, primitives, et cetera, in a code base irrespective of scale?" Uh, yes, absolutely, 100%. Okay. "Do you recommend starting a new product-- project with all these configured in place, or is it better to reach a certain stage, uh, like a proper metric, like having a few users, and then actually add these primitives? What I find is that simple products and projects, adding primitives and code mappings and all would bloat the repo because it's already easy for the agent to find it." Yes. Okay. That's, that's a fair, um, caveat or, or, uh, nuance. So, um, agents are pretty good at finding what they need in the code base. So if you're talking about a 20,000-line code base, then, um, sure, you could probably get away with not worrying about primitives.
The agent can find what it needs. Um, uh, but we're like, we're, uh... And, and sorry, I don't, I don't mean to diminish, you know, um, code bases that are 20,000 lines, but we're talking about serious software that is like, uh, you know, at least 100,000 lines or, or many more. So, um, but, uh, yeah, for a very simple code base, um, yeah, maybe. So the, the benefit of the ADR, though, is, is more than just what's in the code base, but why that is in the code base and what is not in the code base and why. Uh, and so it's more than just, um, what the, what the agent can find. It is a historical record of how we got to where we are, so that when you say, "Hey, agent, I want you to add this feature," um, the agent doesn't, uh, decide to go in a direction that we've already decided in a previous conversation that we're not interested in.
Um, and so, uh, I would-- I, I don't know that primitives are necessarily... Uh, okay, let, let me just tell you what I would do. If I'm planning on a very simple code base, 20,000 lines or less, something like that, um, I would still use ADRs and primitives. Um, and I think that, um, it's because, um... Okay, the, like, the argument against it is because, like, oh, well, the agent can just find it, whatever it needs, uh, immediately, uh, anyway, and, uh, the primitives and ADRs just bloat everything. Um, the fact is that the agent, the implementer agent is not going to be the one actually diving into the ADRs. It's the planning agent, the, like, the one that you're discussing things with.
And for the planning agent, if you, um, tell it to, like, go and refresh its understanding of, of the way the code base is every time, that is actually going to take more tokens than just, like, the high-level thing that you've got in the primitives YAML file and the, uh, or, or however you structure that, um, and the ADRs. So it, it is more token efficient to have those kind of summaries of the architecture. Um, and then, so then the next, uh, argument against it is, uh, well, okay, but it gets out of date. Why does it get out of date? [chuckles] Fix that. Uh, like, that, that is friction, uh, and it, uh, it, it should not get out of date.
Um, and so with the friction logs, we're gonna talk about friction logs, uh, soon, but, um, with the friction logs, uh, agents are told, like, "Was there anything conflicting?" Like, "Did you get any, uh, documentation that was incorrect?" Um, "Did you get any documentation that was, um, like, overloaded context, so it was too much, more than you needed?" Uh, that sort of thing. So, uh, to an-- like, the short answer to the question is yes, I would absolutely use ADRs and primitives, um, in a code base irrespective of scale. Now, uh, you know what? Let me, let me just caveat that again. Uh, personal software, like a little dashboard I built for myself or whatever, no, I'm not doing that.
Um, but that's like a couple thousand lines probably, uh, and it's not like a long-term product that I'm trying to ship. But, but if I'm building a product that I care about long-term maintainability, I will start with ADRs and primitives. Okay, I'm seeing a whole bunch of questions here in the chat, um, but I, I-- Wojtek, is there any that I should really, um, look at before we move on to the next question? I just don't want to miss anything that's relevant to what we're talking about.
30:49 Wojtek: Yeah. Well, in general, I think we should prioritize the ones that were voted on, uh, Discord.
30:55 Kent C. Dodds: Okay.
30:55 Wojtek: So if you see anything that grabs your attention, sure, go ahead. Uh, just let's always move back to the Discord ones. And then we can-
31:03 Kent C. Dodds: Oh, we got a couple thumbs up and-
31:03 Wojtek: if we have time left, we can, we can come back to them.
31:08 Kent C. Dodds: We, we got a couple thumbs up here in the chat, and, uh, please take this one. So I, I will answer this one here really quick about in-house code review versus, uh, agents versus commercial solutions. Um, I th-- Oh, okay. I think that long term, um, AI code review will not be a thing. Um, I, I think that in, in a year from now, the agents will just get it right, and they won't need to be reviewed. That's, that's maybe a hot-- that's definitely a hot take. Uh, lots of you are probably rolling your eyes at that, but I, I believe that that's the case. Um, so, um, would, would I spend a lot of time, um, building up and fine-tuning the, um, like an in-house, uh, AI review process?
Probably not. Um, uh, I would instead jump on with one of the, uh, AI review tools, the commercial products, to tide me over until the agents are just that good. Um, yeah, CodeRabbit is probably not happy with me, uh, right now. I, uh, yeah. [laughs] Uh, I mean, I, I like them, and you'll notice, actually watch them, watch what they're doing. Every one of them is, um, is expanding beyond code review because they, they believe this as well. They're all building software factories. Um, and, uh, and it's because they, they know that code review is, is-- got a shelf life. So I would not build my own thing. And on top of that, um, code review is a hard problem.
Uh, it is, uh, you can't just say, "Review this." Uh, it, it is-- there's a lot more to it, um, that these agents are doing. And so I wouldn't, I wouldn't bother wasting my time. Now, if you're at a giant company, it's really hard to get the-- through vendoring and all of that stuff. Like, some review is better than no review, uh, and, and you could have some time to, um, improve that process. But, uh, but yeah, I think long term, we won't need to worry about it. Um, okay, sweet. Let's go to the next one. How do you keep ADRs, principles, et cetera, up to date? Oh, yeah. So this is what we're going to talk about with the friction log.
Uh, the, that, uh, I think that's tomorrow. Um, but, uh, the idea is that when the agent is finished doing its work, it is told to, um, uh, to, um, tell me what problems it had as it was going through things, like if there was any documentation that was out of date or whatever, uh, and it logs that as an issue, and then I've got an agent that goes and fixes that. We'll go deeper in this, uh, later. Um, but, uh, the other thing is that, um, the, the agent that does the planning with me helps me write those ADRs, and, um, and so, like, that-- and, and those are committed to the repo.
And so every day, I also have an agent that's responsible for keeping documentation up to date. And so it will look at changes in documentation and changes in implementation, and it will verify that those things are in line, um, uh, and that they are correct. And if they're not, then it will fix it. Um, so yeah, that's the, like, quick answer, how I do that. I, I didn't read the whole text, so let me make sure I'm not missing some nuance here. Uh, so far I was convinced that, uh, the code should be the source of truth and everything that can be picked up from the code should be picked up from the code.
Uh, from my experience, uh, docs on code, um, lists of files and primitives quickly become out of date and obsolete and confuse the model, uh, more than help. Have you noticed anything like this? What are your thoughts? Yeah, so, uh, one other thing that I'll add to this is that, um, um, if you are-- if, if you're operating in a, a code base that's like 100,000 lines long or, uh, less than, uh, having everything be in, uh, in code as the source of truth, maybe that's reasonable, um, for the agent to find what it needs. But, like, there are people in this call who-- I, I'm actually curious. Throw it in the chat, uh, if you have more than a million lines of code in your code base.
Uh, then say how many lines of code you do have in your code base. So I've got 1.5 million in my code base, uh, right now. Uh, and I'm curious. I, I expect we have at least one person who's over 10 million lines and maybe over 20. Uh, and so while you're tossing those in the chat, the, um, you, you cannot expect the agent to be able to find the right primitives, um, in a code base that's, uh, 20 million lines, uh, long. That just is not gonna happen. Um, yeah, we got 2+, probably 10 million+, yeah. Yep, I expected we had at least one. Um, so yeah, like, we're, we're dealing with some, um, significant-- like, I would say once you get to the, uh, point of at least half a million lines, even honestly a quarter of a million lines, uh, now you're just wasting a lot of, um, context, um, and you are running the risk of agents re-implementing something that already exists in the code base because it couldn't find it.
So, um, yeah, that's how I feel about that. Uh, and, and you can get things, uh, or keep things up to date. And if you're worried about it, then you can just have a weekly agent that goes through and just, uh, looks at all of the ADRs and, and the primitives and makes sure that they're still accurate. Um, like everything that you are-- this, like, you can just throw tokens at the problem. Um, it's, [laughs] it's kind of crazy. Um, uh, and it, it will-- like, they're really capable. So, okay, yeah, we can go on to the next. Yeah, we got, like, tons of people here with millions of lines. Um, all right. How do we track ADRs that-- Uh, and I would say also, not every product needs to be, um, like millions of lines long worth of code base.
Um, but if your code base is still, like, really relatively small, um, then I, like, I would, I would, um, do some introspection on your ambition. Um, you, you don't, you, you, um, products are almost better when you're good at saying no to things. So, like, this is a, a really important skill. You need to be able to say no to ideas. Um, but, uh, we've got, uh, a lot of really great things that we can do for our customers now, um, and, uh, and that's gonna necessitate more code than ever before, and you can be really ambitious now. Uh, so be ambitious. Uh, that doesn't necessarily need, mean that you need massive code base, but, um, but just think about it.
How do we track ADRs that only partly supersede earlier ones? Um, yeah, this is a good question. In a real product, ADR n+1 often changes only some points of ADR n, um, so both stay valid, uh, but parts of ADR n are stale, sometimes across many ADRs in between. How do we track that and keep up to date? So here's what, uh, so I haven't, I haven't run into this, uh, as a problem yet, um, but, uh, I said earlier that I'm going to start gardening my ADRs, and what that, um, what that means is, um, we, I, I effectively... Here's what I'm thinking I'm gonna do. So I haven't done this yet. Take that for what that's worth.
Um, but I'm going to tell an agent to go through all of the ADRs and intelligently split it up into, um, the, um, like, the, the current final state of all the decisions and put it into, like, a new set of ADRs, and then take the ones that we currently have, uh, and make a document that references those and the commit where they were deleted. Uh, and so if an agent really needs to go back in history and figure out, okay, how did we get to the state we are now, then they can. Um, but, uh, but we can kind of st- It's almost like, uh, squashing your migration history or something. I, I did that early on in the project too.
Um, you could say, okay, take whatever, like, the, here we've got all these mi- these database migrations. We're just gonna squash them into one, um, you know, one initial migration so that we can reset the clock to zero. Even in retrospect, I feel like that was kind of stupid and unnecessary, but I, I don't know. Like, you, you get, um, like, thousands of migrations and then setting up a new database, uh, can be a, a pain. And so, uh, occasionally having some sort of compaction or something, uh, can be useful. So that's kind of what I'm planning on doing with my ADRs, um, eventually. Um, and if you're having problems with, uh, ADRs that, like, um, that are conflicting with other ones, then that might be something.
Another thing that I do sometimes, um, is instead of superseding an ADR, I will actually update an ADR. Uh, I know that's kind of like heresy. That's kind of like not, not the way you're supposed to, uh, technically do it, but, like, we do have git history, and if you really need to see the history, you can. You could even update the ADR and say, "We made this, like, slight modification to this ADR. Uh, if you need to see what it was before, here's the commit SHA or whatever, uh, from what it was before." Um, uh, so yeah, if you're not ready to just, like, do, okay, well, we're making a slight change, let's do a whole, like, uh, rebase of these ADRs or something, um, then you can, you can kind of cheat a little bit.
Uh, okay, let me see. We got a couple, uh, a couple other questions. Um, yeah. So, uh, Abraham is, uh, asking about the software factories. Software factories were less of a term when I was planning these, uh, lessons, um, and it became kind of a turn, term as I was recording them. And so, um, what we're doing in these lessons is effectively building a software factory, so that's why, um, why I'm referencing it here and why you don't really see it in the lessons. Um, but yeah, that, like, it, everybody's doing this now, um, which is good for you because you, um, you're, like, right on the, the cusp of what everybody else is doing, uh, and, like, the direction that the industry is moving, so this is cool.
Um, let me make sure that... Uh, ADRs can have an amend as a function. There, there's no rule, uh, Holly, there's no rule about whether, um, like, how ADRs are supposed to function. Um, it's, uh, there's, uh, like, there's no linter for it. You could build a linter for it yourself. You can make your own template. You can do it however you want to. And so, yeah, you can have an amend. In fact, I think, um, I think that in a past project, I did have an amend, um, uh, like, status, um, in one of my ADRs. Um, okay, great. Let's go ahead and go on to the next one. Okay. Clear ways to improve code base quality to have better results with agents.
How do you improve and maintain code base quality to have more predictable results of your tests performed by your agents? Um, so let's say, uh, like, if I were, um, jumping into a legacy code base right now, and my, like, I, I get hired by a company to either be a consultant or, like, I, I'm starting day one on the job, um, I'm going to have a nice long chat with an agent and, and many chats, uh, with an agent about the code base before making any changes to it at all. And, um, I'm going to, uh, try to piece together how we arrived at things, and I'm going to, uh, like, assuming that it doesn't have any ADRs or any map of the features or map of the primitives or anything like that, and I'm going to build out the, um, those things as, uh, as the code base stands today.
Um, I would actually probably, you know, this is, this is an idea that just came to me now, so take it for what it's worth, but one thing that I think would be really useful is to have a section in documentation that describes the patterns that we are trying to get rid of. Um, and- Um, and I would discuss those with an agent and, and like, when can we get rid of these? Uh, we do have, uh, one thing that I talk about in my lessons here is the cleanup issues. So an agent making a PR will say, like, the, um, uh, the things that need to be cleaned up as a, uh, result of what we're doing.
Like, we're migrating from this old way of doing things, but we-- it has to be a migration over time because we've got user data and we've-- all of that. So, um, so we, we create a cleanup issue. Um, and so I would, uh, create cleanup issues. I would get those prioritized, and the agent can do all of this. Um, you just talk to it about those things, and the agent can create the issues for you. They can prioritize the issues for you. Um, you, uh, just dump your context into that. I would have-- uh, if I were just a new hire, I would come in and have lots of, um, meetings with the, um, like, leadership, uh, and make sure I understand what the product vision is and the, the goals and all of that too.
Um, and, uh, and yeah, just get myself very, very organized. And then once you have that done, then you just say, "Okay, great. Let's start going through these." And, um, um, and like, I would-- of course, you're manually reviewing things, but like, the, uh, until you develop that trust in the agent that you're, um, constantly trying to refactor. Like, let's make all of our, um, our implicit state machines that exist in the code base, let's make those explicit. And let's, um, now test those, um, and make sure that they match the, the outcomes that, um, are intentional. 'Cause there are a lot of non-intentional outcomes that are in your code base right now, um, that the-- and the agent can't tell the difference, um, between what's intentional and what's, uh, what's, uh, incidental.
Um, and maybe one day it will be able to, but right now you need to, um, uh, to discuss with the agent, like, uh, this is, this is possible. Is this, uh, something that we actually want? Is this, uh, the intent? And so that's why you need to have, um, discussions with the, the leadership and make sure that you understand what is intended. Um, it helps a lot if you have tests that communicate that intent, but if-- there are lots of legacy code bases that don't have tests. And so that would be another thing, um, that you could do is, like, you have conversations with product team about, like, "Okay, how are these features supposed to work?"
Um, and then you encode those, uh, intentions in the tests. Um, you also, like, tell the agent, "Here's what-- like, here's our transcript of our recording. This is how it's supposed to work." And the agent will be like, "Oh, well, here's a, a, a couple of thing-- here are some things that you may have missed that this feature is able to do or, or can do. Uh, do, do we actually want to do that?" And it, it will probably do a reasonable job of identifying things that are incidental and things that are intentional. Um, but yeah, then you bring that back to product and, uh, or you just know from your own understanding of how the product's supposed to work, and you say, "No, no, no, we don't necessarily want those."
You write the test for the things that you want, and then you make an explicit state machine for, um, the things that, uh, are intentional about how the product is supposed to work. And you just work through the entire code base that way, trimming away the, uh, old bad patterns. Um, that's how I would do it. So I haven't done it, uh, like in, in a, a legacy code base or anything like that. I, I always like to be very clear about, um, where my advice is coming from, um, but that's what I would do. Good question. Boy, I'm, I am slow at answering questions, y'all. Um, I, [laughs] I talk a lot, so, uh, hopefully it's useful.
Uh, let's go on to the next one. What time is it? Oh, goodness. Okay. Uh, what's the best model for, uh, PR reviews? Oh, gosh. Um, so I, I use, uh, third-party services. I do not know what models they're use-- uh, they're using. Um, uh, Mythos, uh, would be a good one for security. [laughs] So if you, if you have access to Mythos, um, then, uh, yeah, that'd be great. But, uh, yeah, honestly, um, I can't answer this question. I, I don't-- I, I have no idea, uh, what I would use, um, if I didn't have the third parties. I would have to experiment and play around with them. Um, yeah. Like, uh, if Theo says that OpenAI models are, uh, thorough, then that sounds good to me too.
Uh, let's go to the next one. How do you start a new project? Uh, I have a, uh, um, uh, a project starter for, uh, I, I used to have one called the Epic Stack. Uh, I mean, it still exists, but I don't really use that. And I made a new one called epicflare, uh, which is my starter for Cloudflare projects, and that's where I'm doing everything now. Uh, if I were to start a brand new project today, the first thing I would do is go to, over to epicflare, and I'd have an agent just revamp the whole thing, like upgrade all the dependencies, uh, get it on autopilot based on the work that I did in my last project, which is Kody, um, and, uh, and make sure that it's in, in a good state, and then it'd be like, "Awesome."
Now let's-- uh, I'll generate my new project based off of this, and I'll just, you know, off into the sunset. Uh, and that will include all of the, um, the strategies that you're learning in this week. Um, m-my design, testing strategies, and verification strategies is everything that you're learning this week, um, is exactly what, what it would be. Uh, great. Next one. How do you test the correctness and quality of the code? Um, do I look at the code anymore? No, I do not. How do you know it's implemented to expectations? Uh, because it, it has verification. Uh, so, uh, first of all, the agent understands my intent. We get aligned, uh, largely because, uh, the primitives that we have and the ADRs that we have, um, uh, built up over the, you know, the time that we've been working in the code base.
And, um, and then we have, uh, sometimes a really long conversation about, uh, a new feature, um, or if it's a simple thing, a very short one. Um, but in any case, uh, we are aligned on the intent, and then, um, we have the necessary verification to, uh, verify that the intent, um, or the outcome matches the intent. Um, and that's everything that, uh, I'm teaching you this week. Um- Uh, as far as, like, understanding each other, like, it's, it really feels like a conversation h- you're having with a coworker. And, um, like, you say something, and they send-- say something back, and you're like, "Whoa, whoa, whoa. No, that's not what I meant."
Or, like, you just know. You kind of can feel that. Um, and I, I feel that experience when I'm talking with the agents. Um, and then, like, they are keeping a planning document, and they, they have gaps and, and, uh, that we discuss. And once those gaps are filled, then I say go. Um, and, uh, uh, and if it's a really complicated thing, then I have it iterate with another agent just to make sure we're not, uh, missing something. And sometimes it'll find something, sometimes it won't. It'll be like, "Nope, this looks solid. Let's go." Um, uh, that its tests haven't missed anything, descriptions are reflective, accurate, uh, uh, complete. Um, uh, that it hasn't changed, um, more than something you considered.
Do you care about readability? How does it reflect ability to, uh, review? Uh, how big a change is acceptable, uh, for you to commit the change? Um, hmm. Um, so my, my priority is to get the outcomes that I'm looking for. Um, the, the way that we get there has become a lot less important, um, for me, um, as long as long term we're not putting-- painting ourself into a corner. So there are a couple of things that we do with that. Uh, one is you want to limit the number of one-way doors that you're going through and try to make them two-way doors if you can. Um, I believe we talked about this already in, in the week.
Um, but the idea, uh, is that a one-way door, door is something that's very difficult to, um, to back out of once it's been deployed. Um, and you can use a feature flag. I think we're going to talk about this tomorrow. Uh, use a feature flag to help you turn a one-way door into a two-way door. Um, and, uh, and there are other mechanisms that you can use, like long-term migrations and, and stuff like that too, and like backups, um, things like that. Um, the, uh, uh, another thing, um, to, to think about, if you were like back 10 years ago and you became, um, like a, um, architect or like principal engineer, or you go up to, uh, even technical fellow if you're like Microsoft level.
At that level, you're still very technical, and you're very-- you understand the architecture at a high level, but you do not under-- uh, you do not look at the code at all. Um, you, you are not in the weeds of implementation, um, and you rely on the, the people who are, uh, individual contributors, um, uh, to like focus on the implementation. Um, so it is not a new thing for a, uh, an engineering leader with like an engineering mind to, um, elevate above the actual implementation of the code. Um, think about it this way too. I can-- and I think I talk about this in, in the videos, but, um, it-- when I hop into my Tesla, um, I can tell it to take me to the airport, and it can take me all kinds of different ways.
And some of those ways I, um, I don't-- I wouldn't do. I wouldn't go that way, or I wouldn't change the lanes in this way, um, if I had been driving. And early on when I started getting used to that, um, I would be like, "I don't want to-- like, I don't like that. No, we-- let's, let's get around this guy," or whatever, and I would disengage self-driving, and I'd get around the, the guy and reengage it. But I found that, um, my life got better, and I got less stressed when I just decided to just let the car do what it was going to do. Um, and I still get to the airport, and it works like just fine, and now I can relax and, uh, like, uh, talk to my Grok Bot.
That's what I do now. Uh, or listen to a podcast or a book or whatever and, and not worry about, um, the implementation side of getting me to where I want to go. So, um, uh, I think that we do need to elevate above the code a little bit, and when it comes down to, um, whether, um, the, like, everything is exactly implemented the way that we want to be, um, uh, I think we got to stop caring about that so much. Um, I do care about maintainability though. Uh, I care a great deal about that because my agents, um, are, are like my little buddies, and I want them to have a good time working in my code base.
And so, um, uh, what I do with that is, um, I, I think of the agents as kind of like your pet, where you buy your pet like little toys that it's going to enjoy, um, because like you, you care about your pet's enjoyment of life or whatever. Um, and so this is like libraries that you encourage it to use or patterns that you want it to use or, um, or whatever. Uh, and so having, um, like still being involved in the, like understanding what, um, tools are available for agents, I think is still valuable. Um, I, like I, in fact, I just recently, um, had my agents fix up the, uh, implicit state machines in our code base, uh, to move those over to explicit state machines with XState, uh, because I think that XState is a good way to model that sort of thing, uh, based on my past experience.
Um, or even like if I didn't have past experience, based on like the smart people that I know that, um, that tell me that this is a good idea and that they've had good results. Um, I've been evaluating Effect. Uh, I know that like just me mentioning Effect, somebody in the chat is going to be like, "Effect rocks," uh, because, uh, you Effect people really love Effect. Um, but I've been evaluating that because smart people that I know, um, are, are saying that Effect is, is very good and, and helps their agents do everything, um, that, uh, like, or do things better. And now that I don't care about readability of the code or, or anything like that, like it's, it's irrelevant to me, then yeah, there it is.
I knew it. I, I knew you were there. Um, but, uh, yeah. Um, I, I could tell my agent, "Hey, you, why don't you bring in Effect here?" Or I could even just ask it, "Do you think that Effect would, um, have a positive effect on this code base?" Um, so that, like, that is, um, how far I care about the, um, uh, the readability. For me, it's how good is this for my agents to work in? Um, oh, there was-- Oh, yeah, the other thing is the friction log. Um, that is a really, really valuable piece. Like, did you have a hard time working in this code base? Why, and, like, how, how would you fix it?
And then having an agent go through and fix all those things. Um, that helps a lot too. So, um, as far as how big a change is acceptable for me to commit, um, I've committed some massive, massive changes. I migrated my own, um, personal site, which has, like, gets a lot of traffic, by the way, and it's not a small, uh... Well, let's see. I think it's like a 40,000-line code base, so it's not like a, a small portfolio site. It's like a, a, you know, it's a bit of software. Um, but, uh, yeah, I migrated it from a long-running server on fly.io to Cloudflare Workers, um, with, like, totally different, um, uh, image hosting and different, uh, pipeline and different, uh, I mean, it's a different runtime, um, and, like, a different database.
Like, so, uh, so many things were different, and I did that in one PR with Fable 5. Um, and so I'm not afraid to merge, uh, massive change. The-- What I am afraid to merge is something that hasn't been properly verified. And so that's what, uh, what you want to focus your time on is verification. Okay, I think that's enough. Let's go on to the next one. We're not gonna get through all of these. I- I'm confident. Uh, it's just I talk a lot. So make sure you're upvoting the ones that you really, really want me to talk about. Uh, what are your, uh, what are top three things in your AI development that have the biggest outcome, uh, impact on your outcomes?
Interested in learning what you find to be most helpful. Um, okay. It is, uh, the three pillars. [laughs] I'm cheating. Uh, I'll, I'll give you something more specific, but making sure agents build the right thing in the first place, making sure that if they build the wrong thing, you catch that, and making sure that if you didn't catch it, it's not bad. That's-- Those are the three pillars we're talking about this week. But the, the things that help that the most, uh, I'm focused most on, like, the, uh, limiting blast radius is literally just like disaster recovery and feature flags. Like, those are pretty much the biggest, uh, biggest two things. You, you set that up, and now we're good.
And it's, um, it's something that you do have to maintain long term and everything, but, but that's, that's kind of it. It's kind of a simple thing. The, um, the thing that I'm focused on most, though, is getting agents to build the right thing in the first place, and that is the ADRs and primitives. Um, those are definitely the biggest things. Um, if I were still re- um, needing to review things more closely... Oh, you know what? I- I'm gonna-- Here, I'll, I'll give you three. The ADRs, primitives, and, uh, friction log. If I could only do three things, that would stink. I, I don't wanna only do three things, 'cause I'd also-- Verification is also very important.
Um, so yeah, I- I'm gonna give you four. Uh, A- ADRs, um, primitives, uh, friction log, and verification, uh, through, like, a CLI, um, and, like, um, uh, like, the feature map is kind of useful, uh, that helps the agent find, um, the UI or whatever, the APIs, whatever it needs to hit to, to test out its stuff. But, like, the CLI helps, uh, a lot with that. So those are the things. Next question. Tips on improving my ability to do software design when AI writes most of the code. Ooh, that's a good one. How can I improve my software design skills when AI writes most of the code and handles the task m- almost end-to-end?
Recommendation for resources would be helpful too. Okay, I don't have recommendations for resources. Here's, here's my big secret. Uh, I do read books. Uh, I listen to books, actually. Um, but, uh, but most of my experience, um, I get from just, like, trying a lot of different things and seeing how that goes and talking to a lot of people. So I'm, I'm not much of a reader, so I can't... The, the reason I say that is I just, I don't have many good books to give you. Um, uh, hmm. I did have, uh, a-- Actually, there was an interesting one, um, that I did listen to that I, I thought was pretty fascinating. Let me see if I can find it really quick.
Um, while I'm looking for that, the other, um, uh, the agent... Okay. Right now, the, the skill that you need the most is your ability to communicate your intent and, um, your ability to design and understand a system. I cannot promise that those will be valuable skills in a year. Um, and just think about that for a second while I, I look, uh, look up this book. Um, so there is... I have so many books. Thinking in Systems, uh, is a great book. That, uh, I think that's the one. Yeah, it's Thinking in Systems is the one. Um, it is by, uh, Donella H. Meadows, and, um, it's, it's way bigger than software systems.
It's systems like... She talks about... I think it's a she, uh, Donella. I, I don't know. The author t- author talks about, um, the, uh, um, like, systems in biology and, like, just it's a broader systems, uh, thing, but it was just really fascinating, um, fascinating read. So that would be one that you could look at. And, um, there was another one. Oh, The Design of Everyday, uh, Things, also a really interesting... And, and that's, uh, more on the technical side, um, but also als- very general. Um, another really good one on, uh, designing good products in general. But anyway, so- Back to intent and systems design. Um, agents, um... The problem is that agents don't have the context that you have, and you're not good at giving it that context yet.
Uh, you, um, they're, they're not good at pulling that context out of you yet. And so this is why you have to have, uh, good systems design skills, because you need to tell the agent what to do in those, um, high, uh, leverage, high, like, high-impact areas. And so this is why systems design matters. Eventually, the agents will have-- Like, if the agents had all the same context that you do, they would design a better system than you will. Um, and, and so it, it's a matter of getting them the right context about the, the product and the vision and, and the future and, uh, and everything. And, um, the, the problem is that there is context that is in your mind about what this product is for and, and how you're thinking about the direction and everything that you don't quite articulate, like, that you can't quite articulate well yet.
And so you don't think to mention everything, uh, that would be relevant in the decisions, uh, for the system that you go for. So, for example, if you were to say, uh, "Hey, should-- Uh, I'm gonna build a new app. Should we go serverless or should we do a long-running server?" And the agent is going to ask, um, ask you questions to help determine what, uh, which one of those makes the most sense. And if you, um, fail to mention that you need to be running FFmpeg in the background or, or, like, uh, you're gonna be concatenating audio files together, uh, using FFmpeg, um, then the agent might say, "Well, serverless sounds like a good thing.
Like, you said that this needs to scale or whatever." And, like, in, in your mind, you just hadn't mentioned, uh, or, or thought about the fact that you need some sort of long-running service. Um, the agent, uh, not knowing that might make the wrong recommendation. But eventually, the agent is gonna get really good at asking you, um, the right questions. And it's not just the grill-me skill. I think the grill-me skill's days are, are numbered as well. I don't-- Like, th-this is a, um, a crutch. Um, and I don't use the grill-me skill myself either, because I think agents are actually already getting pretty good at this anyway. But, um, but the agent is going to be able to pull all of the necessary context from you, uh, asking you the right questions naturally, in, in a way that doesn't feel as taxing as, uh, going through, like, a giant grill-me session does.
Um, they're gonna just get really, really good at this. And so, um, when they are, uh, really good at, at pulling your intent, um, then they will just design, uh, the right system. And the, the question is, like, can you continuously feed it the, uh, the necessary context that it needs to make the adjustments as, uh, requirements change and all of that? Um, so, uh, I'm not saying, like, uh, you're asking me how do I improve my, uh, software design and, and systems design skills, and I-I'm not here to say don't, uh, 'cause I do think that right now, right now it's valuable. Um, and the way that I would do that is, um, by talking about systems with the agent.
Um, and you could even practice it on existing code bases. You'd be like, "Hey, uh, how do you suppose the DoorDash app is designed?" Or, "How, how is Bolt designed? How is, um, the, this banking app d- uh, designed from a systems level?" Or, "If we were to reimplement this, how would-- what would be a good design for the system here? Where are the good module boundaries? What are the pieces that we need to, uh, consider and everything?" And, um, and you can just practice talking about them. Uh, have it draw you diagrams and get comfortable with the language. Um, and then what, what's cool about that is once you're comfortable with the language, then you can use that language in future sessions when you're talking about your own system.
Uh, and you could say, "Oh, you know, like, how, um, how x.com does this?" Uh, like, "We, we talked about that and, and, like, their subscription model is kind of modeled after this. I want to do a similar thing in our, our thing." Like, referencing existing, um, prior art ha-has always been, uh, the way that, like, we, we do stuff, uh, in the software world, like referencing prior art. Um, and so I would, I would do that sort of thing with systems, um, thinking as well. Okay, great. Next question. "When agents write the, uh, code in minutes, what actually replaces traditional QA in your workflow? If we scale agent feature outcomes, manual verification breaks down instantly.
What's your experience with this problem? Um, do you see testers evolving into eval authors who write system boundary conditions, or do you rely entirely on auto-automated agent-agent verification? Which approach might be accepted by industry?" Uh, I asked this question as a CEO. I want a different perspective. Okay, yeah. Um, so I don't do manual verification. Like, I, I very rarely do any manual verification anymore. Um, I will manually verify, uh, the homepage, uh, if-- or, or anything that has, like, a, um, a unique user experience interaction sort of thing. And the way that I do that typically is the agent will build something and, um, that it's proud of, and then it will send me, send me the preview environment, and I can, um, play with it myself.
Um, uh, this is especially useful on mobile where you want, like, uh, on, on the kody.codes homepage, if you pull that up, there's a, um, a, uh, w-where is that? Um, like a little-- It's not a carousel. I guess it's kind of a carousel. Uh, but it just kind of goes through all of the, uh, I don't know if you can see this, but it's, like, going through these agents, and they're just kind of scrolling through. Uh, and I wanted to be able to flick it with my, uh, finger and just, like, you know, feel what that feels like. And so that's the sort of thing that I want to verify myself. It's a user interaction, uh, thing that, uh, I want to just make sure it feels good, um, in the way that, like, that interacts.
Um, uh, so that's the sort of thing that I would manually verify. Lots of things, if it's a visual thing- Actually, um, my best results come from, uh, telling the, the-- when I'm planning this with the agent, I'll just tell that planning agent to, um, uh, to make a mock-up of what the UI would look like, and then I iterate with that mock-up and I say, "Yeah, this is what I want it to look like." And then, um, from there, I'm confident that it will make it look like that, and so I don't worry about it. Um, but if I, if I really want to, the agents will take screenshots and then I can look at those.
They'll even take videos and I can watch those. Um, and so, like, as far as, um, the traditional manual verifications and QA and stuff like that, um, that, that has been automated. Um, and you, like, your AI reviewers can also, uh, there are, uh, some products that will a- actually, like, run the app and verify, uh, that things are operating according to the plan. Um, I don't think that the ones that I have set up actually do that. Um, but, uh, yeah, and I, I don't think that I ever will invest in that. But if that's important to you, then that's something you can do. Um, so, um, uh, seeing testers turning into eval authors, I don't think so.
Uh, no. Uh, I mean, like, if you're building an agent, um, then having evals seems like a good idea, but I, I don't build agents, so I can't, uh, advise you on that. This is why this isn't just the Kent Co- like, uh, MEGA.dev, but you're getting other instructors who have experience with that sort of thing. So I'll, I'll defer to them. But, uh, but yeah, um, I don't think that a, a manual QA tester is gonna turn to an eval person. Okay, next question. Uh, increa- increasing the trust in the work of your agents. How do you improve the verification system and guardrail to build more trust in your agent's work, uh, to even come to the point of merging automatically for certain types of merge requests?
Um, so this ha- the, the way that this happens is, um, uh, there are a couple of things. So first, that friction log helps a ton. Uh, like you say, "Hey, agent, good job on this one. W- what made this hard? Let's file an issue and then we'll, we'll grease the wheels and make it, uh, make things smoother for you." And, um, that process does wonders for improving, uh, verification systems, because the agent is like, "Well, I had to, like, click through the UI to create a new user, and then I had to, like, add a couple re- you know, uh, items to their to-do list so I could test this, and it would just be nice if I didn't have to do that.
That was kind of wasteful. It took me about five minutes to go through all that." Great. We'll add that to the CLI, and now the agent has like a, a 500 millisecond CLI command that it runs, and now it can just do that every time. Um, and so that, that's the verification piece. Um, now getting trust in the agent comes from, um, finding that, uh, you, you tell the agent, "Now hold off on this. Let me approve. Uh, let me," you know, whatever. And, um, you get to the point where you're just saying yes every time, like approved, approved, approved, approved. All right, great. Now you've gotten to this point where you trust the agent.
Like, every time they say that they're confident in something, you're confident in, uh, their ability, uh, for that thing. Uh, this is not different from gaining trust in humans. Uh, it's the same sort of thing. Like, um, over time, you just, uh, like, yep, you nailed it. You nail it every time. Um, and, uh, and eventually, like, you don't have to look at the code anymore. Good question. Next question. Make sure you're upvoting questions, by the way, um, so we make sure we, um, get to the ones that, that you most want. Uh, how do you use AI in your personal life and in your business? Uh, oh, man. Yeah, talk about, uh, use cases where your AI in your personal life and business, how your workflow works.
Uh, Grok Bot is like my primary, but I do also use, um, Claude and ChatGPT, um, and, uh, Devin sometimes for like personal life. But I, I use Devin for, as my implementer. Um, most recently, like, I've been using Devin a lot. Um, but, uh, yeah, uh, I'm only able to use all of these because I have Kody. Um, and this is why, uh, part of why I built Kody. But, uh, uh, I know that y'all really don't like it when I talk about Kody 'cause you think I'm selling something. I am actually selling Kody, so I, I appreciate that. But, uh, but like, the-- I'm just answering the question. Uh, I, I use Kody to manage all of that.
But, uh, yeah, so with, um, I, I use AI to, to do just like everything. Um, for example, I have a, uh, solar, uh, a solar roof on my house, and it's not producing what I was promised, uh, when we, um, paid to have it built. And so I've got an agent that is hounding the installer to get them out here to fix things and, and everything. So that happened literally this morning. And I've just got this, this repo that holds all the documentation and everything for it, um, and like, here's, here's our conversation, here's the transcript, here, like all of that stuff is sitting in that, uh, that repo. Um, and, uh, and so the agent can just keep on hounding them until eventually, like, they'll come over and fix things.
Um, I have it, uh, I have an MCP server in, um, on my NAS in my closet, and that's exposed to Kody. And so, um, I have every day, Kody will control when my shades go up in my home, um, because, like, if, if it's gonna be a hot day, then I want the east-facing shades to be down so that it doesn't heat up the house. But if it's gonna be a cold day, then I want them to be up so it does warm up the house. Uh, in the afternoon, the sun is gonna come shining through the west side and reflect off of the kitchen countertops and blind our eyes, and so I want those shades to come down in the afternoon, whether it's hot or not.
Uh, and like the southern-facing shades need to come down at a certain time, uh, but then they go up. But like the position of the sun changes throughout the year, so what time is that gonna be? Uh, it's gonna depend on every single day. And so like, uh, every morning, it wakes up and it schedules everything, um, and then, uh, uh, but, uh, but the cool thing about that is that it is all deterministic. So the agent just builds the, the software that determines all of that and, and plans that, schedules it, and then the agent goes away. And so this is actually another thing that I, I try to do, uh, deeply, is what are the things that I have handed off to my agents that the agents should hand off to software?
Um, and, uh, and so, like, you want to, uh, make deterministic software. My video next week, uh, shows how I made a massive improvement to one of, uh, Matt Pocock's, uh, skills, as just an example, um, by turning a bunch of it into deterministic software. Um, and so we should be doing that a lot more. Um, I, I also use-- Like, I, I've got a, um, a, uh, another, like, uh, I'm in a play, uh, and I've got a bot that's responsible for keeping up with the schedule. They post, like, our practice schedule, uh, every Sunday night. And so that bot will just, uh, go and look at the practice schedule and get it in my calendar.
Um, uh, like, I, I'm using agents for all kinds of-- Like, somebody, uh, buys one of my products, and so I've got an agent that will go off and, and look through my email and see if we've ever had a conversation. And if it makes sense, it will, um, draft a reply to that conversation. Like, "Oh, we were talking about Kody in this thread, so I'm gonna draft a reply to that," rather than make a brand new reply and, and make it look like it's not personal. And then, of course, I read it and I edit it, uh, to make it actually personal, and then I'll, I'll send that. If somebody cancels a subscription, then it will do the same thing and say, "Sorry to see you go," like, give me some feedback and, uh, and all of that, uh, as well.
Um, the, uh, like, I, I am using agents to do, uh, just a, a silly amount of things in, in both my business and, and personal life. Um, and I'm sharing a lot of that stuff on my YouTube channel, so, uh, subscribe. Okay, next one. Pretty much, like, you wanna-- What are the things that you're doing right now that an agent could do? Try to see if you can get the agent to do it. That's, like, um, that's the whole goal. Okay, 100% AI written but not AI native. Where do you start? Hello, I have products that evolve, uh, through several generations of AI tooling, some dating back to before AI. Today, agents write all the code, but these projects, uh, weren't designed for autonomous development.
It feels like a new kind of, um, technical debt. Agents keep shipping features, but I'm not always sure the code base is built around clear primitives and con- uh, consistent principles. Uh, I understand the approach taught in Mega, but after exploring Kody's agentic setup today, and I struggled to work out how to get there from an existing project. There are so many interconnected pieces, and it's hard to know what to introduce first. If you inherited a mature code base like this, what would your first three steps be to move it to rel- uh, toward reliable autonomous development while still shipping features? What needs to come first, and what can wait? Um, so, uh, I would run the autopilot, um, CLI.
Uh, so npx kcd-skills autopilot. Uh, so you don't, you don't run this. You tell your agent to run this, 'cause it'll get instructions, and then it will do an audit on your code base. Um, and, um, that, uh, that will tell you, like, what, what, uh, what are the areas of opportunity, um, especially for legacy code base. Like, we've kind of talked about this already, and I mentioned that this question was asked before we answered all, all those things, so I'm-- we'll, we'll get through this one quick. But, um, um, ADRs, uh, like, map out the, um, the way that things are currently, uh, the primitives, um, and, uh, and patterns that you don't want to keep around, like, that you want to clean up.
That's-- Those are the things that I would, um, start with. Uh, next question. Oh, I, I just realized I put my, uh, my message as a reply to somebody else's message. Sorry. [laughs] Uh, okay. Um, let's see. Um, in what areas do you think that AI should not be overused? Uh, where do you think AI could do more harm than good by promising short-term gains at the expense of long-term success? Um, I, I would say, um, making-- putting your code base on autopilot before you've developed trust that it's building the right thing, uh, and that it, like, has the ver- the, the tools that it needs to verify its work. Um, and the, uh, yeah, like, skipping the part of making ADRs and primitives, um, mapping and a, um, um, and the verification.
Like, if you skip that, then you're, you're gonna be in trouble. So, um, that's what I would say to that. Next question. Uh, okay. How can we optimize an AI-native, uh, software development life cycle? We're experimenting with AI-native SDLC, where, uh, product teams focus more on defining intent and specifications rather than implementation. Our current workflow looks like this. Product spec, we collaboratively shape, um, the initial intent into a product spec, uh, focused by the w- uh, focused on the why and what using grilling skills to shape the intent. Then implementation spec, we take the product spec, refine it against the existing code base, and create a more technical specification describing how the solution should be implemented.
Then we hand off the implementation spec to a coding agent, um, to execute. How would you optimize this workflow for AI-native, uh, engineering organization? In particular, is the separation between product and implementation a useful abstraction, or we, uh, unnecessary handoffs and overhead? How much technical decision-making should we delegate directly to agents versus explicitly defining up front? Great question. Um, this is a hard one, hard one to answer. The-- You need to develop trust, uh, in your agent first, and so you want to be very involved in, uh, looking at, at how the agent is going to implement it. Because, uh, um, especially as you're building out your ADRs and your, um, uh, your primitives, mappings, and all of that, um, the agent might not, um, know or, like, it might be harder to find alignment with you, uh, on that.
So, um, this is, like, kind of how you test how good your ADRs and, and primitives are, is, um, by if the agent, um, is, like, able to take a product spec and turn it into an implementation spec that it, um, references the same primitives that you would. Um, I wouldn't make those separate documents. Um, uh, yeah, I, I, uh, maybe in an organization, in a team setting, it makes sense, um, but like in part just because it's useful for you to, um, to see or to come into alignment with the product people. Um, so you have a meeting with product people, you've got an agent sitting on the desk there, like recording the meeting or whatever, and you have a conversation about what the, the product should be.
Um, you have the agent, um, asking questions as well, like trying-- It's just another teammate, um, that, um, can like think up other ideas. Like I, I was actually helping somebody, um, with a product that they were building, uh, the other day, like a couple months ago. And, uh, we just sat at my computer, and we chatted with the agent. So I would say, you know, like, "We're building this," and it would ask a question. I'd turn to the, the product person, and I would explain, like help-- It was actually a business owner. Uh, and I would explain to them what the technical jargon was that the agent was using, and, and we'd talk about it.
He'd answer like, "Well, we want this, that, the other," or like, "We can defer that," or whatever, and then I'd turn to the agent, and I'd give that to the agent, and we'd kind of go back and forth like that. So, uh, by the end of it though, um, like you, you could say that that was a product document, but it was also just the implementation spec, um, as well. Um, and so I don't, I don't know that I see clearly the, the value of having those be separate documents. The agent can manage it fine. Uh, like here's the product requirements, and here's the implementation. Like it's all one document. Um, but, uh, but yeah, like this is the way you want to be, um, thinking about it.
Um, uh, um, like typically, once I, once I feel like I've communicated what my goals for the product are, um, then I will ask the agent, "What are the gaps? What's left?" Like, and like I said, if it's, if it's an important thing, then I will ask it to, to reach out to another agent or a couple agents and have them look over the, um, the requirements and the implementation spec and see if they could find holes. And those same agents also have access to the code base and, and all the primitives and everything too. Um, and, and, and then I tell them, "Okay, iterate." Now, like at this point, I'm not, um, looking too closely even at the implementation spec, um, because like every single time I would look, I'd be like, "Yep, that's how I would do it."
So most of the time, the agent, um, will make the technical decisions. Uh, I'll say like, "Just go with your recommendations. What do you think? And, and, uh, is there anything that, um, is a gap in the product, like in how the product works, uh, that we need to discuss?" And we'll go through those. And then sometimes, uh, I will look at the, like the decisions that the agent said it made by itself, um, that it's like, "Well, it was pretty obvious that this..." In fact, I will often say like, "Anything that you're 80% confident or 90% confident, uh, is the right decision, just make those decisions and just ask me the questions about things that you're not 90% confident in."
Um, and then I will like peruse the things that it said, "Oh, yeah, I'm, I'm like positive this is the way we need to go." Um, and, uh, and yeah, it's right. Um, it, it nails it. So, um, th- this just takes some time of, uh, like do it-- Like I-- So to answer the question, I think that you are doing it well. Continue doing it this way until you find yourself, um, uh, and I would probably combine the product and implementation spec. I, I don't think that that's necessary to separate. Um, especially I think the implementer should understand the background on the product. So, um, uh, provided that it's not just some massive document, at least a summary of what the product goals are would be a good idea, I think.
Um, I, frankly, I'm not looking at the prompts, the-- I'm planning. Um, the-- So my planning agent, uh, is, uh, my Grok Bot is responsible for spawning out the, the sub- or the, the implementers. And so I don't look at those prompts ever. Um, so I'm not exactly sure what, uh, what it's doing. The results are good. The outcomes are, are what I, I'm looking for. But, um, yeah, so like develop that level of trust, like read the, the plans and everything, and, and once you're like, "Yep, every time it's right," then you can stop looking at them. So, cool. Uh, next question. Um, somebody is complaining that, uh, the, uh, question queue seem to be using out-dated vote counts.
So I, I want-- I do want to make sure, since we are limited on our time, I want to make sure that we are getting the questions that most people see valuable. Um, uh, so I'm going to look at this one from Sean that's getting two hands up and, and a thumbs up. Uh, what would be interesting to create an agent that pruned through all the question? Oh, yeah. Okay. Yes. [laughs] That's not a question. Very good. Uh, okay. Um, and I agree. I, I think that would be cool, uh, to kind of, um, uh, have an agent go through and, and, uh, because some of these questions have overlap and stuff too. Um, somebody's asking about how the ShipPR package works.
I actually have a video. I'll post, uh, a video, uh, about that, uh, to Discord later. Good question. Um, let's see. I just, I want to make sure that we're answering the right questions. So, uh, I'll, I'll keep going through this, assuming, Wojtek, that you're getting me the questions that people actually want me to answer.
1:26:06 Wojtek: Yeah. Uh, uh, if I may cut in for a second. Um, the upvotes that are added now during this session on Discord, they do not update, unfortunately. So if there was a question or a new question appeared, and during this session, it was upvoted, uh, you know, by a couple spots, then un- unfortunately, that is not seen here. But- New questions are still being added. So as we go on and you guys add questions, they, uh, they appear on our list, but we won't be able to go through all the list like, like you said, Kent. So I guess at one point you decide when, um, we, you know, we, we will stop the session and probably answer, you could answer the rest, the remaining questions, you know, on Discord directly.
1:26:57 Kent C. Dodds: Uh, yes, there are a lot of them. But yeah, I, I could probably get through, uh, those, um, at, uh, some point between, uh, talking with people. Uh, I'm, I'm scrolling through it right now just to, to make sure I'm checking all the ones that have a lot of, um, upvotes on them. And I think we actually got most of the ones that have a lot of, um, upvotes. So, um, yeah, I, and I'm scrolling through it quickly, but if, if there's just like, "Kent, you really need to see this one," then just put it in the chat and I'll, I'll keep an eye on that. Uh, okay. Uh, how should software engineers... Oh, actually, what time is it?
It's 11:27. Is that, um, we're at 90 minutes. Uh, how much time should we keep going, um, before we take-
1:27:39 Wojtek: Your decision.
1:27:41 Kent C. Dodds: My decision?
1:27:42 Wojtek: Probably, probably, probably not, uh, three or four hours, you know, but I guess you have another 15 minutes to spare, then go ahead.
1:27:50 Kent C. Dodds: Yeah. Holly's over here, "One more question." Kind of like the one more prompt. Come on, just one more prompt, bro. It'll get it this time. Uh, one more question. Okay, we'll, we'll do this one. How should software engineers develop product thinking? Um, here's how I did it. I started a podcast. Um, uh, that has helped a lot. Uh, so the, the principle is talk to people who have really good product thinking. Um, take notes on products that you use, um, uh, and like the things that you love, the things that you hate. Um, and, uh, um, yeah, honestly, just talk to... Like you, you gotta talk to the salespeople. You gotta talk to the marketing people, uh, the support people.
Like you, you should go to your support team and just be like, "Can I listen on the calls or can I read the transcripts?" Or like whatever. Um, that will help you so much with the specifics of your product. Um, uh, and then also there are some books. Uh, I think that, uh, Adam posted some books, uh, in the Discord. Uh, The Mom Test is good. Um, the, uh, um, uh, Jobs Theory is another good one you wanna look at. Um, oh, what, what's the, uh... Oh, like look into the Kano model, uh, for prioritizing things and, and RICE, uh, it's a, uh, it's risk impact, uh, I forget what C is. Yeah. Anyway, but, uh, yeah, that's another good model.
Um, but yeah, like software engineers, uh, it, we, it was always good, uh, like the best software engineers always had good product thinking. Um, but now it's even more critical because the thing that we used to offer in addition to that was, uh, implementation skills. And now, like, you're just not as good as the agents, uh, anymore. They're better at coding than you are. Um, and that is fine. Um, uh, okay. Uh, but yeah, like looking at really good products and, uh, and you don't have to know why it's good necessarily. You just know that it's good and take notes on it. And then, um, talk, like you could just talk to an agent and say, "Why is this good?"
[laughs] And, and, uh, have a con- like talking about it helps just a ton. Um, and so this is the Become an Epic Product Engineer podcast. Go listen to that. Um, The Design of Everyday Things is another really great book I mentioned already. Uh, go listen to that one. All right, we answered this one. Um, here's somebody, uh, "How do you deal with mental fatigue of working with AI agents? I love working with AI, uh, with agents, but handling several tasks within the same project leaves me drained after a few hours. Coding itself used to give me a mental break between decisions." Uh, this is, yes, uh, very much relatable. Um, and yes, I do have, um, problems with mental fatigue.
One thing that helps me a lot, um, going on walks and making, uh, like doing these, you know, if you want to call it grill, grill-me sessions. Like I said, I don't have a, a formal grill-me thing, but, uh, doing those sorts of sessions while you're walking and talking, uh, is refreshing. Uh, I, I love that. Um, so that, that helps. It's still like you're still thinking a lot. You're, uh, and sometimes you need to visualize things, so pull out your phone and tell it to give you a, a chart or something. Um, but, uh, yeah, that, that has an outsized, uh, impact on, on my mental health, uh, and mental fatigue. Um, I, I maybe don't suffer from this quite so much because I have also like the content that I'm creating for my YouTube channel and stuff.
And so I find myself wishing that I had more time to go through, uh, more grill-me session sort of things. Um, uh, more of those sorts of planning sessions. Um, the other thing is, um, when you're, when you spend a lot of time, um, designing the system and, and, uh, like, or rather I should say the implementation details, if, if, if your sessions talking with the agent are so focused on the implementation details, then that can get exhausting. But when you're, uh, able to focus more on the product decisions, um, and, um, like the, the trade-offs that you're making and, and that, for me, that's invigorating. Um, and so like everybody's gonna have a, like a different feeling on that, I suppose.
But, um, it, it, it used to be, uh, really fatiguing to have a bunch of context switching. Like I've got one agent, uh, that I'm working on over here and okay, we're gonna, I'm gonna read what it said and I'm gonna send a response and send it off. And then I go over here to this agent and now I gotta get back up into context of what it's talking about and then read what it says and send a response. And now I gotta go back to this one and I've got like four agents going on and it's like spinning plates. That was exhausting. I don't do that anymore because I just have the one agent that is responsible for those others.
And so I, I delegate that responsibility to that one agent. So it gets less tiring the more your code base becomes on autopilot. Uh, good question. Cool. Um, we'll take a-- Thank you for, for flagging that one for me. Um, okay. "How do agents review their work before the PR stage? Uh, what depth of review process does your agent or other agents on its behalf do on its own work before the code reaches a pull request stage? Because agents can almost always find something to flag. I find it difficult, uh, or arbitrary to figure out what standard, um, determines, uh, work is, uh, ready to be, uh, PR and avoid getting stuck in the endless cycle of review, fix, repeat."
Um, so I actually just added this yesterday or the day before. Um, uh, I added a, um, like a simplify step. So in the ship PR process, uh, in that video that I'm gonna, uh, share, like please remind me if I forget, uh, to, to put it in there. But, uh, in that ship PR process, the first step used to be, okay, now mark your PR as ready for review so the AI review goes and CI goes. But, um, but now I actually have a step before that, which is basically, um, you might not need to change anything, but look at everything that you've done, and if there's an opportunity to simplify stuff, then go ahead and, and simplify it.
Um, and then the next step is to, to mark the PR ready for review. And so, um, having the agent review its own work, it's, it's the same with humans. You don't want it to just be the only one to review its own work, because, like, it's got that context. It's already kind of decided this is the way that we're gonna do it and, and everything. Um, but like, like a human, as you go through and you're implementing something, you discover that like, "Oh, you know, that was the wrong direction. Let's go this way." But you leave some, uh, some code changes behind, um, that, uh, weren't necessarily necessary. Um, and you realize later that like, "Oh, I probably should have gone, uh, and done this instead."
And so, um, having a, a, like a one last check before it marks the PR as ready for review, um, uh, feels like a good idea, which is why I added it to the ship PR process. Um, but, uh, yeah, that's, that's pretty much it. Um, I, I haven't really felt a huge amount of pain on the-- once the agent gets its PR up. And like I said, I, I think that, uh, PR review is, is close to over. In fact, um, Amp, the coding agent, uh, that, that company, they do not do PRs. They do, um, trunk-based development. You, you push to main. And, um, of course, they've got CI before it deploys everything.
But, um, but yeah, I, I'm very intrigued by that idea, um, that you've got review that happens, um, before you commit and push. Um, I like using PRs because I have these third-party services, but I think that even my days are, are numbered on that. Uh, the Git history is enough to keep track of what's going on in the code base. I, I think eventually we'll just be pushing to main. Uh, okay, great. Next question. "How can we manage, um, harness from smartphone?" Oh, yeah. Uh, "Hello. Since our harness can, uh, run multiple agents at once on a VPS, we can, uh, we have to provide proper feedback to some of them. Um, when we are on a PC or laptop, we can connect, uh, there via herdr, any term- uh, other terminal, so we can see each terminal it's running.
But how can we manage them via s- uh, smartphone? I suppose, uh, there have to, uh, exist other tools for that, like TC- T3 code. Uh, is this the end goal, to combine all agents into a single thread conversation and talk only with a single agent on the smartphone, like Grok Bot? Or is there another pr- option? The harness is prepared, uh, so very good architecture, no gaps, so it works almost independently after initial step, uh, setup. But in case of necessity for an answer, I can, uh, wait a long time, uh, processing other tasks." Okay, um, uh, so I already-- Like, I've, I've actually been developing the, um, probably I would say the majority of the, um, stuff that I build, um, happens from my phone since the beginning of the year.
Like January, I got early access to Cloudflare, uh, or, or to, uh, Cursor Cloud Agents. And, um, and since then, I have just been Cloud Agent piled. In fact, I skipped over the work trees thing. I, I have not, uh, I never did like Git work trees. That was a whole big thing, and people are still doing, uh, developing like that. And, and it's a, it's a fine way to do it. Uh, but I went straight to Cloud Agents, um, because I have always, as an open source person, I've always felt like it was really important that you have your, uh, project in a state that anybody could just pick it up, spin it up really quickly, and, and get to work.
And so Cloud Agents fit really well with that, where it's a brand new fresh environment. You clone it, and boom, now you're off to the races. And of course, now, like, you can have the environment where it's already cloned and everything too. Um, but, uh, but the nice thing about that is you can have thousands of agents running if you have the money for it, and they're not gonna be stepping on each other's toes. And, um, when you have that sort of environment, then yeah, you can manage it from your phone. Uh, and you don't even need a Tailscale network because, like, it's running in, uh, in the cloud. Um, and so, like, uh, a lot of the workflows that, um, a lot of people are, uh, using, like herdr and stuff like that, I, I just have never gotten into, um, because I just use the cloud.
Now, it's expensive, uh, to be clear. Um, and I, I should say also, I, I like to caveat that with, uh, they give me, uh, credits in exchange for feedback, and so I'm, I'm not paying for that myself. Um, and so, yeah, that, um, that hopefully frames or gives enough context to, uh, why I've done things the way that I have. Um, so, um, managing it, uh, I, I do personally believe that the future is just talking to an agent, and that agent manages the other agents. Um, I have not felt the need to, uh, dive deep into my, uh, the agents that are actually implementing things for about, like- Maybe 70% of the time, I don't look at, uh, maybe 80% of the time, I don't look at their threads.
Maybe even 90. Like, mo- the vast majority of, uh, agents that are actually implementing stuff, I don't look at those threads. Um, I will occasionally jump into, uh, one of those, but, um, but yeah, um, I can manage it all from my computer, from my smartphone, but it's all through a Grok Bot. And you could do the same thing with Muse. You could do it, uh, Cursor projects are effectively this. Um, I was using, uh, Cursor like a Grok Bot before Grok Bot was a thing, um, where, like, you just have one, one thread that manages these other, um, other, uh, cloud agents. Um, yeah, and I, I think that's the way that it's gonna go.
Uh, next question.
1:39:41 Wojtek: Kent, uh, maybe you can answer the question that was posted on chat, uh, that Martina posted with MEGA.dev. How can juniors build engineering taste in the AI area? Era.
1:39:54 Kent C. Dodds: Yeah. Okay, there it is. Um, ooh, yeah. Uh, in the lesson, you mentioned proposing options the agent hadn't considered. My engineering taste came from years of coding and painful mistakes, um, before moving into management, where delegation became my normal way of working. AI now fits that style. How can juniors develop that taste if they delegate implementation from day one? Um, that's a really fair question. Um, I would say, first of all, um, that juniors... Uh, so I, I appreciate the framing of your question, because you're asking about taste. But let me first address the, the elephant in the room that a lot of people are worried that juniors are just gonna be, like, you're-- we're not gonna get new people in this industry because everything we would ask a junior to do is now done by agents.
So, um, juniors were never coders, um, and neither were we. We were never coders. We were problem solvers. And it just so happened that coding was how we solved those problems. But, uh, the way a junior becomes a senior isn't by coding, it's by pretending to be a senior and, like, doing the things that seniors do. They do it worse, and so we've got guardrails around all of that, permissions, whatever. But, uh, but over time, they develop the skills that seniors have by doing things that seniors do, just, um, and hopefully over time, they get better at it. Uh, and so that has not changed. Um, they are still doing what seniors do, but seniors are now, um, uh, building systems with agents.
And, um, and so, uh, juniors can develop in that skill as well. They just do it, uh, less well, and they need direction and guardrails and, uh, and review. And it's the same sort of thing with the agent. Like, you have them go do a thing, and you review what they've done, and, and you say, "Yes, this is good," or, "Here's your feedback." And, and then they, they iterate, iterate on it again, and, and hopefully they get better and better over time, where you're saying yes a lot more, uh, frequently. And then ultimately, you end up, um, never saying no. And, and that's when, like, they've, um, elevated their, uh, their position and everything.
So, as far as, uh, developing ta- engineering taste, um, I think, like, first of all, I think that they are-- that is going to happen. Like, they will, they will naturally get to, uh, uh, like, they'll develop that taste. Even though the, the agent is actually the one implementing, uh, the system and everything, they will develop the skills necessary to get the agent to, uh, to have the outcome that you're looking for, um, by virtue of, of iterating and getting the feedback and, and, like, um, uh, that loop in the same way that they developed coding skills. Uh, but then the other thing to consider is that, like, how long, um, how long before, um, having engineering taste doesn't matter?
Um, I went on a long walk with, uh, with my Grok Bot a few weeks ago, uh, asking about what the future of software is and just thinking about it. Like, I, I, I find that talking with agents, it's like a, um, uh, the rubber duck thing, except the rubber duck talks back, um, where it's just, um... So I would say, like, "What is, what is the future of software development? What is the future of our industry? Um, is there a future for our industry?" And the agent is like, "Well, of course there is." Like, you need, um, you know, you need to be able to communicate your intent, and you need to, um, to be able to understand the design systems.
And I'm like, "Yo, that's just in your training data. That's what everybody's saying. Why is that actually, like, a uniquely human thing?" And, um, eventually I got the agent to finally concede, not that it's being a sycophant, maybe it is, um, but, like, eventually it finally conceded that, no, like, models are gonna get good, and they already are pretty good. It's just a matter of getting them the right context, and they're gonna be able to get good at getting that context, and, and we'll-- the tools we'll build will get really good at getting them the necessary context to, um, uh, to be able to design the right system. So I said, "Well, what is actually a uniquely human thing?"
And, um, [sighs] uh, I'm actually not sure, uh, what's left for us long term. Um, right now, you need to be good at communicating intent. You need to be good at designing systems. Um, like, that-- these skills are valuable. I'm not telling you that you wasted your weeks, uh, with, with MEGA.dev, definitely not. Um, but long term, I just, um, I, I'm not confident. I, I, uh, I could not have predicted where we are now a year ago, um, so I, I have no confidence in my ability to predict what it will be a year from now. But my expectation is that we will be in a much different place a year from now, uh, and that it, like, it will continue to improve, um, to one degree or another.
And also, even if the models don't get any better, we will-- we have so much opportunity to take advantage of how good the models are now. And so, um, with that in mind, I decided that the, the most durable thing that, uh, that you can do for your career is, um, to be next to the door of opportunity as soon as it opens. Um, and that means that you have a deep- ... understanding of the problems that humans are facing, and you also are very engaged in, um, what's going on in the world as far as opportunities to, um, um, to solve those problems. And so, if you're, like, really into dog ownership, then you just understand everything that people struggle with when it comes to owning a dog.
And then you have a, a, um, you have your ear to the ground on the new advancements in agents, in models, and all of that. And because you ha- have such this deep understanding of the problems of people owning dogs, uh, when there's a new advancement, you instantly know, "Oh, this, this can apply to solve this problem," and you're right next to the door when it opens. That's, that's the, uh, um, what I think is maybe uniquely human. Now, how long that, uh, lasts, I don't know. The-- Once, once the agents can just automatically solve our problems, uh, for us, and they, they are al- always ready to apply the new opportunities to these problems, then we're in, like, Star Trek territory, hopefully, not Terminator.
Uh, like, I would prefer Star Trek. But, um, but, uh, like, and who-- We, we can't plan for that. But this is what I, I think, is you just have to have a really deep understanding of, um, of what the agents, uh, or what the opportunities are, what the problems people are facing are. Um, so that doesn't-- Like, I know that's a little tangential to, like, how to, uh, juniors build engineering taste, um, but I'm just not confident that this is a long-term problem. Um, it i- it is something that you need today, and the way that they do that is by practicing, uh, talking to, uh, to agents and, and other people about how systems are designed and everything.
That's how you do it. Um, but, um, and, and I don't think that you should say, "Well, Kent said it's not a long-term problem, so I'm not gonna bother." I think you should bother, because the, the fact is that, like, getting good at this gets you closer to the door of opportunity. Like, being deeply integrated into how, uh, systems work is how you, um, uh, are able to take advantage of the opportunities of technological advancements. So, it's not a waste of time, um, let's be clear. Um, but just don't think that you're safe because you understand systems design, 'cause I don't think that's, that's the case. I, I think that, um, that's something that agents will be able to do in the relatively near future.
Great question. Uh, any other question that I need to make sure that I, I get to?
1:47:26 Wojtek: Yeah, you can check in the chat.
1:47:29 Kent C. Dodds: Okay.
1:47:29 Wojtek: Another screenshot.
1:47:30 Kent C. Dodds: This is helpful. Thanks, Wojtek. Uh, or Wojciech. Um, so, uh, practical methods for maintaining an efficient and effective toolkit. Use Kody. [laughing] Oh, yeah, you, y'all are making it hard for me not to sell Kody to you. Uh, okay. So, um, Sean is saying, "As my library of MCP skill, MCPs skills, CLI tools, and other, uh, augmentations grows, uh, context usage climbs considerably. I've started developing a skill router to reduce the, uh, quantity of skills that get preloaded into context, and it helps." I love this question so much. [chuckles] "I've also started adapting many external skills, like, uh, potatoes and Pocock skill libraries, into my own custom skills to trim skill count. But there's a qualitative nature to this that makes it weirdly difficult to decide on a direction for how a router or gateway looks in a multi-computer, multi-provider system.
Speaking of gateways, MCPs are more difficult to manage, especially since many of them are plugging into external servers I have no ability to, uh, customize, and they're typically more context-hungry compared to skills, uh, descriptions, um, uh, to skill descriptions. I haven't tested any MCP gateways, uh, but would this be the next step?" All right. I'm, I'm looking. We have 160 participants, and I'm just waiting for the number of people who drop off, because I'm about to sell you on Kody. [chuckles] The-- Kody is the solution. Like, this, that-- I built Kody to solve this problem. And it's not just-- You are not the only ones experiencing this problem. Your entire-- All of your coworkers are-- No, wait, we're at 161 participants.
I wasn't expecting that number to go up when I started talking about Kody. Um, all, all of your teammates are also experiencing this problem. Um, and, and you want to be able to share those things, uh, between each other as well. So, um, so this is, yeah, this is not a unique problem to you. Um, so how does-- H- how do we, how do we solve this problem? First of all, let's talk about skills. Now, this is, this is general thing. This has nothing to do with Kody. Let's talk about skills. My video next week is about skills and how they're the, the worst. They are bad. Skills are bad. You should not want skills.
Try to avoid skills. What do you want instead? You want software. Um, so back in the day, we would say, "Okay, um, I am doing this manual thing over and over and over again. Um, I understand this problem space. Um, so, like, let me implement a software solution to automate that process so I don't have to worry about it anymore." Great. Okay. And so now we have software companies and, and we're building generalized solutions that can solve problems for people. Um, that is what you should be doing. You, like, you should be trying to move things to software. Now, um, and, and sometimes people will do it too quickly, and so their software will fail because they don't understand the problem properly or whatever.
But ultimately, like, software is a good thing. So the, um, what I find interesting about this is that we got agents in the mix. And so now, this handoff, like, I'm doing this thing manually, instead of handing it off to software to do it deterministically every time, we're now hand- handing it off to an agent. And the agent is like, "Great, I will do this, and I will do it manually every time. And I will never complain because I'm an agent and I love eating tokens. Like, I'm not gonna tell you-- I'm not gonna turn this into software." Now, that is gonna change eventually, and, and I, I believe this is, this is already happening.
Agents will turn that into software kind of behind the scenes. Like, this, this will happen. Unless you tell it not to, because when you give it a skill, you're not telling it to generate, um, uh, software. You're saying, "Do things this way." Like, "This is how I want it done." And then you're just like, "Okay, great. Not very efficient, but I'll do it this way." Uh, so you need to, uh, like explicitly find ways to get the agent to build it as software. That's what my video next week is about. It's, uh, Matt Pocock's triage skill, um, and I, um, I build it as deterministic software and it, like, it's way faster, it's way cheaper, it's, uh, more reliable.
And, and then you move it into Kody, and now it's more, uh, secure and, uh, shareable and everything too. So that's what-- Yeah, look forward to that video on Monday or Tuesday. Um, but, uh, yeah. So, um, turn- turning it into deterministic software is, is kind of how you manage that. Now, um, how-- The, the side of this is like, how do I-- Like, we've got the skill router and, like, the context is exploding. That's the part where it's all just Kody. Stick it all into Kody and, um, and the Kody is structured in such a way that the agent-- You think about it like a home. You tell an agent, "Hey, I want you to, um, to draft an email for me."
Okay, great. So the agent goes into your home and it looks around and it's like, "What do I have? Oh, sweet. Here's a secret that I can reference that is how, like, you communicate with Google. Great. Okay. And so I will use that tool and I will, um, draft the email for you using all the em dashes and everything." [laughs] And then, um, I, uh, and then, like, I'll put that in your drafts folder. Awesome. Um, but, like, what if, uh, like the-- You, you can ask the agent when it comes back out of the house and it's like, "I'm done. I, I did the tool call and everything looks great." Uh, you can say, "Was there anything that, like, could have been better?"
And the agent could say, "Well, I, I had the secret access, but it, uh, I had to figure out the API myself and it was kind of hard. Like, I, I kept on calling the API differently and, and eventually I figured it out. But yeah, that would be nice to, to not have to do that." Okay, great. So you can either tell the agent, "Okay, write down instructions and put it on, like, on the, the bulletin board so that the next agent knows how to do that." Or you can say, "How about you build a new tool?" Get-- Make a new tool, put it on the shelf, and so then the next time you have an agent go into the house, the-- uh, and you say, "Hey, draft me an email," the agent goes in and, and it searches around and it's like, "Oh, sweet.
There's a tool that's exactly what I need for this task. I'm just gonna use that tool. And look, it's got documentation. Oh, he doesn't want to use em dashes. Great, I won't use em dashes." Or, like, I ignored that and the tool itself, it, like, said, "Nope, no em dashes." Like, "I rejected your input. Try again." Um, but, uh, but yeah. So now it's, it's a tool that the, the agent can use in your house. And this house could just fill up with millions of tools and the, um, like infinite tools. And the agent that walks into the house, as long as the search process can surface the thing that it needs, then it doesn't matter how many tools are in the house because the agent can just ins- like, it uses the search capability to find exactly what it needs.
Um, and so this is how, how I have managed it. I've, um-- And you do this with code. So, uh, Kody works by telling the agent, uh, "Here's the search tool," and it uses the search tool to find what it needs. And what it finds is packages of code that it can, uh, that are like NPM packages. And then it writes TypeScript that, um, says, "Okay, import this package and import these, uh, like all these different pieces to, like, grab stuff from, uh, my, uh, kit mailing list and grab stuff from my Instagram and grab stuff from my Google Calendar and whatever, and put those things together in this block of code that, like, those inputs go into the, or those outputs go into those inputs," whatever.
It's, it's just code. It's the most composable thing in the world. And then it sends it off to, to Kody, and Kody will execute it, uh, for them, and then they get just the response. And so it's highly token efficient. Uh, it's deterministic code. The credentials are all, like, beautifully structured so that you can say, "Oh, I've got like six Gmails. Uh, I want you to use my personal one." It's like, "Great, I can use that package with this other, with this specific credential." Um, or like, "We've got a team service level account in our Linear, and I want you to use that one in, um, our, our team Linear account." Great. Okay. And that's, oh, that's the default for the package, so I don't have to map that at all.
It'll just happen, um, by use-- virtue of using the package. So all of the, those credentials are, are swappable and mappable, uh, for the task at hand. Uh, and the agent can just focus on, uh, on actually just building, um, the, the, the ad hoc piece of software that's needed for the specific task. And then it can say, "You're gonna do this a lot," right? Or, or you can say, "I'm gonna do this a lot." So put that in- into another package, and now the agent just uses that tool. So it's just composing tools together that it goes into the house and it, it can find the tools. You don't have to use Kody to do this.
Kody is the best way to do this, um, by a long shot. But, um, but you could act- absolutely just make yourself, uh, like a web of scripts. The biggest problem you're gonna have is credentials and, and how to make sure that, uh, that agents don't, um, have access to the credentials. They can just reference it and, and then sharing it with your team. That's gonna be another problem that you'll have. Um, but, uh, um, yeah. Like, um, a lot of people are diving deep into, well, memory. Okay, I'm gonna have skills, I'm gonna have memory, and, and this is just like more stuff that your agent has to think about upfront. And so now you're thinking about, okay, well, skills routers and, and memory search and everything.
What you really want is just for the agent to find what it needs at the time that it needs it. Um, and so this is where, like, uh, deterministic software, it really comes in, into play, uh, that is also documented with the progressively d- like, the stuff that it needs. Uh, and so, like, the, um, it's like, "Oh, I need to go into your toolbox for this particular thing. And look, there's some instructions on how to, uh, operate this machinery." Uh, and then it, it, uh, goes and, and does it that way. But it doesn't need to know that upfront. And so, um, having it be progressively disclosed is valuable. You don't want to put that in memory necessarily.
Um, the, yeah, there, there was something else I wanted... Oh, and sometimes you want a visual piece of that too, as a, a way to visualize that. And so, like, all of these packages can have a, a visual component that's like a web app sort of thing. Um, so yeah, there you go. I just sold you on Kody. I only lost six people. Brilliant. So, um, there you go. [laughs] You asked for it. [laughs]
1:57:43 Wojtek: Um, I think we went through all the questions that got any upvotes. The remaining questions have like one or max two upvotes. So I think it's best we leave the rest of the questions, uh, for Discord, if that's okay with you.
1:57:57 Kent C. Dodds: All right. Yeah. Hit me up on Discord if, if I didn't answer your question. Uh, I would like to, uh, to get through all of them. Uh, you all paid a lot of money for this, and so I want to make sure that you get your money's worth. So, uh, thank you so much. Uh, I hope that you're really enjoying this, uh, and, um, yeah, it's been just a, a pleasure to be a part of it. Um, yeah, thanks a lot.
1:58:18 Wojtek: Great. Thank you, guys. Have a good day or evening.
1:58:24 Kent C. Dodds: See ya.