S02E05 · Agent Reliability: MEGA lesson
MEGA’s written lesson explains how guardrails, evaluations, and observability help manage the risks and variability of LLM agents. It covers deterministic checks, tracing context and tool use, and balancing agent flexibility with code-level control.
textMEGA · hq.mega.dev · Saved Nov 19, 2026
mega course, agent reliability, agent observability, evaluation design, agent permissions
Written lesson S02E05 of the MEGA.dev course, by MEGA. Agenda: Guardrails · Evals · Observability. Its lesson video is presented by Angie Jones and saved separately. (MEGA Student HQ, needs a login.)
S02E05 · Agent Reliability
Logic that involves LLMs can't be debugged the way we debug code. Even if all the variables stay the same, the system may behave differently, and we can't identify the root cause or make a change that fixes the issue the same way we do in code. This also means that whenever something in the system changes, its behavior may improve or degrade, and no one can tell why. But there's an answer to this.
The good news is that LLM behavior largely depends on the context, and most issues come from the logic that manages it. That logic can be debugged. The bad news is that to find out what is really wrong, we need to see the exact state of the system at a given moment - and this leads us to observability.
Observability alone isn't a solution, because some issues may come not from the surrounding logic but from the context itself: instructions, interactions, and any dynamic content being processed. Changes there can't simply be verified. Even if we can see that the problem is gone, there's a risk that the system's behavior has changed so much that it won't handle many other cases - and this leads us to evals.
And lastly, even if the system is observed and we have evals in place, there is still a non-zero chance that the agent will do something it shouldn't. To reduce that risk, we need mechanics that prevent the agent from taking inappropriate actions - and this leads us to guardrails.
Outline
- Guardrails: domain rules, input and output checks, tool-level restrictions, permissions, approval boundaries, blocking, redaction, prompt-injection risks
- Evals: datasets, task completion, tool use, boundary tests, deterministic checks, model-based scorers, human calibration, online and offline evaluation, versioning and regressions
- Observability: traces, aggregate metrics, cost, latency, failures, provenance, execution records, debugging, recovery, feedback-driven improvements

Guardrails: preventing the unpreventable
In S01E05, we discussed mechanisms for protecting against prompt injection. They're examples of guardrails: logic that aims to reduce behavior we don't want or increase the chance that we'll get what we want. In most cases, we want both. Also, wherever possible, it's worth looking for deterministic checks.
The most important part of designing guardrails is understanding that they're about either affecting the odds that something will or won't happen or deterministically blocking actions that shouldn't be taken. And even if this sounds obvious at this point, in practice it isn't, because as programmers we build things that have to be reliable. What's more, the business expects this from us. Realizing this shift will help us change our expectations, along with how we see the system and how the agents can be steered.
Let's say we have an agent that interacts with the user not only through text but also through generative UI. That UI needs to follow our design system without being restricted to just a few predefined components. So we need to let the agent shape the UI however it needs to, but at the same time prevent it from building logic that may be harmful to the user - like for example a component that submits private data to external API.

Now, the question is: how to achieve this?
The first idea could be to set a system prompt that tells the agent to follow patterns from the design system and to limit external interactions to the APIs the user mentioned. This will obviously fail, because the user may unknowingly paste context that tricks the agent into sending their data without their permission. In other words, this approach may work for our personal agent, but it would be irresponsible to deploy it in production.

Let's see what happens when we look at this problem from primitives perspective. Then, our agent could have the following capabilities:
- Template: components agent can create and reuse to create apps.
- Apps: interactive interfaces that can be built based on templates.
- Packages: scripts agent can write and reuse within the apps.
- Integrations: authorized connections agent can use within the packages.
Those primitives do not entirely prevent us from drifting away from the design system, or users from authorizing connections they don't understand, but the odds of that happening are much lower than if we relied on instructions alone.

Building systems this way is also more efficient because existing resources can be reused or predefined, so not everything has to be designed from scratch. In fact, the agent can also be limited to, for example, using only existing packages and integrations and steering only how they're presented to the user. This approach gives us a balance between agentic flexibility and the control that comes from code.

Guardrails can also rely on the model itself, or even on specialized models such as decision models (like Jev) or moderation models (like omni-moderation), which can verify user inputs and agent outputs to detect terms of service violations or various types of mistakes agent can make.
In production, moderation is critical as user inputs may get our entire account banned, and the risk of that is relatively high. We also want to know how users use our product and whether they stay within the scope it's designed for. Because of this, it's in our interest not only to have proper guardrails but also to know exactly which input came from which user.
Observability in agentic systems
S02E04 mentioned that "activity tracking" helps us understand the logic and behavior of multi-agent systems. In fact, observability is critical because it opens the door to evaluations, moderation and self-improvement.
As programmers we can start to think about observability from the logging perspective, as everything we know about managing logs in apps stays, but things we want to observe changes, or expands.
The difference comes from the fact that in classical apps, logs are about what happens within the code, but in agentic systems we want to see what happens outside our code: the processed context and its transformations. Additionally, observability should include metrics such as latency and cost, as well as metadata describing the entries.

One of the goals of good observability is the ability to travel back in time and see the exact state of the system, its context, and its settings. And the interface should let us interact with the agent to see how it would behave with new settings or a modified context.
In fact, it's a good idea to ask the model about its behavior. In theory, this shouldn't work, but in practice it does. AI can explain its reasoning and point out potential issues. While the explanation isn't proof of the cause, we can check it against recorded inputs, tool results, code, etc. Then we can generalize what it tells us and use that to update the prompts and logic, or even create new evals to observe the agent's behavior in this context.

In previous lessons, we discussed how agents can read and learn from the conversation history. The same can happen here, but the available context can be much richer thanks to metadata and additional product context (such as the user's profile, their subscription plan, etc.). This way, we can gain valuable insights not only into the logic but also into the product itself, including the user experience and suggestions for optimizing the business model.
But first and foremost, self-reflection can focus on optimization and issue detection. Its findings can then be turned into a plan and handed off to cloud agents, which will run experiments and implement improvements. This is important because:
- agents may fail silently due to hallucinations, using tools the wrong way, etc.
- agents may take a longer path, which affects the user experience and increases costs
- in some cases, a task can be handled by smaller, faster models
- token usage may increase without our knowledge
- quality metrics and user feedback may contain valuable insights
All of this, combined with an agentic system that can verify its own work and iterate through self-play, can become an invaluable source of context, especially if it comes from a production environment. We just need to be careful, though, not to include data we don't want to send to the model.
Now, the question is: how do you get observability right?
Since observability and evals are critical aspects of agentic logic, we already have many platforms on the market worth considering for production use, for example braintrust or langfuse (also available as open source). Both of them can be integrated through API, so our current tech stack is not an issue.
But in S02E04, I mentioned that tracing needs to include the entire context of the interaction, not just the events that happen at a given point in the code. To get this right, the whole system architecture should be built with observability in mind, and interactions with the model should be as centralized as possible to ensure consistency of the entire context.

When planning this, it's worth considering what information we want to have available, especially information that may be useful in the near future or help with product decisions on the horizon.
Then, together with AI we can map data from our product into events in observability platform, even in form of synthetic simulations. The goal here is to get the best possible structure that will let us search, filter and explore insights from both individual interactions and from many sessions - and this needs to be considered both from developer, business and agent perspectives.

And lastly, observability gives us a broad overview of the system and its most important metrics related to user activity, usage, costs, latency, prompt cache hits, overall reliability and more. Based on this, we can answer business questions like how much it would cost if we (...) or how many users use feature (...), or even deeper ones like which use cases of the app are the most critical for users.

Evaluation
Agentic systems can be tested, but not in exactly the same way as classic apps, because their behavior varies between runs and their outcomes often cannot be verified with code alone. This leads us to the concept of evals, which are repeatable tests that run a model or agent against a set of defined cases and score the outputs, either deterministically (if that's possible) or with a model (LLM-as-Judge). Evals let us measure quality and catch regressions.
Today we can generate tests, cases and datasets using LLMs, which can take data from the observability system into account to align the results even more closely with how users actually interact with our product. It's also feasible to generate the entire logic that runs evals, although it's even easier to rely on existing tools such as Promptfoo or on features that already exist in platforms like the previously mentioned Braintrust. When deciding, it's worth consider the fact that evals can be offline (local / development) and online (production monitoring) - below we see the difference between both.

Evals may focus on individual components, such as tools, retrieval, planning, grounding, handoff, error recovery or compaction. They may also verify the process as a whole, for example the tool-call sequence and its efficiency, or the system's end-to-end behavior, including its outcome.
Even though AI can help us with every single step needed to prepare, run, and analyze evals, the work is still hard and time-consuming. It simply takes lots of practice and iteration to find out whether the evals reflect reality well enough and give us useful results. And we need to observe those results across multiple runs to see how reliable they are. Since we're dealing with non-deterministic behavior, we measure reliability with the pass^k metric (success on every one of k trials) instead of just pass@1. Also, the entire process has to be properly optimized, because it takes time and costs money.
Building datasets, especially for multi-turn / multi-session interactions, also requires an extraordinary understanding of the product and its use cases, because even if we can observe real users, we still need to generalize their behaviors and simulate changing environments. Otherwise, we end up with too many cases that try to cover everything, or too few that cover too little.

It should be clear that evals can easily become a waste of time and money, and even harm the product. They may lie about the system's condition, because they can easily become biased, or too weak to signal potential issues. On the other hand, evals done well are an extremely valuable part of the system, especially when we're dealing with long-horizon tasks and a high level of autonomy. That's because even the smallest changes in the logic may result in issues that are really hard to solve, especially since they may only become visible after a while.
Now, the best thing to do is to create our own custom evals for anything that comes to mind, whether it's a skill, CLI/MCP tools, or a custom agent. We could even think about building our own benchmark that tests model or harness capabilities in specific areas. Whatever we decide, the steps are:
- Target: what we want to evaluate (a skill, tool, MCP server, agent, or multi-agent system) and which environment, model, and harness we want to consider.
- Dataset: everything that goes into the context, including the system prompt, skills, conversation history, tools, attachments, etc.
- Cases: a set of inputs and expected outputs, with optional constraints such as a maximum number of steps, budget, allowed tools, or forbidden behaviors.
- Score: how we verify the results and what determines success. This may include deterministic checks, model-graded checks, and quantitative metrics.
- Results: how we tell whether the verification succeeded or not. This may include a pass/fail status, score, reason, aggregate metrics, and a side-by-side comparison.

When getting started with evals, it's better to start small, even with 10-20 cases that are well thought out and general enough to cover more than predefined scenarios and tell us how the model behaves across the entire category of tasks.
It's also worth using deterministic graders wherever possible. First, we can ask the agent for a structured result and validate it against a schema. Then we can check whether the application state looks the way it should. In some cases, we can also use keyword matching, but only when we're clearly looking for something specific that must appear in the model's output. And if that's not possible or is too fragile, we can consider agentic evals that involve an LLM.
Agentic evals have to tell us not only whether the agent succeeded but also how reliable and efficient it was. And evals have to include the full trajectory (tool calls, reasoning/intermediate answers, final state), because outcome-only scoring often hides lots of problems.
Takeaways
Observability, guardrails and evals are crucial parts of agentic systems, especially those deployed to production and running at scale. But even if we keep working with existing harnesses, we may still want to evaluate our tooling (CLI, MCP and skills), even when it's used only within a team.
Most harnesses, such as Claude Code or Pi, can be easily integrated with platforms like Langfuse with minimal configuration. This alone lets us observe activity within a team (even anonymously). We can then use agents to analyze how people work with AI and find insights we can use to improve our workflows, especially if we compare the observability logs with PRs. The analysis can also reveal potential issues, such as exposed API keys, or give us ideas for skills, tools and MCPs we can build.