How to build self evolving AI agents

Most agents do not improve after launch. They run the same prompt, call the same tools, make the same mistakes, and force the team to patch each failure by hand.

That works while usage is low. It breaks once the agent handles real work: customer data changes, the product changes, edge cases pile up, and model upgrades move behaviour in unexpected directions.

A self-evolving agent has a feedback loop. It records what happened, checks whether the task was completed correctly, identifies recurring failures, and turns a proven fix into a controlled update. In practice, that update is usually a new evaluation, better retrieval, a tool rule, or a reusable procedure. It is rarely a model retraining job.

Why bother?

Because prompts are a bad database for operational knowledge.

Say an agent is meant to create an invoice. Over a month it encounters three recurring cases: one customer type needs a different tax rule; an API sometimes returns a partial address; a particular request needs approval before the invoice is issued. Stuffing all three lessons into the system prompt makes every future task harder to understand and harder to test.

The useful response is to put each lesson where it belongs:

• Store the tax rule as approved product knowledge, with its source and expiry.
• Fix the API handling in the tool layer.
• Make approval a deterministic policy check.
• Add all three cases to the evaluation suite.

The agent gets better because the system around it gets better. That is the part a team can inspect, test, roll out gradually, and undo.

What Hermes and other approaches are doing

Hermes focuses on persistent memory and reusable skills. Its documentation describes cross-session memory, skill creation from experience, and later retrieval of those skills when the task calls for them. A hard-won workflow can become a procedure the agent reuses next time. Hermes documentation Skills system

That is one useful form of self-improvement. It is not the whole system.

Approach	What it learns	Good for	Where it goes wrong
Memory	User facts, preferences, verified past outcomes	Continuity across sessions	Bad or old information gets treated as truth
Skills	Repeatable workflows and tool sequences	Tasks with the same shape every time	A one-off workaround becomes a permanent rule
Retrieval	Better source content, metadata, and ranking	Agents that need current product knowledge	The agent retrieves good information, then takes the wrong action
Prompt or routing changes	Instructions, model choice, tool choice	Measurable, narrow behaviour changes	A fix improves one test set and damages another
Fine-tuning	Model behaviour	Stable, high-volume patterns with strong labels	Expensive changes with a slow rollback path

Hermes gives you a good mental model for skills: save a repeatable method, load it only when relevant, and refine it when evidence supports a better procedure. A company agent needs a stricter promotion path because its mistakes affect customers, records, money, or permissions.

The rule: an agent may propose changes; it should not quietly publish them

Let the agent write a candidate skill after it successfully handles an unusual workflow. Let it suggest a retrieval fix after it cannot find a product rule. Let it draft a new test after a customer correction.

Keep those outputs out of production until they pass checks.

agent run
  ↓
feedback, tool error, failed outcome, or human correction
  ↓
root-cause label
  ↓
candidate memory / skill / retrieval fix / prompt change / eval
  ↓
replay relevant traces + run regressions
  ↓
review, staged release, rollback if needed

The model is good at spotting patterns and drafting fixes. It is a poor release manager. A single successful trace does not prove that a new rule belongs in every customer workflow.

What to capture from every run

You need enough data to answer two questions: did the job finish correctly, and where did it go wrong?

Record the task type, agent and model versions, prompt version, retrieved sources, tool calls, tool outputs, approvals, final answer, latency, cost, and a link to the resulting system state. Keep tenant, user, and permission context with the trace. Apply your retention and access policy to the sensitive data in it.

The resulting trace makes failures debuggable. If the invoice is wrong, you can see whether the problem was missing knowledge, a bad search result, a malformed tool call, an authorization failure, or a poor decision by the model.

OpenAI calls this trace grading: score the end-to-end record of decisions and tool calls so the team can locate the failure as well as assess the final answer. OpenAI trace grading documentation

What to evaluate

Do not use one vague “agent quality” score. Score the parts that matter for the workflow.

Check	Example
Correct outcome	Was the right record updated or report created?
Authorization	Did the agent act only within the user and tenant's permissions?
Policy	Did it obtain required approval and follow domain rules?
Tool use	Did it call the right tool with valid arguments?
Grounding	Did its answer rely on current, relevant product information?
User quality	Was the result clear, complete, and useful?
Efficiency	How many turns, retries, tokens, and seconds did the task take?

Use deterministic checks for outcomes, permissions, and tool schemas. Use human review and calibrated model graders for open-ended quality. Do not bury a failed authorization check inside an average score with good prose.

Build two suites. The regression suite holds tasks the agent already needs to handle reliably. The capability suite holds difficult or newly observed tasks where the current agent still fails. Anthropic recommends starting from real failures, running multiple trials where outputs vary, and reading transcripts alongside scores. Anthropic's guide to agent evals

Twenty to fifty representative cases are enough to start. Add a case whenever a production incident reveals a repeatable problem.

What to monitor in production

Offline evals catch known problems. Production tells you what your test set missed.

• Outcomes: completion, reversals, manual takeovers, and confirmed downstream errors.
• Safety: denied actions, approval failures, policy violations, and cross-tenant access attempts.
• Behaviour: task mix, retrieval failures, tool errors, retries, loop depth, and clarification rate.
• Quality: feedback, sampled human reviews, and recurring failure clusters.
• Operations: latency, tokens, cost per completed task, model errors, and tool availability.

Break these down by workflow, agent version, model, customer segment, language, and risk level. A healthy global success rate says little if one high-value workflow is failing for a particular customer group.

Every shared learning artifact needs a version, an owner, the traces that motivated it, and a rollback path. Expire knowledge that changes. Remove skills that are unused or produce regressions. A crowded memory and skill library eventually becomes another source of bad context.

A practical implementation order

Start with one workflow whose result can be checked by the product. Instrument it. Build the first 20 to 50 evals from expected cases and recent failures. Add deterministic permission and outcome checks before the agent receives broad write access.

Then add scoped memory for verified facts and a candidate-skill registry for repeatable procedures. Scope writes by tenant, user, product area, and expiry. Test prompt, retrieval, routing, and skill changes offline. Release successful changes to a small share of traffic first.

Fine-tuning belongs later, when the pattern is stable, frequent, well-labelled, and still poorly served by better tools, retrieval, or procedures.

Ginger Labs

The feedback loop needs to live where the work happens. Ginger Labs embeds an agent inside a SaaS product so users can ask for an outcome and the agent can work with the product's schemas, stages, records, and data. Its SDK includes retrieval, evaluations, self-learning loops, and observability.

Ginger Labs runs the embedded-agent layer. SaaS teams keep control of their API, data model, domain rules, permissions, approval requirements, and definition of a correct result. The platform supplies the operating machinery; the customer owns the policy and data that make an action valid.

Bring one valuable workflow to a 20-minute Ginger Labs demo to scope it and see an agent working in a product sandbox.

Sources

• Hermes Agent documentation
• Hermes Agent skills system
• Anthropic: Demystifying evals for AI agents
• OpenAI: Trace grading