How the Insights agent designs itself

By Samir Shukri, Founder · 4 min read

Most AI customer-service products are static. You write rules. The bot follows them. Visitors ask questions the rules don't cover, the bot deflects, and the gaps quietly accumulate until somebody on your team reviews the transcript queue — usually weeks later, usually never.

AxiomAI's Insights agent is a second AI that runs against the first on a weekly cycle. It reads the conversations that happened since the last run, compares them to the bot's current rules, and proposes the next rule — concretely enough that you apply it with one click. Here's how it works under the hood.

Why we built it

Every customer we've onboarded so far went through the same loop in their first month. They'd ship a bot trained on their site. It would answer 80% of the questions. The other 20% would land in the Unanswered queue. Somebody on the customer's team — usually overworked — would scan that queue, mentally aggregate the patterns, write a new rule, and apply it.

That loop is a tax on the customer's attention. It's also exactly the kind of work an LLM is good at — bounded scope, structured input, structured output. So we built a second agent to do it.

The design constraint we held throughout: the customer stays in control. Insights never modifies the bot. It only proposes. The customer reviews, edits if needed, and clicks apply. Draft/publish workflow on every change. Audit log on every accepted proposal.

The loop, in plain terms

Every Sunday at midnight UTC, Insights wakes up for each active tenant and runs four steps:

1. Pull the conversations that closed since the last run, filtered to those where the bot fell back, escalated, or got a thumbs-down. Skip conversations the bot resolved cleanly — they're not signal.
2. Pull the bot's current rules and knowledge sources — the named rules, the crawled URLs, the uploaded PDFs. This is the "what the bot knew" snapshot.
3. Ask a separate LLM to compare the two with a structured prompt: "Given these new conversations and the bot's current rules, what's the smallest concrete change that would have improved the most of them?"
4. Format the proposal in the same rule grammar the human team uses — when X → cite Y → route_to Z. Drop it into the Insights workspace as a draft for review.

The output looks like this: "12 visitors asked about tuition deposit deadlines this week. Your rules don't cover this. Suggested addition: when visitor asks deposit deadlines, cite Admissions Calendar PDF page 4, route to admissions@." It's actionable, it's verifiable, and it took the customer's team zero minutes to produce.

What we learned from the first cohort

After a few months running Insights against real production deployments, most proposals fell into four patterns:

• Missing citation The answer was in a document; the bot wasn't trained on that document yet. Insights surfaces which file to add.
• Missing route The bot answered correctly but didn't know which human to escalate to. Insights proposes the routing rule.
• Missing language The bot only had English content; visitors kept asking in Arabic or French. Insights flags the translation gap.
• Tone mismatch The bot answered correctly but felt off-brand. Insights proposes a tone-adjustment rule with examples.

Roughly 70% of proposals get applied as-is. About 20% get edited (usually to soften a phrasing or tighten the routing). About 10% get dismissed — sometimes the customer has a reason the agent can't see, sometimes the agent over-generalized from a small sample. The dismiss action is itself signal: next week's run sees it and weights similar proposals lower.

What's next

The version we ship today only looks at one tenant at a time. The obvious next step is anonymized cross-tenant insights — "schools running AxiomAI typically need a rule for tuition deposit deadlines by week 3; here's a starter template." We have to be careful with how that data flows; we don't move customer content between tenants, and we don't intend to. But aggregated structural patterns are fair game and could shorten the first-month learning curve.

Further out: Insights proposing not just rule changes but content additions. "Twelve families asked about the new science wing this week and no public page covers it. Draft an article?" That's a CMS workflow more than a chatbot one — which is exactly why the platform is designed as an extensible portal rather than a single-purpose chatbot product. The chatbot is the first surface. The operations layer is the durable one.