Agentic AI

Agentic AI and Multi-Agent Systems: Architecture, Governance, and Enterprise Use Cases

When to keep workflows deterministic, when one tool-calling agent is enough, when a supervisor-worker system earns its cost, and how Governance-as-Code keeps agents inside business policy

AuthorHarshil S.Read Time18 minPublished13 November 2026

Quick Answer

Start with the simplest design that reliably works. Keep fixed, rule-based steps in deterministic code or RPA. Use a single tool-calling agent when the task sits in one domain with a manageable set of tools. Move to a multi-agent system only when tools or instructions start to conflict, read and write permissions must be separated, or independent subtasks can run in parallel. Whichever design you choose, enforce business rules outside the model with Governance-as-Code: a policy engine that checks every tool call, sandboxed execution, audit logs and human approval for high-risk actions.

Introduction

Picture a mid-market manufacturer and distributor that rolls out an internal AI assistant to answer staff questions about standard operating procedures. It works, so within weeks operations managers ask for more: can the same assistant inspect supplier invoices, check stock levels in the ERP, verify contract pricing and adjust purchase orders?

Bolting database queries, document parsers, email tools and write permissions onto that one assistant exposes the strain quickly. The context fills with overlapping tool definitions, the model sometimes calls the wrong endpoint, and the security team raises the objection that matters most: nothing stops the assistant from approving an out-of-policy financial adjustment if a supplier PDF contains misleading instructions.

This guide works through the three decisions that determine whether a project like that succeeds: when a workflow should stay deterministic, when one agent is enough and when a multi-agent system earns its extra cost, and how to enforce business policy outside the model’s reasoning. It assumes you already know the basics; if not, start with our beginner guide to AI agents and how agentic AI differs from traditional AI.

What agentic AI means in production

An AI agent combines a foundation model with instructions, guardrails and executable tools to pursue a goal over several steps. Where a chat completion turns one prompt into one reply, an agent runs a loop: inspect the current state, plan, call a tool or API, read the result, and decide whether the goal is met or another step is needed.

Anthropic’s engineering guidance draws a boundary that is useful for every architecture decision in this article:

Workflows: models and tools orchestrated through predefined code paths. The application decides the order of steps.
Agents: the model dynamically directs its own process and tool use, choosing what to do next based on what it has observed.

Why production systems are mostly bounded agents

Reliable enterprise systems rarely give a model open-ended autonomy. They wrap short agentic loops inside deterministic state machines that control permissions, save checkpoints and stop for human review at defined points. The question is not “agent or no agent” but how much of the control flow the model should own, and which parts must stay in code.

Single-agent vs multi-agent systems

Once a workflow outgrows a single prompt, architects face a choice between giving one agent more capability and splitting the work across several agents. Microsoft’s Azure Architecture Center advises using the lowest level of complexity that reliably meets your requirements, and describes a single agent with tools as often the right default for enterprise use cases. That is a sensible starting point: multi-agent design solves real problems, but it is not an upgrade by default.

How a single-agent architecture works

One model instance handles planning, reasoning and tool execution within one state loop. The application passes the request, system instructions, conversation history and the available tool schemas to the model, which calls tools, inspects the results and repeats until it produces an answer or reaches a stop condition.

Strengths: one shared context means no message passing between agents, lower latency, fewer tokens and a single trace to debug.
Limits: as the tool list grows and tools start to overlap, or as instructions for different departments begin to contradict each other, tool-selection accuracy drops and prompts become hard to maintain.

How multi-agent systems coordinate work

A multi-agent system divides a workflow among agents that each have a narrow role, their own instructions, scoped memory and a restricted subset of tools. LangGraph, Microsoft and AWS document similar orchestration topologies:

Supervisor-worker: a supervisor receives the objective, breaks it into subtasks, hands each to a specialist worker and combines their results. Anthropic calls the same idea orchestrator-workers.
Sequential pipeline: agents process the state in a fixed order, for example template selection, clause drafting, compliance review and risk scoring, with code controlling each transition.
Concurrent execution: an orchestrator sends independent subtasks to specialists at the same time and aggregates the findings, cutting wall-clock time.
Handoff: control passes from a triage agent to a domain specialist, which then owns the conversation without routing every turn back through a central agent.
Group chat and magentic patterns: agents collaborate in a shared thread or under a manager that builds and revises a task plan. These suit open-ended problems and are the hardest to make predictable.

What a multi-agent system costs you

Splitting a task across agents adds coordination overhead: extra model calls for routing, context repeated in several prompts, serialisation errors when one agent’s output becomes another’s input, and failures that only show up when you correlate traces across agents. Each agent is also another prompt, tool set and evaluation suite to version and maintain. The comparison table later in this guide sets these costs against the gains in specialisation, parallelism and permission isolation.

Example: one invoice discrepancy workflow, two designs

Take a hypothetical workflow that compares a vendor’s invoice against the master service agreement and the warehouse delivery receipts:

Single agent: one agent has read access to the contract store, SQL tools for warehouse receipts and an ERP tool that drafts credit memos. It fetches the contract, queries receipts, calculates the variance and drafts the memo in one thread.
Supervisor-worker: a supervisor dispatches two read-only workers in parallel, a contract analysis agent with document search only and a receiving agent with read-only SQL. Their structured summaries go to a finance reconciliation agent that is allowed to create ERP drafts and nothing else.

When to move from one agent to a multi-agent system

Keep the single agent while it meets your accuracy target with clean tool definitions and good prompts. Consider splitting it when you see one or more of these signals:

Tool confusion: evaluation runs show the agent regularly picking the wrong tool among similar ones, and better tool descriptions no longer fix it.
Conflicting instructions: rules for finance, procurement and customer communication pull the same prompt in different directions.
Permission separation: research steps should only read data, while one final step may write, and you want those privileges held by different identities.
Parallel work: independent lookups can run at the same time and latency matters more than token cost.
Noisy context: large tool outputs crowd out the information the planner needs, and isolating them in a worker produces cleaner decisions.

Governance-as-Code for business AI agents

Handing an agent direct credentials to production databases or business APIs is the riskiest shortcut in agentic AI. Language models are probabilistic and susceptible to indirect prompt injection, where untrusted content such as an email, an invoice PDF or a web page carries hidden instructions that steer the agent’s next tool call.

Writing “never approve refunds above $500” in a system prompt is an instruction to the model, not a security control. Governance-as-Code means defining, versioning, testing and enforcing operating permissions in deterministic software that sits outside the model, so the rule holds even when the model is wrong or manipulated. Teams that already run policy-as-code for infrastructure will recognise the approach.

The six-stage Governance-as-Code pipeline

Every tool call an agent proposes passes through the same sequence before it can touch a business system:

1. Business policy: legal, finance and operations define explicit rules, such as spending thresholds, permitted data scopes and execution windows, kept as version-controlled policy files.
2. Agent identity and permissions: each agent runs under its own machine identity with role- and attribute-based access, never a shared administrator token.
3. Policy evaluation: when the agent emits a structured tool call, a policy engine such as Open Policy Agent checks the agent identity, tool name and arguments against deny-by-default rules.
4. Approved execution: allowed calls run in an isolated sandbox with restricted network egress, short-lived credentials from a secrets vault and strict time and resource limits.
5. Monitoring and audit: every policy decision, tool input, tool output and state change is written to an append-only log tagged with trace, session and agent identifiers.
6. Human review where required: if the policy returns “escalate”, for example for a high-value journal entry or an email to an external customer, the orchestrator saves a checkpoint, pauses and sends a structured approval request to an authorised person.

Example policy: limits on ERP credit memos

Open Policy Agent decouples policy decisions from application code and expresses rules in a declarative language called Rego. An illustrative policy for the finance reconciliation agent from the invoice example would set both “allow” and “require human approval” to false by default, then add three narrow rules:

Allow the finance_reconciliation_worker role to call get_invoice_status in production, because it only reads ledger data.
Allow the same role to call create_credit_memo_draft only when the currency is USD and the amount is 1,000 or less.
Require human approval when that role requests a credit memo draft above 1,000.

What the policy layer changes in practice

Any call that matches none of those rules is denied before a request reaches the ERP, regardless of what the model was persuaded to attempt. Because the policy is code, finance can review a change to the threshold in a pull request, it can be unit-tested against sample tool calls, and the audit log records which policy version made each decision. Building this layer usually means proper backend work: an execution proxy, identity mapping and approval screens, which is the kind of custom software development with role-based access most agent projects underestimate.

Sandboxing, secrets and execution isolation

Passing the policy check is necessary but not sufficient. Runtime execution must stay contained:

Code isolation: agents that run generated Python or shell scripts for analysis should do so in short-lived containers or microVMs with no host filesystem mounts.
Network egress control: block arbitrary outbound connections and allow traffic only to explicitly listed internal API gateways.
No secrets in context: never place API keys or database passwords in the model’s context. The execution proxy injects short-lived credentials only after the policy check succeeds.

Frameworks that structure the controls

No single framework or engine makes an agent secure. The NIST AI Risk Management Framework (AI RMF 1.0, currently under revision) organises risk work into four functions, Govern, Map, Measure and Manage, which are useful for assigning ownership and review cadence. The OWASP GenAI Security Project’s Agentic Security Initiative publishes threat-model-based guidance on agentic threats and mitigations that is useful for technical design reviews. Use the first for governance and the second for engineering controls, and treat both as starting points rather than certifications.

Agentic AI vs traditional RPA

Leaders modernising back-office work often ask whether agentic AI should replace their robotic process automation. In most cases it should not: deterministic automation and probabilistic agents solve different classes of problem, and the strongest designs use both.

RPA and scripted automation fit when inputs are structured, rules are fixed, interfaces or APIs are stable, and regulation demands the same execution path every time, with no inference cost.
Agentic AI fits when inputs are unstructured, such as emails, varied PDFs and free-text tickets, when the answer requires combining several sources, and when exceptions need investigation rather than a fixed rule.

Where deterministic automation stays the better choice

Payroll batches, fixed EDI file transfers, deterministic tax calculations and scheduled backups gain nothing from a language model and lose predictability. If you can write the rule down completely, write it as code. Our list of high-priority business processes to automate is mostly in this category.

The hybrid automation pattern

The most practical enterprise pattern is a state graph in which deterministic code or RPA bots handle the standard path, and an agent is invoked only when an unstructured document or an exception appears. The agent turns messy input into structured data or a proposed resolution, and deterministic steps execute the transaction. Token costs are then paid only on ambiguous cases, and the audit trail stays exact for everything else. Before choosing which steps belong where, start by mapping workflow steps and exceptions.

Enterprise use cases in manufacturing, finance and logistics

The three architectural sketches below are hypothetical, not client case studies. They show how orchestration choices and approval boundaries change with the risk of each action.

Manufacturing: production exception and supplier discrepancy triage

Quality engineers lose hours cross-referencing shop-floor sensor alerts, batch inspection PDFs and supplier material certificates whenever a lot fails tolerance checks.

Architecture: a supervisor triggers two read-only workers in parallel, a telemetry inspection agent and a supplier documentation search agent, followed by a report drafting agent.
Integrations: read-only access to the MES time-series API, vector search over supplier quality manuals and draft creation in the quality management system.
Approval boundary: the system proposes a root cause and drafts the non-conformance report and supplier notice; a certified quality engineer authorises any line stop or vendor chargeback.
Failure modes to test: OCR errors on scanned mill test reports, false correlation between unrelated sensor spikes and outdated specifications in the search index.
Success metrics: mean time to investigate exceptions, share of draft reports accepted without major edits and zero unauthorised ERP status changes.

Finance: three-way match exception investigation

Accounts payable teams manually investigate invoices that fail three-way matching against purchase orders and goods-received notes because of freight surcharges, partial deliveries or unit-of-measure mismatches.

Architecture: a hybrid pipeline where deterministic code clears exact matches and routes only mismatches to a single reconciliation agent with read-only ERP tools and contract lookup. One agent is enough here.
Integrations: read-only ledger APIs, the procurement contract repository and a ticketing webhook for approval routing.
Approval boundary: the agent performs variance analysis and attaches the supporting contract clauses; ledger posting, vendor master changes and payment release need a human controller.
Failure modes to test: injected instructions inside a fraudulent invoice PDF that try to change remittance details, and multi-currency tax splits.
Success metrics: exception resolution time, false escalation rate, audit trail completeness and cost per resolved exception.

Logistics: shipment delay and route disruption coordination

Operations coordinators watch carrier webhooks, weather advisories, customs holds and dock schedules to reallocate inventory when shipments slip.

Architecture: an event-driven supervisor in which a disruption triage agent scores each delay and calls a route and inventory analysis agent to simulate fulfilment alternatives.
Integrations: TMS status feeds, WMS inventory APIs and CRM notification templates.
Approval boundary: internal warehouse alerts run automatically; carrier rebooking with expedite fees, or changed delivery commitments to key customers, needs a logistics manager’s approval.
Failure modes to test: retry storms during carrier API outages, duplicate webhook events and split shipments that breach customer contract terms.
Success metrics: share of disruptions triaged within an agreed window, on-time recovery rate for affected orders and avoided expedite spend.

Production architecture for governed agents

Running agents in production is a distributed-systems problem more than a prompting problem. A governed request typically follows this path:

A business event or user request arrives through an API gateway that authenticates the caller.
A stateful graph orchestrator loads or creates the workflow state, saving checkpoints to a durable store and emitting traces and token counts to an observability collector.
A supervisor or router node assigns work to a read-only research agent or an action planning agent.
Every proposed tool call goes to the policy engine, which returns allow, escalate or deny.
Allowed calls run through a sandboxed tool proxy that fetches short-lived credentials from a vault and calls the ERP, WMS or CRM.
Escalated calls wait in a human approval queue; an approval releases the call with a signed token, while a rejection returns control to the orchestrator.
Denied calls are blocked and recorded in the security audit log, and tool results flow back to the orchestrator as untrusted input.

Reliability and security controls to build in

These controls decide whether an agent survives its first month of real traffic. Hosting them well is part of scalable cloud architecture and reliability operations, not an afterthought.

Checkpointing: durable checkpoints, for example in Postgres, let long-running workflows survive restarts and resume at the exact step that was interrupted, including after a human approval.
Idempotency: derive an idempotency key from the run ID and step for every state-changing call, so a retry never creates a duplicate invoice or shipment.
Retries and budgets: use exponential backoff with jitter for rate limits and transient errors, and enforce a maximum number of iterations so stuck loops end instead of burning tokens.
Untrusted outputs: treat every tool result, retrieved document and API response as untrusted input, validate it against a schema and strip markup before it re-enters the model’s context.
Observability: trace every node and tool call with OpenTelemetry trace and span IDs, recording arguments, latency and token use per run so you can explain any decision afterwards.

How to evaluate cost, latency and reliability before launch

Before promoting either design, run both against the same set of historical business tasks with the same models, tools and grading rules. Track five groups of metrics:

End-to-end latency: median and P95 or P99 completion time for the automated part of the workflow, excluding time spent waiting for a human.
Model calls and tokens: input, cached input and output tokens across all agent turns per task.
Cost per successful task: total inference, tool and compute spend divided by verified successful completions, not by raw runs.
Completion and recovery: the share of runs that reach a valid end state, and the share of tool failures recovered by retries or fallbacks.
Policy and human intervention: how often the policy engine blocks a call and what share of runs needs escalation.

Reading the results

Expect a supervisor-worker design to make more model calls, use more tokens and show higher tail latency than a single agent on the same task, because routing and handoffs add inference hops. What it has to buy back is accuracy: fewer wrong tool calls, fewer bad drafts and less human rework. If the single agent reaches comparable accuracy after you tighten its prompts and tool definitions, keep it. If the multi-agent design measurably reduces costly errors, the extra tokens are usually worth paying for. Decide from your own evaluation data rather than vendor benchmarks, because results depend heavily on your tools, data and task mix.

Summary

Key Takeaways

Start with deterministic code where rules are fixed, then a single tool-calling agent, and add agents only when there is a measured reason.
Multi-agent systems buy specialisation, parallelism and permission isolation at the price of latency, tokens and debugging effort.
System prompts are not controls: enforce permissions, limits and approvals in a policy engine outside the model.
Give every agent its own identity, keep secrets out of the context and run tools in a sandbox.
Agentic AI complements RPA; hybrid pipelines run the standard path deterministically and call an agent only for exceptions.
Checkpoints, idempotency keys, iteration budgets and tracing matter as much as model choice.

Single agent vs multi-agent system: dimension by dimension

Dimension	Single agent	Multi-agent system	Trade-off
Architecture complexity	Low	High	One loop vs a distributed state graph
End-to-end latency	Lower	Higher unless work runs in parallel	Handoffs add sequential inference hops
Token consumption	Lower	Higher	Context is repeated across agent prompts
Task specialisation	Moderate	High	Narrow tool sets improve selection accuracy
Parallelism	Limited	High	Independent workers can run concurrently
Context management	One shared context	Scoped per agent	Keeps noisy tool output away from the planner
Error recovery	Retry the whole run	Retry one node	Checkpoints isolate failures to one worker
Debugging and observability	One trace tree	Multi-agent trace graph	Needs trace correlation across handoffs
Security boundary	One shared permission set	Least privilege per agent	No single agent holds every key
Maintenance	One prompt and tool set	Several prompts, schemas and evals	More artefacts to version and test
Best fit	Focused, sequential tasks	Cross-domain or parallel tasks	Start single; split when tools or roles clash

Traditional RPA vs agentic AI vs hybrid automation

Criterion	Traditional RPA	Agentic AI	Hybrid (RPA + agents)
Flexibility	Rigid, rule-bound	Dynamic, goal-driven	Fixed backbone with adaptive steps
Predictability	Deterministic	Probabilistic	Deterministic core, bounded AI steps
Input handling	Structured fields and tables	Unstructured text and documents	Agents structure input for scripted execution
Exception handling	Fails or queues for a person	Investigates and proposes a fix	Agents triage, scripts execute
Speed	Milliseconds to seconds	Seconds to minutes	Fast standard path, slower exception path
Running cost	Licences and compute	Per-token inference plus compute	Inference paid only on ambiguous cases
Maintenance	Breaks on UI or schema changes	Needs prompt and evaluation tuning	Separates parsing from transaction scripts
Auditability	Exact step log	Reasoning trace plus policy logs	Checkpoints plus policy audit
Best fit	High-volume repeatable transfers	Research, triage, complex routing	End-to-end business operations

Pitfalls

Common Mistakes to Avoid

Choosing a multi-agent architecture because it sounds advanced, before a single agent has been measured on real tasks.
Treating system prompt instructions as the only control on what an agent may do.
Running every agent under one shared service account with write access to everything.
Placing API keys or database credentials in the model’s context.
Retrying state-changing tool calls without idempotency keys, creating duplicate records.
Replacing stable, deterministic RPA with an agent where the rules were already complete.
Shipping without an evaluation set, so latency, cost and accuracy trade-offs are guesses.

FAQ

Frequently Asked Questions

What is the main difference between a single-agent and a multi-agent AI system?+

A single-agent system uses one model loop to plan, reason and call tools within a shared context. A multi-agent system splits the work among specialised agents, such as a supervisor and domain workers, each with its own instructions, scoped memory and least-privilege tool access.

When should a business use a single AI agent instead of a multi-agent system?+

Use a single agent when the workflow sits in one domain, the tools are well defined and do not overlap, and the steps are mostly sequential. Introduce multiple agents when evaluations show tool confusion or conflicting instructions, when read and write permissions must be held by different identities, or when independent subtasks benefit from running in parallel.

What is Governance-as-Code for AI agents?+

Governance-as-Code is the practice of writing business rules, tool permissions, spending limits and escalation thresholds as version-controlled policy, for example in Open Policy Agent’s Rego language, and enforcing them deterministically outside the model before any tool call runs.

How do you align AI agents with business policies?+

Translate each policy into a machine-checkable rule, give each agent its own identity and minimum permissions, route every tool call through a policy engine, log every decision and require human approval for actions above defined risk thresholds. Test the rules against sample tool calls just as you would test application code.

Does agentic AI replace traditional RPA?+

No. RPA remains faster, cheaper and more predictable for high-volume tasks with structured inputs and fixed rules. Agentic AI complements it by handling unstructured documents, exception triage and reasoning across systems within a hybrid pipeline.

How do human-in-the-loop approvals work in agent workflows?+

When an agent requests a high-risk action, such as an ERP adjustment or a change to a customer order, the policy engine returns an escalation and the orchestrator saves a checkpoint and pauses. An authorised person reviews the context in an approval screen, and the workflow resumes from the checkpoint with the action either approved or rejected.

Wrap-up

Final Thoughts

Moving from an AI assistant to governed automation is mostly an engineering decision, not a model decision. Keep complete rules in deterministic code, start with one well-instrumented agent, split it only when evaluation data shows a reason, and put every consequential action behind a policy engine, a sandbox and, where the stakes justify it, a human approval.

If you are deciding which of your processes belong in deterministic automation and which in governed agents, map one workflow end to end first. The exceptions you find usually tell you where an agent adds value and where it only adds risk.

References

Sources

Let's talk

Build Your Next Digital Product

Vaqnova designs and builds governed agentic AI systems, from single tool-calling agents to policy-controlled multi-agent workflows, integrated with your ERP and operational systems and scoped around measurable milestones.

Talk to Our Team

Related Services

Agentic AI Services Custom Software Automation

Browse All Articles Talk to Our Team