← Reference · Nestor G Pestelos Jr · Print this page

Artificial Intelligence · Autonomous Agents

AI Agents (LLM-Based)

Definitions, execution patterns, tool boundaries, memory, and multi-agent trade-offs.

Reference entry · last updated 20260908

Previous version (before the POMDP extraction, 20260908).

An LLM-based AI agent is a system in which a language model selects actions and uses their results to pursue a task. Its host controls which tools and permissions are available. [1]

1. First Principles: Choosing the Next Action

This entry covers tool-using LLM systems. A workflow follows steps set in code. An agent lets the model choose the next action from the information available. A system can combine both. Explicit plans and persistent memory are optional. [1]

For example, an agent given the task “fix a failing test” might:

Read the failure message.
Choose a source file to inspect.
Edit the suspected cause and run the test.
Use the result to decide whether to investigate further or finish.

The model chooses each next step from what it finds; the host software executes permitted tool calls and returns their results. The host also enforces limits on what the agent can do. [1, 7]

The agent has observations, such as file contents and test output. The state is the underlying situation, including code and environment details it may not have inspected. A failure message gives evidence about a bug without revealing its cause.

A partially observable Markov decision process (POMDP) formalizes decisions under incomplete information. It is an optional mathematical model for this loop; building an agent does not require a POMDP solver. [2]

2. Agent Subsystems

The four-part grouping below is this entry's organizational model. Implementations can combine or omit parts.

Planning & Reasoning:

The model selects a next action or produces a plan. A host can add task decomposition and programmatic checks. [1]

Memory:

Working context: Instructions, observations, and intermediate results supplied for the current model call.
External memory: Stored records retrieved for later calls. Text buffers, files, and vector search are possible implementations.
Parametric knowledge: Information encoded in model weights. Ordinary context updates do not themselves change those weights.

Reflexion is one implementation that stores textual feedback across trials without updating model weights. [4]

Tool & Action Interface:

Host code checks a requested action, invokes an allowed tool, and returns its result. A tool request alone grants no authority to execute it. [7]

Observation Processing:

For example, a coding host can select file diffs or compiler output for the next call. Truncation and summaries trade detail for context space.

3. Execution Patterns

ReAct (Yao et al.):

Interleaves generated reasoning, actions, and observations:

$$\text{Thought}_t \to \text{Action}_t \to \text{Observation}_t \to \text{Thought}_{t+1}$$

The paper reports reduced hallucination and error propagation on HotpotQA and FEVER using a Wikipedia API, and improved task success on ALFWorld and WebShop against its tested baselines. Observations can correct unsupported reasoning; they do not guarantee that later steps are correct. [3]

Reflexion (Shinn et al.):

Uses feedback to generate textual reflections stored for later trials. Feedback can be scalar or free-form, and externally supplied or internally simulated. It does not require a binary failure signal or weight updates. [4]

Plan-and-Solve (Wang et al.):

A prompting strategy that asks a language model to devise a plan, then solve its subtasks. The paper evaluates GPT-3 on ten reasoning datasets. Separate planner and executor agents, progress tracking, and verification gates are additional architecture choices. They are not requirements of Plan-and-Solve prompting. [5]

For example, a host could pass a plan to a separate executor and require a passing test before the next task. This is an illustrative gated workflow.

4. Tool Calling & Sandboxing

A tool interface describes available actions and their inputs. The following is an illustrative tool definition with a JSON Schema parameter object; provider formats vary. A schema checks input shape, not whether an edit is authorized or correct.

{
  "name": "replace_file_content",
  "description": "Edits an existing file via exact block match replacement.",
  "parameters": {
    "type": "object",
    "properties": {
      "target_file": { "type": "string", "description": "Absolute path" },
      "target_content": { "type": "string", "description": "Exact text to replace" },
      "replacement_content": { "type": "string", "description": "New content" }
    },
    "required": ["target_file", "target_content", "replacement_content"]
  }
}

Containment depends on controls enforced outside the model. Anthropic describes permissions, execution isolation, and network restrictions as ways to limit an agent's reach. These controls have implementation limits and require testing. [7]

Permissions: Restrict file access and tool operations; require approval where policy calls for it.
Isolation: Limit access from the execution environment to the host and other workloads.
Network controls: Restrict reachable services and outbound destinations, including alternate paths available to tools.

5. Multi-Agent Topologies & Orchestration

Multiple agents can divide work or compare candidate answers. Coordination adds cost and handoff risks; a single agent can be sufficient. [1]

The topology descriptions and comparison below are this entry's design synthesis, except for the cited debate result.

Hierarchical (Supervisor-Worker):

An orchestrator assigns work and collects results. Separate contexts can limit irrelevant input, but workers still share any files, tools, and permissions the host exposes.

Sequential Pipeline:

One stage's artifact becomes the next stage's input. For example, a specification can pass to an implementer, then a reviewer. Fixed stage routing is a workflow even when its stages use agents.

Blackboard Architecture:

Agents coordinate through a shared state store. The host must define ownership and conflict handling for concurrent writes.

Multi-Agent Debate (Du et al.):

Multiple model instances propose answers and critique them over rounds. The paper reports improved reasoning and factuality on its evaluated tasks. Those results do not establish a general correctness guarantee. [6]

Agreement can preserve a shared mistake. Claims still need source checks or task-specific tests; a formal verification claim requires a formal specification and a checked proof.

6. Failure Modes & Defenses

These are example failure scenarios and mitigations, not measured guarantees.

Failure Mode	Example	Mitigation & Limit
Error-Fix Loop	The same failing action repeats.	A host counter can stop repeated attempts. Three identical errors is an example cutoff; the suitable limit depends on the task.
Context Contamination	Irrelevant or misleading tool output enters later model calls.	Select observations and summarize long output. Summaries can omit evidence, so retain access to originals.
Trajectory Drift	An early false assumption affects later edits.	Check intermediate artifacts against requirements. Tests only cover their assertions.
Indirect Prompt Injection	An untrusted file or webpage contains instructions that redirect the task.	XML delimiters organize input; they do not enforce permissions. Host-enforced file, tool, and network limits constrain possible actions. [7]

A reader with no action privileges can reduce direct exposure of a privileged actor to untrusted text. This design only helps if the host constrains their communication and validates what crosses the boundary. Passing unrestricted reader output to the actor can carry the injection onward.

7. Architecture Comparison

Architecture	Conditional Benefit	Failure Condition	Verification Requirement
Single Agent	Simple coordination for tasks that fit its context and tools.	Errors persist if the same assumptions drive all checks.	Check the output against external evidence or executable tests.
Hierarchical	Workers can handle separable tasks in focused contexts.	Bad decomposition or shared writes can spread errors.	Validate worker outputs and the combined result; control shared writes.
Sequential Pipeline	Explicit artifacts and checks at each handoff.	A faulty artifact passes a weak gate.	Check handoff contracts and final acceptance.
Multi-Agent Debate	Critiques may expose errors in candidate answers.	Participants share a false premise or accept an unsupported answer.	Verify claims independently of agreement. Debate itself supplies no formal proof.