STRATA
Skip to study
XLVII

IV.12 / AI

Large models and agents

Which actions depend on a model’s output, and which depend on permissions granted by its surrounding software?

A source-led introduction1 source references0 related experiments

Start with the question.

An agentic system connects a model's outputs to actions in an environment and to observations of the results. ReAct provides a documented approach to interleaving task-oriented reasoning and external actions. The practical research question is whether this loop completes a task reliably under defined permissions, tool failures, and incomplete information. A fluent plan and a successfully observed result are separate evaluation events.

Evaluate a public-document lookup task.

Give a system a fixed set of documents and ask it to find a date with its supporting passage. Record the chosen tool call, returned evidence, final answer, and whether the citation actually supports the date. Introduce a missing document and inspect whether the system correctly preserves the unresolved state rather than inventing a result.

What would you inspect?

  • Task completion, evidence support, tool errors, and unnecessary actions.
  • The permission boundary and the point at which success is independently observed.

Take the next step.

Build a small task set with verifiable outcomes and one failure condition, then compare the agent's claims with the recorded environment state.

Read the supporting material.

  1. ReAct: Synergizing Reasoning and Acting in Language Models

    Shunyu Yao and coauthors · Interleaving language-model task planning and external actions in evaluated environments.

Continue through the versioned study records.

Follow the comparisons.

Read the neighboring questions.