# AI Engineering Glossary

This glossary is a quick reference for software engineers who are new to AI engineering on Gaia.

## Core concepts

- **Probabilistic behavior:** The model can produce different outputs for similar inputs.
- **Deterministic control:** Explicit system constraints that stay stable (tool allow-lists, policies, validation rules, role permissions).
- **Grounding:** Tying responses to approved, traceable data sources instead of plausible language.
- **Hallucination:** Confident-sounding output not supported by trusted data or tools.
- **Context:** The information given to the model for a turn (instructions, history, tool outputs, retrieved records).
- **Prompt injection:** Malicious or conflicting instructions in user/external content that try to override policy.

## Runtime and agent concepts

- **System instructions:** High-priority behavior rules for an agent/configuration.
- **Tool call:** A model-requested function invocation to read/write data or trigger behavior.
- **Handoff:** Routing from one agent role to another (for example, orchestrator to specialist).
- **Refusal:** Intentionally declining unsafe or unsupported requests.
- **Escalation:** Handing off to human/operational flow when policy or confidence thresholds are not met.

## Data and workflow concepts

- **Entity:** Structured domain model used by assistants and workflows.
- **Ingestion:** A subtype of data workflow focused on bringing external data into Gaia (manual, context-key, webhook-driven).
- **Pipeline:** A bounded data transformation step.
- **Workflow:** Orchestration of ordered pipeline steps plus failure/rerun behavior.
- **Idempotency:** Re-running the same input does not create duplicate or conflicting state.

## Evaluation concepts

- **Dataset:** Collection of evaluation tasks representing expected and edge behavior.
- **Grader:** Mechanism that scores tasks (deterministic checks, rubric checks, human review).
- **Regression gate:** Must-pass eval checks used to block unsafe releases.
- **Calibration:** Aligning grader outcomes with human judgment to reduce false confidence.

## Operations concepts

- **Latency:** Time to complete a response/run.
- **Throughput:** Amount of work completed per time window.
- **Token usage:** Input/output token consumption used for model billing and optimization.
- **Cost governance:** Budget and control policy for quality/latency/cost tradeoffs.
- **Rollback trigger:** Predefined condition that pauses or reverts a release candidate.

## Gaia mapping cheat sheet

- Behavior controls: **AI Agents**
- Data and automation: **Data Model (Entities, Storage, Pipelines, Workflows, Runs)**
- Runtime evidence: **Conversations (Timeline, Inside Info)**
- Quality evidence: **Evals (Datasets, Graders, Runs, Reports)**
- Delivery governance: **Delivery Management (Discussions, Process, Tasks, Milestones, Timeline, Status)**

---

# Chapter 1: AI Engineer Foundations

Status: ready

Role, mindset, and Gaia context

## Sections

- [Role and Scope](#doc-ch01-ai-engineer-foundations-01-role-and-scope)
- [Core AI Systems Concepts](#doc-ch01-ai-engineer-foundations-02-core-ai-systems-concepts)
- [Gaia Platform Mental Model](#doc-ch01-ai-engineer-foundations-03-gaia-platform-mental-model)
- [Learning Path and Capstone](#doc-ch01-ai-engineer-foundations-04-learning-path-and-capstone)

## Supporting Resource

- [AI Engineering Glossary](#doc-foundations-ai-engineering-glossary)

## Fast path inside Gaia

Use this chapter to build the mental model, then ground it immediately in the product.

1. Read [Getting started](../../user-guide/README.md) so the signed-in platform layout is concrete.
2. Read [Build an AI application](../../user-guide/building-an-ai-application.md) so the end-to-end Gaia workflow has a practical shape.
3. Use [Gaia](../../user-guide/platform-assistant.md) and [Tutorials](../../user-guide/tutorials/README.md) at `/platform/support/tutorials` as lightweight orientation aids while you work through the later chapters.

If the foundational terms in this chapter still feel abstract, open the matching user-guide pages before moving on. The rest of the handbook assumes you can picture the Gaia surfaces being discussed.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Role and Scope

## Learning objectives

By the end of this section, you should be able to:

- Define the Gaia AI Engineer role in measurable terms across design, implementation, quality, and operations.
- Draw clear ownership boundaries between product, domain experts, platform admins, and AI engineers for a single Gaia project.
- Convert a vague request ("build an assistant for X") into a scoped delivery plan with explicit in-scope and out-of-scope items.
- Execute a reproducible Gaia scoping lab and produce evidence that the scope is technically feasible and evaluable.

## Prerequisites

- You can navigate Gaia at a basic level.
- You understand core software engineering concepts: requirements, testability, versioning, and incident handling.
- If you already have a Gaia project, you can open at least one project workspace.
- If you are starting from zero, complete Chapter 2 Sections 1-2 first, then return to this section lab.

## In Gaia

- [Getting started](../../user-guide/README.md) for the actual team and project entry path
- [Build an AI application](../../user-guide/building-an-ai-application.md) for the end-to-end Gaia delivery shape
- [AI Agents](../../user-guide/agents/README.md), [Data Model](../../user-guide/data-model/README.md), [Conversations](../../user-guide/conversations/README.md), and [Evals](../../user-guide/evals/README.md) for the four operating systems this section describes

Use this section to define the scope contract, then verify that each responsibility you claim really maps to one of those Gaia surfaces.

## Concept brief

Many teams fail with AI projects for a simple reason: they start by asking what model to use, not what operating responsibility they are taking on. On Gaia, the AI Engineer role is not "prompt writer" and not "model selector." It is a delivery role with end-to-end accountability for a bounded AI capability: defining behavior, making it observable, evaluating it, and operating it safely over time.

In practical terms, a Gaia AI Engineer sits at the intersection of four systems that must remain coherent:

1. **Behavior system** (agents, prompts, tools, handoffs): What the assistant is allowed to do and how it reasons.
2. **Data system** (entities, storage, pipelines, workflows): What information the assistant can retrieve, update, and trust.
3. **Experience system** (channels, UI layouts, conversation UX): Where users interact and how output is consumed.
4. **Assurance system** (evals, metrics, audit trail, role controls): How the team proves quality and manages risk.

If one of these systems is missing, your project can still "demo" but cannot reliably ship. For example, an agent can answer questions impressively in a single conversation, yet still be unfit for production if there is no eval baseline, no policy boundaries, or no observability for failures.

### What the role owns

A Gaia AI Engineer owns decisions that make AI behavior production-capable, not just impressive.

Core ownership areas:

- **Problem framing**: Translate business goals into bounded assistant tasks and quality criteria.
- **Capability design**: Decide tool/data dependencies and constrain the behavior surface.
- **Configuration quality**: Ensure prompt fragments, tool wiring, and handoff rules are coherent.
- **Evaluation design**: Define what success/failure means before launch and run repeatable checks.
- **Operational reliability**: Detect regressions, diagnose failures, and coordinate safe changes.

A useful test is: "If this assistant causes a costly mistake tomorrow, would I know why and how to fix it quickly?" If the answer is no, the role is not complete yet.

### What the role does not own alone

AI Engineers are accountable for technical fitness, but they do not own all project decisions.

- **Product owner** defines business priority and acceptable tradeoffs.
- **Domain experts** define factual correctness and policy constraints.
- **Project/platform admins** define access controls and team governance.
- **Operations/support teams** own incident communication and escalation procedures.

The AI Engineer’s job is to make these constraints executable inside Gaia. In other words, you convert intent into enforceable system behavior.

### Scope as a contract

In early AI work, teams often say "let's start broad and narrow later." On real timelines, that strategy produces endless churn. On Gaia, you should treat scope as a contract with three explicit components:

1. **In-scope tasks**: the assistant workflows that must work reliably.
2. **Out-of-scope tasks**: things the assistant must decline, defer, or route elsewhere.
3. **Success evidence**: the measurable outputs that prove readiness.

Example (customer support assistant):

- In scope: answer policy lookup questions from approved data sources, draft response suggestions, tag conversations for review.
- Out of scope: legal advice, account deletion operations, external API actions not implemented via approved tools.
- Success evidence: pass-rate threshold on eval set, no policy-violation class in sampled production conversations, stable tool latency budget.

Without this contract, teams overfit to anecdotes. With it, you can evaluate and improve.

### Role maturity expectation

This handbook targets engineers who want to move from demo-quality builds to quality-owned delivery. The expectation by the end of the handbook is that you can define explicit success criteria, diagnose behavior using evidence, and manage scoped changes without destabilizing production behavior.

### Key scope decisions you must make early

Before writing detailed prompts, lock these decisions:

- **User persona and environment**: Who asks questions, where, and under what urgency.
- **Critical failure classes**: What mistakes are unacceptable (compliance, financial, safety, trust).
- **Action boundaries**: What the assistant can change vs. what it can only suggest.
- **Evidence boundaries**: What data sources are authoritative and current.
- **Escalation path**: What to do when confidence is low or tool/data checks fail.

Documenting these decisions first prevents expensive rework later in Data Model, AI Agents, and Evals.

### Observable quality over perceived intelligence

A common trap for new AI Engineers is optimizing for "smart sounding" output. In Gaia production work, your north star is not eloquence. It is **observable quality**:

- Correctness on task-relevant facts.
- Appropriate tool usage and data grounding.
- Predictable behavior under repeated runs.
- Traceable failures with actionable diagnostics.
- Acceptable cost/latency for target channel.

The right question is not "Is the answer good?" It is "Can we repeatedly produce acceptable outcomes under expected load and constraints?"

### Relationship to software engineering fundamentals

The Gaia AI Engineer role extends traditional engineering principles rather than replacing them.

- Requirements become behavior and evaluation contracts.
- Integration work becomes tool/data/channel orchestration.
- Testing becomes eval design plus scenario-based runtime verification.
- Observability extends from system metrics to turn-level and tool-level behavior.
- Release management includes prompt/config changes as first-class operational changes.

If you are already strong in software engineering fundamentals, this role is a natural extension. The key difference is dealing with probabilistic behavior while preserving deterministic guardrails where they matter.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab is a scoping exercise that converts a broad request into an executable Gaia role-and-scope baseline.

### Scenario

Use this scenario (or adapt with equivalent complexity):

"Build a Renewal Operations Assistant that helps account teams answer renewal-policy questions, summarize account status, and draft internal next steps."

### Phase A: Create the scope contract

1. Open your target project in Gaia.
2. Create a short scope note (outside or inside your team notes) with these headings:
   - User persona
   - In-scope tasks
   - Out-of-scope tasks
   - Critical failure classes
   - Success evidence
3. Keep each heading concrete. Avoid vague items like "be helpful" without measurable conditions.

Checkpoint A:

- You have a written scope contract with at least 3 in-scope tasks, 3 out-of-scope tasks, and 3 success evidence metrics.

### Phase B: Map scope to Gaia components

4. Open **AI Agents** and identify the orchestrator agent and one active config that will be used for this capability.
5. Open **Data Model** and list the minimum entities/data sources required to satisfy the in-scope tasks.
6. Open **Conversations -> Channels** and choose one primary delivery channel for first rollout (typically text).
7. Open **Evals** and define one initial dataset theme aligned to the critical failure classes.

Checkpoint B:

- You can point to one candidate config, one minimum data footprint, one launch channel, and one eval theme.

### Phase C: Write acceptance boundaries

8. In your scope note, add an "Acceptance boundaries" section with:
   - Behavior boundaries (what the assistant must refuse or escalate).
   - Data boundaries (what it can cite vs. must not assume).
   - Action boundaries (read-only vs. write/update actions).
9. Add a "Rollback trigger" section with 2-3 conditions that would pause rollout (for example, eval pass rate drops below threshold, policy violation class appears, tool error rate spikes).

Checkpoint C:

- Boundaries and rollback triggers are documented and mapped to observable signals.

### Phase D: Run a minimum viability check in Gaia

10. In **Conversations**, run 3 prompts:

- one clearly in-scope,
- one ambiguous,
- one clearly out-of-scope.

11. Review responses for boundary compliance. Confirm out-of-scope handling is explicit and safe.
12. Capture any mismatch between desired and observed behavior as a short issue list.

Checkpoint D:

- You have evidence for three behavior classes (in-scope, ambiguous, out-of-scope) and at least one improvement item.

### Phase E: Prepare section-level QA evidence

13. Save your scope contract and issue list in a durable location used by your team.
14. Record these summary values:

- Count of in-scope tasks
- Count of out-of-scope tasks
- Number of critical failure classes
- Number of minimum viability checks passed

Checkpoint E:

- You can hand your scope artifact to another engineer and they can reproduce the same intent without verbal clarification.

## Expected outputs

At the end of the lab, you should have:

- A written role-and-scope contract for one assistant capability.
- Explicit in-scope and out-of-scope boundaries tied to Gaia features.
- A mapped implementation surface:
  - one target agent config,
  - one minimum data footprint,
  - one launch channel,
  - one eval theme.
- A minimum viability behavior check (3 conversation cases) with captured results.
- A short issue backlog that can guide the next sections in this chapter.

Evidence examples that count:

- A markdown artifact or team note with the scope contract.
- A transcript sample for the three test prompts.
- A checklist proving boundaries and rollback triggers are defined.

## Failure modes

Below are common scope failures in early Gaia projects and how to recover.

1. **Scope drift through stakeholder requests**
   - Symptom: New asks keep appearing mid-implementation with no priority filter.
   - Recovery: Freeze the current scope contract and classify new asks as backlog items tied to a future chapter or release.

2. **No clear out-of-scope behavior**
   - Symptom: Assistant attempts risky answers when it should decline or escalate.
   - Recovery: Add explicit refusal/escalation rules and test them with out-of-scope prompts in Conversations.

3. **Data assumptions without source boundaries**
   - Symptom: Agent answers confidently with unverifiable claims.
   - Recovery: Document authoritative sources and add eval cases that penalize unsupported claims.

4. **Action surface too broad for first release**
   - Symptom: Team tries to support many write actions before read/grounding is stable.
   - Recovery: Restrict v1 to read-heavy workflows and add writes only after eval stability is demonstrated.

5. **Unmeasurable success criteria**
   - Symptom: Team debates quality subjectively and cannot decide readiness.
   - Recovery: Replace generic goals with concrete thresholds (pass rates, violation counts, latency/cost budgets).

6. **No rollback conditions**
   - Symptom: Team discovers regressions but has no pre-agreed pause trigger.
   - Recovery: Define rollback triggers now and map each trigger to an observable signal.

7. **Ownership ambiguity across teams**
   - Symptom: Product, domain, and engineering disagree on who decides policy behavior.
   - Recovery: Add a simple ownership matrix to the scope contract and align decision rights before further build work.

## Completion checklist

Mark this section complete only when all items are true.

- [ ] I can explain the Gaia AI Engineer role as an accountable delivery role, not a prompt-only role.
- [ ] I documented at least 3 in-scope tasks and 3 out-of-scope tasks for one assistant scenario.
- [ ] I defined at least 3 critical failure classes relevant to my scenario.
- [ ] I mapped one target agent config, one minimum data footprint, one launch channel, and one eval theme.
- [ ] I wrote behavior, data, and action boundaries in explicit terms.
- [ ] I defined at least 2 rollback triggers tied to observable signals.
- [ ] I executed the 3-case minimum viability check (in-scope, ambiguous, out-of-scope).
- [ ] I captured observed mismatches as an issue list for follow-up.
- [ ] Another engineer could reproduce my intent from the artifact without live explanation.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)

---

# Core AI Systems Concepts

## Learning objectives

By the end of this section, you should be able to:

- Explain the difference between probabilistic model behavior and deterministic system controls in a Gaia project.
- Map core AI system concepts (context, tools, grounding, evaluation, and operations) to the exact Gaia workspaces used to manage them.
- Diagnose a weak assistant behavior by classifying the issue as data, configuration, tooling, evaluation, or operational discipline.
- Execute a reproducible Gaia lab that produces evidence for behavior quality, boundary enforcement, and measurement readiness.

## Prerequisites

- You completed [Role and Scope](#doc-ch01-ai-engineer-foundations-01-role-and-scope) and have a scope contract for one assistant capability.
- You can inspect at least one agent configuration in **AI Agents**.
- You can run at least one conversation in **Conversations**.
- You can access **Data Model** and **Evals** in the same project.
- If you do not yet have this workspace access, complete Chapter 2 setup sections first.

## In Gaia

- [AI Agents](../../user-guide/agents/README.md) and [Agent Configuration](../../user-guide/agents/configs.md) for prompt, tool, and handoff boundaries
- [Conversations](../../user-guide/conversations/README.md) for live behavior inspection
- [Data Model](../../user-guide/data-model/README.md) for grounding and workflow substrate
- [Evals](../../user-guide/evals/README.md) for measurable quality evidence

Read this section while keeping one live project open. The concepts here are easiest to absorb when you can point to the actual Gaia surfaces they describe.

## Concept brief

AI systems fail less often because of model intelligence limits and more often because of weak system design around the model. Gaia is designed to make that system explicit. This section introduces the concepts that separate "interesting replies" from production-capable behavior.

A useful framing is:

- The model is a **reasoning engine**.
- Gaia is the **execution and control system** around that engine.

Your job as an AI Engineer is to shape the combined system, not just the model prompt.

### 1) Probabilistic core, deterministic shell

Large language models are probabilistic. Given similar inputs, they can produce different outputs. Production systems cannot rely on hope. They need deterministic boundaries.

In Gaia, deterministic controls include:

- Allowed tool set per configuration
- Entity/data boundaries
- Handoff rules
- On-topic and safety checks
- Evaluation gates and operational thresholds

Think of this as a shell around the model. The shell defines what must remain stable even when language behavior is flexible.

### 2) Capability surface vs conversation style

Teams often optimize style first: tone, fluency, and personality. Those matter, but they are secondary to capability design.

Capability design answers:

- What tasks the assistant can complete
- What data it can use
- What actions it can trigger
- What it must refuse or escalate

Conversation style answers:

- How the assistant communicates while performing those tasks

If capability is weak, polished style creates false confidence. In Gaia, stabilize capability first, then tune style.

### 3) Context architecture

Most AI behavior quality comes from context architecture, not model brand choice alone.

Context architecture in Gaia includes:

- System prompt fragments and variables
- Conversation history and turn state
- Retrieved entity/search results
- Tool outputs and post-step artifacts
- Optional memory signals

Poor context architecture leads to classic errors: contradictory replies, stale references, or unsupported claims.

Design rule:

- Keep context minimal but sufficient.
- Include only the state needed for the current decision.
- Prefer explicit retrieved evidence over broad hidden assumptions.

### 4) Tooling as controlled extension

Tools turn an assistant from "text generator" into a workflow participant. They also expand risk.

Good tooling design requires:

- Clear input/output contracts
- Narrow side-effect scope
- Predictable error handling
- Logging and traceability

In Gaia, tools should be enabled only when they are required for the target scope. Over-enabling tools increases failure surface and complicates evaluation.

A practical boundary model:

- Read tools first
- Write tools later
- Destructive actions last, with explicit safeguards

### 5) Grounding and evidence freshness

A production assistant must be grounded in reliable information, not plausible language.

Grounding means:

- Data comes from approved sources
- Claims can be traced back to known records or tool outputs
- Updates are reflected in future responses with acceptable freshness

In Gaia terms, this ties directly to:

- Data Model correctness
- Storage and source-data quality
- Tool behavior that fetches current state
- Config choices that avoid stale assumptions

A grounded assistant may still be imperfect, but its failures are diagnosable and correctable.

### 6) Evaluation is part of design, not a final step

Without evals, teams optimize to anecdotes. With evals, teams optimize to evidence.

Core evaluation concepts:

- Task sets represent expected and edge-case behavior
- Graders encode success criteria
- Runs reveal reliability across trials
- Reports help compare changes across versions

On Gaia, this moves quality from subjective debate to measurable decisions. Evaluation does not replace judgment. It structures it.

### 7) Operational budgets: latency, cost, reliability

Even accurate assistants fail in production when they are too slow, too expensive, or too fragile.

You need explicit budgets for:

- Turn latency
- Tool execution behavior
- Token/cost usage
- Error rates and recoverability

The right target is not maximum intelligence per reply. It is acceptable quality under real constraints.

### 8) Failure taxonomy for faster debugging

When behavior fails, categorize first, then fix. A simple taxonomy avoids random edits.

Useful categories:

- **Scope failure**: doing tasks it should not do
- **Context failure**: missing or conflicting instructions/state
- **Data failure**: wrong, stale, or unavailable source data
- **Tool failure**: tool call misuse or tool/runtime errors
- **Evaluation failure**: no measurable quality gate
- **Operations failure**: no alerting, rollback trigger, or control limit

This taxonomy maps cleanly to Gaia workspaces and accelerates root-cause analysis.

### 9) Concept-to-Gaia mapping

Use this mapping when deciding where to work next:

- **Behavior definition** -> AI Agents / Agent Configuration
- **Data grounding** -> Data Model (Entities, Storage, Pipelines, Workflows)
- **Real interaction checks** -> Conversations
- **Quality measurement** -> Evals
- **Access/governance boundaries** -> Settings / Project roles / Audit

If you cannot map a concept to a Gaia surface, the implementation path is not ready yet.

### 10) Practical mental model

Use this compact model in daily work:

1. Define the behavior contract.
2. Define the data and tool boundaries.
3. Run representative conversations.
4. Measure with evals.
5. Operate with explicit budgets and rollback triggers.

Repeat this loop until the assistant is predictable enough for your risk profile.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab turns the concepts above into a concrete system check for your scoped assistant.

### Scenario

Continue with your Chapter 1 scenario (for example, Renewal Operations Assistant).

### Phase A: Build a concept map for your project

1. Open your scope artifact from Section 1.
2. Create a "Core concepts map" with five rows:
   - Context architecture
   - Tool boundaries
   - Data grounding
   - Evaluation strategy
   - Operational budgets
3. For each row, write the exact Gaia location you will use (page/tab) and one measurable signal.

Checkpoint A:

- You have a concept map where each concept is tied to at least one Gaia surface and one measurable signal.

### Phase B: Validate behavior path in AI Agents

4. Open **AI Agents** and choose the configuration you mapped in Section 1.
5. Verify that the current tool set matches your scoped tasks (remove any obvious non-essential tools from your working notes if present).
6. Review instruction fragments and confirm they include explicit boundaries (what to do and what to refuse).
7. Confirm handoff or escalation behavior is defined if your scenario needs specialist routing.

Checkpoint B:

- You can explain why each enabled capability exists and how it supports in-scope behavior.

### Phase C: Validate data grounding path in Data Model

8. Open **Data Model** and list the minimum entities/data sources used for your scenario.
9. For each source, answer:
   - Is it authoritative?
   - How fresh is it?
   - How can the assistant reference or fetch it?
10. Identify one stale-data risk and one missing-data risk.

Checkpoint C:

- You have explicit grounding assumptions and at least two documented data risks.

### Phase D: Run concept-driven conversation tests

11. Open **Conversations** and run four prompts:

- Standard in-scope query
- Ambiguous query requiring clarification
- Out-of-scope query requiring refusal/escalation
- Query that should force tool/data grounding

12. For each turn, record:

- Whether behavior stayed within scope
- Whether a tool/data path was used appropriately
- Whether response quality matched your expectations

Checkpoint D:

- You have four categorized transcript samples and a short diagnosis per sample.

### Phase E: Create an initial eval baseline

13. Open **Evals** and define one small dataset theme aligned to your scenario.
14. Add at least five candidate task ideas covering:

- nominal behavior
- ambiguity handling
- out-of-scope behavior
- grounding-critical behavior
- one edge case

15. Define one pass criterion and one fail criterion you can apply consistently.

Checkpoint E:

- You have a first-pass eval baseline that can detect at least one regression class.

### Phase F: Define operational budgets

16. In your scope artifact, add explicit initial budgets for:

- max acceptable latency range
- acceptable tool failure rate
- acceptable cost range per turn class (rough)

17. Add two rollback triggers tied to these budgets.

Checkpoint F:

- You can state exactly when rollout should pause and what signal triggers that decision.

## Expected outputs

By the end of this lab, you should produce:

- A concept map linking five core AI system concepts to Gaia surfaces and measurable signals.
- A reviewed capability snapshot for one active agent configuration.
- A grounding risk note with at least one stale-data and one missing-data risk.
- Four labeled conversation test cases with diagnoses.
- A starter eval baseline (at least five task ideas plus clear pass/fail criteria).
- A first operational budget note with at least two rollback triggers.

Evidence that qualifies:

- A markdown note or artifact containing the concept map and budgets.
- Conversation samples or summaries tied to the four test classes.
- Eval draft entries that another engineer can execute without reinterpretation.

## Failure modes

1. **Model-centric thinking without system controls**
   - Symptom: Team keeps changing models/prompts but quality remains unstable.
   - Recovery: Re-anchor on boundaries, tool contracts, and eval criteria before further model tuning.

2. **Context bloat**
   - Symptom: Replies become inconsistent, slow, or noisy as prompts and state grow.
   - Recovery: Reduce context to decision-relevant inputs and remove redundant instructions.

3. **Unbounded tool surface**
   - Symptom: Assistant calls unnecessary tools or takes risky actions unexpectedly.
   - Recovery: Restrict enabled tools to minimum required scope and retest out-of-scope behavior.

4. **Weak grounding discipline**
   - Symptom: Confident replies are not traceable to approved data.
   - Recovery: Enforce source boundaries and add eval cases that fail unsupported claims.

5. **No ambiguity strategy**
   - Symptom: Assistant fabricates specifics instead of asking clarifying questions.
   - Recovery: Add explicit clarification behavior and validate with ambiguity-focused test prompts.

6. **Evaluation as an afterthought**
   - Symptom: Team cannot decide whether recent changes improved or degraded behavior.
   - Recovery: Build a small, repeatable eval baseline immediately and run before major changes.

7. **No operational budget**
   - Symptom: System appears correct but becomes impractical under latency/cost constraints.
   - Recovery: Define budget thresholds now and monitor against rollout triggers.

8. **Diagnosis by guesswork**
   - Symptom: Fixes are random and regressions recur.
   - Recovery: Classify each failure by taxonomy (scope/context/data/tool/eval/ops) before patching.

## Completion checklist

- [ ] I can explain probabilistic model behavior vs deterministic system controls in Gaia.
- [ ] I mapped core concepts (context, tools, grounding, evals, operations) to explicit Gaia surfaces.
- [ ] I validated one agent configuration against capability boundaries and documented why enabled components exist.
- [ ] I documented at least two grounding risks (stale data and missing data).
- [ ] I executed four concept-driven conversation tests and captured diagnoses.
- [ ] I drafted an eval baseline with at least five task ideas and explicit pass/fail criteria.
- [ ] I defined initial latency/cost/reliability budgets and at least two rollback triggers.
- [ ] My artifacts are clear enough for another engineer to reproduce the same assessment.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Timeline dialog](../../user-guide/conversations/dialogs/timeline.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Entities](../../user-guide/data-model/entities.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval design process](../../user-guide/evals/eval-design-process.md)
- [Run details](../../user-guide/evals/run-details.md)
- [Project roles](../../user-guide/settings/project-roles.md)

---

# Gaia Platform Mental Model

## Learning objectives

By the end of this section, you should be able to:

- Describe Gaia as an integrated system of capabilities rather than a set of disconnected screens.
- Trace one user request end-to-end across channel, conversation runtime, tools/data, and evaluation surfaces.
- Identify where to make a change depending on whether the issue is behavior, data, UX, quality, or governance.
- Execute a reproducible platform-mapping lab that produces a project-specific operating diagram and validation evidence.

## Prerequisites

- You completed [Role and Scope](#doc-ch01-ai-engineer-foundations-01-role-and-scope) and [Core AI Systems Concepts](#doc-ch01-ai-engineer-foundations-02-core-ai-systems-concepts).
- You can open a project in Gaia and access Conversations, AI Agents, Data Model, and Evals.
- You can run a conversation turn and inspect the configuration used.
- If you do not yet have a project/workspace, complete Chapter 2 setup sections first.

## In Gaia

- [Conversations](../../user-guide/conversations/README.md) and [Channels](../../user-guide/conversations/channels/README.md) for entry and interaction context
- [AI Agents](../../user-guide/agents/README.md) for orchestration and behavioral control
- [Data Model](../../user-guide/data-model/README.md) for execution substrate and workflow state
- [Evals](../../user-guide/evals/README.md), [Delivery Management](../../user-guide/delivery/README.md), and [Audit Trail](../../user-guide/audit/README.md) for evidence, change control, and operating follow-through

Walk one real user request across these surfaces while you read. That turns the mental model from an abstraction into a practical debugging map.

## Concept brief

New Gaia users often understand each feature page in isolation but still struggle to ship a reliable AI capability. The gap is usually not missing functionality. It is missing **mental model**.

A mental model is your internal map of how the platform behaves as one system when a real user request enters, is processed, and produces an outcome.

### 1) Think in flow, not pages

Gaia provides many powerful surfaces. But production behavior emerges from **flow between surfaces**, not from any one page.

A minimal flow for most projects:

1. User enters through a channel or internal conversation interface.
2. Conversation runtime applies active configuration and guardrails.
3. Tools and data surfaces are invoked when needed.
4. Response is returned to user in channel-appropriate form.
5. Quality is measured in evals and operational logs.
6. Team iterates configurations and data/workflow design.

If you think in pages, you tune fragments. If you think in flow, you tune outcomes.

### 2) Five-layer Gaia operating model

Use this five-layer model whenever you design, debug, or review changes.

#### Layer A: Entry and intent capture

Where it lives:

- Channels
- Conversations UI
- End-user app surfaces

What it does:

- Captures user request and context
- Defines interaction constraints (channel format, input type)
- Starts the turn with project and configuration context

Typical failure here:

- Wrong channel assumptions, poor prompt framing from users, or missing interaction constraints.

#### Layer B: Behavior orchestration

Where it lives:

- AI Agents
- Agent Configuration (prompts, tools, handoffs, settings)

What it does:

- Determines how the assistant reasons
- Applies prompt policy and behavior boundaries
- Selects and executes tools
- Routes requests via handoff rules when needed

Typical failure here:

- Misaligned instructions, over-broad tool access, or ambiguous handoff conditions.

#### Layer C: Data and execution substrate

Where it lives:

- Data Model (Entities, Storage, Pipelines, Workflows, Runs)
- Tool Registry / Skill Registry

What it does:

- Provides authoritative data and transformation paths
- Supplies reusable execution primitives (tools, skills)
- Maintains operational data freshness

Typical failure here:

- Stale/incomplete data, weak entity design, or brittle workflow execution.

#### Layer D: Evidence and quality control

Where it lives:

- Evals (Datasets, Runs, Graders, Reports)
- Conversation review practices

What it does:

- Converts behavior expectations into measurable checks
- Detects regressions and policy violations
- Supports promotion decisions with evidence

Typical failure here:

- No baseline, inconsistent pass/fail logic, or over-reliance on anecdotal testing.

#### Layer E: Governance and operations

Where it lives:

- Settings (roles/access)
- Audit-related controls
- Delivery/task workflows

What it does:

- Controls who can change what
- Provides traceability for critical actions
- Aligns release and rollback behavior with team process

Typical failure here:

- Unclear ownership, over-permissioned changes, and no clear rollback trigger.

### 3) The "single request" thought experiment

To test your platform mental model, run this exercise:

"A user asks: 'Can you summarize open renewal risks for account A and suggest next actions?'"

You should be able to answer, without guessing:

- Entry: where the request came from and which conversation context applies.
- Behavior: which agent/config is active and what boundaries govern response.
- Data: which entities/workflows/tools are required for accurate grounding.
- Evidence: how success would be measured for this type of request.
- Governance: who is allowed to change behavior if result quality is unacceptable.

If you cannot answer one of these, your mental model is incomplete.

### 4) Control points vs observation points

A strong operator distinguishes where they **change behavior** from where they **observe behavior**.

Control points (you can modify behavior):

- Agent configuration (instructions, tools, settings)
- Data Model definitions and workflows
- Eval definitions and grading criteria
- Role/access and process controls

Observation points (you measure/inspect behavior):

- Conversation traces and outcomes
- Eval run results and reports
- Workflow run records
- Operational incidents and support feedback

Teams get stuck when they confuse these. Example: trying to "observe quality" by editing prompts repeatedly, or trying to "fix behavior" only by reading traces without changing any control point.

### 5) Change routing logic

When something goes wrong, route changes by failure type instead of instinct.

- If the assistant says the wrong thing with correct data available -> start in Agent Configuration.
- If the assistant cannot retrieve needed facts -> start in Data Model or tool wiring.
- If users misunderstand outputs -> start in channel/UI and interaction design.
- If you cannot prove improvement -> start in Evals design.
- If changes keep breaking production unexpectedly -> start in governance/operations controls.

This routing discipline reduces random edits and shortens debug loops.

### 6) Coupling awareness

Treat AI Agents, Data Model, and Evals as a tightly coupled triad. Any major change in one should trigger a review in the other two.

### 7) Progressive reliability mindset

Do not attempt full-system perfection in one pass. Build reliability progressively.

Suggested maturity loop:

1. Establish a narrow, testable capability slice.
2. Validate end-to-end flow for that slice.
3. Add eval coverage for nominal + edge behavior.
4. Add operational budgets and rollback conditions.
5. Expand capability surface only after stable evidence.

This is how a student or junior engineer can ship meaningful systems without overextending complexity too early.

### 8) Practical model for daily work

Use this daily checklist before making changes:

- What layer am I changing?
- What adjacent layer could this break?
- What evidence will prove this helped?
- What rollback condition protects users if it did not?

If any answer is missing, pause and define it first.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a concrete "platform mental model" artifact for your current project.

### Scenario

Continue with your Chapter 1 scenario and current scoped assistant capability.

### Phase A: Build your five-layer map

1. Create a one-page map with five sections:
   - Entry and intent capture
   - Behavior orchestration
   - Data and execution substrate
   - Evidence and quality control
   - Governance and operations
2. Under each section, list the exact Gaia pages/tabs you will use.
3. For each section, add one primary risk and one observable signal.

Checkpoint A:

- Your map covers all five layers with concrete Gaia surfaces and measurable signals.

### Phase B: Trace one request end-to-end

4. Choose one representative user request from your scope.
5. In **Conversations**, run that request and document:
   - active agent/configuration
   - whether tools/data were needed
   - whether boundaries were respected
6. Map each part of the turn back to your five-layer model.

Checkpoint B:

- You can narrate one complete request path without leaving unexplained gaps.

### Phase C: Validate control and observation points

7. For the same request path, identify at least:
   - two control points you could modify,
   - two observation points you would inspect first.
8. Write one example change and one example metric/trace for each point.

Checkpoint C:

- Control and observation points are explicit and non-overlapping.

### Phase D: Run failure-routing drills

9. Simulate three failures from this list (or similar):
   - wrong answer with apparently correct data available,
   - missing required business fact,
   - inconsistent quality across similar prompts.
10. For each, document where you would start:

- AI Agents
- Data Model/tooling
- Evals
- channel/UX
- governance/ops

11. Add a one-line reason for each route decision.

Checkpoint D:

- You can route failures systematically instead of editing prompts by default.

### Phase E: Add coupling checks

12. Pick one planned change (for example, add a tool or modify prompt boundaries).
13. List cross-impact checks in the other two coupled areas:

- If changing AI Agents, what Data Model/Evals checks are required?
- If changing Data Model, what Agent/Evals checks are required?
- If changing Evals, what Agent/Data assumptions must be revalidated?

Checkpoint E:

- Every planned change includes at least two cross-layer checks.

### Phase F: Operationalize the model

14. Add a short "mental model playbook" note to your project docs with:

- your five-layer map
- failure routing table
- coupling checks
- rollback trigger summary

15. Share it with one teammate and ask them to trace one request using your map.

Checkpoint F:

- Another engineer can use your artifact to reason about the same system path with minimal clarification.

## Expected outputs

By the end of this lab, you should have:

- A five-layer Gaia platform map specific to your project.
- One end-to-end request trace mapped across all layers.
- A control-vs-observation matrix for your assistant capability.
- A failure-routing table covering at least three realistic issues.
- A short team-facing playbook note with rollback-trigger summary.

Evidence that qualifies:

- A markdown or document artifact containing the map, routing table, and coupling checks.
- Conversation sample references used for end-to-end tracing.
- Clear route decisions that another engineer can execute.

## Failure modes

1. **Page-by-page thinking**
   - Symptom: Team optimizes one screen while overall behavior quality degrades.
   - Recovery: Return to the five-layer flow and validate cross-layer impact before changes.

2. **No end-to-end tracing discipline**
   - Symptom: Problems are described vaguely ("assistant is weird") with no request path evidence.
   - Recovery: Require one concrete request trace for each reported quality issue.

3. **Prompt-first debugging by default**
   - Symptom: Teams repeatedly tweak instructions for failures caused by data/tooling gaps.
   - Recovery: Use failure-routing logic to select the right starting surface.

4. **Unclear control vs observation boundaries**
   - Symptom: Teams collect metrics but do not know where to intervene, or edit configs without measurable feedback.
   - Recovery: Document control and observation points explicitly per capability.

5. **Coupling blind spots**
   - Symptom: A "small" change causes downstream regressions in evals or data behaviors.
   - Recovery: Enforce coupled triad checks (AI Agents, Data Model, Evals) for major changes.

6. **No ownership path for governance decisions**
   - Symptom: Disputes over who can approve behavior changes or rollback decisions.
   - Recovery: Add role-based ownership rules and escalation paths to the playbook.

7. **Mental model exists only in one person’s head**
   - Symptom: Progress stalls when one engineer is unavailable.
   - Recovery: Publish the model artifact and test peer reproducibility.

8. **No rollback-aware change planning**
   - Symptom: Teams deploy changes without pre-defined stop conditions.
   - Recovery: Attach rollback-trigger checks to every significant behavior/data change.

## Completion checklist

- [ ] I documented a five-layer Gaia platform model for my project.
- [ ] I traced at least one real request across all five layers.
- [ ] I identified clear control points and observation points for that request path.
- [ ] I created a failure-routing table with at least three realistic issues and route decisions.
- [ ] I defined coupling checks for AI Agents, Data Model, and Evals before major changes.
- [ ] I captured rollback-aware operational notes in a shareable playbook artifact.
- [ ] A teammate can follow my artifact and reproduce the same reasoning path.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [Gaia](../../user-guide/platform-assistant.md)
- [Conversations](../../user-guide/conversations/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Entities](../../user-guide/data-model/entities.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval design process](../../user-guide/evals/eval-design-process.md)
- [Settings and project roles](../../user-guide/settings/README.md)

---

# Learning Path and Capstone

## Learning objectives

By the end of this section, you should be able to:

- Translate the handbook into a realistic execution plan with milestones, evidence gates, and review checkpoints.
- Define a capstone scope that is ambitious enough to demonstrate end-to-end Gaia engineering skills but constrained enough to finish.
- Build a chapter-by-chapter learning backlog that links each future chapter to concrete capstone outcomes.
- Execute a reproducible planning lab that results in a capstone charter, success rubric, and rollout/rollback plan.

## Prerequisites

- You completed [Role and Scope](#doc-ch01-ai-engineer-foundations-01-role-and-scope), [Core AI Systems Concepts](#doc-ch01-ai-engineer-foundations-02-core-ai-systems-concepts), and [Gaia Platform Mental Model](#doc-ch01-ai-engineer-foundations-03-gaia-platform-mental-model).
- You have a Gaia project where you can test agent, data model, and eval changes.
- You can run at least one conversation scenario and record outcomes.
- If you do not yet have a project workspace, complete Chapter 2 setup sections first.

## In Gaia

- [Delivery Management](../../user-guide/delivery/README.md) and [Tasks](../../user-guide/tasks/README.md) for the learning backlog and milestone logic
- [AI Agents](../../user-guide/agents/README.md), [Data Model](../../user-guide/data-model/README.md), and [Evals](../../user-guide/evals/README.md) for mapping future chapter outputs to concrete platform work
- [Governance](../../user-guide/governance/README.md) and [Dashboard](../../user-guide/dashboard/README.md) when your capstone needs release evidence and operating review

Treat the capstone plan as a real Gaia delivery program. If a future chapter output cannot be represented as project work, evidence, or a platform artifact, tighten the plan before moving on.

## Concept brief

Most learners fail to become production-capable AI engineers not because they lack intelligence, but because they follow an unstructured path. They read concepts, try random experiments, and collect disconnected skills. A capstone solves this by forcing integration: you must combine scope, system design, implementation, evaluation, and operations into one coherent delivery.

This section gives you the bridge from "learning topics" to "shipping capability."

### 1) The handbook is a delivery program, not a reading list

Treat the 11 chapters as an execution program with increasing responsibility:

- Chapters 1–3 establish fundamentals and system grounding.
- Chapters 4–6 establish behavior design and quality discipline.
- Chapters 7–9 establish operational and delivery discipline.
- Chapters 10–11 establish production playbooks and end-to-end demonstration.

If you only read chapters and skip artifacts, you gain vocabulary but not delivery confidence. If you produce artifacts at each stage, you build reusable engineering muscle.

### 2) Capstone-first planning principle

A strong learning path starts with the final demonstration in mind.

Define the capstone first at a high level:

- Who uses the assistant?
- What problem does it solve?
- What must be demonstrably true at the end?

Then map chapter work backward from capstone requirements. This keeps learning focused and prevents over-investing in low-impact details.

### 3) What a strong Gaia capstone must prove

A credible Gaia AI Engineer capstone should prove competence in five dimensions:

1. **Behavior design**: agent configuration and boundaries are explicit and testable.
2. **Data grounding**: entity/data model supports assistant decisions with traceable sources.
3. **Quality assurance**: evals exist and influence release decisions.
4. **Operational readiness**: latency/cost/reliability budgets and rollback triggers are defined.
5. **Delivery clarity**: artifacts, rationale, and change history are understandable by other engineers.

Any capstone missing one of these dimensions is usually demo-level, not production-level.

### 4) Scope sizing rules for success

A common failure is capstone scope inflation. Use these sizing rules:

- One primary user persona for v1.
- One primary assistant workflow plus one secondary workflow.
- Limited action surface (prefer read-heavy flows first).
- Clear out-of-scope list.
- Clear success metrics with thresholds.

Examples of healthy capstone scope:

- "Renewal Operations Assistant" for internal account teams.
- "Support Triage Assistant" that classifies and drafts response proposals.
- "Policy Lookup Assistant" with strict source grounding and refusal boundaries.

Examples of unhealthy capstone scope:

- "General company assistant for everything."
- Multi-persona, multi-channel, high-autonomy action assistant in first attempt.

### 5) Chapter-to-capstone dependency map

Use this dependency map as your build order logic.

- **Chapter 1**: role/scope contract + mental model + capstone charter draft.
- **Chapter 2**: environment readiness + first running project baseline.
- **Chapter 3**: data structures and source-entry paths that capstone relies on.
- **Chapter 4**: agent behavior and tool boundaries for capstone workflows.
- **Chapter 5**: channel and UX shape for target users.
- **Chapter 6**: measurable quality baselines and regression checks.
- **Chapter 7**: observability and budget controls.
- **Chapter 8**: governance and safety posture.
- **Chapter 9**: delivery process and release readiness mechanics.
- **Chapter 10**: incident and operational playbooks.
- **Chapter 11**: integrated demonstration with evidence package.

Each chapter should produce at least one capstone input artifact.

### 6) Evidence-driven learning loop

Use this loop for each chapter:

1. Define what artifact the chapter should produce.
2. Implement the smallest viable version.
3. Validate with a checklist and measurable signals.
4. Record gaps and feed them to the next chapter backlog.

This transforms the handbook from passive learning to active systems building.

### 7) Skill progression expectations

Your progression should look like this:

- Early chapters: clarity and system mapping.
- Mid chapters: implementation and quality control.
- Late chapters: operational reliability and change management.

Do not expect polished production outcomes in Chapter 2. Do expect higher rigor by Chapters 6–9. The objective is disciplined progression, not instant perfection.

### 8) Capstone acceptance rubric

Use a simple rubric to decide readiness:

- **Scope clarity**: in-scope/out-of-scope and ownership are explicit.
- **Behavior reliability**: representative scenarios behave predictably.
- **Grounding quality**: claims map to approved sources.
- **Eval discipline**: pass/fail criteria and run evidence are available.
- **Ops readiness**: budgets and rollback triggers are documented.
- **Delivery quality**: another engineer can review and reproduce your path.

Score each dimension 0–2:

- 0 = absent
- 1 = partial
- 2 = production-ready baseline

Target score for capstone sign-off: at least 10/12 with no zero in critical dimensions (grounding, evals, ops readiness).

### 9) Common planning mistakes

- Planning too much implementation before defining quality criteria.
- Defining quality criteria without operational thresholds.
- Treating chapter completion as reading completion instead of artifact completion.
- Deferring evaluation setup until late stages.
- Avoiding rollback planning because "we are still learning."

Your capstone should be designed to tolerate learning mistakes while still preserving responsible system behavior.

### 10) Operating cadence recommendation

Use a stable weekly cadence (adapt to your schedule):

- Session 1: chapter implementation artifact work.
- Session 2: validation and issue triage.
- Session 3: chapter write-up and quality evidence update.

At the end of each chapter:

- update chapter-level artifact index,
- record unresolved risks,
- define next chapter entry criteria.

This cadence reduces context switching and keeps momentum.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab turns the handbook into your personal delivery program and creates a capstone charter that can drive Chapters 2–11.

### Scenario

Use your existing assistant scenario from Sections 1–3. If needed, refine it to meet healthy scope rules.

### Phase A: Create a capstone charter

1. Create a new markdown note named "Capstone Charter" in your working docs.
2. Add these sections:
   - Problem statement
   - Primary user persona
   - In-scope workflows (max 2 for v1)
   - Out-of-scope workflows
   - Critical failure classes
   - Success metrics (initial targets)
3. Ensure each section contains concrete statements, not generic goals.

Checkpoint A:

- You have a single-page charter that can be reviewed by another engineer.

### Phase B: Map chapter outputs to capstone needs

4. Create a table with columns:
   - Chapter
   - Capstone dependency
   - Planned artifact
   - Completion signal
5. Fill rows for Chapters 2–11.
6. For each row, define at least one measurable completion signal.

Checkpoint B:

- Every future chapter has a clear capstone contribution and completion signal.

### Phase C: Define quality and release gates

7. Add a "Quality gates" section to the charter with:
   - minimum eval pass criteria,
   - unacceptable failure classes,
   - required review artifacts before rollout.
8. Add a "Release gate" section specifying what must be true for capstone demo readiness.

Checkpoint C:

- You can decide go/no-go using defined gates instead of subjective judgment.

### Phase D: Define operations and rollback plan

9. Add an "Operational budgets" section with initial targets for:
   - latency range,
   - tool error tolerance,
   - cost range per turn class.
10. Add at least two rollback triggers linked to those targets.
11. Add owner/escalation roles for rollback decisions.

Checkpoint D:

- Rollback is preplanned, with trigger signals and accountable owners.

### Phase E: Create chapter 2 entry criteria

12. Add "Chapter 2 entry criteria" to your charter:

- required setup prerequisites,
- baseline project state,
- first workflow definition.

13. Add a small risk register with at least 3 initial risks and mitigations.

Checkpoint E:

- You can begin Chapter 2 with explicit entry conditions and known risks.

### Phase F: Peer reproducibility check

14. Share your charter with one teammate (or review it as if you are a new contributor).
15. Ask: "Could this person run Chapters 2–11 against this same capstone without asking for hidden context?"
16. Revise ambiguous sections until the answer is yes.

Checkpoint F:

- The charter is explicit enough to support team execution and review.

## Expected outputs

By the end of this lab, you should have:

- A capstone charter with scope, boundaries, metrics, and failure classes.
- A chapter-to-capstone dependency table for Chapters 2–11.
- A quality and release gate definition for demo readiness.
- An operations and rollback plan with owner mapping.
- Chapter 2 entry criteria and initial risk register.

Evidence that qualifies:

- One shareable markdown charter artifact.
- A table mapping each chapter to a measurable capstone output.
- Explicit go/no-go and rollback criteria.

## Failure modes

1. **Capstone defined as "build something useful"**
   - Symptom: Team cannot align on what done means.
   - Recovery: Replace broad goals with explicit workflows, boundaries, and thresholds.

2. **Scope creep disguised as ambition**
   - Symptom: New workflows are added each week without reprioritization.
   - Recovery: Freeze v1 scope and move extras to post-capstone backlog.

3. **No measurable chapter outputs**
   - Symptom: Chapters feel complete but no artifacts support progress.
   - Recovery: Require one artifact and one measurable signal per chapter.

4. **Eval and operations deferred too late**
   - Symptom: Quality debates appear near the end with no baseline evidence.
   - Recovery: Define quality and rollback gates now, then refine later.

5. **Capstone depends on unclear ownership**
   - Symptom: Important decisions stall because accountability is ambiguous.
   - Recovery: Assign explicit owners for scope, quality approval, and rollback decisions.

6. **Planning only for happy-path demo**
   - Symptom: System appears strong in rehearsed cases but breaks on realistic edge cases.
   - Recovery: Include ambiguity, refusal, and failure drills in chapter artifacts.

7. **Artifacts not reproducible by others**
   - Symptom: Progress exists only in one person’s memory.
   - Recovery: Enforce peer-readability checks and remove hidden assumptions.

8. **No risk register**
   - Symptom: Known constraints surprise the team repeatedly.
   - Recovery: Maintain a living risk list with mitigation and owner fields.

## Completion checklist

- [ ] I created a capstone charter with explicit scope, out-of-scope boundaries, and critical failure classes.
- [ ] I mapped Chapters 2–11 to concrete capstone artifacts and measurable completion signals.
- [ ] I defined quality and release gates that support objective go/no-go decisions.
- [ ] I documented latency/cost/error budgets and at least two rollback triggers.
- [ ] I assigned ownership and escalation expectations for critical capstone decisions.
- [ ] I created Chapter 2 entry criteria and a starter risk register.
- [ ] I validated that another engineer could execute the plan without hidden context.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval design process](../../user-guide/evals/eval-design-process.md)
- [Settings and project roles](../../user-guide/settings/README.md)
- [Gaia](../../user-guide/platform-assistant.md)

---

# Chapter 2: Gaia Setup and First Project

Status: current

Setup and first runnable workspace

## Sections

- [Access And Roles](#doc-ch02-gaia-setup-and-first-project-01-access-and-roles)
- [Create Team And Project](#doc-ch02-gaia-setup-and-first-project-02-create-team-and-project)
- [First Agent First Config](#doc-ch02-gaia-setup-and-first-project-03-first-agent-first-config)
- [First Conversation Checkpoint](#doc-ch02-gaia-setup-and-first-project-04-first-conversation-checkpoint)

## Alignment with User Guide Setup Surfaces

- [Getting started](../../user-guide/README.md)
- [Cross-instance project transfer](../../user-guide/settings/README.md#cross-instance-project-transfer)
- [Discussions](../../user-guide/discuss/README.md)
- [Tutorials](../../user-guide/tutorials/README.md)
- [Settings](../../user-guide/settings/README.md)

This chapter now follows the same spine as the canonical landing guide: Teams and projects first, then project settings and transfer/import decisions, then the first runnable path into agents and conversations.

The signed-in landing page for this chapter is `/platform/user/teams`.

Use [Tutorials](../../user-guide/tutorials/README.md) at `/platform/support/tutorials` as an optional accelerator only. When a walkthrough is still catching up to product behavior or media ownership, use the owning user-guide page above as the canonical prerequisite.

## Fast path inside Gaia

1. Start in [Getting started](../../user-guide/README.md) and complete the team and project selection flow.
2. Use [Settings](../../user-guide/settings/README.md) when you need to inspect transfer, import, roles, or project-level setup.
3. Move into [AI Agents](../../user-guide/agents/README.md) and [Conversations](../../user-guide/conversations/README.md) only after the project shell is real and accessible.

This chapter is complete only when the setup exists in a live Gaia workspace, not when the setup logic merely makes sense on paper.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed, including the Teams landing guide and the transfer/import flow

---

# Access and Roles

## Learning objectives

By the end of this section, you should be able to:

- Explain how access boundaries and role permissions shape what an AI Engineer can safely build and operate in Gaia.
- Validate project/team access before implementation work starts, so you do not discover permission blockers mid-delivery.
- Design a minimum viable role setup for a small AI delivery team with clear separation of responsibilities.
- Execute a reproducible access-and-roles readiness lab and produce evidence that your project can support Chapters 2–4 without governance ambiguity.

## Prerequisites

- You completed Chapter 1 and have a capstone charter with clear scope and ownership assumptions.
- You can sign in to Gaia and access the Home/Teams view.
- You have at least one project available for setup work.
- You can coordinate with a project or team admin if permission changes are needed.

## In Gaia

- [Getting started](../../user-guide/README.md) for the Teams entry path
- [Settings](../../user-guide/settings/README.md), [Platform users and roles](../../user-guide/settings/platform-users.md), [Project users](../../user-guide/settings/project-users.md), and [Project roles](../../user-guide/settings/project-roles.md) for access configuration
- [Audit Trail](../../user-guide/audit/README.md) for proof that critical access changes happened as intended

Do the access design work in Gaia before deeper implementation. This section should end with a usable role boundary, not just a role discussion.

## Concept brief

Access and roles are often treated as administrative details. In practice, they are engineering constraints that determine whether your system design is implementable, auditable, and safe.

For AI engineering on Gaia, permission design matters for three reasons:

1. **Execution safety**: only the right people should change behavior-critical settings.
2. **Delivery speed**: missing permissions can block development at the worst moment.
3. **Operational accountability**: role boundaries make ownership explicit when quality degrades or incidents occur.

If you get roles wrong, even excellent technical work becomes fragile. If you get roles right, teams move faster with less risk.

One practical example: a user with project access should land in the first platform workspace their role can actually use, while an app-only user or signed-out end user should be routed to the relevant app login entrypoint instead of the internal platform shell or an automatic guest session. Entry routing is part of access design, not a cosmetic detail.

### 1) Access is part of system design

Traditional teams separate "technical design" and "access configuration." For production AI systems, this separation creates blind spots.

Examples:

- You design a robust evaluation workflow, but no one in the QA role can start or review runs.
- You define strict configuration controls, but too many users can still modify active agent settings.
- You rely on data workflows, but the engineer responsible cannot view run diagnostics.

In each case, the system appears designed on paper but is non-operational in practice.

Access planning should happen before deep implementation, not after.

### 2) Gaia role reality: visibility and authority

In Gaia, role settings influence what users can see and do across critical surfaces such as AI Agents, Data Model, and Evals.

At the platform level, Gaia now also distinguishes central operator lifecycle and delegated platform capabilities through the **Users** page. That means platform admins can suspend or archive operators without deleting historical ownership links, and they can see when a lifecycle change would strand a team, project, or delegated platform responsibility.

The same **Users** page is also the fastest place to troubleshoot identity-to-role mismatches. For a selected user, platform admins can review direct organization, team, project-user, and app-channel assignments, then change or remove the listed role. This matters when a person can sign in to Gaia but cannot reach a project app channel, because app-channel access is intentionally separate from internal project-user access.

When two platform user rows represent the same person with different email casing, merge the duplicate into the canonical user instead of only archiving it. The merge preserves the canonical identity, moves login and direct access records, and prevents future case-insensitive login lookup from resolving to the archived duplicate.

Delegated platform roles can now separate view authority from mutation authority across platform-wide surfaces such as **Platform Dashboard**, **Models**, **Settings**, **Tasks**, **Access Requests**, and discussion moderation. Use that split intentionally: broad read access helps operators monitor the estate, while narrower manage access limits who can change cross-project behavior.

Treat the **Models** mutation permission as production-impacting. Operators with that authority can add models manually, import catalog JSON, refresh provider metadata, or sync selected Microsoft Foundry deployments through API-key or managed-identity authentication. Gaia keeps Foundry deployment names separate from source model names so metadata refresh can update model type, size, modalities, context windows, pricing, API metadata, built-in tools, and reasoning capability notes from public provider pages where available, with default-reasoning-model web search as a fallback for incomplete page structures or missing Foundry pricing. That changes which deployments builders can select across projects, so assign it to people who own model availability, credential rotation, and rollout timing.

You should think in two layers:

- **Visibility authority**: who can view sensitive surfaces and diagnostics.
- **Mutation authority**: who can change behavior-impacting settings.

For early project stages, many teams overgrant mutation authority "to move fast." This usually causes expensive rework because unstable changes are hard to attribute and control. A better approach is constrained mutation with explicit ownership.

### 3) Minimum viable role model for handbook work

For Chapter 2 and early implementation, use a small, clear role model.

Suggested baseline:

- **Project Admin / Team Admin**
  - Owns configuration, role assignments, and critical setup changes.
  - Can unblock permission issues quickly.
- **AI Engineer (builder)**
  - Owns day-to-day implementation and evaluation design.
  - Needs broad working access but not unrestricted governance authority in larger teams.
- **Reviewer / Domain Approver**
  - Validates quality, policy alignment, and acceptance criteria.
  - Needs visibility and eval/review capabilities; mutation scope can be limited.

The exact role names may differ in your project, but the responsibility split should remain explicit.

### 4) Responsibility boundaries reduce delivery risk

A common source of quality regressions is unclear decision rights.

Define up front:

- Who can activate/deactivate agent configurations.
- Who can change data model structures.
- Who can approve eval baselines and thresholds.
- Who can approve release and rollback decisions.

When these boundaries are explicit, engineering decisions become faster because you avoid social ambiguity during high-pressure moments.

### 5) Role design must follow the capstone, not ideology

Avoid role models copied from unrelated teams. Design roles around your capstone requirements.

Questions to ask:

- Which surfaces will be edited frequently in next 2 chapters?
- Which changes are high-risk and require approval?
- Which tasks can be parallelized safely?

For a single-engineer learning project, broad access may be acceptable. For collaborative projects, tighter mutation control usually improves reliability.

### 6) Principle of least privilege with delivery realism

Least privilege means users get only what they need. But rigid least-privilege models can slow delivery if poorly designed.

Use practical least privilege:

- Start with narrow permissions.
- Add access only when a blocked workflow is confirmed.
- Document why each privilege expansion exists.

This gives you both safety and traceable decision making.

### 7) Access readiness before implementation sprints

Before Chapter 2 implementation tasks, run an access readiness check.

Minimum check items:

- Can the builder open and edit the intended agent configuration?
- Can the builder access required data model surfaces?
- Can the reviewer access conversation and eval evidence?
- Can an admin rapidly adjust roles if needed?

This prevents losing implementation time to avoidable permission friction.

### 8) Access anti-patterns to avoid

- **Shared admin account usage**: destroys accountability and audit clarity.
- **Permanent emergency access**: temporary exceptions become permanent risk.
- **Late-stage role cleanup**: doing governance after rollout usually breaks trust.
- **Unowned role definitions**: nobody maintains alignment as scope evolves.

Treat these as system-level defects, not process quirks.

### 9) Signals that your role model is working

Your access model is healthy when:

- Engineers can execute planned work without repeated access tickets.
- High-impact changes require explicit approval or accountable owner action.
- Reviewers can inspect sufficient evidence without overbroad mutation rights.
- Incidents can be triaged with clear ownership rather than cross-team confusion.

These are practical outcomes, not compliance slogans.

### 10) Positioning for the next sections

This section sets up Chapter 2 execution. If access readiness is weak, you should fix it now before creating teams/projects/configurations in subsequent sections.

In short:

- Access and roles are not setup bureaucracy.
- They are preconditions for reliable AI engineering.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab creates an "Access Readiness Baseline" artifact for your capstone project.

### Scenario

Use your selected capstone project (or create one if not available) and assume a small team with at least one admin and one builder role.

### Phase A: Audit current access posture

1. Open the **Teams** area and identify your target team and project.
2. List current members who will participate in capstone delivery.
3. For each member, record current effective role and intended delivery responsibility.
4. Mark mismatches where role does not match responsibility.

Checkpoint A:

- You have a role/responsibility table with at least one explicit mismatch or confirmation per participant.

### Phase B: Validate builder-critical permissions

5. As the primary builder (or using role assumptions), validate access to:
   - AI Agents
   - Agent Configuration edit flow
   - Data Model tabs needed for capstone
   - Conversations workspace
   - Evals workspace
6. For each surface, mark status as:
   - accessible and sufficient,
   - accessible but insufficient,
   - blocked.

Checkpoint B:

- You have a permission matrix for builder-critical surfaces with status labels.

### Phase C: Validate reviewer evidence access

7. Define one reviewer profile (domain expert or QA reviewer).
8. Confirm whether reviewer can inspect:
   - conversation outcomes,
   - evaluation runs/reports,
   - relevant configuration summaries.
9. If reviewer can mutate high-risk settings unnecessarily, note this as over-privilege.

Checkpoint C:

- Reviewer evidence access is validated, and over-privilege risks are documented.

### Phase D: Define governance boundaries

10. In your artifact, define who can perform each action:

- activate/deactivate agent configuration,
- modify role assignments,
- approve eval pass thresholds,
- authorize rollout,
- authorize rollback.

11. Add a backup owner for each critical action.

Checkpoint D:

- Critical decisions have a primary and backup owner.

### Phase E: Apply minimum viable corrections

12. Resolve the highest-impact mismatches first:

- blocked builder access on required surfaces,
- unclear ownership for rollout/rollback,
- major reviewer over-privilege.

13. Record what changed and why.
14. Keep this change log concise and traceable.

Checkpoint E:

- At least one high-impact access gap has been corrected or escalated with owner/date.

### Phase F: Produce access readiness artifact

15. Create a final "Access Readiness Baseline" summary with:

- role/responsibility table,
- builder permission matrix,
- reviewer evidence access check,
- governance ownership map,
- change log and open gaps.

16. Add Chapter 2 go/no-go statement:

- Go if required builder surfaces are available and ownership map is explicit.
- No-go if critical blockers remain unresolved.

Checkpoint F:

- You have a reusable access baseline artifact that another engineer can apply to a similar project.

## Expected outputs

By the end of this lab, you should have:

- A role/responsibility matrix for your capstone team.
- A builder permission matrix covering core Gaia workspaces.
- A reviewer evidence-access check with over-privilege notes.
- A governance ownership map for high-impact decisions.
- An access change log and unresolved risk list.
- A Chapter 2 go/no-go statement based on explicit criteria.

Evidence that qualifies:

- One markdown artifact containing all matrices/maps/checklists.
- Clear owner assignments and escalation path for unresolved blockers.
- Explicit rationale for any privilege changes.

## Failure modes

1. **Roles assigned by title, not responsibility**
   - Symptom: People have access that does not match their actual delivery tasks.
   - Recovery: Re-map permissions to responsibilities and validate surfaces required by each role.

2. **Builder blocked on core surfaces**
   - Symptom: Work stalls because AI Agents/Data Model/Evals access is partial or missing.
   - Recovery: Resolve blockers before implementation sprint; do not defer to "later."

3. **Reviewer cannot inspect evidence**
   - Symptom: Approval decisions are made without direct visibility into outcomes.
   - Recovery: Grant reviewer visibility to conversations/evals while preserving mutation controls.

4. **Reviewer over-privileged**
   - Symptom: Review roles can alter high-risk settings unintentionally.
   - Recovery: Separate evidence access from mutation authority.

5. **No owner for rollout/rollback decisions**
   - Symptom: Incidents escalate chaotically with delayed decisions.
   - Recovery: Assign primary and backup owners for critical decisions.

6. **Access exceptions are undocumented**
   - Symptom: Temporary permission changes become permanent and untraceable.
   - Recovery: Log every privilege exception with reason, owner, and expiration intent.

7. **Shared admin usage**
   - Symptom: Change attribution is impossible and audit confidence drops.
   - Recovery: Enforce named accounts for all privileged actions.

8. **Go-live without readiness baseline**
   - Symptom: Teams begin build work with unresolved governance and access blockers.
   - Recovery: Require explicit go/no-go checklist completion before Chapter 2 implementation milestones.

## Completion checklist

- [ ] I mapped team roles to actual capstone responsibilities.
- [ ] I validated builder access across AI Agents, Data Model, Conversations, and Evals.
- [ ] I validated reviewer evidence visibility and checked for over-privilege.
- [ ] I assigned primary and backup owners for rollout and rollback decisions.
- [ ] I resolved or escalated high-impact access mismatches with traceable ownership.
- [ ] I created an Access Readiness Baseline artifact that another engineer can follow.
- [ ] I recorded an explicit Chapter 2 go/no-go decision based on readiness criteria.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Getting started](../../user-guide/README.md)
- [Settings](../../user-guide/settings/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)

---

# Create Team and Project

## Learning objectives

By the end of this section, you should be able to:

- Create a team/project structure in Gaia that matches your capstone delivery needs instead of ad-hoc experimentation.
- Apply practical naming, ownership, and scope boundaries at team and project level to reduce setup churn later.
- Validate that your newly created project is "implementation-ready" for agent, data model, and eval work.
- Execute a reproducible project-bootstrap lab and produce setup evidence that supports the next two Chapter 2 sections.

## Prerequisites

- You completed [Access and Roles](#doc-ch02-gaia-setup-and-first-project-01-access-and-roles) and have an access readiness baseline.
- You can access the **Teams** view and have permission to create teams/projects (or have an admin available).
- You have a capstone charter draft from Chapter 1 with primary persona and workflow scope.

## In Gaia

- [Getting started](../../user-guide/README.md) for Teams and project creation flow
- [Settings](../../user-guide/settings/README.md) for project-level configuration after creation
- [Cross-instance project transfer](../../user-guide/settings/README.md#cross-instance-project-transfer) only when import is a real requirement

Create the team and project while reading this section. The naming and scope rules here should be reflected immediately in Gaia, not written down for later.

## Concept brief

Creating a team and project sounds simple, but it is one of the highest-leverage decisions in a Gaia implementation. Early structure determines how cleanly you can evolve agents, data model changes, evaluations, and release governance.

When setup is weak, teams experience:

- naming confusion,
- cross-project contamination,
- unclear ownership,
- and slow onboarding for new collaborators.

When setup is strong, later engineering work compounds in value because everything has a clear boundary.

### 1) Team is an ownership boundary, project is a delivery boundary

A useful mental model:

- **Team** groups people with shared governance and accountability.
- **Project** contains a specific AI delivery unit (scope, config surface, data/evals behavior).

You should create a project when the delivery boundary is clear enough to evolve independently. If two efforts require different quality criteria, data boundaries, or release cadence, they should not share the same project by default.

### 2) Avoid "single giant project" anti-pattern

Many new deployments start with one project for everything. This feels efficient short-term, but creates long-term friction:

- unrelated experiments modify shared configs,
- eval results become noisy across workflows,
- access policies become difficult to enforce precisely.

Prefer focused projects with explicit scope. You can always add additional projects when new capabilities need independent change velocity.

### 3) Setup should reflect capstone outcomes

Your capstone charter already defines user persona, in-scope workflows, and failure classes. Use that to drive setup choices:

- Team name should indicate accountable delivery unit.
- Project name should indicate capability intent.
- Project description should express scope, not marketing language.

Bad example:

- Team: "AI"
- Project: "Assistant"

Better example:

- Team: "Revenue Operations"
- Project: "Renewal Operations Assistant"

Names are not cosmetic. They become anchors in communication, triage, and review workflows.

### 4) Scope boundaries at project creation time

Before clicking "Create project," define these four boundaries explicitly:

1. **Problem boundary**: what problem this project solves.
2. **Data boundary**: what data families this project depends on.
3. **Behavior boundary**: what assistant should and should not do.
4. **Release boundary**: what evidence is required before rollout.

If these are unclear, creation can proceed, but implementation speed will degrade in Chapter 3+ because setup assumptions keep changing.

### 5) Parent/child team structure as a scaling lever

Gaia supports hierarchical teams. Use this intentionally:

- Parent teams for broad organizational ownership.
- Sub-teams for domain-specific execution.

Choose hierarchy only when it improves clarity. Do not over-model organization charts for their own sake.

A practical rule:

- If a sub-group needs distinct admin ownership or distinct project backlog cadence, a sub-team can help.

### 6) "First runnable workspace" criteria

A newly created project is not ready just because it exists. It becomes runnable when:

- access is aligned,
- naming and scope are clear,
- project settings baseline is validated,
- the canonical project spec reflects the intended starter structure,
- and the team can run one minimal conversation setup path without blockers.

Treat project creation as the start of delivery, not a checkbox step.

### 7) Create vs import decision

Gaia supports importing projects from another instance. For handbook progression:

- use **Create** when learning and defining a clean capstone baseline,
- use **Import** when migration or replication is required.

Do not import unless you need cross-instance continuity. Imported complexity can hide learning gaps in early chapters.

When you do need import mode, use the canonical pair of user-guide flows together: the Teams landing guide for the destination **Import** dialog and [Cross-instance project transfer](../../user-guide/settings/README.md#cross-instance-project-transfer) for the source token, allow-list prerequisites, and preflight checks.

### 8) Project metadata quality improves delivery clarity

Include basic metadata discipline from day one:

- clear project description,
- owner notes,
- initial status assumptions,
- and short setup timestamp/change note.

This reduces ambiguity when new collaborators join or when you revisit decisions later.

### 9) Setup decisions should be reversible

Not every setup decision will be perfect. Optimize for reversible decisions:

- choose clear names early (renaming later is possible but disruptive to communication),
- document why choices were made,
- keep scope narrow so pivoting is affordable.

A reversible setup mindset prevents "frozen wrong structure" as your capstone evolves.

### 10) Hand-off readiness for next sections

This section should end with a project that is ready for:

- first agent/config creation,
- first conversation checkpoint,
- and chapter-level setup validation.

If those cannot happen without structural changes, setup is incomplete and should be corrected now.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab creates a project bootstrap artifact and validates that the project is ready for Section 3 and Section 4 work.

### Scenario

Use your capstone plan and bootstrap one clean project workspace for it.

### Phase A: Team/project planning worksheet

1. Create a short planning note before creating anything.
2. Add fields:
   - Team name
   - Team purpose
   - Project name
   - Project scope summary
   - Primary owner
   - Backup owner
3. Add one line for each boundary:
   - problem boundary,
   - data boundary,
   - behavior boundary,
   - release boundary.

Checkpoint A:

- You have a pre-creation worksheet with concrete names, owners, and boundaries.

### Phase B: Create or select team

4. Open **Teams** in Gaia.
5. If needed, create a new team (or sub-team) using your worksheet.
6. Verify team selection loads as expected and intended owners can access it.
7. If hierarchy is used, confirm parent team choice is intentional and documented.

Checkpoint B:

- The chosen team boundary and ownership model are explicit and validated.

### Phase C: Create project

8. In the selected team, create a new project with your planned name/description.
   - Use **Blank project** when you want to assemble the first agents, entities, folders, and workflows manually.
   - Use **Design With Gaia** on the Teams page when you want Gaia to reason through the architecture in conversation first, keep the work in a persisted pre-project design session, review unresolved questions in the panel as the design matures, and provision only after explicit approval.
   - Use **Ask Gaia** when you want Gaia to draft the first scaffold from your brief, preview the proposed structure, and create the workspace only after approval.
9. Confirm the project appears in team project list and opens correctly.
10. Record project identifier details needed for future references.
11. If you used **Ask Gaia**, review the project preview carefully before approval and note which optional starter structure you accepted or removed. Treat that approved preview as the first canonical project-spec baseline for later evolution.
12. If you used import mode, follow the transfer-token and destination import steps from the canonical User Guide pages, then capture source/destination rationale and any migration constraints.

Checkpoint C:

- Project exists with scope-aligned metadata, can be opened reliably, and has an intentional bootstrap path documented (blank, Ask Gaia, or import).

### Phase D: Baseline settings and access validation

12. Open **Settings** for the project and verify:

- required users can access the project,
- role assignments match your access baseline,
- language/default settings are acceptable for initial build,
- and the **Project Spec** page in **Delivery Management** matches the intended managed configuration when the project was created through **Ask Gaia**.

13. Record any mismatch and resolve highest-impact blockers.

Checkpoint D:

- Settings/access baseline is aligned with chapter execution needs.

### Phase E: First runnable readiness probe

14. Run a minimal readiness probe:

- open AI Agents page,
- open Data Model page,
- open Conversations page,
- open Evals page.

15. Confirm no blocking permission or navigation errors in this path.
16. Capture one screenshot or short log summary proving each page loads.

Checkpoint E:

- Project is runnable across core workspaces needed for upcoming sections.

### Phase F: Produce project bootstrap artifact

17. Create a final "Project Bootstrap" artifact with:

- team/project naming rationale,
- ownership map,
- boundary definitions,
- settings/access check summary,
- readiness probe results,
- open risks.

18. Add explicit go/no-go for proceeding to Section 3.

Checkpoint F:

- Another engineer can review your artifact and understand exactly why this project is ready (or not ready) for implementation work.

## Expected outputs

By the end of this lab, you should have:

- A team/project planning worksheet.
- A created and validated project workspace aligned to capstone scope.
- A settings/access baseline confirmation.
- A four-surface readiness probe result (AI Agents, Data Model, Conversations, Evals).
- A Project Bootstrap artifact with go/no-go decision and open risks.

Evidence that qualifies:

- One markdown bootstrap document.
- Clear owner and boundary statements.
- Explicit readiness probe proof.

## Failure modes

1. **Project name is generic and non-functional**
   - Symptom: Team discussions repeatedly ask "which assistant project are we talking about?"
   - Recovery: Rename or recreate with scope-explicit naming before deeper build work.

2. **Team/project boundary mismatch**
   - Symptom: Unrelated workflows compete in one project, creating noisy changes.
   - Recovery: Split into focused project boundaries based on delivery and quality criteria.

3. **Creation without boundary definitions**
   - Symptom: Scope keeps changing because setup intent was never documented.
   - Recovery: Add boundary worksheet and revisit naming/ownership immediately.

4. **Owners are undefined or symbolic only**
   - Symptom: No clear decision-maker for setup blockers.
   - Recovery: Assign primary and backup owners with explicit responsibility.

5. **Settings baseline skipped**
   - Symptom: Access mismatches discovered only when implementation begins.
   - Recovery: Run settings/access validation immediately after project creation.

6. **Readiness probe omitted**
   - Symptom: Team assumes readiness; later discovers missing permissions on critical pages.
   - Recovery: Require page-level readiness checks before section handoff.

7. **Import used without migration rationale**
   - Symptom: Unexpected constraints from imported state complicate learning flow.
   - Recovery: Document import intent and constraints; prefer fresh create for learning-first setups.

8. **No go/no-go decision recorded**
   - Symptom: Section transitions happen without objective readiness criteria.
   - Recovery: Add explicit decision statement with unresolved risks and owners.

## Completion checklist

- [ ] I produced a team/project setup worksheet with clear names, owners, and boundaries.
- [ ] I created (or intentionally selected) the right team boundary for the capstone project.
- [ ] I created the project with scope-aligned metadata.
- [ ] I validated settings/access alignment for upcoming chapter work.
- [ ] I executed and recorded a readiness probe across AI Agents, Data Model, Conversations, and Evals.
- [ ] I documented a Project Bootstrap artifact with open risks and a go/no-go decision.
- [ ] Another engineer could follow my artifact and recreate the same setup choices.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Teams and projects](../../user-guide/README.md)
- [Cross-instance project transfer](../../user-guide/settings/README.md#cross-instance-project-transfer)
- [Settings](../../user-guide/settings/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Evals](../../user-guide/evals/README.md)

---

# First Agent First Config

## Learning objectives

By the end of this section, you should be able to:

- Define a first agent scope that is narrow enough to validate quickly and strong enough to support the Chapter 2 checkpoint.
- Configure one primary model and conversation behavior baseline in Gaia without overfitting to edge cases too early.
- Enable only the minimum tool set needed for your first workflow so you can observe reasoning and execution quality clearly.
- Run a reproducible first-config lab and produce evidence that the project is ready for a meaningful first conversation validation.

## Prerequisites

- You completed [Access and Roles](#doc-ch02-gaia-setup-and-first-project-01-access-and-roles) and [Create Team and Project](#doc-ch02-gaia-setup-and-first-project-02-create-team-and-project).
- You have one project with validated access across AI Agents, Conversations, Data Model, and Evals.
- You can create or edit an agent configuration in the project.
- You have a short capstone problem statement with one primary workflow and one explicit out-of-scope area.

## In Gaia

- [AI Agents](../../user-guide/agents/README.md) and [Agent Configuration](../../user-guide/agents/configs.md) for the first configuration
- [Tool registry](../../user-guide/data-model/tool-registry.md) when the first workflow needs controlled tool access
- [Conversations](../../user-guide/conversations/README.md) for the first live validation turns

Build the first config in Gaia while keeping the workflow deliberately narrow. This section is about creating a measurable baseline, not a feature-rich assistant.

## Concept brief

The first agent configuration is where your capstone shifts from setup to behavior engineering. This step is often rushed, which creates confusion later when teams cannot explain why the assistant behaves inconsistently.

The goal is not to produce a "perfect assistant." The goal is to produce a controlled baseline that can be measured, critiqued, and improved.

### 1) First config is a baseline contract

Treat your first config as an explicit contract between:

- user intent,
- model behavior,
- tool boundaries,
- and quality expectations.

If this contract is vague, every test result becomes ambiguous. If it is concrete, failures become diagnosable.

### 2) Keep the first workflow intentionally narrow

For first configuration work, pick one workflow that is:

- frequent enough to matter,
- simple enough to evaluate quickly,
- and bounded enough to avoid uncontrolled tool sprawl.

Examples:

- "Draft a status summary from known project facts."
- "Answer role and process questions with references."
- "Create a structured action list from a user request."

Avoid starting with broad "do everything" behavior. Breadth hides quality gaps.

### 3) Scope controls quality velocity

Use a three-line scope statement in your config notes:

1. **In scope:** exact user job to be done.
2. **Out of scope:** what assistant must decline or hand off.
3. **Success signal:** what a "good first response" looks like.

This statement prevents prompt drift and helps reviewers evaluate outputs consistently.

### 4) System instructions should guide, not overprescribe

Early configurations often fail in two opposite ways:

- too vague: model improvises unpredictably,
- too rigid: model becomes brittle and unnatural.

A strong baseline prompt should include:

- role and objective,
- response style constraints,
- refusal/handoff boundaries,
- tool use policy.

It should not try to encode every edge case on day one.

### 5) Model and reasoning choices should match task risk

Choose a model and reasoning level based on:

- response criticality,
- cost tolerance,
- latency expectations.

For first config, optimize for observability and repeatability over raw sophistication. You can upgrade model strategy after checkpoint evidence is available.

### 6) Start with minimum viable tools

Only enable tools that are strictly necessary for the selected workflow.

Benefits:

- easier debugging,
- lower risk of unsafe or irrelevant actions,
- clearer attribution when quality changes.

Tool minimalism in early stages is a quality accelerator, not a limitation.

### 7) Define fallback and refusal behavior explicitly

Your first config must state what happens when:

- required data is missing,
- tool execution fails,
- user asks for out-of-scope actions.

Without fallback rules, the assistant may hallucinate confidence instead of signaling uncertainty.

### 8) Version every meaningful change

Record each config change with:

- what changed,
- why it changed,
- expected impact.

This simple discipline turns "it seems better" into measurable iteration.

### 9) Configuration quality is proven in conversation, not in editor

A config that looks good in a form is not validated until you run realistic prompts and inspect outcomes.

Section 4 formalizes this checkpoint. In this section, you prepare the config so that checkpoint evidence is meaningful.

### 10) Exit criteria for this section

You are done with Section 3 when:

- one agent config is active,
- scope boundaries are explicit,
- tool set is intentional and minimal,
- and you can justify your model/behavior choices in writing.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "First Agent Config Baseline" artifact for the next conversation checkpoint.

### Scenario

Use the project created in Section 2 and configure one agent for the primary capstone workflow.

### Phase A: Define workflow and boundaries

1. Write a one-paragraph workflow target (who asks, what outcome is needed, what constraints apply).
2. Add three boundary lines:
   - in scope tasks,
   - out-of-scope tasks,
   - refusal/handoff conditions.
3. Add one measurable success signal for first-response quality.

Checkpoint A:

- You can explain exactly what this first agent should do and should not do.

### Phase B: Create agent and baseline instructions

4. Open **AI Agents** and create a new agent aligned with your workflow target.
5. Add a concise role description and baseline instruction set.
6. Ensure instruction text includes:
   - response quality expectations,
   - handling of uncertainty,
   - respect for scope boundaries.
7. Save the configuration draft.

Checkpoint B:

- Agent role and behavior intent are explicit and reviewable.

### Phase C: Configure model and behavior controls

8. Select the primary model and reasoning level appropriate for the workflow.
9. Set baseline response behavior options (language/tone/style where relevant).
10. Record why this model/behavior setup was chosen now (not in abstract).

Checkpoint C:

- Model and reasoning choices have practical justification tied to task risk and cost.

### Phase D: Enable minimum viable tools

11. Identify required tools for the selected workflow.
12. Enable only those tools in the first config.
13. For each enabled tool, document:

- why it is needed,
- what could go wrong if misused.

14. Confirm unnecessary tools remain disabled.

Checkpoint D:

- Tool surface is intentionally minimal and justified.

### Phase E: Define fallback and escalation behavior

15. Add explicit handling rules for:

- missing inputs,
- unavailable tool/data paths,
- out-of-scope requests.

16. Add a clear escalation path (for example: request clarification, defer, or route to human owner).
17. Save and activate the config revision.

Checkpoint E:

- Uncertainty and failure behavior are defined before live conversation checks.

### Phase F: Produce baseline artifact

18. Create a "First Agent Config Baseline" markdown artifact containing:

- workflow target and boundaries,
- agent instruction summary,
- model and reasoning rationale,
- enabled tool list with risks,
- fallback/escalation rules,
- config version note.

19. Add a go/no-go decision for Section 4 checkpoint execution.

Checkpoint F:

- Another engineer can understand exactly what behavior you intend to validate next.

## Expected outputs

By the end of this lab, you should have:

- One active first agent configuration for your capstone workflow.
- A scope statement with explicit in-scope and out-of-scope boundaries.
- A model and reasoning rationale linked to risk/cost assumptions.
- A minimum viable tool set with risk notes.
- Fallback and escalation behavior documented.
- A First Agent Config Baseline artifact with go/no-go for Section 4.

Evidence that qualifies:

- One markdown baseline document with all decisions and rationale.
- A saved active config in Gaia that matches the document.
- Clear owner accountability for future config changes.

## Failure modes

1. **Config scope is undefined**
   - Symptom: assistant handles unrelated tasks inconsistently.
   - Recovery: rewrite in-scope/out-of-scope lines before further tuning.

2. **Prompt overfitting before baseline validation**
   - Symptom: long, brittle instructions with no measurement plan.
   - Recovery: simplify to core behavior contract and validate with checkpoint prompts first.

3. **Too many tools enabled early**
   - Symptom: unpredictable tool usage and hard-to-debug responses.
   - Recovery: reduce to minimum viable tool set; add tools only with explicit evidence need.

4. **Model selection by preference instead of task fit**
   - Symptom: unnecessary cost/latency or weak reasoning for task criticality.
   - Recovery: tie model choice to workflow risk and measurable expectations.

5. **No fallback behavior**
   - Symptom: confident answers when data is missing or requests are out of scope.
   - Recovery: add explicit uncertainty, clarification, and handoff rules.

6. **Unversioned config changes**
   - Symptom: quality shifts cannot be traced to specific edits.
   - Recovery: log every meaningful change with reason and expected effect.

7. **Activation without ownership**
   - Symptom: no one accountable when first checkpoint fails.
   - Recovery: assign owner and review cadence before Section 4.

## Completion checklist

- [ ] I defined a narrow first workflow with explicit in-scope and out-of-scope boundaries.
- [ ] I created and saved a first agent configuration aligned with that workflow.
- [ ] I selected model and reasoning behavior with practical rationale.
- [ ] I enabled only the minimum required tools and documented tool risks.
- [ ] I defined fallback and escalation behavior for uncertainty and failures.
- [ ] I produced a First Agent Config Baseline artifact with go/no-go for Section 4.
- [ ] Another engineer could review my artifact and understand exactly what is being validated next.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Evals](../../user-guide/evals/README.md)

---

# First Conversation Checkpoint

## Learning objectives

By the end of this section, you should be able to:

- Run a first realistic conversation checkpoint against your newly configured agent and collect evidence beyond subjective impressions.
- Evaluate response quality across scope fit, correctness, safety, and operational behavior using a consistent rubric.
- Identify whether failures originate in prompt/config/tool boundaries or in missing project context and convert them into concrete fixes.
- Produce a checkpoint report with an explicit go/no-go decision for advancing from Chapter 2 setup to Chapter 3 implementation work.

## Prerequisites

- You completed [Access and Roles](#doc-ch02-gaia-setup-and-first-project-01-access-and-roles), [Create Team and Project](#doc-ch02-gaia-setup-and-first-project-02-create-team-and-project), and [First Agent First Config](#doc-ch02-gaia-setup-and-first-project-03-first-agent-first-config).
- You have one active agent configuration with explicit scope boundaries and minimum viable tools.
- You can access Conversations and Evals surfaces for the project.
- You have at least 8-12 representative prompts for the first workflow checkpoint.

## In Gaia

- [Conversations](../../user-guide/conversations/README.md) for running representative prompts
- [Give feedback on a reply](../../user-guide/conversations/dialogs/feedback.md) and [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md) for evidence capture
- [Evals](../../user-guide/evals/README.md) and [Promote Conversation to Eval](../../user-guide/evals/dialogs/promote-to-eval.md) when checkpoint findings should become repeatable quality assets

Use this section to produce a real checkpoint report from Gaia conversation evidence, not just personal impressions from a few replies.

## Concept brief

The first conversation checkpoint is your first quality gate. Its purpose is not to prove the assistant is production-ready. Its purpose is to prove that your baseline behaves predictably enough to iterate with confidence.

Without this checkpoint, teams often continue building on unstable assumptions. This increases rework and hides quality risks until later chapters.

### 1) A checkpoint is an evidence step, not a demo

A demo asks "does it look good?"  
A checkpoint asks "is behavior acceptable against explicit criteria?"

Use the checkpoint to produce evidence that another engineer can inspect and challenge.

### 2) Start with representative prompts

Test prompts should reflect realistic user intent, not idealized happy paths only.

Include:

- normal requests,
- ambiguous requests,
- missing-context requests,
- out-of-scope requests.

If your checkpoint ignores ambiguity and boundary pressure, results will be falsely optimistic.

### 3) Use a fixed quality rubric

Evaluate each conversation turn against stable criteria:

1. **Scope adherence**: stays in defined role boundaries.
2. **Task correctness**: response is directionally and factually sound for available context.
3. **Response quality**: clarity, structure, and actionability.
4. **Safety and refusal behavior**: uncertainty and out-of-scope handling are explicit.
5. **Tool discipline**: tools are used only when needed, with sensible results.

This rubric turns opinion into repeatable engineering judgment.

### 4) Distinguish error classes before fixing

Do not patch blindly. First classify failures:

- instruction error (prompt ambiguity),
- configuration error (model/reasoning/behavior mismatch),
- tool boundary error (wrong tool set or misuse),
- context/data error (missing or stale project information).

Classification determines which layer to change and prevents accidental regressions.

### 5) Evaluate end-to-end conversation flow

Single-turn quality is not enough. Check multi-turn behavior:

- consistency across follow-ups,
- memory of prior constraints,
- graceful clarification behavior,
- stable handling under corrections.

Many first configs pass single-turn checks but fail once conversation state compounds.

### 6) Treat refusals as a core quality signal

Early teams often reward "always answers." That creates hidden risk.

A high-quality baseline should:

- refuse unsafe or out-of-scope actions clearly,
- ask clarifying questions when input is incomplete,
- avoid fabricated certainty when context is missing.

Refusal quality is part of product reliability.

### 7) Capture observability artifacts, not just notes

Checkpoint evidence should include:

- prompt set used,
- response snapshots,
- failure annotations,
- config version tested.

This enables repeatability and supports later eval design in Chapter 6.

### 8) Convert findings to prioritized actions

Each failure should become one of:

- fix now (blocking),
- backlog next section (non-blocking but important),
- monitor (acceptable for baseline).

Unprioritized findings become forgotten findings.

### 9) Declare explicit go/no-go criteria

Before running the checkpoint, define objective criteria such as:

- no critical safety boundary violations,
- acceptable quality on a target percentage of representative prompts,
- no unresolved blocking tool/config errors.

If criteria are undefined, go/no-go decisions become arbitrary.

### 10) Section 4 closes Chapter 2

Chapter 2 is complete when your setup and first config are validated by conversation evidence, not just by successful form completion.

The checkpoint output becomes the bridge to Chapter 3 data-model implementation priorities.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "First Conversation Checkpoint Report" and a Chapter 2 go/no-go decision.

### Scenario

Use your active first config and run a controlled checkpoint against representative prompts for your primary workflow.

### Phase A: Prepare checkpoint protocol

1. Assemble 8-12 prompts covering normal, ambiguous, missing-context, and out-of-scope cases.
2. Freeze the tested config version and record it in the checkpoint document.
3. Define pass/fail thresholds for the rubric dimensions before execution.

Checkpoint A:

- Protocol, prompt set, and thresholds are fixed before you begin testing.

### Phase B: Execute baseline conversation set

4. Open **Conversations** and run the full prompt set with the active config.
5. Capture each response (or a concise structured summary per prompt).
6. Tag each run with rubric scores and short rationale.
7. Include at least 2 multi-turn prompts that require follow-up clarifications.

Checkpoint B:

- You have comparable evidence across a consistent prompt set.

### Phase C: Analyze failures by layer

8. For every failed or weak case, classify root cause:

- instruction,
- config,
- tool,
- context/data.

9. Mark severity:

- blocking,
- major,
- minor.

10. Record proposed fix and owner for each issue.

Checkpoint C:

- Findings are actionable, prioritized, and assigned.

### Phase D: Apply minimal corrective pass

11. Apply only high-impact, low-risk fixes first (for example: clarify boundary instructions, adjust tool usage guidance).
12. Re-run a targeted subset of failed prompts.
13. Compare before/after outcomes and note regressions if any.

Checkpoint D:

- At least one corrective iteration is validated with targeted rechecks.

### Phase E: Validate safety and boundary behavior

14. Run explicit out-of-scope and uncertainty prompts.
15. Confirm refusal and clarification behavior aligns with Section 3 boundaries.
16. Verify assistant does not overstep tool/policy limits.

Checkpoint E:

- Boundary handling is tested and evidenced, not assumed.

### Phase F: Produce final checkpoint report

17. Create a "First Conversation Checkpoint Report" containing:

- prompt set and rubric definition,
- baseline results,
- failure classification table,
- fixes applied and recheck results,
- unresolved risks and owners.

18. Add final Chapter 2 decision:

- **Go** to Chapter 3 if blocking issues are resolved and thresholds are met.
- **No-go** if critical gaps remain unresolved.

Checkpoint F:

- A reviewer can audit your decision path and reproduce the checkpoint.

## Expected outputs

By the end of this lab, you should have:

- A representative checkpoint prompt set with rubric and thresholds.
- Baseline conversation evidence for the active config.
- A classified issue list with severity, owner, and proposed fixes.
- A corrective pass with targeted recheck results.
- A First Conversation Checkpoint Report with explicit Chapter 2 go/no-go.

Evidence that qualifies:

- One markdown checkpoint report with attached response excerpts or structured logs.
- Clear before/after evidence for at least one fix.
- Explicit unresolved risk list with ownership.

## Failure modes

1. **Checkpoint uses only happy paths**
   - Symptom: quality appears strong but fails on ambiguous real-world prompts.
   - Recovery: require explicit ambiguous and out-of-scope cases in the prompt set.

2. **Rubric defined after execution**
   - Symptom: scoring shifts to justify preferred outcomes.
   - Recovery: freeze rubric and thresholds before running prompts.

3. **Root causes are not classified**
   - Symptom: random config edits without measurable improvement.
   - Recovery: classify each failure by instruction/config/tool/context layer first.

4. **No corrective recheck**
   - Symptom: fixes are assumed effective but unverified.
   - Recovery: rerun failed prompt subset after each high-impact fix.

5. **Boundary prompts skipped**
   - Symptom: unsafe overreach discovered only later in production-like flows.
   - Recovery: include explicit refusal and uncertainty tests as mandatory.

6. **No ownership for unresolved issues**
   - Symptom: important risks stay open with no timeline.
   - Recovery: assign owner and next action for every non-closed finding.

7. **Go/no-go without criteria**
   - Symptom: chapter transition is subjective and non-repeatable.
   - Recovery: define and enforce objective entry criteria for Chapter 3.

## Completion checklist

- [ ] I created a representative prompt set and fixed rubric before running the checkpoint.
- [ ] I executed baseline conversations and captured evidence across single-turn and multi-turn flows.
- [ ] I classified failures by instruction/config/tool/context and assigned severity.
- [ ] I applied at least one corrective pass and rechecked targeted failures.
- [ ] I validated refusal, clarification, and boundary behavior explicitly.
- [ ] I produced a First Conversation Checkpoint Report with unresolved risks and owners.
- [ ] I recorded a clear Chapter 2 go/no-go decision based on objective criteria.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Agent config checklist](../../user-guide/conversations/agent-config-checklist.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval design process](../../user-guide/evals/eval-design-process.md)

---

# Chapter 3: Data Modeling on Gaia

Status: draft

Entities, storage, pipelines, and workflow fundamentals

## Sections

- [Entity Design](#doc-ch03-data-modeling-on-gaia-01-entity-design)
- [Storage And Ingestion](#doc-ch03-data-modeling-on-gaia-02-storage-and-ingestion)
- [Pipelines And Workflows](#doc-ch03-data-modeling-on-gaia-03-pipelines-and-workflows)
- [Run Debugging And Data Quality](#doc-ch03-data-modeling-on-gaia-04-run-debugging-and-data-quality)

## Alignment with User Guide Data Model Surfaces

- [Data Model](../../user-guide/data-model/README.md)
- [Entities](../../user-guide/data-model/entities.md)
- [Storage](../../user-guide/data-model/storage.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Tool registry](../../user-guide/data-model/tool-registry.md)
- [Scheduled jobs](../../user-guide/data-model/scheduled.md)
- [Engineering reference: TypeScript in data pipelines](../../user-guide/data-model/typescript-pipelines.md)
- [TypeScript in pipelines index](../../user-guide/data-model/typescript/README.md)
- [Record formats](../../user-guide/data-model/typescript/record-formats.md)
- [TypeScript source](../../user-guide/data-model/typescript/source.md)
- [TypeScript transform](../../user-guide/data-model/typescript/transform.md)
- [TypeScript target](../../user-guide/data-model/typescript/target.md)
- [AI transform aggregation](../../user-guide/data-model/typescript/ai-transform-aggregation.md)
- [Workflow context sharing](../../user-guide/data-model/typescript/workflow-context-sharing.md)

## Fast path inside Gaia

1. Start in [Data Model](../../user-guide/data-model/README.md) and define the entity and workflow boundary before you worry about implementation detail.
2. Use [Entities](../../user-guide/data-model/entities.md), [Storage](../../user-guide/data-model/storage.md), and [Pipelines](../../user-guide/data-model/pipelines.md) to create the data flow and retained structure.
3. Use [Workflows](../../user-guide/data-model/workflows.md) and [Runs](../../user-guide/data-model/runs.md) to prove the model works on live records instead of remaining a schema sketch.
4. Use the TypeScript references only when the no-code surfaces are no longer enough for the required transform logic.

If you cannot point to a real entity, run, or workflow in Gaia, this chapter is still theoretical.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Entity Design

## Learning objectives

By the end of this section, you should be able to:

- Define entity boundaries that reflect business reality and produce stable records over time.
- Design properties, constraints, and relationships that support both agent workflows and downstream analytics.
- Choose between relational and graph entity modeling patterns in Gaia based on query shape and operational risk.
- Run a reproducible entity-design lab that produces a reviewable model artifact with explicit quality gates.

## Prerequisites

- You completed Chapter 2 and can work in a project with access to **Data Model → Entities**.
- You have one concrete capstone workflow with clear input/output expectations.
- You can describe at least one decision the assistant must make using structured data, not only free text.
- You have baseline familiarity with the Gaia User Guide pages for entities, pipelines, and workflows.

## In Gaia

- [Entities](../../user-guide/data-model/entities.md) for the actual record model
- [Workflows](../../user-guide/data-model/workflows.md) and [Tool registry](../../user-guide/data-model/tool-registry.md) for downstream consumers of the entity model
- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md) when the entity must support operator-facing views later

Entity editing, relational data browsing, graph queries, and schema visualization now each open in their own Data Model pages, so you can review and compare those surfaces without stacking dialogs.

Design entities against live Gaia records and expected workflow usage, not against a detached schema sketch.

## Concept brief

Entity design is where AI application ambition meets operational reality. If your entity model is vague, every downstream layer becomes fragile: prompts get overloaded, pipelines overfit, workflows become opaque, and debugging turns into guesswork. If your entity model is explicit, the rest of the system becomes testable.

In Gaia, entities are not just "tables you happen to have." They are system contracts that connect:

- what users ask,
- what tools can safely read and write,
- what workflows can process repeatedly,
- and what evaluators can verify later.

A strong model gives your assistant memory with structure. A weak model gives the illusion of memory while quality silently degrades.

### 1) Entity design starts with business nouns, not UI screens

A common early mistake is defining entities from current page layouts: "DashboardCard," "SidebarItem," "WizardStep." These are presentation constructs, not durable business concepts. Model drift starts here.

Instead, define entities around durable business nouns:

- customer, account, invoice, incident, contract,
- project, milestone, task, risk,
- policy, control, audit event.

The test is simple: if the UI changes, does the entity still make sense? If not, you modeled the interface, not the domain.

In Gaia, this matters because entities are reused by conversations, pipelines, workflows, and eval logic. UI-specific naming creates cross-surface confusion quickly. Durable nouns reduce refactor cost and make traces readable months later.

### 2) Choose record grain deliberately

Record grain means "one record equals what?" If grain is ambiguous, duplicates and inconsistent updates become inevitable.

Examples:

- One `Order` record per checkout transaction (good).
- One `Order` record per customer per month (only good if that is the business truth).
- One `Order` record per import file row without identity normalization (usually risky).

Define grain in one sentence per entity:

- "One record represents one signed customer contract version."
- "One record represents one execution run of a workflow."

Then validate with edge cases:

- What happens when the same source object arrives twice?
- What happens when a source object is updated after initial ingest?
- What happens when two systems disagree on naming or timestamps?

If grain is clear, you can design uniqueness and upsert logic. If grain is fuzzy, ingestion quality cannot stabilize.

### 3) Use identifiers as reliability primitives, not implementation details

IDs are often treated as plumbing, but in data platforms they are reliability controls.

In Gaia, every entity gets system-managed fields like `id`, timestamps, and audit ownership metadata. You still need business identity strategy:

- natural key (for example external CRM ID),
- synthetic key (UUID assigned internally),
- composite uniqueness (for example `accountId + billingPeriod`).

Guideline:

- Keep one internal stable identifier.
- Keep explicit source identifiers for traceability.
- Add unique constraints where duplicate creation would materially harm workflow outcomes.

Do not rely on fuzzy text matching for identity in production flows. Text similarity can assist enrichment, not replace identity contracts.

### 4) Separate required, optional, derived, and operational properties

Property sprawl is another frequent failure point. If all fields are equal, no field is trustworthy.

Classify properties before implementation:

- **Required:** needed for the record to be valid for core use.
- **Optional:** useful but not required for base workflow.
- **Derived:** computed from other fields or enrichment steps.
- **Operational:** run metadata, status flags, diagnostics.

This classification improves several things at once:

- pipeline validation logic,
- assistant response confidence,
- eval expectations,
- migration planning.

When you review a bad run, this taxonomy helps isolate whether failures are schema violations, enrichment gaps, or operational timing issues.

### 5) Model lifecycle states explicitly

Most business objects are not static. They move through states: draft, active, paused, completed, canceled, archived. If lifecycle state is implicit, assistants and workflows make inconsistent decisions.

For each entity, document:

- valid states,
- allowed transitions,
- transition triggers,
- side effects.

Example:

- `Task` can move from `planned -> in_progress -> blocked|done`.
- `done -> in_progress` may be allowed only with an explicit reopening reason.

You do not need an enterprise state-machine framework for first delivery, but you do need explicit transition rules. This reduces contradictory updates across manual edits, workflow targets, and tool-driven automation.

### 6) Relationships are first-class design, not post-processing

Teams often design entities first and "add relationships later." In practice, that creates brittle joins, confusing navigation, and duplicated attributes.

In Gaia, relationship planning should answer:

- one-to-many or many-to-many?
- should the relationship itself carry properties?
- what foreign key or relationship field naming is expected?
- does the query pattern need traversal depth (graph) or mostly bounded joins (relational)?

If the business question is mostly "what is directly related to this record," relational modeling is typically enough. If the core question is "find paths, influence chains, or multi-hop dependencies," graph entities may be more appropriate.

The wrong relationship strategy usually appears later as:

- duplicated denormalized fields,
- expensive pipeline transforms to recover missing links,
- incomplete conversation answers because grounding cannot traverse needed context.

### 7) Decide relational vs graph based on dominant questions

Gaia supports relational and graph entities, and both are valid. The choice should follow question shape, not trend preference.

Prefer **relational** when:

- records are form-like and transactional,
- filtering/sorting/reporting dominates,
- updates follow stable keyed operations.

Prefer **graph** when:

- relationship traversal is central,
- multi-hop context drives decisions,
- edge properties are semantically important.

Avoid premature graph adoption "just in case." Graph adds conceptual overhead and should be justified by query patterns you actually run.

A pragmatic approach for most teams:

- start relational for the core operating model,
- introduce graph where evidence shows traversal is the bottleneck.

### 8) Design for ingestion and tooling together

Entity design cannot be separated from ingestion behavior. If your schema cannot absorb real source variation safely, your model will either reject useful data or silently accept bad data.

Before finalizing an entity, answer:

- What are expected source payload shapes?
- Which fields must be normalized?
- Which fields may be missing temporarily?
- How do we detect malformed or stale records?

Also validate tool implications:

- Which tools will create/update records?
- Which tools only read?
- What are safe default behaviors when required fields are missing?

In Gaia, strong data models make tool behavior predictable. Weak models force prompt-level compensations, which are difficult to audit and easy to break.

### 9) Versioning and migration should be planned from day one

The first model is rarely the final model. Growth changes requirements. If you ignore migration planning early, every schema change becomes risky.

Define minimal versioning discipline:

- record model version in design docs,
- log schema-affecting changes and rationale,
- describe compatibility expectations for pipelines and workflows,
- stage risky changes behind explicit validation runs.

When changing property types or uniqueness behavior, ensure you can answer:

- what happens to existing records?
- what breaks in transforms?
- what needs backfill or repair?
- who approves rollout?

Migrations are not a database-only concern; they are workflow and quality concerns.

### 10) Definition of done for entity design

Entity design is done when it is operationally testable, not when diagrams look tidy.

A practical definition of done includes:

- every entity has explicit grain and identity rules,
- required vs optional vs derived properties are documented,
- relationship semantics are clear,
- lifecycle states are defined,
- ingestion assumptions are explicit,
- one reproducible lab verifies the design against realistic records.

If another engineer can read your model artifact and predict ingestion/update behavior without asking clarifying questions, your design is mature enough for pipeline work.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces an "Entity Design Pack" artifact that feeds Sections 2 and 3.

### Scenario

You are building the first stable data model for a capstone assistant that supports operational decision-making (for example delivery, support, or account planning). The assistant must answer questions using structured records and trigger follow-up actions through workflows.

### Phase A: Domain decomposition and noun selection

1. Write the top five user decisions your assistant must support.
2. For each decision, list the business nouns involved.
3. Consolidate overlapping nouns and remove UI-only terms.
4. Choose 3-6 initial entities that cover core decisions.
5. For each entity, write:
   - one-sentence purpose,
   - one-sentence record grain,
   - one sentence explaining why it exists now.

Checkpoint A:

- Your entity set is domain-first, not UI-first.
- Each entity has explicit grain language.

### Phase B: Property and identity design

6. For each entity, define candidate properties.
7. Classify each property as required, optional, derived, or operational.
8. Mark candidate business identifiers (source IDs, natural keys).
9. Define uniqueness strategy where duplicates are unacceptable.
10. Record which properties are expected to be populated at ingest vs later enrichment.

Checkpoint B:

- Identity strategy is explicit per entity.
- Required properties are realistic for first ingest.

### Phase C: Relationship and lifecycle design

11. Define relationships between entities (cardinality and direction).
12. Identify relationships that need their own attributes.
13. Choose relational or graph modeling per entity group based on dominant query questions.
14. For two critical entities, define lifecycle states and allowed transitions.
15. Capture one risky edge case per relationship (for example stale links, orphan records, or circular dependency risk).

Checkpoint C:

- Relationship choices are justified by query patterns.
- Lifecycle transitions are explicit for critical entities.

### Phase D: Implement entities in Gaia

16. Open **Data Model → Entities**.
17. Create the selected entities with descriptions aligned to your design notes.
18. Add properties with clear names and appropriate types.
19. Define relationships and review auto-generated relationship fields where applicable.
20. Add unique constraints where your design requires strict identity protection.
21. Save changes and run **Sync Database** or **Sync Graph** as appropriate.

Checkpoint D:

- Gaia entity definitions reflect your design document.
- Structural sync completes without ambiguous naming conflicts.

### Phase E: Validate with representative records

22. Insert or ingest a small representative dataset (10-30 records per core entity).
23. Use **Data Model → Entities** data views to verify:

- required fields are present,
- uniqueness assumptions hold,
- relationships resolve as expected.

24. Run 3-5 conversation prompts that depend on these entities.
25. Note where assistant behavior fails due to model gaps rather than prompt quality.

Checkpoint E:

- You can trace at least one conversation response back to specific entity records.
- At least one concrete model improvement opportunity is identified.

### Phase F: Produce the Entity Design Pack

26. Create a markdown artifact titled `chapter-03-section-01-entity-design-pack.md` containing:

- entity catalog (purpose + grain),
- property taxonomy per entity,
- identity and uniqueness strategy,
- relationship map,
- lifecycle states for critical entities,
- relational/graph rationale,
- validation findings from Phase E.

27. Add a go/no-go recommendation for moving to Storage and Ingestion design.

Checkpoint F:

- Another engineer can review the artifact and implement ingestion/pipeline work without reinterpreting your assumptions.

## Expected outputs

By the end of this lab, you should have:

- 3-6 implemented Gaia entity definitions aligned with explicit business nouns.
- Written grain and identity rules for each entity.
- Property classification (required/optional/derived/operational) documented per entity.
- Relationship definitions with cardinality and query-pattern rationale.
- At least one validated lifecycle model for a critical entity.
- A reviewable Entity Design Pack artifact and go/no-go recommendation for Section 2.

Evidence that qualifies:

- Entity list visible in **Data Model → Entities** with synchronized schema.
- Sample data that proves identities and relationships behave as expected.
- A markdown design artifact that another engineer can execute against.

## Failure modes

1. **UI-driven entities instead of domain-driven entities**
   - Symptom: names mirror screens/components rather than durable business concepts.
   - Recovery: refactor around stable nouns tied to decisions and workflows.

2. **Ambiguous record grain**
   - Symptom: duplicate records appear and ownership of updates is unclear.
   - Recovery: rewrite grain statements and add identity tests before more ingestion work.

3. **Identity strategy missing or weak**
   - Symptom: merges and upserts create conflicts or silent duplicates.
   - Recovery: introduce explicit key strategy and enforce uniqueness where needed.

4. **Property sprawl without criticality classification**
   - Symptom: pipelines cannot distinguish blocking errors from optional enrichment gaps.
   - Recovery: classify properties into required/optional/derived/operational and update validation rules.

5. **Relationships added as an afterthought**
   - Symptom: repeated denormalized fields and fragile cross-entity joins.
   - Recovery: formalize relationship design and migrate duplicated attributes to explicit links.

6. **Relational/graph choice based on preference, not query shape**
   - Symptom: unnecessary complexity or poor traversal capability.
   - Recovery: re-evaluate dominant questions and choose storage model accordingly.

7. **No lifecycle state model**
   - Symptom: workflows and manual edits transition records inconsistently.
   - Recovery: define allowed states and transitions for critical entities, then enforce in workflows.

8. **Design artifact not reproducible**
   - Symptom: reviewers cannot validate assumptions or rerun checks.
   - Recovery: include explicit sample data, checkpoints, and pass/fail criteria in the artifact.

## Completion checklist

- [ ] I defined 3-6 domain-first entities with explicit record grain.
- [ ] I documented identity and uniqueness strategy for each core entity.
- [ ] I classified properties as required, optional, derived, or operational.
- [ ] I designed relationships with justified cardinality and storage-model choices.
- [ ] I implemented the entities in Gaia and synchronized structure successfully.
- [ ] I validated representative records and traced at least one conversation behavior to the model.
- [ ] I produced an Entity Design Pack artifact with go/no-go for Section 2.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [Data Model Overview](../../user-guide/data-model/README.md)
- [Entities](../../user-guide/data-model/entities.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Conversations](../../user-guide/conversations/README.md)

---

# Storage And Ingestion

## Learning objectives

By the end of this section, you should be able to:

- Design project storage structure and source-entry paths that keep raw inputs, processed outputs, and evidence traceable.
- Choose a source-entry strategy in Gaia (manual upload, context-key pipelines, webhook-triggered workflows) based on data volatility and operational risk.
- Define validation checks that detect malformed, stale, duplicated, or incomplete payloads before they damage downstream workflows.
- Execute a reproducible source-entry lab and produce an auditable handoff artifact for pipeline/workflow implementation.

## Prerequisites

- You completed [Entity Design](#doc-ch03-data-modeling-on-gaia-01-entity-design) and have at least one synced entity model in Gaia.
- You can access **Data Model → Storage**, **Pipelines**, and **Workflows** in your project.
- You have one sample source dataset (CSV, JSON, JSONL, or API payload export) representative of real operations.

## Concept brief

Storage and ingestion are where "nice diagrams" confront the disorder of real data. Source systems send inconsistent fields, late updates, partial records, and duplicates. If your ingestion layer is weak, every downstream assistant behavior looks unreliable even when prompts and tools are written well.

In Gaia, storage and source-entry design should answer one operational question:

- Can we bring external data into the project repeatedly, safely, and observably?

That requires decisions across folder structure, payload contracts, trigger strategies, validation checkpoints, and recovery behavior.

### 1) Structure storage around traceable entry paths

Many teams treat storage paths as temporary clutter. That becomes expensive quickly when debugging runs.

Use Storage as a project-scoped interface with explicit intent:

- `raw/` for untouched source payloads,
- `processed/` for outputs ready for entities/workflows,
- `evidence/` for audit snapshots and review artifacts.

This convention does three things:

- reduces ambiguity during replay and debugging,
- makes handoffs clearer across operators and reviewers,
- improves reviewer confidence because artifacts have predictable location.

Your folder naming is not cosmetic. It is part of your observability strategy.

### 1.1) Use Document Folders for collaboration-first intake

For source packs that require shared review, use **Document Folders** instead of ad-hoc personal uploads.

- Folder membership controls who can view/edit files, join folder-linked conversations, and collaborate on artifacts published into the folder.
- Treat folder access as the canonical control plane: files uploaded directly, files extracted from ZIP archives, and folder-published artifacts inherit the folder ACL immediately.
- Gaia supports two folder modes. Channel-defined folders inherit their ingestion policy from their assigned Personal Assistant channel, including folder indexing mode, reranking policy, language, embeddings, automatic PDF analysis policy, and file-extractor routing.
- Project-level folders created from **Conversations -> Folders** define that ingestion policy on the folder itself so the same folder can be reused across multiple configs and channels.
- The folder create dialog now lets operators choose that mode explicitly: project-level folders expose the full indexing configuration on the dialog, while channel-defined folders only require selecting the governing channel.
- On the folder page, channel-defined folders keep those inherited settings read-only until an owner or editor uses **Convert to project-level**. After conversion, the same dialog exposes the full folder-owned indexing editor.
- Use owners for governance changes, editors for day-to-day collaboration, and viewers for read-only review. Gaia now supports multiple owners, but always preserves at least one owner on a folder.
- ZIP uploads unpack into a dedicated subfolder, which preserves package boundaries for later audits.
- Plan manual ZIP intake within Gaia's bounded extraction policy: at most 500 files, 20 MB per expanded entry, and 100 MB expanded in total. Split larger source packs or use a governed provider-native intake path.
- Folder indexing is automatic (background + fallback), metadata + full-text/structure across text and spreadsheet documents, so review operators can search across uploaded artifacts quickly.
- Folder file lists expose per-file indexing state so operators can see when extraction is still in progress.
- When a file fails indexing, the folder file list now carries the recorded failure reason inline next to the failed file state, which shortens triage loops during review.
- Folder indexing overviews summarize whichever policy currently governs the folder: either the assigned channel configuration for channel-defined folders or the folder's own indexing settings for project-level folders.
- Folder indexing overviews now include search diagnostics from the stored index rows, which helps distinguish between "hybrid configured" and "embeddings actually present" during troubleshooting.
- Structure Search-ready folders also expose a structural tree, which gives operators a faster route to the exact indexed section instead of treating the folder as a flat chunk list.
- Folder-search tool cards now also report rerank telemetry, which gives operators a faster read on whether query fusion ran, whether model reranking was attempted, and whether latency came from the rerank pass or the base retrieval path.
- Folder search can add bounded multilingual query rewrites before retrieval when the user's wording and indexed document languages differ. This helps mixed-language source packs, such as local-language procedures that keep technical terms in English, while preserving the original query as the first search. Use the folder or channel indexing settings to choose a preferred multilingual rewrite model when a specific text model should own that bridge.
- Treat `search_document_folder` as the default evidence-discovery path across indexed text and spreadsheet documents in linked and attached knowledge folders. Use `document_folder_spreadsheet_query` after the relevant workbook or sheet is known and the operator or agent needs structured spreadsheet analysis, exact A1 range reads, grouped totals, or aggregations.
- If a channel-defined folder shows missing channel configuration in the overview, recreate it under the correct channel or use **Convert to project-level** to preserve the folder while moving it onto folder-owned settings.
- In Personal Assistant threads, folder retrieval should be configured explicitly. From **Conversations -> Folders**, use **Activate** on the target folder when you want Gaia to create or repair the Personal Assistant channel, agent, active configuration, structured-document extractor routing, folder participant link, indexing settings, and runtime service-account binding in one flow. For manual setups, turn on the folder workspace, attach any shared channel knowledge folders, add the reusable folder-retrieval prompt fragment, attach the Document knowledge grounding validator, and enable the folder search tools on the target agent before relying on folder-grounded answers. Gaia enables folder search automatically when the folder workspace is on.
- Inline AI and Agent file extractors can contribute structured indexing hints for Document Folders. Use this to add safe aliases, logical document spans, anchored section labels, sheet summaries, or PDF visual-analysis preferences without surrendering canonical extraction or chunking to custom logic.
- When documents have reader-facing units that span pages or chunks, have the extractor return `indexing.documentStructure.spans` so Structure Search can expose the logical outline while Gaia keeps canonical evidence extraction domain-neutral.
- Gaia can also infer a structural outline directly from a readable PDF table of contents or clear page-top headings during indexing, which helps large textbooks, manuals, and policy packets expose chapter and section trees even when no custom extractor span hints were supplied.
- Oversized spreadsheets no longer need to fit the interactive workbook grid limits to be searchable in Document Folders. Gaia now falls back to row-chunked spreadsheet indexing for large CSV/XLSX files, preserving sheet and row citations while keeping the stored preview bounded.
- Oversized spreadsheet uploads in conversations no longer fail artifact creation either. Gaia now stores them as read-only workbook-preview artifacts, keeps the original CSV/XLSX source for export, and makes the limit condition explicit in artifact metadata instead of dropping the import.
- Treat extractor hints as additive: Gaia still owns canonical text, retrieval-unit boundaries, embeddings, and evidence safety checks. If a hint cannot be anchored to visible file content, it is ignored.
- Use hybrid retrieval when the task needs semantic similarity across concepts or paraphrases. Use multilingual expansion as the default language-bridging layer for otherwise concrete evidence requests; it feeds better wording into the same keyword, hybrid, and Structure Search paths rather than replacing them.

### 1.2) Use channel file routing for conversational uploads

When source files arrive from end-user conversations, configure the entrypoint channel as part of the source-entry design.

- In **Conversations -> Channels -> Text** or **Conversations -> Channels -> Personal Assistant**, use the **Files** tab to decide whether users can attach files to the model, upload files to conversation storage, or both.
- Design the intake around Gaia's per-message boundary: at most five files, 10 MB per file, and 20 MB combined.
- Add routing rules with:
  - a filename pattern,
  - one or more supported file types (`doc`, `docx`, `txt`, `md`, `pdf`, `xlsx`, `csv`),
  - one routing target: direct workflow, reusable registry extractor, or inline extractor.
- Use **Conversations -> File Extractors** when multiple channels should share the same extraction behavior.
- Use **Inline extractor** when the extraction logic should remain local to one channel configuration.
- If those extractors should improve Document Folder retrieval, return the reserved `metadata` + `indexing` envelope instead of metadata-only JSON.
- Each matching rule starts a workflow run with the seeded context key `conversation_storage_upload`.

Folder-reindex implication:

- channel rule edits and linked registry extractor updates can trigger broad folder reindexing in the same project,
- this is intentional and correctness-first; prefer temporary extra indexing work over stale retrieval evidence.

Operational implication:

- one matching upload can trigger a reproducible source-entry path.
- supported files are expanded into structured records before downstream transforms run,
- the last target receives the final transformed record stream without an extra wrapper layer.

This routing model is preferable to ad-hoc per-config upload toggles because it keeps source-entry policy aligned with the public channel where the files actually enter the system.

### 2) Source-entry strategy should follow data arrival pattern

There is no single best source-entry mode. Choose based on how data appears:

- **Manual upload:** good for controlled pilots, one-off backfills, or early model validation.
- **Context-key intake pipeline:** useful when upstream systems can push payloads into a known context-key pattern (for example `orders-*`).
- **Webhook-triggered workflow:** preferred for external event-driven or scheduled sync automation.

Decision rule:

- start with the simplest mode that satisfies reliability and freshness needs,
- move to automated triggers when manual execution becomes riskier than automation.

Avoid over-automating before data quality baselines exist. Early automation can scale mistakes.

### 3) Define payload contracts before writing transforms

Transforms fail or silently degrade when payload expectations are undocumented.

For each source, define a lightweight contract:

- required top-level fields,
- optional extension fields,
- identifier fields used for dedupe/upsert,
- timestamp semantics (event time vs ingest time),
- allowed nullability.

A payload contract is not bureaucracy. It is what lets you distinguish source issues from transform bugs.

When contracts evolve, version them in your source-entry notes and communicate expected downstream impact.

### 4) Freshness is a product requirement, not a data engineering detail

Many assistant failures are actually freshness failures. The answer can be logically correct for last week and still wrong now.

Define freshness objectives per entity class:

- high-volatility operational entities (often hourly or event-driven),
- planning entities (daily/weekly),
- archive/reference entities (on-demand or scheduled low frequency).

Then encode freshness checks into ingestion validation:

- latest-source timestamp seen,
- ingestion lag threshold,
- stale-record percentage.

Without explicit freshness policy, users lose trust even if technical runs "succeed."

### 5) Validate before write whenever practical

A useful ingestion principle: reject obviously bad payloads early, before they contaminate entity records.

Practical pre-write checks:

- schema shape sanity,
- required identifiers present,
- timestamp parseability,
- impossible values (negative quantities where not allowed),
- enum/domain conformance.

Not every check must block writes. Some should quarantine suspect records while allowing healthy records through.

Design three outcomes for incoming records:

- accepted,
- accepted-with-warning,
- rejected/quarantined.

This triage model makes quality behavior explicit and avoids "all-or-nothing" ingestion brittleness.

### 6) Idempotency and deduplication must be explicit

In event-driven systems, retries are normal. If your ingest path is not idempotent, retries create duplicate records and downstream confusion.

Define idempotency strategy per source:

- event ID-based dedupe,
- natural-key upsert,
- hash-based duplicate detection for payload snapshots.

Also define duplicate tolerance:

- where duplicates are forbidden,
- where temporary duplicates are acceptable with later reconciliation,
- what reconciliation workflow looks like.

In Gaia, unique constraints and clear key mapping in entities help enforce this discipline.

### 7) Security and access boundaries apply to ingestion design

Storage and webhook setup can accidentally bypass governance if treated as purely technical setup.

Minimum controls:

- least-privilege access to Storage actions,
- authenticated webhook usage with project-scoped keys,
- clear ownership for key rotation and secret handling,
- documented source allow-list and operational contacts.

Do not embed credentials in scripts or handbook artifacts. Use project-approved key handling paths and keep ownership explicit.

### 8) Observability starts at ingestion, not after failures

If you only inspect Runs after incidents, you are operating reactively.

Define ingestion observability counters now:

- payloads received,
- payloads accepted/rejected,
- processing duration,
- duplicates detected,
- stale payload rate,
- schema mismatch frequency.

Link these counters to review cadence. Even lightweight weekly review catches drift earlier than ad hoc firefighting.

### 9) Recovery playbooks are part of ingestion architecture

Ingestion incidents are inevitable. Good teams plan rollback and replay behavior before production pressure.

Document, at minimum:

- how to pause triggers safely,
- how to replay a bounded payload set,
- how to backfill missing time windows,
- how to isolate and reprocess quarantined records.

The goal is controlled recovery without introducing second-order damage. If recovery requires guessing folder paths and key names, your architecture is incomplete.

### 10) Definition of done for storage and ingestion design

Section completion means you can prove source-entry reliability in a controlled run, not merely configure screens.

You are done when:

- storage structure is intentional and documented,
- payload contract exists for each entry path,
- trigger strategy matches freshness needs,
- dedupe/idempotency strategy is explicit,
- validation and quarantine behavior is defined,
- one reproducible source-entry cycle produces auditable evidence.

At that point, pipeline and workflow orchestration can proceed with lower risk and clearer debugging boundaries.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Storage and Ingestion Readiness Pack" used by Section 3.

### Scenario

You are integrating an external operational feed into Gaia so assistant answers can rely on recent structured data. The feed arrives either on schedule (batch file) or on events (webhook posts).

### Phase A: Storage topology and ownership

1. Open **Data Model → Storage**.
2. Create a baseline folder structure such as:
   - `raw/<source-name>/`,
   - `staged/<source-name>/`,
   - `processed/<source-name>/`,
   - `evidence/ch03/section-02/`.
3. Upload one representative sample payload into `raw/<source-name>/`.
4. Document owner and update cadence for this source.

Checkpoint A:

- Folder structure reflects lifecycle stages, not ad hoc upload habits.
- Source owner and cadence are recorded.

### Phase B: Payload contract and freshness policy

5. Write a payload contract for the sample source:
   - required fields,
   - optional fields,
   - identifier fields,
   - timestamp fields and timezone assumptions.
6. Define freshness SLO for this source (for example: "data must be no older than 24h").
7. Define a stale-data threshold and failure action.

Checkpoint B:

- Contract and freshness policy are explicit and reviewable.

### Phase C: Ingestion path selection and setup

8. Decide source-entry mode for this source:
   - manual upload pilot,
   - context-key intake pipeline (pattern-based),
   - webhook-triggered workflow.
9. If webhook mode is selected:
   - configure webhook channel with a concrete webhook key,
   - configure pipeline context-source pattern separately (for example `orders-*`),
   - link to the intended workflow,
   - capture URL and auth handling notes,
   - document the workflow input contract as `{ headers, payload }` by default; multipart uploads also include `webhook_files` entries with base64 file data for workflow tools, while workflow logs redact that binary field; if webhook signature validation is required, enable raw-body preservation so the contract also includes `rawBody`, then use that field for exact-body signature validation,
   - if array splitting is needed, point the webhook `JSON Array Path` into `payload` (for example `payload.items`); array results are stored under indexed keys, while object results stay under the exact webhook key,
   - decide whether webhook callers need **Dynamic response** mode (`{ status, data, headers? }`, with a restricted set of safe response headers) or the default `202 Accepted` acknowledgment path.
10. If manual/context mode is selected:

- define exact trigger sequence and operator responsibilities.

Checkpoint C:

- Trigger path is clear, with no ambiguity about who/what starts ingestion.

### Phase D: Validation and duplicate strategy

11. Define validation rules for pre-write checks:

- required identifiers,
- timestamp validity,
- schema shape sanity.

12. Define duplicate handling strategy (upsert key or dedupe rule).
13. Identify quarantine behavior for rejected records and where evidence is stored.

Checkpoint D:

- You can explain how malformed and duplicate records are handled without manual guesswork.

### Phase E: Execute ingestion trial and inspect runs

14. Run one controlled ingestion cycle using the selected trigger path.
15. Open **Data Model → Runs** and inspect:

- status,
- duration,
- record count,
- warnings/errors.

16. Validate entity data after the run:

- required fields populated,
- duplicate behavior matches design,
- freshness target met for ingested records.

17. Capture screenshots or structured notes for evidence.

Checkpoint E:

- Ingest trial produces measurable evidence, not only a success message.

### Phase F: Produce the readiness pack

18. Create a markdown artifact named `chapter-03-section-02-storage-ingestion-readiness-pack.md` containing:

- storage topology,
- payload contract,
- freshness policy,
- trigger strategy,
- validation and dedupe rules,
- trial run evidence,
- open risks and next actions.

19. Add go/no-go recommendation for workflow composition in Section 3.

Checkpoint F:

- Another engineer can execute the same ingest procedure and evaluate quality using your artifact.

## Expected outputs

By the end of this lab, you should have:

- A project storage structure aligned with ingestion lifecycle stages.
- One documented ingestion contract for a representative source.
- A selected and configured trigger path (manual/context/webhook) with clear ownership.
- Validation, duplicate handling, and quarantine strategy defined.
- One successful controlled ingestion run with evidence in Runs and entity data views.
- A Storage and Ingestion Readiness Pack artifact with explicit go/no-go for Section 3.

Evidence that qualifies:

- Named folders and sample files visible in Storage.
- Recorded run metadata (status, duration, counts, warnings).
- Entity-level verification notes linked to payload contract expectations.

## Failure modes

1. **Storage organized by convenience rather than lifecycle**
   - Symptom: raw and processed artifacts are mixed, making replay/debugging slow.
   - Recovery: restructure folders by ingestion stage and document conventions.

2. **No explicit payload contract**
   - Symptom: transforms break unexpectedly when source fields drift.
   - Recovery: create versioned payload contract and gate ingest against required fields.

3. **Freshness not defined**
   - Symptom: assistants answer from stale records while runs still appear successful.
   - Recovery: define freshness thresholds and alerting/escalation actions.

4. **Ingestion trigger ownership is ambiguous**
   - Symptom: missed runs or duplicate runs caused by manual/operator confusion.
   - Recovery: assign clear trigger responsibility and document operational schedule.

5. **Duplicate handling absent**
   - Symptom: retries create duplicate records that distort metrics and agent output.
   - Recovery: implement deterministic dedupe/upsert strategy tied to business identity.

6. **Rejected records silently dropped**
   - Symptom: data loss is noticed only after downstream defects.
   - Recovery: quarantine rejected payloads and log reject reasons for review.

7. **Security controls bypassed in integration scripts**
   - Symptom: keys shared informally, difficult audit trail.
   - Recovery: enforce key ownership, rotation policy, and approved auth paths.

8. **Run success treated as sufficient evidence**
   - Symptom: low-quality records still enter entities despite green status.
   - Recovery: require post-run entity-quality checks as a completion gate.

## Completion checklist

- [ ] I defined a storage topology that separates raw, staged, processed, and evidence artifacts.
- [ ] I documented a payload contract with required fields, identifiers, and timestamp semantics.
- [ ] I selected an ingestion trigger strategy matched to source cadence and risk.
- [ ] I defined freshness, validation, and duplicate-handling policies.
- [ ] I executed a controlled ingest trial and inspected run evidence.
- [ ] I validated post-ingest entity quality against contract expectations.
- [ ] I produced a Storage and Ingestion Readiness Pack artifact with go/no-go for Section 3.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Data Model Overview](../../user-guide/data-model/README.md)
- [Storage](../../user-guide/data-model/storage.md)
- [Ingestion Webhook](../../user-guide/data-model/ingestion-webhook.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Entities](../../user-guide/data-model/entities.md)

---

# Pipelines And Workflows

## Learning objectives

By the end of this section, you should be able to:

- Decompose a data automation goal into pipeline stages with clear source, transform, and target responsibilities.
- Compose Gaia workflows that sequence pipelines safely, with explicit error-handling and rerun semantics.
- Define operational budgets (latency, throughput, and failure tolerance) for data workflows that support assistant quality.
- Execute a full pipeline-to-workflow lab and produce an evidence-based runbook for recurring operations.

## Prerequisites

- You completed [Entity Design](#doc-ch03-data-modeling-on-gaia-01-entity-design) and [Storage And Ingestion](#doc-ch03-data-modeling-on-gaia-02-storage-and-ingestion).
- At least one representative source payload or folder-backed document set can enter your Gaia project through a defined workflow entry path.
- You can access **Data Model → Pipelines**, **Workflows**, and **Runs**.
- You have one prioritized business flow where data must move from an entry path through transformation and into an entity update or downstream action.

## In Gaia

- [Pipelines](../../user-guide/data-model/pipelines.md) for bounded transformation logic
- [Workflows](../../user-guide/data-model/workflows.md) for orchestration and failure behavior
- [Runs](../../user-guide/data-model/runs.md) and [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md) for execution evidence and rerun analysis

Use this section to shape one real workflow graph and one real pipeline sequence in Gaia. Keep the transform boundary and orchestration boundary separate.

## Concept brief

Pipelines and workflows are the execution spine of data modeling on Gaia. Entity design defines what data should look like. Storage and entry paths define how it arrives. Pipelines and workflows define how it becomes dependable operational state.

In the current implementation, workflow orchestration is graph-based rather than a simple ordered list. That matters operationally because routing, loops, approvals, and tool calls are modeled explicitly as nodes and named ports instead of being implied in notes or stage ordering.

The design mistake to avoid is building one oversized pipeline that tries to do everything. Oversized pipelines are hard to reason about, hard to test, and hard to recover when partial failures happen.

A strong delivery pattern separates concerns:

- pipelines perform bounded transformation responsibilities,
- workflows orchestrate sequence and control behavior,
- runs provide execution evidence and operational feedback.

### 1) Design pipelines as bounded units of meaning

A pipeline should do one coherent job, not every possible job in the domain.

Examples of bounded jobs:

- parse and normalize source records,
- enrich records with derived fields,
- map normalized records to a target entity,
- publish notifications or follow-up context.

If a pipeline has multiple unrelated responsibilities, debugging and ownership become ambiguous.

A practical heuristic: if two teams would own two parts of the logic, split them into separate pipelines and orchestrate with a workflow.

### 2) Separate normalization from enrichment

Normalization and enrichment are often mixed, which creates hidden coupling.

- **Normalization:** convert raw source into consistent base schema.
- **Enrichment:** add derived values, external lookups, or AI-assisted transformations.

Why separate them:

- normalization failures indicate source-shape issues,
- enrichment failures indicate computation/integration issues,
- reruns can target only the affected stage.

This split is operationally valuable because it reduces the blast radius of failures and makes run evidence interpretable.

### 3) Keep transforms deterministic where possible

Pipelines that produce different outputs from identical inputs without clear reason are difficult to trust.

Deterministic transform discipline:

- avoid hidden external state dependencies,
- isolate non-deterministic AI transforms behind explicit prompts and settings,
- record model/version assumptions when AI transforms are used,
- for TypeScript pipeline scripts, treat module usage as constrained runtime capability (only
  built-in `crypto` is allowlisted; prefer `context.utils.*` for everything else),
- prefer the reduced-risk **QuickJS WebAssembly** runtime for compatible scripts, test the full
  source-transform-target path after switching, and reserve the high-risk legacy runtime for
  trusted scripts whose compatibility requirements justify its shared Node.js-process access,
- design fallbacks when enrichment is unavailable.

You may still use probabilistic transforms, but do so deliberately and with quality checks. Determinism is not always possible; observability and containment are mandatory.

### 4) Workflow orchestration is where reliability is won or lost

Pipelines alone do not guarantee robust execution. Workflows provide control over:

- stage order,
- trigger semantics,
- failure behavior,
- rerun boundaries.

On Gaia, represent those decisions directly in the workflow graph:

- use **If** for explicit `then` and `else` routing,
- use **Switch** when more than two ordered cases should route to different paths,
- use **Branch** and **Join** when independent paths can run from the same workflow state and converge before the next single-thread step,
- use **While** for repeated evaluation with `loop` and `exit`,
- use **For each** when you need to fan across items from workflow context,
- use tool and human-action nodes when orchestration requires stateful checks or approvals.
- attach workflow-level knowledge folders when the graph depends on approved documents, and override that scope only on nodes that need a narrower evidence set.
- attach runtime document folders when each run should use a different document set, and seed initial context data when routing or TypeScript nodes need run-specific values.
- use saved UI Layouts for human-action nodes when the reviewer needs a read-only decision packet, a guided response form, or both instead of a generic response form.

Define each workflow as a contract:

- input trigger,
- ordered stage sequence,
- evidence and knowledge scope,
- human review surface,
- expected output states,
- failure escalation behavior.

If stage ordering is implicit, subtle regressions appear when teams edit pipelines independently.

### 5) Model failure behavior explicitly

A workflow should specify what happens when a stage fails:

- fail fast and stop,
- skip optional stage with warning,
- retry with bounded attempts,
- quarantine and continue for valid subset.

Do not leave these behaviors as tribal knowledge. Capture them in workflow notes and runbook artifacts.

Explicit failure policies improve incident response because operators do not need to improvise under pressure.

### 6) Rerun strategy is part of design, not emergency procedure

Reruns are inevitable. The key is safe reruns.

Define rerun semantics ahead of time:

- rerun full workflow vs selected stage,
- idempotency expectations,
- duplicate prevention mechanism,
- post-rerun quality checks.

Without a rerun strategy, "fix and rerun" can amplify data damage. A correct rerun should converge toward correct state, not multiply inconsistencies.

### 7) Latency and throughput budgets should be practical

Teams often ignore performance until users complain. For assistant-centered applications, data freshness and update latency directly affect response quality.

Set budget targets:

- max acceptable run duration,
- max accepted queue or backlog delay,
- throughput expectations for peak workflow windows.

Then measure against runs. If budgets are repeatedly missed, you need stage decomposition, batch tuning, or schedule adjustments.

### 8) Pipeline naming and metadata are quality tools

Naming conventions feel mundane but materially impact operations.

Use names that expose intent and order:

- `crm-normalize-leads-v1`,
- `crm-enrich-segment-v1`,
- `crm-target-lead-entity-v1`.

Add concise descriptions with:

- source assumptions,
- transform purpose,
- target guarantees.

Good naming reduces onboarding time and lowers misconfiguration risk in workflow composition.

### 9) Connect pipeline evidence to eval and conversation quality

Pipelines are not isolated backend mechanics. Their outputs influence conversation truthfulness, completeness, and actionability.

After major workflow changes, validate with:

- entity-level data checks,
- representative conversation prompts,
- targeted evals for known fragile scenarios.

This closes the loop from data automation to user-visible quality. Teams that skip this loop often debug "assistant behavior" when the root cause is pipeline drift.

### 10) Definition of done for pipelines and workflows

This section is done when you can reliably execute a workflow that updates real entity state and is diagnosable on failure.

Definition of done includes:

- staged pipelines with clear responsibilities,
- workflow orchestration with explicit failure and rerun behavior,
- run evidence across at least one normal and one edge scenario,
- operational runbook artifact with ownership and escalation notes.

If another engineer can run your workflow, interpret outcomes, and recover from a failure using only your artifact, your design is production-credible.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Pipeline and Workflow Operations Runbook" artifact.

### Scenario

You need to automate a recurring update from incoming operational data into one or more Gaia entities so that assistants can answer with recent structured context.

### Phase A: Stage decomposition

1. Choose one end-to-end business flow (for example lead updates, incident updates, or project status refresh).
2. Decompose flow into 2-4 pipeline stages:
   - normalization,
   - enrichment/derivation,
   - target write/update,
   - optional notification/context output.
3. For each stage, define:
   - source input shape,
   - transformation intent,
   - target output contract.

Checkpoint A:

- Stage boundaries are clear and non-overlapping.
- Each stage has a measurable success condition.

### Phase B: Build pipelines in Gaia

4. Open **Data Model → Pipelines**.
5. Create one pipeline per stage with descriptive names and concise descriptions.
6. Configure source and target settings per stage.
7. Add transforms needed for normalization/enrichment.
8. Set initial batch sizing and any required options.
9. Save each pipeline and verify they appear in the pipeline list.

Checkpoint B:

- Pipelines are created and individually understandable from names/descriptions.

### Phase C: Compose workflow orchestration

10. Open **Data Model → Workflows**.
11. Click **New Graph** and replace the default placeholder flow with the stage nodes you actually need.
12. Create a workflow graph that sequences the stage pipelines in intended order.
13. Add control blocks where orchestration decisions are real system behavior rather than tribal knowledge.
14. Configure trigger mode (manual, schedule, or event-driven path).
15. Define failure behavior per stage (stop, retry, continue with warnings, or quarantine path).
16. Save the workflow and record the version note.

Checkpoint C:

- Workflow sequence and failure behavior are explicit and documented.
- Named control-flow ports are wired intentionally, not inferred.

### Phase D: Execute baseline run and inspect evidence

17. Trigger workflow run.
18. If the workflow graph needs run-specific inputs, choose the runtime document folder and provide the initial context JSON before starting the run.
19. Open **Data Model → Runs** and inspect run details:

- status transitions,
- per-stage timing,
- record counts,
- warnings/errors.

20. Validate target entities after run:

- required fields populated,
- duplicate behavior consistent,
- transformed values match expected semantics.

Checkpoint D:

- Baseline run evidence confirms expected data movement and quality.

### Phase E: Failure injection and safe rerun

21. Introduce one controlled failure (for example bad mapping, missing field, or transform mismatch).
22. Re-run workflow and confirm failure is isolated and diagnosable.
23. Apply minimal fix.
24. Execute rerun from workflow or run details view.
25. Validate idempotency and duplicate prevention after rerun.

Checkpoint E:

- You can demonstrate a full fail-fix-rerun cycle with clear causal evidence.

### Phase F: Produce operations runbook artifact

26. Create `chapter-03-section-03-pipeline-workflow-operations-runbook.md` containing:

- stage architecture diagram/text map,
- pipeline list with purpose,
- workflow trigger and failure policy,
- baseline and failure-run metrics,
- rerun procedure,
- escalation and ownership notes,
- next optimization backlog.

27. Add go/no-go recommendation for Section 4 debugging and quality hardening.

Checkpoint F:

- Another engineer can operate and troubleshoot the workflow using your runbook without additional walkthrough.

## Expected outputs

By the end of this lab, you should have:

- 2-4 pipelines implementing a coherent staged automation flow.
- One workflow orchestrating these pipelines with explicit sequencing and failure rules.
- Baseline run evidence with status, timing, and record-level validation.
- One controlled failure/recovery demonstration proving safe rerun behavior.
- A Pipeline and Workflow Operations Runbook artifact with ownership and escalation details.
- A clear optimization backlog for throughput, latency, or data-quality improvement.

Evidence that qualifies:

- Pipeline and workflow definitions visible in Gaia.
- Run records showing both successful and failure/recovery cycles.
- Entity-level before/after checks tied to workflow outputs.

## Failure modes

1. **Monolithic pipeline design**
   - Symptom: one pipeline mixes normalization, enrichment, and target logic.
   - Recovery: split into bounded stages and orchestrate with workflow order.

2. **Stage contracts are implicit**
   - Symptom: downstream stage assumptions drift and failures become intermittent.
   - Recovery: document explicit input/output contracts per stage.

3. **Non-deterministic transforms without guardrails**
   - Symptom: same input produces inconsistent outputs across runs.
   - Recovery: isolate probabilistic transforms, capture settings, and add verification checks.

4. **Workflow failure policy undefined**
   - Symptom: operators do not know whether to retry, stop, or continue.
   - Recovery: codify failure behavior per stage and reflect it in runbook.

5. **Unsafe reruns create duplicates**
   - Symptom: incident recovery doubles record counts.
   - Recovery: enforce idempotent keys/upsert rules and validate duplicate metrics after rerun.

6. **Performance budgets absent**
   - Symptom: workflow technically succeeds but freshness targets are missed.
   - Recovery: define latency/throughput budgets and track them in run reviews.

7. **Poor naming/metadata**
   - Symptom: wrong pipelines selected in workflow edits.
   - Recovery: adopt descriptive naming conventions and explicit descriptions.

8. **No connection to user-visible quality checks**
   - Symptom: data automation changes degrade assistant output unnoticed.
   - Recovery: run representative conversations/evals after material workflow changes.

## Completion checklist

- [ ] I decomposed one business flow into bounded pipeline stages with explicit contracts.
- [ ] I implemented 2-4 pipelines in Gaia with clear names and descriptions.
- [ ] I composed a workflow with explicit trigger, ordering, and failure behavior.
- [ ] I executed and analyzed a baseline run with entity-level validation.
- [ ] I ran a controlled failure and demonstrated safe fix/rerun recovery.
- [ ] I documented latency/throughput expectations and observed run metrics.
- [ ] I produced a Pipeline and Workflow Operations Runbook artifact with go/no-go for Section 4.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Data Model Overview](../../user-guide/data-model/README.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [TypeScript in Data Pipelines](../../user-guide/data-model/typescript-pipelines.md)
- [Evals](../../user-guide/evals/README.md)

---

# Run Debugging And Data Quality

## Learning objectives

By the end of this section, you should be able to:

- Diagnose workflow run failures by isolating the exact failing stage, payload condition, or configuration drift.
- Apply a repeatable debug loop that preserves evidence and avoids introducing second-order regressions.
- Evaluate post-run data quality using explicit dimensions (completeness, validity, uniqueness, timeliness, lineage) rather than status alone.
- Produce a run-review artifact that supports operational handoff, recurrence prevention, and Chapter 3 completion.

## Prerequisites

- You completed [Entity Design](#doc-ch03-data-modeling-on-gaia-01-entity-design), [Storage And Ingestion](#doc-ch03-data-modeling-on-gaia-02-storage-and-ingestion), and [Pipelines And Workflows](#doc-ch03-data-modeling-on-gaia-03-pipelines-and-workflows).
- Your project has at least one workflow with recent runs visible in **Data Model → Runs**.
- You can edit pipelines/workflows and trigger reruns.
- You have baseline expectations for run duration, expected record counts, and minimum data quality standards.

## In Gaia

- [Runs](../../user-guide/data-model/runs.md) and [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md) for evidence-first debugging
- [Entities](../../user-guide/data-model/entities.md) and [Storage](../../user-guide/data-model/storage.md) for checking whether bad runs produced bad persisted state
- [Tasks](../../user-guide/tasks/README.md) when recurring run defects need owned follow-up

Debug from live Gaia evidence first, then patch. This section should produce a repeatable run-review loop, not just a list of hypotheses.

## Concept brief

Debugging and data quality are not separate activities in Gaia operations. A run can complete with no hard error and still inject broken, stale, or duplicated records. Conversely, a run can fail early while preserving overall system quality by preventing bad writes.

Strong teams treat run diagnostics and quality verification as a single reliability loop:

1. detect abnormal behavior,
2. isolate root cause,
3. fix minimally,
4. rerun safely,
5. verify data quality and user impact,
6. prevent recurrence.

The goal is not to make every run green. The goal is to make run outcomes trustworthy.

### 1) Start from evidence, not hypotheses

Under pressure, teams jump straight to assumptions: "the model changed," "webhook dropped payloads," "database latency spiked." Sometimes true, often wrong.

Start with observable evidence in Runs:

- status transitions,
- stage durations,
- error/warning logs,
- record counts,
- trigger context.

Only then form hypotheses. Evidence-first debugging reduces time-to-fix and prevents random changes that mask the actual fault.

### 2) Define what a "bad run" means before incidents

If your team has no explicit bad-run criteria, discussions become subjective.

Define bad-run thresholds in advance:

- failed/stopped status,
- duration beyond agreed limit,
- output count outside expected range,
- stale timestamp ratio above threshold,
- duplicate-rate increase beyond tolerance.

This lets operators detect subtle degradations before customers notice. It also gives a shared language for escalation.

### 3) Use stage-level isolation to narrow fault domains

Most workflows have multiple stages. Root-cause speed depends on narrowing the fault domain quickly.

Isolation checklist:

- which stage first deviated from baseline timing?
- where did record volume shift unexpectedly?
- where did required field completeness drop?
- where did errors begin (source parse, transform, target write)?

When stage boundaries were designed well in Section 3, this analysis is straightforward. If stages are monolithic, debugging cost increases dramatically.

### 4) Preserve failing evidence before patching

A classic anti-pattern is fixing immediately and losing the only diagnostic signal.

Before changing anything, capture:

- failing run ID,
- relevant log excerpts,
- baseline metrics (duration, counts),
- sample bad output records,
- trigger/payload metadata.

Preserved evidence enables:

- causal comparison after fix,
- peer review,
- recurrence analysis,
- future runbook improvement.

Without preserved evidence, every incident "resolution" is anecdotal.

### 5) Apply minimal fixes with explicit intent

During incident response, broad refactors are dangerous. Apply the smallest patch that addresses the identified fault.

Examples of minimal fixes:

- correct one mapping key,
- add one null guard in transform,
- adjust one stage ordering issue,
- tighten one filter that admitted malformed payloads.

Record the fix intent in one line:

- "Fix timestamp parsing for source `X` where timezone suffix was optional."

This keeps change rationale clear and improves reversibility if the patch underperforms.

### 6) Rerun with idempotency awareness

Rerun safety determines whether debugging recovers quality or creates new defects.

Before rerun, verify:

- duplicate prevention mechanism is active,
- upsert keys are unchanged,
- quarantine behavior remains intact,
- downstream stages will not replay side effects unintentionally.

After rerun, compare against failing baseline:

- status improvement,
- duration delta,
- count correction,
- removal of specific error signature.

A green rerun without these checks may still hide data damage.

### 7) Evaluate data quality dimensions, not only run status

A disciplined quality rubric for Gaia runs should include:

- **Completeness:** expected records and required fields are present.
- **Validity:** values conform to schema/domain constraints.
- **Uniqueness:** no unintended duplicate keys or duplicate relationship entries.
- **Timeliness:** data freshness and run latency meet operating targets.
- **Lineage:** record origin is traceable to source and stage path.

Treat this rubric as a release gate for workflow changes. If quality dimensions fail, the workflow is not healthy regardless of status color.

### 8) Connect run failures to user-visible impact

Not every run anomaly has equal business impact.

Classify incidents by user consequence:

- informational/no visible impact,
- degraded assistant completeness,
- incorrect recommendations,
- high-risk action errors.

This classification helps prioritize fixes and communication. It also aligns technical response with product risk rather than infrastructure metrics alone.

### 9) Convert incidents into preventive controls

A resolved incident has limited value if it repeats next week.

For each meaningful incident, add one preventive control:

- stricter validation rule,
- new alert threshold,
- targeted eval scenario,
- schema constraint update,
- runbook step refinement.

This turns debugging from reactive maintenance into reliability engineering.

### 10) Definition of done for run debugging and quality hardening

This section is complete when you can demonstrate a full detect-isolate-fix-rerun-verify-prevent cycle with evidence.

Definition of done includes:

- one investigated failed or degraded run,
- preserved baseline evidence,
- minimal fix with rationale,
- validated rerun with improved metrics,
- post-run quality rubric pass/fail results,
- preventive control added to reduce recurrence.

If your artifact allows another engineer to understand the incident and reproduce your recovery decision, the section meets handbook quality expectations.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Run Debugging and Data Quality Review" artifact.

### Scenario

A workflow that updates production-relevant entities has started producing inconsistent outcomes: either failures, delayed completion, or suspicious output quality. You must restore reliability without introducing duplicate or stale records.

### Phase A: Incident framing and baseline capture

1. Open **Data Model → Runs** and select one recent problematic run.
2. Record:
   - run ID,
   - trigger type,
   - status,
   - total duration,
   - affected workflow/pipeline stages.
3. Capture preliminary incident severity based on user impact (low/medium/high).
4. Save these notes in a working markdown incident log.

Checkpoint A:

- Incident scope and baseline are explicit before any edits.

### Phase B: Stage isolation and hypothesis set

5. Open run details and inspect stage-level timing and log data.
6. Identify first stage where observed behavior diverges from expected baseline.
7. Build 2-3 plausible hypotheses tied to evidence (not guesses).
8. For each hypothesis, define one validation step.

Checkpoint B:

- You can point to the first deviating stage and explain why it is suspect.

### Phase C: Root-cause confirmation

9. Validate hypotheses using available metadata and entity output checks.
10. Confirm one primary root cause and optionally one contributing factor.
11. Capture a concise root-cause statement:
    - "Because X condition occurred at stage Y, field Z became invalid, causing downstream write failures."

Checkpoint C:

- Root cause is specific and testable, not broad or generic.

### Phase D: Minimal fix and controlled rerun

12. Patch the smallest configuration/transform/workflow element that addresses root cause.
13. Record exactly what changed and why.
14. Trigger rerun.
15. Re-open run details and compare baseline vs rerun on:
    - status,
    - duration,
    - stage error profile,
    - output count.

Checkpoint D:

- Rerun shows measurable improvement tied to the intended fix.

### Phase E: Data quality validation

16. In **Data Model → Entities**, inspect affected records after rerun.
17. Evaluate at least five quality dimensions:
    - completeness,
    - validity,
    - uniqueness,
    - timeliness,
    - lineage.
18. Record pass/fail with concrete evidence per dimension.
19. Run 3-5 representative conversation prompts dependent on the updated records and verify whether user-visible quality improved.

Checkpoint E:

- Technical recovery is verified against actual data and user-facing behavior.

### Phase F: Preventive hardening and artifact publication

20. Add one preventive control (for example a validation gate, alert threshold, or eval test case).
21. Create `chapter-03-section-04-run-debugging-data-quality-review.md` containing:
    - incident summary,
    - baseline metrics,
    - root cause,
    - applied fix,
    - rerun comparison,
    - quality rubric results,
    - preventive action,
    - owner and review cadence.
22. Add a Chapter 3 completion recommendation based on evidence quality.

Checkpoint F:

- Another engineer can follow your artifact to understand and audit the reliability decision.

## Expected outputs

By the end of this lab, you should have:

- One complete incident investigation with preserved run evidence.
- A root-cause statement tied to specific stage/payload behavior.
- A minimal corrective change and validated rerun comparison.
- A structured data quality assessment across five dimensions.
- One preventive control committed to reduce recurrence.
- A Run Debugging and Data Quality Review artifact suitable for chapter QA.

Evidence that qualifies:

- Run IDs and metric snapshots (before/after).
- Entity-level verification notes tied to quality rubric.
- Conversation sanity checks confirming user-visible quality impact.

## Failure modes

1. **Fixing before baseline evidence capture**
   - Symptom: no way to prove cause or measure improvement.
   - Recovery: enforce incident template requiring run ID and baseline metrics first.

2. **Treating any green run as resolved**
   - Symptom: bad records persist despite successful status.
   - Recovery: require post-run quality rubric validation as a close condition.

3. **Changing multiple variables in one patch**
   - Symptom: unclear which change resolved the issue.
   - Recovery: apply minimal fixes and rerun iteratively.

4. **Ignoring duplicate risk during rerun**
   - Symptom: incident recovery introduces inflated counts and contradictory records.
   - Recovery: verify idempotency keys and uniqueness checks before rerun.

5. **Overlooking timeliness regressions**
   - Symptom: workflows succeed but miss freshness windows.
   - Recovery: track duration and freshness thresholds as first-class quality signals.

6. **Root-cause statements too generic**
   - Symptom: repeated incidents labeled as "pipeline instability."
   - Recovery: require stage/field/condition specificity in incident write-up.

7. **No preventive follow-through**
   - Symptom: same class of incident repeats.
   - Recovery: attach at least one guardrail action with owner and due date.

8. **No user-impact validation**
   - Symptom: technical metrics improve while assistant answers remain degraded.
   - Recovery: include representative conversation checks in incident closure criteria.

## Completion checklist

- [ ] I selected and documented one problematic run with baseline evidence.
- [ ] I isolated the first failing/degrading stage and confirmed root cause.
- [ ] I applied a minimal fix and captured before/after rerun metrics.
- [ ] I validated completeness, validity, uniqueness, timeliness, and lineage on affected data.
- [ ] I checked user-visible impact through representative conversation prompts.
- [ ] I added one preventive control with ownership and review cadence.
- [ ] I produced a Run Debugging and Data Quality Review artifact and Chapter 3 completion recommendation.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Data Model Overview](../../user-guide/data-model/README.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Entities](../../user-guide/data-model/entities.md)
- [Evals](../../user-guide/evals/README.md)

---

# Chapter 4: Agent Engineering

Status: draft

Prompt, config, protocol, tool, and handoff engineering

## Sections

- [Agent Architecture](#doc-ch04-agent-engineering-01-agent-architecture)
- [Config Patterns](#doc-ch04-agent-engineering-02-config-patterns)
- [Tooling Patterns](#doc-ch04-agent-engineering-03-tooling-patterns)
- [Handoffs And Multi-Agent Control](#doc-ch04-agent-engineering-04-handoffs-and-multi-agent-control)
- [Execution Protocols](#doc-ch04-agent-engineering-05-execution-protocols)

## Alignment with User Guide Agent Surfaces

- [AI Agents](../../user-guide/agents/README.md)
- [Protocols](../../user-guide/agents/protocols.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Skills](../../user-guide/agents/skills.md)
- [Guardrails](../../user-guide/agents/guardrails.md)
- [Tool registry](../../user-guide/data-model/tool-registry.md)
- [Evals](../../user-guide/evals/README.md)

## Fast path inside Gaia

1. Start in [AI Agents](../../user-guide/agents/README.md) and make the agent topology explicit before tuning prompts.
2. Use [Agent Configuration](../../user-guide/agents/configs.md) and [Protocols](../../user-guide/agents/protocols.md) to define authority, staged execution, model behavior, and handoff posture.
3. Use [Tool registry](../../user-guide/data-model/tool-registry.md) and [Skills](../../user-guide/agents/skills.md) to control what the agent can actually do.
4. If the design problem is reusable routing, branching, approvals, or data preparation around the agent, compare with [Workflows](../../user-guide/data-model/workflows.md) and [Pipelines And Workflows](#doc-ch03-data-modeling-on-gaia-03-pipelines-and-workflows) instead of forcing that logic into execution protocols.
5. Use [Evals](../../user-guide/evals/README.md) to validate the design before calling it production-ready.

The chapter is not done when the prompt sounds good. It is done when the agent behavior is reviewable in Gaia and validated against the intended workflow.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Agent Architecture

## Learning objectives

By the end of this section, you should be able to:

- Design an agent architecture for Gaia that separates orchestration, specialization, and execution responsibilities.
- Define explicit boundaries between prompts, tools, data model dependencies, and runtime safeguards.
- Trace a user request through Gaia's conversation path and identify control points where behavior quality is enforced.
- Produce an architecture artifact that another engineer can use to implement, review, and operate your agent system.

## Prerequisites

- You completed Chapter 3 and have a working data model with at least one realistic workflow.
- You can access **AI Agents**, **Conversations**, **Data Model**, and **Evals** in your project.
- You have one capstone use case with at least two distinct intent clusters (for example: "general triage" and "specialized execution").
- You can define acceptable latency/cost/reliability tradeoffs for your first production-like assistant flow.

## In Gaia

- [AI Agents](../../user-guide/agents/README.md) and [Agent Configuration](../../user-guide/agents/configs.md) for role topology and runtime authority
- [Tool registry](../../user-guide/data-model/tool-registry.md) and [Data Model](../../user-guide/data-model/README.md) for execution boundaries and state dependencies
- [Conversations](../../user-guide/conversations/README.md) and [Evals](../../user-guide/evals/README.md) for architecture validation

Map the architecture on real Gaia surfaces while reading. Each role, tool boundary, and observation point should already have a home in the platform.

## Concept brief

Agent architecture is where your application stops being a "single prompt" and becomes a controlled system. In Gaia, architecture quality determines whether you get predictable routing, safe tool use, measurable improvements, and reliable operations.

A weak architecture often looks impressive in demos but fails under realistic traffic: requests route unpredictably, tools are overused, edge cases trigger hallucinated confidence, and fixes become entangled across settings. A strong architecture makes behavior explainable and evolvable.

In practical terms, agent architecture should answer four questions clearly:

- Which agent receives the request first?
- Which agent should handle which class of intent?
- Which tools/data can each agent use?
- How do we detect and recover when behavior drifts?

### 1) Think in role topology, not one "smart" agent

New teams often attempt one universal agent. That approach usually accumulates conflicting instructions and overbroad tool access.

A better pattern is role topology:

- **Orchestrator agent:** receives initial user message, interprets intent, routes when needed.
- **Specialist agents:** own narrow tasks with focused prompts and tool surfaces.
- **Fallback behavior:** handles uncertainty, ambiguity, or unsupported requests.

This topology improves maintainability because each role can evolve independently. It also helps evaluation: failures can be attributed to routing logic vs specialist behavior.

In Gaia, this maps naturally to orchestrator selection and handoff rules in AI Agents.

### 2) Architecture should separate cognition, execution, and state

Agent behavior has three layers that should be designed intentionally:

- **Cognition layer:** prompt fragments, reasoning settings, and response style rules.
- **Execution layer:** tool calls, workflow triggers, and side-effect actions.
- **State layer:** entity records, memory artifacts, and run/eval evidence.

Many reliability issues come from mixing these concerns. For example, trying to encode data-correction logic entirely in prompt text when it should be a tool + entity constraint.

A healthy architecture keeps these layers loosely coupled:

- prompt decides intent and policy,
- tool executes controlled action,
- data model enforces structure,
- observability verifies outcome.

### 3) Define intent domains with explicit handoff boundaries

Architecture decisions are easiest when intent domains are explicit. For each candidate specialist, define:

- in-scope request types,
- out-of-scope request types,
- required context,
- expected output contract.

Example split:

- orchestrator: classify, clarify, route, summarize.
- data-ops specialist: run data checks, report anomalies.
- project specialist: create/update delivery records.

Without explicit boundaries, orchestrator prompts become overloaded and handoff behavior turns inconsistent. Boundaries reduce policy conflict and improve user trust because routing appears coherent.

### 4) Tool access is an architectural decision, not a UI toggle

In Gaia, tools can read data, change state, navigate UI, and delegate work. Tool selection is therefore architecture, not configuration housekeeping.

Per-agent tool policy should specify:

- minimal required tools,
- prohibited tools,
- conditions for high-impact actions,
- expected failure responses.

Pattern:

- orchestrator: mostly read/search/navigation + delegation tools.
- specialist: focused write tools for its domain.
- high-risk tool access: isolated to specific specialist with stricter instructions.

When all tools are enabled everywhere, you lose control over causality and incident containment.

### 5) Model strategy should follow risk profile per role

Do not choose one model strategy for every agent role by default. Role risk differs:

- orchestrator may prioritize fast intent routing with clear safety behavior.
- specialist may require deeper reasoning for constrained tasks.
- post-step processors may use cheaper models for classification or extraction.

For each role, define:

- response criticality,
- acceptable latency,
- cost budget,
- expected turn complexity.

In Gaia configs, this translates into primary/fast model choices, reasoning effort, and per-turn limits. Architecture quality improves when model choices are justified by role function rather than habit.

### 6) Prompt architecture should be modular and testable

Long monolithic system prompts are difficult to debug. Gaia supports prompt fragments and shared reusable pieces; use them as architecture primitives.

Recommended structure:

- role mission fragment,
- scope boundaries fragment,
- tool-use policy fragment,
- refusal/escalation fragment,
- style/output contract fragment.

Benefits:

- easier diff/merge and version review,
- targeted edits without destabilizing all behavior,
- clearer mapping between prompt edits and eval outcomes.

Architecture should let you ask: "Which fragment caused this behavior shift?" If you cannot answer, modularity is insufficient.

### 7) Control points and observation points must be designed together

Every architecture needs control points (where behavior is constrained) and observation points (where behavior is measured).

Typical Gaia control points:

- on-topic and off-topic rules,
- handoff rules,
- max tool calls and per-turn budgets,
- tool allow-list per agent.

Typical observation points:

- conversation outcomes,
- tool execution traces,
- run quality signals,
- eval run results.

If you add controls without observation, you cannot verify whether controls work. If you observe without controls, you only document failures. Robust architecture requires both.

### 8) Failure containment is a primary architecture requirement

Architecture is not complete until it addresses failure behavior. Define containment for:

- tool failure,
- missing data,
- ambiguous intent,
- handoff mismatch,
- policy refusal.

Containment design includes:

- clear user-visible fallback messages,
- safe retry/clarification behavior,
- escalation paths,
- no-silent-failure policy.

A good architecture fails gracefully and visibly. A bad architecture fails silently or confidently wrong.

### 9) Architecture should be versioned as a delivery artifact

Teams often version configs but not architecture decisions. That creates hidden drift.

Maintain a lightweight architecture artifact containing:

- role topology,
- boundary contracts,
- tool allocation rationale,
- model strategy by role,
- key guardrails,
- expected quality metrics.

Update it when major behavior changes. This prevents "tribal memory" ownership and accelerates onboarding/review.

### 10) Definition of done for Section 1

Agent architecture is done when you can explain system behavior before implementation details.

Practical done criteria:

- role topology is explicit,
- handoff boundaries are documented,
- tool/model allocation is justified,
- controls and observations are mapped,
- failure containment strategy is defined,
- one reproducible architecture lab validates assumptions.

If another engineer can independently build consistent configs from your architecture artifact, the design is operationally ready for Section 2.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces an "Agent Architecture Blueprint" artifact for config implementation.

### Scenario

Your capstone assistant must support multiple request classes with different risk and execution needs. You need a maintainable architecture that routes requests reliably and keeps tool access controlled.

### Phase A: Intent and role topology mapping

1. List the top 10 user intents expected in your capstone.
2. Group intents into 2-4 domains based on required expertise and risk.
3. Define an orchestrator role and 1-3 specialist roles.
4. For each role, write:
   - mission statement,
   - in-scope intents,
   - out-of-scope intents,
   - success criteria.

Checkpoint A:

- Role topology is explicit and domain-oriented.
- Intent overlap is minimized.

### Phase B: Boundary and handoff contract definition

5. For each domain boundary, define handoff trigger conditions.
6. Define ambiguity handling (when to ask clarification vs handoff).
7. Define refusal and escalation behavior for unsupported/high-risk requests.
8. Add one negative example per role (a request that must not be handled directly).

Checkpoint B:

- Handoff boundaries are concrete, not generic.
- Unsupported intents have explicit behavior.

### Phase C: Tool and model allocation

9. Map minimum viable tools per role.
10. Mark high-impact tools and assign them only to specialist roles where justified.
11. Choose model strategy by role (primary/fast, reasoning level, budget assumptions).
12. Define per-role performance targets (latency/cost/reliability).

Checkpoint C:

- Tool and model allocation matches role risk profile.
- No role has unexplained broad privileges.

### Phase D: Gaia architecture setup draft

13. Open **AI Agents** and create or update agents to reflect the role topology.
14. Assign orchestrator and baseline handoff relationships.
15. Apply initial tool allow-lists per role.
16. Save configuration drafts (do not over-tune prompts yet).

Checkpoint D:

- Gaia setup mirrors architecture design with clear role boundaries.

### Phase E: Request-path validation

17. Run 10 representative conversation prompts spanning all intent domains.
18. For each prompt, record:

- selected route (orchestrator vs specialist),
- tool usage behavior,
- boundary compliance,
- user-visible outcome quality.

19. Identify at least three architecture mismatches and proposed corrections.

Checkpoint E:

- You can trace request path decisions and detect routing/tool boundary defects.

### Phase F: Publish architecture blueprint artifact

20. Create `chapter-04-section-01-agent-architecture-blueprint.md` with:

- role topology diagram/text map,
- intent-domain matrix,
- handoff contracts,
- tool/model allocation rationale,
- control/observation map,
- validation results and corrections.

21. Add go/no-go recommendation for Section 2 config implementation.

Checkpoint F:

- Another engineer can implement the same architecture consistently using your artifact.

## Expected outputs

By the end of this lab, you should have:

- A clearly defined orchestrator-plus-specialist topology.
- Documented intent boundaries and handoff contracts.
- Per-role tool and model allocation with risk rationale.
- Initial Gaia agent setup reflecting the architecture.
- Request-path validation evidence across representative prompts.
- A published Agent Architecture Blueprint with go/no-go for Section 2.

Evidence that qualifies:

- AI Agents configuration state aligned to documented topology.
- Conversation traces showing expected routing behavior.
- A blueprint artifact that supports independent review and implementation.

## Failure modes

1. **Single-agent overloading**
   - Symptom: one agent handles everything with inconsistent quality and slow iteration.
   - Recovery: split roles by intent domain and risk, then introduce handoff boundaries.

2. **Boundary ambiguity between roles**
   - Symptom: similar requests route differently without clear reason.
   - Recovery: define explicit in-scope/out-of-scope contracts and negative examples.

3. **Tool overexposure across all agents**
   - Symptom: unintended state-changing actions from non-specialist roles.
   - Recovery: enforce least-privilege tool allocation and move high-impact tools to specialists.

4. **Model choices based on preference, not role function**
   - Symptom: unnecessary cost/latency for simple routing or weak reasoning for complex tasks.
   - Recovery: define model strategy per role using task criticality and budget constraints.

5. **Prompt monolith design**
   - Symptom: small changes produce unpredictable behavior shifts.
   - Recovery: modularize prompts into role, boundary, tool, and fallback fragments.

6. **No failure containment policy**
   - Symptom: tool/data failures cause silent or misleading responses.
   - Recovery: define fallback, clarification, and escalation behavior per role.

7. **Control points without observability**
   - Symptom: guardrails exist but effectiveness cannot be measured.
   - Recovery: pair each control with concrete observation metrics and review cadence.

8. **Architecture undocumented outside UI settings**
   - Symptom: onboarding and review rely on tribal knowledge.
   - Recovery: publish and maintain a versioned architecture blueprint artifact.

## Completion checklist

- [ ] I defined a role topology with one orchestrator and clear specialist domains.
- [ ] I documented in-scope/out-of-scope boundaries and handoff rules per role.
- [ ] I allocated tools and models per role using explicit risk/cost rationale.
- [ ] I mapped architecture control points and observation points.
- [ ] I validated request paths across representative prompts and captured mismatches.
- [ ] I documented failure containment behavior for ambiguity, tool errors, and out-of-scope requests.
- [ ] I published an Agent Architecture Blueprint artifact with go/no-go for Section 2.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Agent Config Checklist](../../user-guide/conversations/agent-config-checklist.md)
- [Evals](../../user-guide/evals/README.md)

---

# Config Patterns

## Learning objectives

By the end of this section, you should be able to:

- Build agent configurations as explicit behavior contracts instead of ad hoc setting bundles.
- Apply repeatable configuration patterns for scope control, safety, tool use, and prompt modularity.
- Version, compare, and promote config changes using evidence from conversations and evals.
- Produce a configuration pattern pack that enables consistent implementation across multiple agents.

## Prerequisites

- You completed [Agent Architecture](#doc-ch04-agent-engineering-01-agent-architecture) and have a role topology defined.
- You can create/edit agent configurations in **AI Agents**.
- You have at least one baseline prompt set and a small prompt dataset for behavioral checks.
- You can run quick validation in **Conversations** and collect quality signals for comparison.

## In Gaia

- [Agent Configuration](../../user-guide/agents/configs.md) for config versions, prompt fragments, and tool policy
- [Delivery Management](../../user-guide/delivery/README.md) for the project-wide configuration overview when a complaint spans agents, channels, knowledge, governance, evals, or delivery
- [Conversations](../../user-guide/conversations/README.md) for immediate behavior checks
- [Evals](../../user-guide/evals/README.md) when config changes need repeatable validation instead of anecdotal judgment

Apply each pattern to a real Gaia config version as you read. The point is to make configuration changes reviewable and explainable, not just stylistically better.

## Concept brief

If architecture defines "who does what," configuration defines "how each role behaves right now." In Gaia, configuration is the living contract between prompt strategy, model settings, tool policy, and runtime guardrails.

Most teams fail not because they cannot write prompts, but because they cannot manage config evolution. Good responses appear in isolated tests, then disappear after unrelated changes. The missing capability is disciplined config patterning.

Config patterns are reusable decision templates. They reduce randomness and make behavior changes auditable.

One practical way to keep configs disciplined is to review every meaningful change through four lenses:

- **Behavior lens:** does the response quality improve for target intents?
- **Boundary lens:** does refusal, off-topic handling, and escalation remain correct?
- **Execution lens:** does tool usage become more accurate and efficient?
- **Operations lens:** do latency, cost, and support burden stay within limits?

If any lens regresses, the change is not \"done\" even if sample responses look stronger. This four-lens review helps teams avoid shipping prompt-heavy improvements that quietly increase routing errors, tool churn, or operational costs.

When the complaint is vague or clearly cross-surface, do not start by rewriting a prompt fragment. Review the project-wide configuration overview in **Delivery Management -> Overview** first, identify whether the gap belongs to runtime, channels, knowledge, governance, evals, or delivery, and only then change the owning surface.

### 1) Treat every config as an explicit contract

A configuration should answer these questions in plain language:

- What task class does this config optimize for?
- What behavior is intentionally excluded?
- Which tools may be used and under what conditions?
- What constitutes acceptable output quality?

If these are implicit, config edits become guesswork. Explicit contracts make review and debugging faster.

Recommended contract fields in your notes:

- role + scope,
- model strategy,
- tool surface,
- safety rules,
- response/output policy,
- known limitations.

### 2) Start from minimum viable configuration, then layer deliberately

A common anti-pattern is enabling many features at once: advanced prompts, many tools, aggressive handoffs, post-steps, and broad guardrails. This hides causality.

A better pattern:

1. baseline scope prompt,
2. minimal model settings,
3. minimum necessary tools,
4. core guardrails,
5. validation,
6. one incremental enhancement at a time.

This sequence makes outcome shifts attributable. In agent engineering, traceability is more valuable than initial feature completeness.

### 3) Use role-specific config variants, not one-size-fits-all

Different roles require different configuration shapes:

- orchestrator config pattern: routing clarity, concise replies, strict boundary handling.
- specialist config pattern: richer domain instructions, targeted tool policy, stricter output structure.
- assessment/post-step config pattern: classification consistency, low latency, deterministic output schema.

Trying to force one generic pattern across all roles produces compromises that weaken each role's performance.

In Gaia, use separate configuration versions per agent and avoid copying settings blindly without role rationale.

### 4) Prompt fragment architecture is the backbone of maintainable configs

Prompt quality is not only writing quality. It is change quality.

Use fragment composition to separate concerns:

- mission fragment,
- context fragment,
- tool policy fragment,
- refusal/escalation fragment,
- formatting/output fragment.

Pattern benefit:

- smaller diffs,
- targeted experiments,
- easier root-cause mapping after regressions.

When fragments are shared across agents, maintain a version note so you know which config depends on which guidance generation.

### 5) Model settings must align with behavioral intent

Config quality depends on coherence between prompts and model behavior settings.

Examples of alignment checks:

- if outputs must be concise and deterministic, avoid high creativity settings.
- if role performs deep synthesis, reasoning effort should match complexity.
- if off-topic checks use fast model, ensure its instructions are not semantically weaker than primary intent boundaries.

Also define per-role operational limits:

- max tool calls,
- parallel tool execution policy,
- per-turn budget assumptions.

Settings without rationale create accidental performance and cost regressions.

### Staged execution protocols should be configured explicitly

Some roles need more than a good prompt. They need an explicit operating sequence.

That is what execution protocols are for. Instead of burying sequencing inside one long instruction block, define visible ordered stages such as:

- plan -> execute -> verify
- clarify -> research -> synthesize -> verify -> finalize

Treat these stages as configuration behavior, not as hidden reasoning:

- each stage narrows the active skills and tools
- each stage has explicit instructions, expected outputs, and exit criteria
- each stage can define typed checkpoint output fields when downstream review or automation depends on structured payloads
- checkpoints are written as durable session state that another operator can inspect
- approval pauses are explicit instead of being implied only by wording

This matters because teams often mistake “the model reasoned well once” for “the system is operationally controlled.” A staged protocol gives you something reviewable, repeatable, and durable across multiple sessions.

Treat the session surface as part of the operating model, not just the config model. Operators should be able to inspect recent runs in the configuration, filter by status or event type, cancel a stuck active session, and restart a fresh run from the initial stage when the conversation needs another pass. When the same config is being used in a live thread, the main project **Conversations** view should also expose a compact summary of the current stage, waiting-for-user state, and latest checkpoint so reviewers do not have to leave the thread just to understand where execution stands. In today's product that summary is opened from the conversation toolbar's **Execution** button and expanded on demand. Reviewers advance a waiting stage by sending an explicit continue signal in the thread, and blocked stages continue when the missing input or evidence is supplied. Administrative actions such as cancel and restart still live in the configuration's **Recent execution sessions** panel.

When the same staged pattern needs to appear across multiple agents, put it in the Protocols registry first instead of cloning one config into another. That gives the team one reusable stage architecture plus one starter bundle of skills and tools, while still letting each agent customize the copied protocol after it is applied.

### 6) Tool policy must include intent-to-tool routing

Tool enablement alone is insufficient. The config should guide when not to call a tool.

Strong tool policy pattern includes:

- trigger conditions for each tool class,
- preconditions before side-effect tools,
- fallback when tool errors occur,
- user clarification behavior for underspecified actions.

This policy belongs in the instruction system, not just team memory. It reduces over-calling and under-calling patterns that degrade user trust.

For document-grounded assistants, keep knowledge grounding as an explicit configuration choice rather than a hidden platform heuristic. In practice, that means attaching a deterministic validator such as document knowledge grounding only to the configs that should search linked or attached folder evidence before answering. Use config-level knowledge folders for agent-specific policy packs or specialist references, and channel-level knowledge folders for shared standards that every conversation on that channel should inherit. Treat the validator like any other reusable config primitive: the trigger decides when the policy applies, and the validation step decides whether the draft passes. This makes the behavior reviewable, testable, and easy to disable for configs that are conversational but not document-grounded.

### 7) Safety and boundary patterns should be explicit and testable

Configurations should encode safety as normal behavior, not emergency behavior.

Key patterns:

- on-topic rules with clear scope examples,
- off-topic response policy,
- refusal language for prohibited requests,
- escalation/handoff behavior for high-risk intent.

Test these patterns with negative prompts, not just happy paths. A configuration that shines on ideal inputs but fails boundary prompts is not production-ready.

### 8) Config versioning is a delivery process, not a storage feature

Multiple config versions are useful only when changes are meaningful and reviewable.

Adopt a simple version process:

- each version has purpose statement,
- each version has a short description in Gaia that explains what changed,
- each change has expected impact hypothesis,
- each version has validation evidence,
- promotion criteria are explicit.

Use diff/merge intentionally:

- compare only one behavioral dimension at a time when possible,
- avoid bulk merges of unrelated changes,
- document why a merge decision was made.

Without a process, version history becomes noise.

### 9) Promote configs using evidence gates

A config should be promoted to active because of evidence, not confidence.

Recommended promotion gates:

- baseline conversation pass rate across representative prompts,
- boundary prompt behavior pass rate,
- tool-call correctness checks,
- latency/cost within budget,
- no critical regressions in recent eval results.

If a version fails one critical gate, do not promote. Keep an explicit rollback path to the previous stable config.

### 10) Definition of done for Section 2

Config patterning is done when your team can apply consistent configuration logic across roles without re-inventing decisions every sprint.

Done criteria:

- repeatable config templates exist per role,
- prompt/model/tool/safety coherence is documented,
- versioning/promotion process is explicit,
- validation evidence is part of every meaningful change,
- one section lab proves reproducibility.

If another engineer can clone your patterns and reach comparable behavior quality, your configuration discipline is mature enough for tooling patterns in Section 3.

As your project scales, this discipline also enables parallel work. Different engineers can improve separate role configurations without destabilizing shared behavior contracts, because each config change is anchored to template rules, evidence gates, and rollback expectations.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Config Pattern Pack" artifact.

### Scenario

You need to operationalize the architecture from Section 1 by creating reproducible configuration patterns for orchestrator and specialist agents, then validating promotion criteria.

### Phase A: Baseline config contract creation

1. Select one orchestrator and one specialist agent.
2. For each, write a config contract including:
   - mission and scope,
   - output quality expectations,
   - prohibited behavior,
   - required tool classes.
3. Define one measurable hypothesis for improvement vs current active config.

Checkpoint A:

- Both configs have explicit contracts and measurable hypotheses.

### Phase B: Pattern template implementation

4. Open **AI Agents → Configurations** for each selected agent.
5. Implement pattern templates:
   - modular prompt fragments,
   - model settings aligned to role,
   - execution protocol stages when the role needs explicit sequencing,
   - max tool calls and execution policy,
   - on-topic/off-topic and refusal behavior.
6. Save as new versions (do not replace baseline yet).

Checkpoint B:

- New versions are structurally consistent with pattern templates.

### Phase C: Tool policy and guardrail tuning

7. Apply minimal tool allow-list per role.
8. Add intent-to-tool policy instructions for key tool classes.
9. Define fallback behavior for tool failures and missing inputs.
10. Verify high-impact actions require explicit preconditions.

Checkpoint C:

- Tool usage policy is explicit, constrained, and role-appropriate.

### Phase D: Conversation validation pass

11. Run 12-20 test prompts across:

- core in-scope tasks,
- ambiguous tasks,
- out-of-scope/safety boundary tasks.

12. Record per prompt:

- response quality,
- routing correctness,
- tool behavior,
- boundary compliance,
- latency notes.

13. Compare results against prior active config behavior where relevant.

Checkpoint D:

- Validation evidence shows whether hypotheses are supported.

### Phase E: Diff/merge and promotion decision

14. Use config comparison to isolate beneficial changes.
15. Keep successful changes; revert regressions.
16. Apply promotion gates and decide:

- promote to active,
- iterate and re-test,
- rollback.

17. Record decision rationale and risk notes.

Checkpoint E:

- Promotion decision is evidence-driven and reversible.

### Phase F: Publish config pattern pack

18. Create `chapter-04-section-02-config-pattern-pack.md` including:

- role-based config templates,
- prompt fragment patterns,
- model/tool/safety defaults,
- validation dataset summary,
- promotion gate results,
- rollback notes.

19. Add go/no-go recommendation for Section 3 tooling implementation.

Checkpoint F:

- Another engineer can apply the pattern pack and reproduce comparable outcomes.

## Expected outputs

By the end of this lab, you should have:

- At least two new configuration versions built from explicit role patterns.
- Documented prompt/model/tool/guardrail coherence per role.
- A validation dataset and results matrix covering normal and boundary prompts.
- Diff/merge decisions tied to measurable behavior outcomes.
- A promotion decision with rollback path documented.
- A Config Pattern Pack artifact with go/no-go for Section 3.

Evidence that qualifies:

- Saved config versions in AI Agents with clear naming/version intent.
- Conversation evidence showing improved or controlled behavior change.
- Artifact documentation sufficient for independent review.

## Failure modes

1. **Configuration as unstructured settings bundle**
   - Symptom: changes are hard to review and impacts are unclear.
   - Recovery: enforce explicit config contracts and version intent notes.

2. **Too many concurrent changes in one version**
   - Symptom: cannot identify what caused behavior shift.
   - Recovery: sequence changes by dimension (prompt, model, tools, guardrails) and validate incrementally.

3. **Prompt and model settings mismatch**
   - Symptom: response style conflicts with expected policy or consistency.
   - Recovery: align model parameters and reasoning settings with role intent.

4. **Tool policy underspecified**
   - Symptom: tools are called excessively or skipped when needed.
   - Recovery: add intent-to-tool triggers, preconditions, and fallback rules.

5. **Boundary/safety behavior untested**
   - Symptom: strong in-scope performance but poor refusal/handoff behavior.
   - Recovery: include explicit boundary prompt suite in every config validation cycle.

6. **Version proliferation without governance**
   - Symptom: many versions, little clarity on which is stable.
   - Recovery: define promotion gates and archive/deprecate stale experiment versions.

7. **Promotion by intuition**
   - Symptom: active config regresses after subjective approval.
   - Recovery: require quantitative and qualitative evidence before activation.

8. **No rollback discipline**
   - Symptom: recovery from bad promotion is slow and disruptive.
   - Recovery: keep prior stable version identifiable and reversible at all times.

## Completion checklist

- [ ] I created explicit configuration contracts for at least one orchestrator and one specialist role.
- [ ] I applied modular prompt, model, tool, and guardrail patterns consistently.
- [ ] I validated configuration behavior with representative and boundary prompt sets.
- [ ] I used diff/merge decisions to isolate beneficial changes and reject regressions.
- [ ] I made an evidence-based promotion decision with rollback strategy.
- [ ] I documented reusable role-based templates in a Config Pattern Pack artifact.
- [ ] I produced a go/no-go recommendation for Section 3 tooling patterns.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [TypeScript Tools (Engineering Reference)](../../user-guide/agents/typescript-tools.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Agent Config Checklist](../../user-guide/conversations/agent-config-checklist.md)
- [Evals](../../user-guide/evals/README.md)

---

# Tooling Patterns

## Learning objectives

By the end of this section, you should be able to:

- Design tool surfaces that are minimal, safe, and aligned with agent role boundaries.
- Define robust input/output contracts for tools, including failure and fallback semantics.
- Implement a practical tool testing and rollout workflow using Gaia registry and config capabilities.
- Produce a tooling pattern artifact that supports reliable scaling of agent capabilities.

## Prerequisites

- You completed [Config Patterns](#doc-ch04-agent-engineering-02-config-patterns) and have at least one stable baseline config.
- You can access **AI Agents → Configurations**, **Data Model → Tool registry**, and **Conversations**.
- You have one candidate workflow that requires tool-assisted execution (read/search/write/navigation/delegation).
- You can evaluate tool behavior using conversation traces and targeted prompts.

## In Gaia

- [Tool registry](../../user-guide/data-model/tool-registry.md) for reusable tool definitions
- [Agent Configuration](../../user-guide/agents/configs.md) for per-agent tool policy
- [Conversations](../../user-guide/conversations/README.md) and [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md) for tool-call evidence

Use this section to shape one real tool surface and one real attachment policy in Gaia. Keep read, write, and side-effect paths explicit.

## Concept brief

Tooling is where language behavior becomes system behavior. Prompts decide intent, but tools create observable effects: querying entities, updating records, navigating UI, orchestrating workflows, and delegating tasks.

In Gaia, most severe reliability incidents are tool-policy incidents, not text-generation incidents:

- tool called with ambiguous arguments,
- wrong tool selected for task class,
- side-effect action without precondition checks,
- recoverable tool errors treated as final truth.

Tooling patterns reduce these risks by standardizing how tools are designed, enabled, tested, and governed.

They also improve delivery velocity. When tool contracts and validation patterns are consistent, onboarding new engineers and adding new capabilities becomes incremental instead of disruptive. You spend less time reverse-engineering tool behavior and more time improving user outcomes.

### 1) Design tools around user-intent jobs, not API endpoints

A tool should represent a meaningful job from the agent's perspective, not a raw backend primitive.

Weak pattern:

- one generic tool with many optional fields and many implicit modes.

Strong pattern:

- focused tools with clear purpose and predictable argument shape.

Example categories:

- retrieve context,
- inspect record details,
- update structured field set,
- trigger workflow,
- navigate to relevant UI page,
- delegate to specialist.

When tools mirror intent jobs, model selection is more accurate and tool traces are easier to audit.

### 2) Keep tool contracts strict, explicit, and JSON-friendly

Model-generated tool arguments are probabilistic. Your contract must absorb variance safely.

Define for each tool:

- required fields,
- optional fields and defaults,
- type constraints,
- enum/domain constraints,
- expected output schema,
- error object schema.

Contract clarity helps both runtime safety and debugging. If your output shape changes across success/failure arbitrarily, downstream reasoning degrades.

Prefer explicit error objects over opaque text blobs so the agent can choose the next step (retry, clarify, fallback, escalate).

### 3) Separate read tools from write tools in policy and ownership

Read and write tools have different risk profiles and should follow different governance.

Read tools:

- usually safe to call broadly,
- still require query constraints to avoid noisy/expensive retrieval.

Write tools:

- require stricter role boundaries,
- need precondition checks,
- need idempotency considerations,
- often need user confirmation behavior for high-impact actions.

Per-role patterns:

- orchestrator: mostly read + delegate.
- specialist: focused write capabilities in narrow domain.

This separation simplifies incident response because side-effect paths are easier to isolate.

### 4) Tool descriptions are behavior controls, not documentation garnish

The model relies heavily on tool descriptions to decide when and how to call a tool. Poor descriptions cause misuse.

Strong description elements:

- what the tool does,
- when to use it,
- when not to use it,
- required argument expectations,
- key limitations.

Keep descriptions concise but unambiguous. Overly broad language invites incorrect calls. In practice, improving descriptions often produces bigger gains than tweaking model settings.

### 5) Prefer minimal tool sets and progressive enablement

Tool sprawl is a common quality killer. More tools increase ambiguity and selection errors.

Enable tools progressively:

1. start with minimum viable set for one workflow,
2. validate tool-call correctness,
3. add one tool at a time with explicit use case,
4. revalidate before promotion.

Use registry reuse to keep definitions consistent, but do not automatically attach all registry tools to all configs.

### 6) Design for partial failure and graceful degradation

External dependencies fail. Entity queries can return empty. Workflow triggers can time out. Your tooling pattern should define behavior for these states.

Per-tool failure policy:

- retryable vs non-retryable error classes,
- user-visible messaging strategy,
- fallback tool or manual path,
- logging/diagnostic payload included in response.

If tool failures are not machine-readable, the agent often hallucinates continuity. Structured failure outputs help the agent respond safely.

### 7) Build deterministic tests for high-impact tools

You cannot rely only on free-form conversation tests for tooling reliability.

For high-impact tools, test at three levels:

- registry-level manual test payloads,
- configuration-level conversation calls,
- scenario-level end-to-end path with expected state changes.

Track for each test case:

- inputs,
- expected output shape,
- expected side effects,
- failure behavior.

When tests are explicit, refactors become safer and rollback decisions become faster.

### 8) Use registry patterns for reuse with governance

Gaia Tool Registry and Skill Registry support reuse, but reuse without governance can propagate bugs at scale.

Governance pattern:

- keep registry tool owner defined,
- tag tools by domain and risk,
- keep every webservice or MCP destination on a static, reviewed origin; private origins require a
  separate host-operator exception and must never be derived from model arguments,
- track linked configurations,
- test registry changes before broad propagation,
- maintain deprecation notes for replaced tools.

When you copy an agent bundle from another project, treat compatible target-tool matches as an explicit review point. Approve a mapping only when the parameter contract still represents the same business capability; otherwise keep the name and publish the imported definition under a new version so the target project preserves a clear change history.

For composite capabilities, skill bundles can encapsulate prompt + tool combinations, reducing per-config drift.

### 8.1) Use composite tools for synchronous, inspectable orchestration

Composite tools are the right pattern when one user-visible action needs several saved registry tools to run in a fixed order and return one final result.

Good composite cases:

- read, then write, then navigate
- resolve or create a runtime artifact and then persist linked state
- bundle a small, stable sequence into one reusable business action

Conversation artifact tools are a good example of this contract discipline. Agents should create or reuse the right canonical artifact type for the job: `markdown` for documents, `grid-json` for spreadsheets, `slides-json` for presentation decks, `pdf-annotations-json` for PDF review state, and `ui-layout-canvas` for runtime UI sessions. For PPTX sources, agents should work against the normalized `slides-json` canvas artifact and keep warnings visible when the bounded import cannot preserve the full PowerPoint surface. Exports should come from verified artifact tool output, not from links invented in assistant prose.

Prefer a composite tool when:

- the work should finish inside one turn
- the capability needs deterministic step ordering
- authors need registry-level testing with step traces and postconditions
- the same higher-level action should be reused across multiple configs

Prefer a TypeScript tool when:

- most of the value is local computation or reshaping rather than orchestration
- the logic is too bespoke to justify exposing every internal step as a reusable registry tool

For TypeScript tools, make the execution runtime part of the review. Use **QuickJS WebAssembly**
when the script passes its registry test there; it has reduced risk because Gaia capabilities cross
an explicit bridge. Keep **Legacy Node.js** only when a trusted script needs its greater
compatibility, and record acceptance of its high shared-process risk.

Prefer a workflow when:

- work is long-running, approval-based, scheduled, or asynchronous
- you need explicit run tracking beyond one synchronous tool result
- the process is closer to guided execution than a single atomic action

Operating rules for composites:

- author them in **Data Model → Tool registry**, not in config-local copies
- use the structured composite builder for common step, output, and postcondition edits, then drop into advanced JSON only for shapes the builder does not yet model well
- keep step names tied to business jobs, not raw backend implementation details
- aim for one canonical final UI effect rather than several competing step-level navigation outcomes
- run fixed positive and negative tests in the Tool Registry before broad rollout
- treat nested registry-tool references as lifecycle dependencies that must survive import and transfer, not as disposable editor-only UUIDs
- keep nesting shallow so the composite remains inspectable and reviewable

### 9) Observe tool behavior as a first-class quality metric

Tooling quality should be measured explicitly, not inferred from final responses.

Key metrics:

- tool call success/failure rate,
- wrong-tool selection frequency,
- average calls per turn,
- retry frequency,
- side-effect error rate,
- latency contribution by tool class.

Use these metrics to decide:

- whether to simplify tool set,
- whether to split tools,
- whether to tighten descriptions/contracts,
- whether to adjust role boundaries.

### 10) Definition of done for Section 3

Tooling pattern work is done when tool-assisted behavior is predictable, auditable, and safe under normal and edge conditions.

Done criteria:

- minimal tool sets per role are explicit,
- contracts/descriptions are strict and actionable,
- high-impact tools have deterministic test cases,
- registry reuse is governed,
- failure behavior is graceful and structured,
- lab evidence demonstrates reliable tool-enabled flows.

If another engineer can add a new tool using your pattern guide without degrading baseline quality, your tooling discipline is operationally strong.

This is especially important when multiple teams contribute tools in parallel. Pattern consistency prevents local optimizations from becoming system-wide instability, and it makes it easier to tie tool-level changes to measurable quality and cost outcomes.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Tooling Pattern and Validation Matrix" artifact.

### Scenario

You need to support a multi-step assistant behavior that requires both read and write operations, while maintaining strict safety and observability.

### Phase A: Tool inventory and risk classification

1. List all tools currently enabled in your target configuration.
2. Classify each as:
   - read,
   - write,
   - navigation,
   - delegation,
   - external integration.
3. Mark risk level (low/medium/high) based on side-effect impact.
4. Identify redundant or unused tools.

Checkpoint A:

- Tool inventory includes risk and purpose clarity.
- Redundant tools are identified for potential removal.

### Phase B: Contract hardening

5. For the top 5 most-used tools, document input/output contracts.
6. Add explicit required fields and error response structure.
7. Improve tool descriptions to include use/non-use guidance.
8. For high-risk write tools, define preconditions and confirmation behavior.

Checkpoint B:

- Contracts and descriptions reduce ambiguity for model tool selection.

### Phase C: Registry and configuration alignment

9. Review tools in **Data Model → Tool registry** and align naming/ownership.
10. Link reusable tools from registry rather than duplicating definitions where appropriate.
11. Remove or disable unnecessary config-local tools.
12. If helpful, define skill bundles that combine prompt fragments with curated tool sets.

Checkpoint C:

- Tool reuse is intentional and governed.
- Config tool surface is minimal and role-aligned.

### Phase D: Deterministic tool tests

13. For high-impact tools, run registry-level tests with fixed payloads.
14. Record expected vs actual outputs and side effects.
15. Add at least one negative test per high-impact tool (invalid input, missing data, or denied action).
16. Update contracts/descriptions if tests reveal ambiguity.

Checkpoint D:

- High-impact tools pass deterministic positive and negative tests.

### Phase E: Conversation-level validation

17. Run 12-15 prompts that trigger different tool paths.
18. Record:

- tool selection correctness,
- argument quality,
- failure handling behavior,
- user-visible response quality.

19. Compare before/after metrics for tool-call reliability.

Checkpoint E:

- Conversation evidence confirms improved tool behavior and safer failure handling.

### Phase F: Publish tooling artifact

20. Create `chapter-04-section-03-tooling-pattern-validation-matrix.md` with:

- tool inventory + risk matrix,
- hardened contracts,
- registry linkage decisions,
- deterministic test cases/results,
- conversation validation outcomes,
- next hardening backlog.

21. Add go/no-go recommendation for Section 4 handoff/multi-agent control.

Checkpoint F:

- Another engineer can reuse and extend your tooling pattern with predictable outcomes.

## Expected outputs

By the end of this lab, you should have:

- A role-aligned minimal tool surface for at least one key configuration.
- Hardened contracts and descriptions for high-impact tools.
- Registry governance decisions documented (reuse, ownership, deprecations).
- Deterministic test evidence for high-risk tool paths.
- Conversation-level proof of improved tool selection and error handling.
- A Tooling Pattern and Validation Matrix artifact with go/no-go for Section 4.

Evidence that qualifies:

- Updated tool definitions/configs visible in Gaia.
- Test notes with fixed payloads and expected outcomes.
- Prompt-level traces demonstrating better tool-call reliability.

## Failure modes

1. **Tool sprawl without role alignment**
   - Symptom: model picks irrelevant tools and response quality becomes erratic.
   - Recovery: reduce enabled tool set to minimum role-specific capabilities.

2. **Ambiguous tool contracts**
   - Symptom: arguments arrive in inconsistent shapes, causing runtime errors.
   - Recovery: harden required fields, types, and structured error outputs.

3. **Poor tool descriptions**
   - Symptom: wrong tool selected for similar intents.
   - Recovery: rewrite descriptions with explicit usage boundaries and examples.

4. **Write tools exposed too broadly**
   - Symptom: non-specialist agents perform unintended state changes.
   - Recovery: restrict write tools to specialist roles and enforce preconditions.

5. **No deterministic testing for critical tools**
   - Symptom: regressions discovered only in production-like conversations.
   - Recovery: add fixed-payload positive/negative tests for high-impact tools.

6. **Registry reuse without change control**
   - Symptom: one registry edit causes widespread unintended behavior shifts.
   - Recovery: define owner review and staged rollout process for shared tools.

7. **Failure outputs not machine-readable**
   - Symptom: agent cannot decide retry vs fallback and may hallucinate success.
   - Recovery: standardize error object schema for tool responses.

8. **Tool metrics ignored in evaluation**
   - Symptom: final response seems acceptable while tool reliability degrades.
   - Recovery: track tool-call metrics as explicit quality indicators in reviews.

## Completion checklist

- [ ] I classified enabled tools by purpose and risk and removed unnecessary entries.
- [ ] I hardened contracts/descriptions for high-impact tools.
- [ ] I enforced stronger boundaries for write and high-risk actions.
- [ ] I aligned reusable tools through Tool Registry with ownership notes.
- [ ] I executed deterministic positive and negative tests for critical tools.
- [ ] I validated tool behavior through conversation prompts and recorded evidence.
- [ ] I published a Tooling Pattern and Validation Matrix artifact with go/no-go for Section 4.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [TypeScript Tools (Engineering Reference)](../../user-guide/agents/typescript-tools.md)
- [Tool Registry](../../user-guide/data-model/tool-registry.md)
- [Skill Registry](../../user-guide/agents/skills.md)
- [Conversations](../../user-guide/conversations/README.md)

---

# Handoffs And Multi-Agent Control

## Learning objectives

By the end of this section, you should be able to:

- Design handoff rules that route requests across agents predictably and safely.
- Prevent multi-agent failure patterns such as loops, ping-pong routing, and ambiguous ownership.
- Implement governance for delegation depth, escalation behavior, and user-visible continuity.
- Produce a multi-agent control runbook that supports reliable operation and continuous improvement.

## Prerequisites

- You completed [Agent Architecture](#doc-ch04-agent-engineering-01-agent-architecture), [Config Patterns](#doc-ch04-agent-engineering-02-config-patterns), and [Tooling Patterns](#doc-ch04-agent-engineering-03-tooling-patterns).
- You have at least two agents configured in Gaia, including one orchestrator.
- You can edit handoff rules in **AI Agents → Agent Configuration → Handoff**.
- You can test and inspect routing behavior through representative conversation prompts.

## In Gaia

- [AI Agents](../../user-guide/agents/README.md) and [Agent Configuration](../../user-guide/agents/configs.md) for handoff rules and role boundaries
- [Conversations](../../user-guide/conversations/README.md) and [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md) for routing evidence
- [Evals](../../user-guide/evals/README.md) when handoff quality needs repeatable regression protection

This section should end with one real handoff path that can be observed in Gaia without guesswork about who owned which turn.

## Concept brief

Handoffs turn isolated agents into a coordinated system. Without disciplined control, multi-agent setups can degrade quickly:

- requests bounce across agents,
- specialists handle wrong intent classes,
- user context is lost during transfer,
- tool actions execute without clear ownership.

Good handoff design is not just "if intent X then route to agent Y." It includes routing confidence, ambiguity strategy, recursion limits, fallback logic, and user communication continuity.

The objective is simple: each request should be handled by the best available role with minimal confusion and maximal traceability.

In mature Gaia deployments, routing quality often becomes a stronger differentiator than raw prompt sophistication. Two assistants can share similar models and tools, yet the one with better handoff control feels significantly more reliable because users reach the right capability faster and receive fewer contradictory responses.

### 1) Handoffs are routing contracts, not heuristics

Treat each handoff as a contract between source and target agent.

A strong contract includes:

- trigger condition,
- target role rationale,
- required context fields,
- expected target output,
- return/escalation behavior.

If routing rules are only keyword hints without contract semantics, behavior drifts as prompts evolve.

In Gaia, handoff rules should be explicit enough that two reviewers independently predict the same route for the same prompt.

Gaia now supports three routing styles inside the **Handoff** tab:

- text conditions for natural-language routing boundaries,
- regex rules for deterministic pattern matching across the extracted conversation context,
- TypeScript rules for deterministic programmatic routing that can inspect the cloned messages array, use standard transform utilities, and append hidden handoff notes for the transfer.

When a TypeScript handoff rule needs to coordinate with later tools, Bridge Agent hooks, or routing scripts, use shared conversation storage with purpose-prefixed keys. This keeps the transfer state durable without turning hidden handoff notes into an overloaded data bus.

These rules execute in list order. First match wins. That means rule order is part of the contract, not just presentation.

Conversation-level collaboration adds a second, lighter control surface. Users can invite agents into a specific conversation and either mention an agent directly or let Gaia select up to two invited agents based on each agent's name and description. Channel owners must enable **Allow collaboration** on the Text or Personal Assistant channel before that channel exposes participant invites, mention autocomplete, or invited-agent reactions. Treat this as "multi-agent reaction" rather than a durable handoff: the selected agents answer the current turn visibly, while the normal coordinator does not produce its own full answer for that turn.

Use conversation-level invited-agent reactions when:

- multiple specialists may contribute useful parallel perspectives to one user turn,
- the thread needs a named specialist to answer without changing the conversation's default agent,
- the collaboration is local to one transcript and should not alter global handoff rules.

Use formal handoff rules when routing should be repeatable across many conversations, governed by configured conditions, or part of a stable operating boundary between agents.

### 2) Preserve ownership clarity across the route

In multi-agent systems, unclear ownership causes duplicated work and contradictory answers.

Define ownership policy:

- orchestrator owns intent classification and user experience continuity,
- specialist owns domain decision quality,
- escalation owner handles unresolved or high-risk requests.

For each handoff edge, define who owns:

- final response synthesis,
- unresolved ambiguity resolution,
- side-effect authorization.

Ownership clarity is the difference between coordinated delegation and conversational chaos.

### 3) Route by capability + confidence, not keywords alone

Keyword-based routing is brittle. Use capability and confidence framing:

- Does target agent have required tools/data?
- Is user intent sufficiently clear for direct handoff?
- Is clarification cheaper and safer than routing now?

Pattern:

- high confidence + right capability -> handoff.
- low confidence -> ask focused clarification first.
- no capable target -> decline/escalate explicitly.

This reduces unnecessary transfers and improves user trust.

Operational guidance for rule choice:

- Use text conditions when reviewers can describe the boundary clearly in natural language and you still want Gaia to arbitrate ambiguous context.
- Use regex rules when the route depends on stable lexical signals such as ticket prefixes, invoice formats, or product codes.
- Use TypeScript rules when the route depends on deterministic data checks, entity lookups, storage reads, or transfer notes that should be attached to the handoff without appearing as normal transcript messages.
- Keep broad catch-all text rules below narrower deterministic rules unless you intentionally want them to preempt everything else.

### 4) Prevent loops and routing ping-pong

Looping is a core multi-agent failure mode. It often happens when boundary definitions overlap and no depth control exists.

Loop prevention controls:

- explicit non-return paths unless conditions are met,
- depth limits on chained handoffs,
- anti-ping-pong rules (do not hand back without new information),
- ambiguity fallback to orchestrator or human escalation.

In Gaia flows, recursion depth is bounded. Architecture should still avoid relying on hard limits as the primary safeguard.

### 5) Carry context intentionally across handoffs

A handoff should preserve relevant context, not copy entire conversation indiscriminately.

Context transfer pattern:

- intent summary,
- key entities/IDs involved,
- constraints and boundaries,
- unresolved questions,
- expected output shape.

Too little context forces repeated clarification. Too much noisy context increases target confusion and latency. Good handoffs transfer just enough to execute accurately.

### 6) Keep user-visible continuity during delegation

Users should understand that routing happened and why, without exposing internal complexity.

Continuity guidelines:

- provide concise handoff acknowledgment,
- avoid repetitive introductions at each agent hop,
- preserve conversational thread tone and goal,
- explain clarifying questions in relation to user objective.

A system can be technically correct but still feel broken if routing narrative is abrupt or inconsistent.

For invited-agent reactions, continuity is even more important because several visible assistant replies can appear for one user message. Keep invited specialist descriptions specific enough that Gaia selects them only when their role is clear. Encourage users to mention a named agent when they want a particular specialist to answer. Human mentions should be treated as notifications, not as a live human takeover workflow.

### 7) Constrain side effects in delegated paths

Delegated chains can hide side-effect decisions. Ensure high-impact actions remain governed.

Controls:

- restrict write tools to agents with explicit ownership,
- require preconditions and confirmations for critical actions,
- prohibit downstream agent from expanding scope silently,
- require structured action results for auditability.

Delegation should increase precision, not bypass safeguards.

### 8) Monitor routing quality as a first-class metric

Multi-agent quality is not only answer quality. Routing quality must be measured.

Track metrics such as:

- correct first-route rate,
- clarification-before-handoff rate,
- average handoff hops per resolved request,
- loop/near-loop incident count,
- unresolved escalation rate,
- user satisfaction on routed conversations.

These metrics reveal whether architecture boundaries are working or drifting.

### 9) Build escalation and recovery playbooks for routing failures

When routing fails, teams need predictable recovery.

Playbook components:

- detect routing anomaly,
- isolate offending rule/agent boundary,
- apply minimal routing fix,
- run targeted regression prompt set,
- document preventive control.

Without a playbook, multi-agent incidents are often "fixed" by broad prompt rewrites that introduce new regressions.

### 10) Definition of done for Section 4 and Chapter 4

Handoffs and multi-agent control are done when routing is predictable, explainable, and resilient under edge cases.

Done criteria:

- handoff contracts are explicit and role-aligned,
- loop prevention and depth policies are defined,
- context transfer and continuity behavior are validated,
- delegated side effects are governed,
- routing quality metrics and escalation playbook exist,
- section lab demonstrates stable multi-agent behavior.

At this point, Chapter 4 outcomes become actionable inputs for Chapter 5 experience and channel design.

This is also where teams shift from configuration craftsmanship to operational governance. Handoff behavior, escalation quality, and routing drift need a recurring review rhythm so control quality improves over time instead of degrading silently.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Multi-Agent Handoff Control Runbook" artifact.

### Scenario

Your assistant ecosystem includes an orchestrator and multiple specialists. You must guarantee reliable routing across normal, ambiguous, and high-risk requests while keeping user experience coherent.

### Phase A: Handoff map and ownership matrix

1. List all agents participating in multi-agent flow.
2. Build a handoff map showing possible source -> target routes.
3. For each route, define:
   - trigger intent class,
   - required context,
   - target ownership,
   - expected final output.
4. Create an ownership matrix for decision authority and side-effect authority.

Checkpoint A:

- Routing map and ownership matrix cover all critical intent classes.

### Phase B: Rule implementation in Gaia

5. Open each agent configuration and review **Handoff** rules.
6. Implement/update rules to match routing contracts.
7. Add ambiguity-handling policy (clarify before handoff when needed).
8. Ensure no circular rules exist without explicit break conditions.

Checkpoint B:

- Implemented rules align with documented map and avoid obvious loop paths.

### Phase C: Context transfer and continuity checks

9. Define what summary/context each handoff should preserve.
10. Run test prompts that require one-hop and two-hop routing.
11. Validate:

- context needed by specialist is present,
- user-facing continuity is coherent,
- no redundant restarts of conversation intent.

Checkpoint C:

- Delegated conversations remain coherent and context-aware.

### Phase D: Edge-case and loop-resilience testing

12. Run adversarial/ambiguous prompts designed to provoke misrouting.
13. Run prompts that could trigger back-and-forth handoffs.
14. Confirm depth/recursion control and fallback behavior.
15. Capture any ping-pong or unresolved routing incidents.

Checkpoint D:

- Loop/ping-pong risks are identified and mitigated with rule adjustments.

### Phase E: Side-effect governance validation

16. Run prompts that request high-impact actions through delegated paths.
17. Verify only authorized specialist roles execute side-effect tools.
18. Confirm precondition/confirmation behavior before state changes.
19. Validate action results are reported with structured clarity.

Checkpoint E:

- Delegated side effects remain controlled and auditable.

### Phase F: Publish control runbook and chapter closure recommendation

20. Create `chapter-04-section-04-multi-agent-handoff-control-runbook.md` including:

- routing map,
- ownership matrix,
- handoff rules and break conditions,
- test prompt suite and outcomes,
- loop-resilience findings,
- side-effect governance checks,
- escalation playbook,
- routing quality metrics baseline.

21. Add Chapter 4 readiness recommendation and carry-over risks for Chapter 5.

Checkpoint F:

- Another engineer can operate, diagnose, and improve the multi-agent routing system using your runbook.
- The runbook includes a practical review cadence (for example weekly routing review and monthly boundary audit).

## Expected outputs

By the end of this lab, you should have:

- A complete handoff map across orchestrator and specialist agents.
- Documented routing contracts and ownership model.
- Implemented handoff rules validated against representative and adversarial prompts.
- Evidence of loop prevention and controlled delegation depth.
- Verified governance for delegated side-effect actions.
- A Multi-Agent Handoff Control Runbook artifact with Chapter 4 readiness recommendation.

Evidence that qualifies:

- Handoff rules visible in agent configurations and consistent with runbook.
- Conversation traces showing stable one-hop and multi-hop routing.
- Explicit incident notes for misroutes/loops and corresponding mitigations.
- A first routing baseline report (first-route accuracy, average hop count, unresolved escalation rate).

## Failure modes

1. **Keyword-only routing rules**
   - Symptom: similar prompts route inconsistently as wording changes.
   - Recovery: rewrite routing as capability + confidence contracts with clear conditions.

2. **Ownership ambiguity after handoff**
   - Symptom: conflicting responses or duplicate actions between agents.
   - Recovery: define and document authority per route (final response vs action execution).

3. **Handoff loops or ping-pong behavior**
   - Symptom: repeated transfers with no progress.
   - Recovery: add break conditions, anti-return rules, and clearer boundary scopes.

4. **Context loss during transfer**
   - Symptom: specialist asks for information already provided.
   - Recovery: standardize handoff context package and enforce minimal required context fields.

5. **Delegated side-effect bypass**
   - Symptom: unauthorized agent executes write actions via indirect path.
   - Recovery: tighten tool permissions and confirmation policies per role.

6. **User-visible discontinuity**
   - Symptom: conversation feels reset or fragmented after handoff.
   - Recovery: add continuity instructions and concise transfer acknowledgments.

7. **No routing-quality metrics**
   - Symptom: handoff degradation noticed only anecdotally.
   - Recovery: track first-route accuracy, hop count, loop incidents, and escalation outcomes.

8. **No escalation playbook for routing incidents**
   - Symptom: emergency fixes are broad and unstable.
   - Recovery: adopt minimal-fix routing playbook with targeted regression prompt suite.

9. **No cadence for routing drift review**
   - Symptom: routing quality slowly degrades and issues are discovered late.
   - Recovery: establish recurring routing-quality review with named owners and trend tracking.

## Completion checklist

- [ ] I created a complete handoff map and ownership matrix for participating agents.
- [ ] I implemented routing rules that reflect explicit capability and boundary contracts.
- [ ] I validated context continuity across one-hop and multi-hop delegation.
- [ ] I tested and mitigated loop/ping-pong routing risks.
- [ ] I verified governance of delegated side-effect actions.
- [ ] I established routing quality metrics and escalation playbook basics.
- [ ] I published a Multi-Agent Handoff Control Runbook and Chapter 4 readiness recommendation.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Agent Config Checklist](../../user-guide/conversations/agent-config-checklist.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval Design Process](../../user-guide/evals/eval-design-process.md)
- [Eval Runs](../../user-guide/evals/runs.md)

---

# Execution Protocols

## Learning objectives

By the end of this section, you should be able to:

- decide when an agent needs an explicit execution protocol instead of a stronger prompt alone
- compare execution protocols with workflow graphs and decide when one, the other, or both are justified
- design reviewable stage sequences with scoped authority, checkpoints, and approval gates
- frame deep research as a protocol-only design or a hybrid workflow-plus-protocol design
- operate live staged sessions without losing context or auditability
- identify common failure modes such as over-staging, approval fatigue, and meaningless checkpoints

## Prerequisites

- You completed [Config Patterns](#doc-ch04-agent-engineering-02-config-patterns) and [Tooling Patterns](#doc-ch04-agent-engineering-03-tooling-patterns).
- You can edit **AI Agents -> Agent Configuration -> Execution**.
- You can manage reusable templates in **AI Agents -> Protocols**.
- You can test a staged run in [Conversations](../../user-guide/conversations/README.md).

## In Gaia

- [Agent Configuration](../../user-guide/agents/configs.md) for the actual staged behavior that runs on an agent configuration
- [Protocols](../../user-guide/agents/protocols.md) for reusable protocol templates that can be applied across multiple agents
- [Conversations](../../user-guide/conversations/README.md) for observing live stage behavior and inline review actions
- [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md) when you need a turn-by-turn record around a staged run
- [Workflows](../../user-guide/data-model/workflows.md) when the main problem is reusable orchestration around data, tools, approvals, or branching rather than staged agent behavior
- [Evals](../../user-guide/evals/README.md) when the protocol needs repeatable verification instead of anecdotal confidence

This section is done when your team can explain why a staged workflow exists, how it moves forward, and how an operator should intervene when it stalls.

## Concept brief

Execution protocols turn agent behavior into an explicit operating model.

Without a protocol, one turn is mostly governed by prompts, tool policy, and runtime limits. That is often enough. But some work benefits from something more controlled: a visible sequence of stages with clear boundaries, durable checkpoints, and optional approval gates.

That is the purpose of execution protocols in Gaia.

An execution protocol is not hidden chain-of-thought. It is a reviewable state machine that operators can inspect. Each stage has a specific job, a limited authority surface, and a clear record of what happened before the next stage begins.

### 1) Use protocols when sequence changes risk or accountability

Do not add stages because a workflow sounds sophisticated. Add stages when the order itself matters.

Good reasons to use a protocol:

- planning must complete before implementation is allowed
- research and evidence gathering must be separated from final synthesis
- a reviewer must approve work before the next stage can proceed
- structured outputs from one stage are needed by later stages or operators
- the same governed sequence should be reused across multiple agents

Weak reasons to use a protocol:

- the team wants the agent to sound more thoughtful
- a simple assistant already performs well with good prompts and tool policy
- stages exist only to restate the same instructions in different words

If sequencing does not materially improve control, traceability, or operator confidence, keep the agent simpler.

### Comparison: execution protocols vs workflow graphs

Teams often confuse execution protocols with workflow graphs because both introduce visible structure. The important distinction is where the control problem lives.

An execution protocol governs an agent's behavior inside a conversation. It answers questions such as:

- should the agent plan before it writes?
- should a reviewer approve the next step?
- should the same request stay visible as staged work instead of collapsing into one opaque turn?

A workflow graph governs reusable automation outside or around that conversation. It answers questions such as:

- should the system branch based on state?
- should the same run loop over items or wait on a human-action node?
- should evidence, source files, approvals, or downstream actions follow a repeatable path every time the automation runs?

That distinction is easier to see in a comparison table:

| If the main design problem is...                                                             | Prefer...             | Why                                                                                                             |
| -------------------------------------------------------------------------------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------- |
| the agent should not skip from scoping into action                                           | an execution protocol | stage boundaries govern the agent's next move in the conversation                                               |
| the system must branch, loop, route approvals, or prepare evidence repeatedly                | a workflow graph      | the orchestration problem is automation topology, not staged conversation behavior                              |
| the agent needs staged analysis and the surrounding system also needs reusable orchestration | both                  | the workflow graph prepares or routes the work; the protocol governs the agent's staged reasoning and reporting |

Two anti-patterns follow from this:

- do not use protocols to simulate branching, looping, or reusable automation patterns that belong in workflow graphs
- do not build a workflow graph merely to make one agent's analysis feel more deliberate when the real need is staged visibility, checkpoints, and approval control

For the orchestration side of that boundary, see [Pipelines And Workflows](#doc-ch03-data-modeling-on-gaia-03-pipelines-and-workflows).

### 2) Treat each stage as one kind of work

Strong stage design is narrow.

Examples of healthy stage boundaries:

- `Plan`: scope the work, assumptions, dependencies, and unknowns
- `Execute`: make the requested changes or produce the requested artifact
- `Verify`: run checks, capture evidence, and state readiness

Examples of weak stage boundaries:

- `Work on request`
- `Think and act`
- `Plan, implement, verify, and summarize`

If a stage mixes planning, implementation, and verification, another operator will not know what “done” means for that stage.

### 3) Stage authority should shrink, not just stage wording

The main operational value of a protocol is not the labels. It is that each stage can narrow what the agent is allowed to do.

Use stage-level controls deliberately:

- narrow the active skills to the ones needed for the current stage
- narrow the active tools, especially write-capable or side-effecting tools
- keep always-on navigation/help/memory controls only when they still help the operator understand the run
- use expected outputs and exit criteria so completion is explicit instead of inferred

If every stage has the same tools, same skills, and same authority, the protocol is mostly decorative.

### 4) Checkpoints are the durable handoff surface

Every staged turn should leave behind a checkpoint that another operator can read quickly.

Useful checkpoints contain:

- a short summary of what the stage accomplished in that turn
- structured outputs only when they help later stages or reviewers
- open questions when the stage cannot progress safely
- an explicit status: still in progress, ready to move on, or blocked

Structured outputs are most valuable when downstream work depends on them. Use typed fields when another stage, another operator, or a governance review needs predictable data.

Do not force complex structured output schemas onto every stage. Over-specified checkpoints create busywork and degrade adoption.

### 5) Approval gates are a control, not a ritual

Requiring confirmation between stages is useful when the pause changes risk.

Good uses of confirmation gates:

- approving a plan before implementation starts
- approving sensitive work before a write action or external update
- approving the shift from evidence gathering into final recommendation

Bad uses of confirmation gates:

- pausing after every small stage because “more approval feels safer”
- adding approvals when no real decision is being made
- forcing reviewers to approve work that is already obvious from the checkpoint

Approval fatigue is real. If operators click through every transition automatically, the gate is not adding control.

### 6) Live session operations are part of the design

Designing the protocol is only half the job. Teams also need to know how to operate it.

In Gaia, a staged run creates a durable execution session. That matters because the session can:

- continue across multiple user turns
- pause in a waiting state until approval arrives
- remain reviewable when blocked instead of silently resetting
- be cancelled without deleting its history
- be restarted from the initial stage when a fresh pass is the safer option

The session is visible in the configuration editor and, for live threads, in the conversation itself. Operators should not need to guess which stage is active or what the last checkpoint said.

In the main project **Conversations** workspace, the toolbar's **Execution** button surfaces the live run as a compact panel above the transcript. Use it to review the current stage, pending next stage, latest checkpoint summary, and typed outputs without leaving the thread. The panel is intentionally read-first: move past a waiting stage by replying with an explicit approval such as "continue" or "approved, move on", and respond to blocked stages by supplying the missing input or evidence. Use the configuration editor for session-administration actions such as cancel or restart.

The protocol lifecycle is part of that operating model, not an afterthought.

Gaia now lets each protocol choose one of three lifecycle modes:

- **One run per conversation:** The protocol starts once for the conversation and does not auto-restart after completion.
- **Restart after completion:** The next user request starts a fresh run from the initial stage after the previous run has completed.
- **Reset every turn:** Each user turn starts from the initial stage, and Gaia cancels any unfinished session from the previous turn.

Use them deliberately:

- choose **One run per conversation** for governed multi-turn work where the session itself is the operating thread
- choose **Restart after completion** for repeatable specialist tasks where the same governed pattern should begin cleanly on the next request
- choose **Reset every turn** only when continuity between turns would add confusion rather than control

Do not use looping stages as a substitute for lifecycle design. If the real need is “run this governed flow again when the user asks again,” the correct control is the lifecycle mode, not a self-loop that keeps the session artificially open.

### 7) Use the Protocols registry for shared patterns

When one staged pattern should be reused across multiple agents, move it into the [Protocols](../../user-guide/agents/protocols.md) registry.

The registry is for reusable architecture. The configuration is for the agent-specific final shape.

That means:

- put the common stage structure, starter skills, and starter tools in the registry template
- apply the template into each configuration that needs it
- customize the copied protocol per agent when role-specific changes are necessary

Do not hand-copy large stage architectures from one agent to another if the team expects them to stay aligned conceptually.

### 8) Common failure modes

- **Over-staging:** Too many small stages make the workflow slower without improving control.
- **Decorative staging:** The stages have different names but the same instructions, skills, and tools.
- **Checkpoint theater:** The agent records summaries that sound polished but do not help another operator decide what happens next.
- **Approval fatigue:** Reviewers approve every stage automatically because the gate does not represent a meaningful decision.
- **Blocked-session ambiguity:** The stage can be marked blocked, but nobody knows what information or action would unblock it.
- **Template drift by copy-paste:** Teams clone staged flows manually instead of using the registry, then lose consistency across agents.

Treat these as operational defects, not stylistic preferences.

### 9) Definition of done for execution-protocol design

An execution protocol is production-ready when:

- the reason for using staged execution is explicit and defensible
- each stage has one clear job and a bounded authority surface
- checkpoints produce useful summaries and structured outputs only where needed
- approval gates exist only where a real control decision is being made
- the lifecycle mode matches how the work should repeat, resume, or reset inside the same conversation
- operators know how to inspect, approve, resume, cancel, and restart sessions
- a live staged run has been tested in Gaia and reviewed with evidence

If another engineer can operate the protocol confidently without tribal knowledge, the design is mature enough.

## Worked example: delivery implementation agent

Consider a delivery implementation agent that handles requests like:

> Add a risk-review step to the onboarding workflow, update the supporting checklist, and confirm the flow is ready for the next release.

This is a good candidate for `Plan -> Execute -> Verify` because the sequence changes operational risk.

If the same agent tries to do everything in one pass, several problems appear quickly:

- the scope can drift while the agent is already making changes
- implementation can begin before a reviewer agrees on the affected workflow and records
- verification gets mixed with more build work, so readiness is never stated cleanly

A stronger staged design would look like this:

| Stage     | Objective                                               | Authority surface                                                                              | Checkpoint output                                           | Why the boundary matters                                                           |
| --------- | ------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| `Plan`    | turn the request into one approved implementation batch | discovery, existing workflow review, release-task review, related governance or eval context   | plan summary, dependencies, open questions                  | prevents write work before the affected surfaces and assumptions are explicit      |
| `Execute` | make the approved updates in Gaia                       | workflow edits, task/checklist updates, related configuration changes already approved in plan | changed resources, implementation notes, remaining blockers | keeps the stage focused on doing the approved work rather than renegotiating scope |
| `Verify`  | prove the batch is ready or explain why it is not       | eval execution, workflow checks, browser or operator evidence capture, readiness reporting     | checks run, evidence, readiness summary                     | makes the final answer about evidence and readiness instead of more hidden edits   |

The approval gate belongs after `Plan`, not after every stage.

That design choice matters:

- it creates one meaningful control point before write work starts
- it avoids approval fatigue during routine execution and verification
- it still gives the operator a clean summary of intended scope, dependencies, and open questions before changes happen

This example also shows why stage-level authority matters.

In `Plan`, the agent should mostly inspect and summarize. In `Execute`, it can use the write-capable tools required for the approved batch. In `Verify`, it should switch back to checks and evidence capture. If all three stages expose the same broad authority surface, the protocol loses most of its operational value.

This is the pattern to copy when you need a builder-style agent that must not collapse planning, implementation, and readiness evidence into one opaque turn.

## Concept pattern: deep research as light, full, or hybrid design

Deep research is a useful teaching case because teams often misdiagnose where the complexity actually lives.

Sometimes the hard problem is the agent's staged analytical behavior. Sometimes the hard problem is the evidence pipeline around it. Sometimes both are true.

### Light deep research: protocol only

Use a protocol by itself when the main concern is governing how the research conversation unfolds.

That usually means:

- the question needs clarification before evidence gathering starts
- the research stage should stay separate from synthesis
- the answer should be challenged before final delivery
- an operator wants visible checkpoints rather than one polished but opaque answer

In that design, the protocol does most of the work. A sequence such as `Clarify -> Research -> Synthesize -> Verify -> Finalize` is enough because the evidence gathering is still primarily driven from within the governed conversation.

### Full deep research: workflow graph plus protocol

Use both capabilities when the research procedure also depends on repeatable orchestration outside the conversation.

That usually means:

- source collection happens on a schedule or through a recurring intake path
- restricted sources need approval or routing before the agent can use them
- evidence must be normalized, enriched, or filtered before the research agent should see it
- the same preparation path should feed many research runs, not just one conversation

In that design, the workflow graph prepares the evidence landscape and the protocol governs how the research agent works through it. The graph answers, "What evidence path should run?" The protocol answers, "How should the agent analyze and report on that evidence?"

### Hybrid design rule: start where the pressure really is

Many teams should start with the lighter design.

Begin with a protocol when the main risk is that the research agent will blur clarification, evidence gathering, synthesis, and final recommendation into one uncontrolled turn.

Add a workflow graph only when the pressure moves outward from the conversation into reusable orchestration, such as source routing, recurring ingestion, approvals, or multi-step evidence preparation.

This keeps the system honest:

- protocol-first designs avoid inventing automation topology before it is needed
- workflow-first designs are justified only when the orchestration pattern itself is durable and reusable
- hybrid designs are strongest when each layer has one clear responsibility instead of overlapping control logic

A useful diagnostic question is this: if you removed the workflow graph, would the core problem still be staged analytical control? If yes, keep the protocol. If you removed the protocol, would the remaining problem still be recurring orchestration and evidence preparation? If yes, keep the workflow graph.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to create one working execution protocol and run one staged conversation through it.
- Capture one protocol-design risk and one operating improvement before moving on.

### Extended path (recommended)

- Complete all phases, failure-mode drills, and the full checklist.

This lab produces an "Execution Protocol Design Note" artifact.

### Scenario

You have an agent that performs work which should not happen as one uncontrolled turn. You need a visible staged workflow that an operator can understand, review, and, when needed, approve or resume.

### Phase A: Decide whether staged execution is justified

1. Pick one real agent workflow from your project.
2. Write a short answer to three questions:
   - What risk appears when this work happens as one open-ended turn?
   - Which stage boundaries would materially reduce that risk?
   - Which parts of the work still do not need a stage?
3. If you cannot answer those questions clearly, do not add a protocol yet.

Checkpoint A:

- The team can explain why the protocol exists and why a normal config is insufficient.

### Phase B: Design the stage architecture

4. Create the minimum useful stage list.
5. For each stage, define:
   - objective,
   - allowed tools and skills,
   - expected outputs,
   - exit criteria,
   - whether confirmation is required.
6. Add structured output fields only where another stage or reviewer needs predictable data.

Checkpoint B:

- Each stage has a single clear job and the protocol is still small enough to explain on one screen.

### Phase C: Implement in Gaia

7. Open **AI Agents -> Agent Configuration -> Execution**.
8. In agent configurations, Gaia starts from the default template. In the **Protocols** registry, new templates start from a generic custom stage and can optionally apply a built-in preset as a scaffold.
9. Configure the stages, structured outputs, and confirmation gates.
10. Save the configuration and confirm the protocol is reviewable in the Execution tab.

Checkpoint C:

- The configuration shows a coherent staged flow with no duplicate stage ids or ambiguous exit conditions.

### Phase D: Run and inspect a live session

11. Start a conversation that should trigger the protocol.
12. Observe the first stage checkpoint.
13. If the stage should require approval, verify that the run pauses correctly.
14. If the stage is waiting for confirmation, send an explicit approval message such as "continue" or "approved, move on". If the stage is blocked, reply with the missing input or evidence and confirm the stage behavior is still predictable.
15. Review the latest checkpoint, structured outputs, and open questions.

Checkpoint D:

- The live session behaves predictably and the operator can tell what stage is active and why.

### Phase E: Review reuse and operational fit

16. Decide whether the same stage architecture should be reusable across multiple agents.
17. If yes, move the shared version into **Protocols** and document which parts are safe to customize per agent.
18. Record one follow-up improvement for either checkpoint quality, authority scoping, or approval design.

Checkpoint E:

- The team knows whether this protocol is one-off or reusable and has one concrete improvement queued.

## Failure-mode drills

- Remove one stage and test whether control quality actually gets worse. If it does not, the original protocol was too complex.
- Force a stage into `blocked` and confirm the open questions explain what needs to happen next.
- Temporarily over-broaden one stage's tool set and review whether the authority boundary becomes harder to explain.
- Ask a second operator to read the latest checkpoint and predict the next action. If they cannot, the checkpoint is not useful enough.

## Completion checklist

- [ ] The reason for staged execution is explicit.
- [ ] Each stage has one clear responsibility.
- [ ] Stage-level authority is narrower than the overall agent authority where appropriate.
- [ ] Checkpoints are useful to another operator.
- [ ] Approval gates exist only where a real decision is needed.
- [ ] One live staged run has been reviewed in Gaia.
- [ ] The team knows whether the pattern belongs in the Protocols registry.

## Related pages

- [Protocols](../../user-guide/agents/protocols.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Conversations](../../user-guide/conversations/README.md)

---

# Chapter 5: Channel and Experience Design

Status: current

User-facing channel and application experience design

## Sections

- [UI Layouts Basics](#doc-ch05-channel-and-experience-design-01-ui-layouts-basics)
- [Data Binding Patterns](#doc-ch05-channel-and-experience-design-02-data-binding-patterns)
- [Channel Publishing](#doc-ch05-channel-and-experience-design-03-channel-publishing)
- [Conversation UX Patterns](#doc-ch05-channel-and-experience-design-04-conversation-ux-patterns)
- [UI Layout Component Reference](#doc-ch05-channel-and-experience-design-05-ui-layout-component-reference)

## Alignment with User Guide Experience Surfaces

- [Conversations](../../user-guide/conversations/README.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md)
- [Document Folders](../../user-guide/conversations/document-folders/README.md)
- [Workflow actions](../../user-guide/conversations/workflow-actions.md)
- [File extractor registry](../../user-guide/conversations/file-extractor-registry.md)
- [Start a voice conversation](../../user-guide/conversations/dialogs/voice-conversation.md)
- [Tutorials](../../user-guide/tutorials/README.md)

Treat the Conversations header tabs as the canonical navigation spine for this chapter. Move through Channels, UI Layouts, Folders, Actions, and File Extractors in the same order you see them in the product workspace.

Use [Tutorials](../../user-guide/tutorials/README.md) at `/platform/support/tutorials` as an optional accelerator only. If a tutorial path is still catching up to folder, workflow-action, or file-extractor behavior, use the owning user-guide page above as the canonical reference.

## Fast path inside Gaia

1. Start in [Channels](../../user-guide/conversations/channels/README.md) and decide the published experience boundary.
2. Use [UI Layouts](../../user-guide/conversations/ui-layouts/README.md) to shape the operator or end-user surface.
3. Use [Document Folders](../../user-guide/conversations/document-folders/README.md) and [Workflow actions](../../user-guide/conversations/workflow-actions.md) when the experience includes retained documents or human-review work.
4. Return to [Conversations](../../user-guide/conversations/README.md) to validate that the experience reads cleanly in the live channel.

If the user journey is not testable in a published or previewable Gaia channel, this chapter is still a design exercise rather than an implemented experience.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# UI Layouts Basics

## Learning objectives

By the end of this section, you should be able to:

- Design UI layouts in Gaia that match user decision flows instead of static screen mockups.
- Structure layout composition (rows, columns, cards, forms, tables) for readability, responsiveness, and maintainability.
- Align layout design with entity and conversation requirements so agents can reliably open and use canvas views.
- Produce a layout baseline artifact that can be reviewed and reused across channels in Chapter 5.

## Prerequisites

- You completed Chapters 3 and 4 and have at least one working entity model and agent configuration.
- You can access **Conversations -> UI Layouts**, **Data Model → Entities**, and **Conversations** in your project.
- You have one concrete workflow where users must inspect or edit structured records.
- You can define target user roles and the main tasks each role needs to complete in the interface.

## In Gaia

- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md) for the actual layout surface
- [Entities](../../user-guide/data-model/entities.md) for the data semantics the layout must expose
- [Channels](../../user-guide/conversations/channels/README.md) and [Conversations](../../user-guide/conversations/README.md) for where users will actually encounter the layout

Design one real layout in Gaia as you read. The point is to support decisions and write-back flow, not to create detached screen ideas.

## Concept brief

UI layouts in Gaia are not decorative shells around agent responses. They are interaction surfaces where users inspect evidence, review structured records, make edits, and complete decisions. If layout design is weak, the assistant may reason correctly but users still fail to execute work efficiently.

A strong layout design practice treats interface structure as part of system reliability:

- records are easier to validate,
- context is easier to understand,
- write-back actions are less error-prone,
- conversation-to-canvas transitions feel coherent.

In Chapter 5, Section 1 establishes the foundation: how to build layouts that are useful under real operational conditions.

### 1) Start from user decisions, not component catalogs

A common anti-pattern is building layouts by browsing available UI components and assembling attractive screens. This creates visually rich but operationally weak interfaces.

Instead, begin with decision mapping:

- what decisions must users make on this screen,
- what data is required for each decision,
- what actions should be available,
- what confirmation state is needed after action.

For example, a "case review" layout may require:

- severity context,
- related incidents,
- current owner,
- action controls (approve/escalate/request follow-up).

If a field does not support a decision, challenge its presence. Decision-first design reduces clutter and improves completion speed.

### 2) Separate information zones by cognitive purpose

Well-designed layouts group information by user intent. Mixing summary, detail, and action controls in one undifferentiated block increases cognitive load.

Use clear zones:

- **Orientation zone:** status, key identifiers, high-level summary.
- **Evidence zone:** supporting facts, related data, timelines, linked records.
- **Action zone:** editable inputs, checkboxes, switches, radio-group decisions, buttons, confirmations.

This zoning pattern helps users and agents align on where to look and what to do next. It also simplifies iterative improvements because each zone has a clear role.

### 3) Build for scanning first, deep reading second

Most operational users scan before they read deeply. Layouts should support fast signal detection.

Scanning-friendly design patterns:

- concise section headings,
- visible status chips/indicators,
- prioritized ordering of critical fields,
- consistent spacing and typography hierarchy,
- collapsed advanced details where appropriate.

Avoid forcing users through long unstructured text blocks before they can identify key state. Assistants can provide narrative in chat; layout should provide structured visibility.

### 4) Design responsive structure intentionally

Layouts in Gaia may be viewed in different widths and contexts (split canvas, full-page app, channel-specific surfaces). Responsive behavior should be planned, not left to chance.

Practical responsive principles:

- avoid overly nested grids for critical forms,
- ensure key actions remain visible at narrower widths,
- keep label/value readability stable across breakpoints,
- test dense tables and cards for overflow behavior.

A layout that works only at one viewport creates hidden production defects when used in Personal Assistant or text-channel app contexts.

### 5) Treat visual hierarchy as a trust mechanism

Users interpret visual emphasis as product guidance. Poor hierarchy implies uncertainty.

Hierarchy controls include:

- heading scale,
- color contrast,
- grouping boundaries,
- spacing rhythm,
- action prominence.

Consistency matters more than novelty. If similar elements use different styles without rationale, users hesitate and error rates rise. A reliable hierarchy accelerates decision confidence.

### 6) Integrate layout design with entity semantics

UI layouts should reflect the data model's business semantics from Chapter 3.

Alignment checks:

- required entity properties are clearly visible and editable where appropriate,
- lifecycle state is prominent and not buried,
- relationship context is accessible near relevant fields,
- audit-critical attributes are distinguishable from optional metadata.

If layout naming and entity naming diverge, users and agents both suffer confusion. Keep terminology synchronized across entity definitions, prompts, and UI labels.

### 7) Plan write-back interactions for safety and clarity

Editable layouts are high leverage and high risk. Poor write-back design causes accidental state changes and inconsistent records.

Safe write-back patterns:

- clear distinction between read-only and editable regions,
- use radio groups for mutually exclusive decisions, checkboxes for independent flags, and switches for simple on/off state so the control matches the data meaning,
- explicit save/update affordances,
- validation feedback near offending field,
- confirmation messaging after successful write,
- predictable behavior when validation fails.

Do not hide important save consequences in chat text only. The layout should communicate state transitions directly.

### 8) Design conversation-to-canvas continuity

In Gaia, many workflows start in conversation and continue in the canvas. This transition should feel intentional.

Continuity guidelines:

- canvas opens with the relevant record and context already focused,
- layout labels and chat language remain consistent,
- next-step actions are visible after open,
- users can return to chat with clear understanding of current state.

A technically successful `entity_open` call can still produce poor experience if the layout opens in a confusing or overloaded state.

### 9) Use reusable patterns and naming discipline

As projects scale, layout consistency becomes a delivery advantage.

Define reusable standards:

- naming conventions for layouts and folders,
- shared section structure templates,
- default style tokens and spacing rules,
- common component patterns for status, metadata, action bars.

Consistency reduces onboarding time and avoids one-off "special" layouts that are expensive to maintain.

### 10) Definition of done for UI Layout basics

Layout work is done when interfaces support real decisions with low friction and high clarity.

Done criteria:

- layout structure mirrors decision flow,
- hierarchy supports fast scanning,
- responsive behavior is verified,
- write-back behavior is safe and understandable,
- conversation-to-canvas continuity is validated,
- one reproducible lab demonstrates operational usefulness.

If another engineer can use your layout and complete the target task without verbal guidance, Section 1 quality is sufficient for data binding work in Section 2.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "UI Layout Baseline Pack" artifact.

### Scenario

You are designing the first operational layout for a workflow where users must inspect and update structured records during conversation-assisted work.

### Phase A: Decision-flow mapping

1. Select one high-value workflow (for example case review, renewal planning, incident triage).
2. List the top five user decisions on that workflow.
3. For each decision, define required data fields and actions.
4. Group fields into orientation/evidence/action zones.

Checkpoint A:

- Layout requirements are tied to decisions, not generic component preferences.

### Phase B: Layout skeleton design

5. Open **Conversations -> UI Layouts** and create a new layout (or duplicate a baseline template).
6. Build a structural skeleton with clear zones:
   - header/summary area,
   - evidence/data area,
   - action/edit area.
7. Add provisional labels and placeholders before detailed styling.
8. Save as draft version.

Checkpoint B:

- Structural zones are clear and aligned with decision flow.

### Phase C: Visual hierarchy and responsive tuning

9. Apply typography and spacing hierarchy to emphasize critical information.
10. Ensure action controls are visually clear and accessible.
11. Validate layout behavior in narrower canvas widths.
12. Resolve readability/overflow issues.

Checkpoint C:

- Scan-first readability and responsive behavior are acceptable.

### Phase D: Entity alignment and field semantics

13. Map displayed fields to actual entity properties.
14. Confirm required fields and lifecycle/state data are prominent.
15. Check terminology consistency between layout labels and entity model.
16. Save updated layout.

Checkpoint D:

- Layout accurately reflects entity semantics and required operational fields.

### Phase E: Conversation-canvas trial

17. Link layout to target entity in **Data Model → Entities**.
18. Trigger layout from **Conversations** using agent workflow.
19. Execute one read-focused and one edit-focused scenario.
20. Record usability observations:

- clarity of context,
- speed of task completion,
- error/confusion points.

Checkpoint E:

- Conversation-to-canvas experience is coherent and task-completable.

### Phase F: Publish baseline pack

21. Create `chapter-05-section-01-ui-layout-baseline-pack.md` including:

- decision-flow map,
- layout structure rationale,
- hierarchy and responsive choices,
- entity-field mapping,
- conversation trial findings,
- prioritized improvement backlog.

22. Add go/no-go recommendation for Section 2 data binding implementation.

Checkpoint F:

- Another engineer can understand layout intent and continue implementation without reinterpretation.

## Expected outputs

By the end of this lab, you should have:

- One operational layout draft mapped to a real workflow.
- Explicit zone structure (orientation/evidence/action) and hierarchy rationale.
- Verified responsive behavior in conversation canvas contexts.
- Entity-aligned field mapping and terminology consistency.
- One conversation-canvas validation cycle with observations.
- A UI Layout Baseline Pack artifact with go/no-go recommendation for Section 2.

Evidence that qualifies:

- Saved layout visible in **Conversations -> UI Layouts** with clear naming.
- Entity linkage and canvas open behavior working in conversation.
- Written artifact with design rationale and validation notes.

## Failure modes

1. **Component-first design without decision mapping**
   - Symptom: visually complete layout that does not help users complete tasks.
   - Recovery: rebuild structure from user decision flow and required evidence/action mapping.

2. **No information zoning**
   - Symptom: summary, details, and actions are mixed, increasing confusion.
   - Recovery: separate orientation/evidence/action regions and rebalance hierarchy.

3. **Poor scanability**
   - Symptom: users take too long to identify critical state.
   - Recovery: strengthen heading structure, status emphasis, and field prioritization.

4. **Responsive breakdown in canvas widths**
   - Symptom: clipped content, hidden actions, difficult editing.
   - Recovery: simplify grid nesting and retune spacing/priority for narrower layouts.

5. **Entity-layout semantic mismatch**
   - Symptom: labels differ from data model language, causing misinterpretation.
   - Recovery: align naming with entity schema and workflow terminology.

6. **Unsafe write-back interaction design**
   - Symptom: accidental edits or unclear save outcomes.
   - Recovery: clarify editable zones, validation messaging, and post-save feedback.

7. **Conversation-to-canvas discontinuity**
   - Symptom: canvas opens but user cannot infer next step.
   - Recovery: tune open state, labels, and action visibility for immediate task continuation.

8. **No reusable pattern discipline**
   - Symptom: each layout is stylistically inconsistent and hard to maintain.
   - Recovery: establish naming/folder/style conventions and apply consistently.

## Completion checklist

- [ ] I mapped layout requirements to concrete user decisions.
- [ ] I built a layout structure with clear orientation, evidence, and action zones.
- [ ] I validated visual hierarchy and responsive behavior in realistic canvas widths.
- [ ] I aligned layout labels and fields with entity semantics.
- [ ] I validated at least one read path and one edit path from conversation to canvas.
- [ ] I documented usability findings and prioritized improvements.
- [ ] I published a UI Layout Baseline Pack with go/no-go recommendation for Section 2.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md)
- [Data Binding](../../user-guide/conversations/ui-layouts/data-binding.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Entity Canvas Workflow Scenario](../../user-guide/conversations/scenarios/entity-canvas-workflow.md)
- [Entities](../../user-guide/data-model/entities.md)

---

# Data Binding Patterns

## Learning objectives

By the end of this section, you should be able to:

- Design robust data binding strategies for Gaia layouts using repeat scopes, paths, and write-back patterns.
- Choose between scenario-based binding and advanced manual binding based on complexity and risk.
- Prevent common binding failures such as path drift, scope confusion, and unsafe updates.
- Produce a binding validation artifact that supports reliable channel experiences and Section 3 publishing work.

## Prerequisites

- You completed [UI Layouts Basics](#doc-ch05-channel-and-experience-design-01-ui-layouts-basics) with at least one operational layout draft.
- You can access **Conversations -> UI Layouts**, **Data Model → Entities**, and **Conversations**.
- Your target entity has representative sample records so bindings can be tested with realistic data.
- You understand basic path notation and JSON structure conventions used in Gaia.

## In Gaia

- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md) for binding configuration
- [Entities](../../user-guide/data-model/entities.md) and [Runs](../../user-guide/data-model/runs.md) for realistic bound data shapes
- [Conversations](../../user-guide/conversations/README.md) for live conversation-to-canvas validation

Use this section on a real layout and a real sample record. Preview is useful, but the binding should also survive live Gaia context.

## Concept brief

A layout without binding is a mockup. Data binding is what turns interface structure into a working product surface. In Gaia, binding decisions determine whether users see accurate context, whether edits persist safely, and whether conversation-driven canvas workflows remain trustworthy.

Most binding bugs are not dramatic runtime crashes. They are subtle mismatches:

- wrong path resolves silently to empty,
- repeated nodes show incorrect nested values,
- editable controls update the wrong record,
- data appears correct in preview but fails in live conversation context.

Data binding patterns help teams avoid these failures by standardizing how paths, scopes, and write-back contracts are designed.

### 1) Treat binding as a contract between data model and UI intent

Each bound node should have a clear purpose:

- display a specific value,
- display repeated values from a list,
- capture a user edit and write back safely.

For every binding, define:

- source context (where value comes from),
- transformation rules (if any),
- expected display behavior,
- write-back target (if editable).

If you cannot state this contract in one or two lines, the binding is likely too implicit and fragile.

### 2) Prefer scenario-based binding for common patterns

Gaia provides binding scenarios because most layout needs follow repeatable shapes.

Use scenario mode first for:

- single value display,
- repeating text arrays,
- repeating object arrays,
- editable field write-back, including boolean controls such as checkboxes and switches plus single-select controls such as radio groups.

Benefits:

- fewer manual path errors,
- clearer intent for reviewers,
- better maintainability as data structures evolve.

Advanced manual fields should be reserved for edge cases where scenario controls cannot express required behavior.

### 3) Design repeat scopes explicitly

Repeat-based rendering is a high-value and high-confusion area. Many defects come from unclear alias and scope assumptions.

Repeat design checklist:

- define array path explicitly,
- define item alias intentionally,
- define index alias only when needed,
- ensure nested nodes reference the correct active scope.

Avoid mixing top-level shortcuts and nested aliases carelessly. Ambiguous scope leads to "looks right in one row, wrong in others" defects that are hard to detect quickly.

### 4) Path discipline reduces silent failures

Path syntax is powerful but unforgiving. Minor path drift can produce empty or incorrect output without visible errors.

Path discipline practices:

- prefer explicit paths over overly clever expressions,
- validate path existence using preview checks,
- avoid deep brittle paths when schema evolution is expected,
- document critical paths in layout notes for future reviewers.

A path is not just technical syntax; it is an interface dependency. Treat it like code that needs review.

### 5) Write-back binding requires strict safety rules

Editable fields are where UX and data integrity intersect. Unsafe write-back can corrupt records quickly.

For each editable binding, confirm:

- entity target is correct,
- record ID path resolves reliably,
- property target matches expected type,
- the control semantics match the data shape (for example checkbox or switch for boolean state, radio group for one-of-many, not free text),
- parse/format logic is symmetric where needed,
- validation feedback is user-visible.

If record ID path can be null or ambiguous, fail safely and request correction rather than writing uncertain updates.

### 6) Keep display transforms and data transforms distinguishable

Formatting values for display and parsing values for storage are different concerns.

Examples:

- display: format timestamp into human-readable date.
- parse: convert input text back to canonical timestamp.

Combining these concerns loosely can produce subtle data corruption (for example locale-dependent parsing errors). Keep transformation logic explicit and test with edge values.

### 7) Preview validation is necessary but not sufficient

Preview tooling catches many path/scope errors early, but live context can differ in shape and completeness.

Validation should happen in two stages:

- editor preview validation,
- live conversation-canvas validation with actual records.

Test both normal and imperfect records (missing optional fields, empty lists, stale references). Robust binding design anticipates partial data.

### 8) Design for null, empty, and partial states

Operational data is rarely complete. Layouts that assume perfect data create brittle experiences.

Null-safe binding patterns:

- provide default display values for missing data,
- avoid showing raw null/object artifacts,
- communicate missing critical data clearly,
- preserve editability where safe even when some context is incomplete.

A good binding pattern turns partial data into understandable state rather than broken UI.

### 9) Version and review binding changes as behavior changes

Changing binding paths or write-back targets can alter user-visible behavior as much as prompt or tool changes.

Treat binding edits with software discipline:

- capture change intent,
- document expected impact,
- run regression checks on key records,
- include binding changes in release review notes.

This prevents "small UI tweak" incidents that actually break operational workflows.

### 10) Definition of done for Data Binding patterns

Binding work is done when data renders and updates predictably across representative records and live conversation contexts.

Done criteria:

- scenario/manual choice is justified,
- repeat scopes and paths are explicit,
- write-back safety is validated,
- null/partial states are handled gracefully,
- preview and live checks both pass,
- one reproducible lab captures binding decisions and evidence.

If another engineer can modify a bound layout confidently using your pattern notes, Section 2 is ready for channel publishing design.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Data Binding Validation Matrix" artifact.

### Scenario

You need to convert a layout baseline into a production-credible interface where real records display accurately and editable fields persist safely during conversation-assisted workflows.

### Phase A: Binding inventory and criticality mapping

1. Open the target layout and list all nodes with data dependencies.
2. Classify each node as:
   - display-only single value,
   - repeat display,
   - editable write-back,
   - computed/expressive display.
3. Mark critical bindings (those that influence key decisions or writes).
4. Define expected behavior for each critical binding.

Checkpoint A:

- Binding inventory is complete and criticality is explicit.

### Phase B: Scenario-first implementation

5. Configure common bindings using **Binding Scenario** controls.
6. Use **Single value** for straightforward field presentation.
7. Use **Repeat list of text/objects** for repeated sections.
8. Use **Editable field (write-back)** for update controls.
9. Keep a note of any node that requires advanced manual binding.

Checkpoint B:

- Most bindings are implemented via scenario mode with clear intent.

### Phase C: Advanced path and scope hardening

10. For edge-case nodes, switch to advanced manual fields.
11. Define explicit repeat path, aliases, and nested field paths.
12. Validate path resolution in preview:
    - exists,
    - expected type,
    - expected item count.
13. Resolve ambiguous alias/path collisions.

Checkpoint C:

- Advanced bindings are explicit and scope-safe.

### Phase D: Write-back validation

14. Test each editable field with representative updates.
15. Verify record ID path and entity property target correctness.
16. Validate input parsing and output formatting with edge cases.
17. Confirm validation errors are visible and actionable.

Checkpoint D:

- Write-back behavior is accurate, safe, and user-understandable.

### Phase E: Live conversation-canvas tests

18. Trigger layout through a conversation using real records.
19. Run at least five scenarios:
    - complete record,
    - partially missing optional fields,
    - empty repeat list,
    - invalid edit attempt,
    - successful update round-trip.
20. Record rendering and persistence outcomes.

Checkpoint E:

- Binding behavior is stable in live context, not only preview mode.

### Phase F: Publish validation matrix

21. Create `chapter-05-section-02-data-binding-validation-matrix.md` including:
    - binding inventory and criticality,
    - scenario vs advanced decisions,
    - path/scope conventions,
    - write-back test outcomes,
    - live conversation validation results,
    - unresolved risks and mitigation plan.
22. Add go/no-go recommendation for Section 3 channel publishing.

Checkpoint F:

- Another engineer can audit and extend bindings with minimal onboarding.

## Expected outputs

By the end of this lab, you should have:

- A fully bound layout with explicit coverage of display, repeat, and editable nodes.
- Documented scenario-first binding strategy with justified advanced exceptions.
- Verified path and scope behavior across representative data shapes.
- Validated write-back behavior with error handling and round-trip checks.
- Live conversation-canvas evidence for normal and edge scenarios.
- A Data Binding Validation Matrix artifact with go/no-go recommendation for Section 3.

Evidence that qualifies:

- Layout nodes render correct live data across test scenarios.
- Editable fields persist to intended entity records reliably.
- Validation matrix documents both successful cases and known limitations.

## Failure modes

1. **Manual binding overuse for simple cases**
   - Symptom: unnecessary complexity and frequent path mistakes.
   - Recovery: migrate straightforward bindings to scenario mode.

2. **Repeat scope ambiguity**
   - Symptom: nested values appear from wrong records or wrong list items.
   - Recovery: define explicit aliases and audit all nested references.

3. **Fragile deep paths**
   - Symptom: minor schema changes break rendering silently.
   - Recovery: simplify paths, add defaults, and validate on representative records.

4. **Incorrect write-back record targeting**
   - Symptom: edits saved to wrong record or fail unpredictably.
   - Recovery: harden record ID path checks and test with known record IDs.

5. **Transform/parse mismatch**
   - Symptom: displayed values look correct but stored values are malformed.
   - Recovery: separate display formatting from parse logic and test round-trip consistency.

6. **Preview-only validation confidence**
   - Symptom: live conversation behavior differs from editor preview.
   - Recovery: add mandatory live canvas test suite before publishing.

7. **Null and empty states ignored**
   - Symptom: blank or broken sections when data is incomplete.
   - Recovery: define null-safe defaults and explicit missing-data cues.

8. **Undocumented binding changes**
   - Symptom: regression cause is unclear during later maintenance.
   - Recovery: log binding edit intent, impact, and validation evidence in artifact notes.

## Completion checklist

- [ ] I inventoried all bound nodes and identified critical bindings.
- [ ] I implemented common bindings using scenario mode first.
- [ ] I hardened advanced bindings with explicit path and scope conventions.
- [ ] I validated write-back safety, including record targeting and parsing.
- [ ] I tested live conversation-canvas behavior across normal and edge data cases.
- [ ] I documented binding decisions and outcomes in a validation matrix.
- [ ] I published a go/no-go recommendation for Section 3 channel publishing.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md)
- [Data Binding in UI Layouts](../../user-guide/conversations/ui-layouts/data-binding.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Entity Canvas Workflow Scenario](../../user-guide/conversations/scenarios/entity-canvas-workflow.md)
- [Entities](../../user-guide/data-model/entities.md)

---

# Channel Publishing

## Learning objectives

By the end of this section, you should be able to:

- Design a channel publishing strategy that aligns audience, authentication, agent routing, and operational risk.
- Configure and release Gaia channels using versioning and enablement controls safely.
- Define rollout, monitoring, and rollback procedures for channel changes across text and integration surfaces.
- Produce a channel release artifact that supports reproducible publishing decisions.

## Prerequisites

- You completed [Data Binding Patterns](#doc-ch05-channel-and-experience-design-02-data-binding-patterns) and have at least one working conversation-ready experience.
- You can access **Channels**, **AI Agents**, and **Conversations** in your project.
- You have one target audience and use case ready for exposure through a channel.
- You can describe baseline service expectations (availability, response quality, latency, and feedback handling).

## In Gaia

- [Channels](../../user-guide/conversations/channels/README.md) for publishing, versioning, and access posture
- [AI Agents](../../user-guide/agents/README.md) for exposed routing behavior
- [Conversations](../../user-guide/conversations/README.md), [Dashboard](../../user-guide/dashboard/README.md), and [Audit Trail](../../user-guide/audit/README.md) for post-publish validation

Publish as a controlled Gaia release step, not as a final toggle. Version, validate, and keep rollback conditions visible.

## Concept brief

Publishing a channel is the moment your internal assistant design meets real users and real integration traffic. Up to this point, most work is controlled and engineer-facing. Channel publishing introduces external variability:

- different device contexts,
- different authentication states,
- different message formats,
- unpredictable usage patterns.

A safe publishing practice makes channel behavior deliberate rather than accidental. In Gaia, channels are not merely transport settings. They are product contracts that define:

- who can access the assistant,
- which entrypoint they use,
- which agent logic is exposed,
- how changes are versioned and rolled out.

### 1) Start with channel intent and audience segmentation

Do not publish a channel because it exists in the menu. Publish when there is a clear audience and objective.

For each channel candidate, define:

- primary audience (internal operators, end customers, partners, systems),
- interaction style (chat-first, workflow trigger, API integration),
- security expectations,
- response quality expectations.

Example segmentation:

- Text channel for web chatbot users,
- Personal Assistant channel for branded assistant experiences, often authenticated internal workflows,
- Webhook channel for machine-to-machine workflow triggers.

Audience-first framing prevents overexposed endpoints and misaligned UX.

### 2) Treat channel type selection as architecture extension

Channel choice influences behavior boundaries.

- **Text / Personal Assistant:** conversational UX, branding, feedback loops, human-readable continuity.
- **Alternative frontend:** API-first clients with external UI control.
- **Webhook / MCP / ChatGPT:** integration surfaces with stricter protocol and security concerns.
- **SIP/SMS/Email/WhatsApp/Viber:** channel-specific constraints on payload style, latency, and interaction depth.

Choose channel type by interaction requirement, not by convenience. A poor match can force brittle workarounds in prompts and tools.

### 3) Align channel-agent routing explicitly

Publishing exposes whatever routing and agent defaults you attach. Ambiguous defaults create unpredictable user experience.

Per channel, define:

- default agent or routing policy,
- permitted specialist handoffs,
- language and tone expectations,
- channel-specific constraints (for example concise responses on SMS-like surfaces).

Text and Personal Assistant channels may point directly at a specialist agent. Use an orchestrator only when the channel experience actually needs handoff or multi-agent routing, and enforce that requirement before launch instead of relying on a project default to correct the route.

For **MCP** and **ChatGPT** channels, treat routing as per-capability policy rather than a channel default. If a capability invokes an agent, assign that capability explicitly to the correct agent and review the full capability list before publishing.

If channel-level and agent-level assumptions conflict, users see inconsistent behavior across entrypoints.

### 4) Channel versioning is your release safety net

Gaia channel versions allow controlled rollout. Use them as release gates, not historical clutter.

Recommended versioning pattern:

- maintain one clearly stable enabled version,
- create new version for meaningful behavior changes,
- validate before enabling,
- keep rollback candidate identifiable.

Each version should include release notes:

- what's changed,
- expected impact,
- risk level,
- rollback trigger conditions.

Without version discipline, channel publishing becomes irreversible trial-and-error.

### 5) Authentication and access should be designed per channel

Security posture differs by channel and audience.

Access design considerations:

- public vs authenticated access,
- project role visibility,
- API key ownership and rotation,
- endpoint discoverability and slug governance.

A secure-by-default stance is simpler than retroactive hardening. Publish least privilege first, then expand deliberately with evidence.

### 6) Theme and UX consistency are part of publishing quality

For user-facing channels, branding and interaction consistency impact trust. Theme settings are not cosmetic details; they shape perceived reliability.

Publishing checks should include:

- logo and color alignment,
- confirmation that the selected logo still reads clearly at favicon size on project tabs and browser chrome,
- separation between the shared UI theme and any conversation-specific overrides,
- if a Text channel uses a non-default font family, confirmation that the selected Google Font is readable across messages, composer controls, banners, headers, and labels,
- confirmation that conversation overrides are scoped per surface and inherit from the shared UI theme only when left blank,
- if conversation background layers are configured, confirmation that decorative artwork is anchored and scaled intentionally and remains confined to the intended conversation canvas instead of bleeding into rails, drawers, or dialogs,
- confirmation that read-only channel previews still show the same headings, links, inline UI layouts, and thinking states that users will see at runtime,
- explicit decision on whether user avatars and agent avatars are shown in the conversation stream,
- if user avatars are enabled, whether they should use standard Gaia user colors or explicit channel-level background/icon overrides,
- if agent avatars are enabled, whether they should use theme-default colors or explicit overrides,
- explicit review of portal frame chrome when the channel is exposed through the floating portal assistant, including parity between **Portal view** and any hosted or downloaded embed script that will be installed outside Gaia,
- if Text-channel portal sizing is customized, confirmation that the default viewport-relative height still matches the intended deployment and that half-screen maximize expands width without unexpectedly changing height,
- if the portal PDF download button is enabled, confirmation that the icon appears in the embedded header to the left of **New conversation**, matches the approved visual placement, and downloads the expected transcript-only PDF,
- if portal Info or Warning message banners are enabled, confirmation that the localized copy from the **Messages** tab plus the Portal-tab icon asset, accent color, optional background color, text color, dismissal behavior, and placement under the embedded header match the customer-approved portal experience,
- if portal close confirmation is enabled, confirmation that the localized modal copy from the **Messages** tab plus the dialog icon, per-action visibility toggles, primary button color, and close/return/terminate behavior match the customer-approved exit flow,
- if the Text channel shows a pre-first-message Terms & Privacy notice, confirmation that the localized wording and inline link labels from the **Messages** tab plus the Portal-tab link targets and link color match the approved legal copy for that customer deployment,
- if custom portal JavaScript is enabled, confirmation that it targets Gaia's public app origin and passes only the intended iframe params into the embedded experience,
- readability and contrast, including whether the preview warnings identify the exact text/background pair before teams intentionally save low-contrast color choices and whether the team understands those warnings are advisory rather than save-blocking,
- consistency with message style,
- visible state for loading and errors,
- feedback affordance behavior.

A channel can be technically functional yet rejected by users if visual and interaction quality appears inconsistent.

### 7) Define non-happy-path behavior before launch

Publishing decisions should include known failure paths:

- unavailable agent/config,
- auth failures,
- unsupported payloads,
- dependency outages,
- channel disablement behavior.

Document how each channel surfaces failure:

- user-facing message,
- retry guidance,
- escalation route,
- incident ownership.

This avoids panic-driven messaging when real incidents occur.

### 8) Monitor channel health with channel-specific metrics

A single global metric set is insufficient. Different channels require different operational signals.

Examples:

- text/personal assistant: response latency, abandonment rate, feedback trend.
- webhook: request success rate, workflow trigger success, payload rejection rate.
- API channels: auth failures, protocol errors, downstream timeouts.

Define minimum channel health dashboards before scaling usage. Monitoring after launch is mandatory, not optional optimization.

### 9) Rollout and rollback must be explicit procedures

A channel release is a delivery event. Treat it with the same discipline as code release.

Rollout pattern:

- preflight checks,
- limited-scope validation,
- enablement decision,
- post-launch observation window,
- stabilization confirmation.

Rollback triggers should be predefined:

- severe quality regression,
- critical auth/security issue,
- unacceptable latency/error rate,
- unsafe routing behavior.

If rollback criteria are undefined, teams hesitate during incidents and user impact grows.

### 10) Definition of done for channel publishing

Channel publishing is done when release risk is controlled and user experience is validated.

Done criteria:

- channel intent and audience are explicit,
- routing/auth/theme configurations are coherent,
- versioning and rollback paths are documented,
- preflight and post-launch checks are executed,
- health monitoring signals are defined,
- one reproducible lab demonstrates safe publish workflow.

If another engineer can publish a new channel version using your runbook without ad hoc decisions, Section 3 is ready for final conversation UX hardening in Section 4.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Channel Publishing Release Checklist" artifact.

### Scenario

You are preparing a channel launch for a real audience and need to publish safely with clear rollback controls and monitoring.

### Phase A: Channel strategy definition

1. Select one target channel type for release.
2. Define audience, objectives, and usage expectations.
3. Define security posture (auth required, endpoint exposure, key ownership).
4. Define routing policy (default agent, specialist handoffs, constraints).

Checkpoint A:

- Channel intent, audience, and risk model are explicit.

### Phase B: Configuration and version preparation

5. Open **Conversations -> Channels** from the shared Conversations workspace tabs and create a new channel version (or clone the current baseline).
6. Configure channel-specific settings:
   - slug/endpoint,
   - auth/feedback settings,
   - theme/branding as applicable,
   - workflow binding for integration channels.
   - supporting file extractor and folder behavior when uploads or Personal Assistant file access are part of the release.
7. Record release notes with expected behavior changes.

Checkpoint B:

- New version is configured and documented, but not yet broadly exposed.

### Phase C: Preflight validation

8. Validate routing and conversation behavior with representative prompts.
9. Validate auth and access boundaries.
10. Validate non-happy-path messages (missing config, invalid auth, unsupported input).
11. Validate channel-specific UX consistency (for user-facing channels).

Checkpoint C:

- Preflight checks confirm readiness or identify blocking issues before enablement.

### Phase D: Controlled enablement

12. Enable the prepared channel version.
13. Verify endpoint availability and basic interaction flow.
14. Run a smoke test covering:

- one in-scope request,
- one ambiguous request,
- one boundary/failure request.

15. Confirm expected logs/telemetry visibility.

Checkpoint D:

- Channel is live with validated baseline behavior.

### Phase E: Observation window and rollback drill

16. Monitor channel metrics during initial observation window.
17. Compare observed behavior to release expectations.
18. Execute a rollback simulation plan (or real rollback if issues found):

- disable current version,
- re-enable prior stable version,
- verify restoration.

Checkpoint E:

- Team can recover quickly from channel regressions.

### Phase F: Publish release artifact

19. Create `chapter-05-section-03-channel-publishing-release-checklist.md` including:

- channel strategy,
- version config summary,
- preflight outcomes,
- launch metrics baseline,
- rollback triggers and steps,
- open risks and owners.

20. Add go/no-go recommendation for Section 4 conversation UX pattern hardening.

Checkpoint F:

- Another engineer can publish the same channel safely using your checklist.

## Expected outputs

By the end of this lab, you should have:

- One channel release plan tied to audience, security, and routing intent.
- A configured and versioned channel candidate with release notes.
- Preflight validation evidence covering normal and failure paths.
- Controlled launch verification with initial monitoring baseline.
- Documented rollback triggers and tested rollback procedure.
- A Channel Publishing Release Checklist artifact with go/no-go recommendation for Section 4.

Evidence that qualifies:

- Channel version state visible in Gaia with explicit enablement decision.
- Conversation/integration traces confirming expected routing and behavior.
- Release artifact with owner accountability and follow-up actions.

## Failure modes

1. **Publishing without audience/intent clarity**
   - Symptom: channel launches but fails to serve a clear user need.
   - Recovery: define channel objective and audience before config changes.

2. **Channel type misalignment**
   - Symptom: UX and protocol constraints conflict with workflow needs.
   - Recovery: re-evaluate channel selection against interaction requirements.

3. **Unclear routing defaults**
   - Symptom: users experience inconsistent agent behavior across channels.
   - Recovery: document channel-specific routing policy and validate explicitly.

4. **No version release discipline**
   - Symptom: changes are hard to roll back and impact is unclear.
   - Recovery: adopt version notes, preflight gates, and stable rollback candidate policy.

5. **Weak access control configuration**
   - Symptom: unintended exposure or frequent auth incidents.
   - Recovery: apply least-privilege channel access and validate auth boundaries pre-launch.

6. **Non-happy-path behavior untested**
   - Symptom: failure messages are confusing during incidents.
   - Recovery: include failure-path checks in mandatory preflight suite.

7. **No channel-specific monitoring**
   - Symptom: quality degradation noticed only after user complaints.
   - Recovery: define per-channel health metrics and observation cadence.

8. **Rollback plan missing or untested**
   - Symptom: incident recovery is slow and risky.
   - Recovery: define rollback triggers and rehearse version fallback steps.

## Completion checklist

- [ ] I defined channel audience, objective, and security posture.
- [ ] I prepared a versioned channel configuration with release notes.
- [ ] I validated routing, auth, UX, and failure paths before launch.
- [ ] I performed controlled enablement and recorded initial health metrics.
- [ ] I documented and tested rollback criteria and procedure.
- [ ] I assigned owners for launch risks and follow-up actions.
- [ ] I published a Channel Publishing Release Checklist with go/no-go recommendation for Section 4.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md)
- [Document Folders](../../user-guide/conversations/document-folders/README.md)
- [Workflow actions](../../user-guide/conversations/workflow-actions.md)
- [File extractor registry](../../user-guide/conversations/file-extractor-registry.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Settings](../../user-guide/settings/README.md)
- [Runs](../../user-guide/data-model/runs.md)

---

# Conversation UX Patterns

## Learning objectives

By the end of this section, you should be able to:

- Design conversation experiences in Gaia that remain clear, trustworthy, and efficient across channels.
- Apply reusable UX patterns for prompts, clarifications, tool-result narration, and canvas collaboration.
- Define interaction guardrails for ambiguity, refusal, feedback collection, and review workflows.
- Produce a conversation UX runbook that closes Chapter 5 and prepares for Chapter 6 eval hardening.

## Prerequisites

- You completed [Channel Publishing](#doc-ch05-channel-and-experience-design-03-channel-publishing) and have at least one live or launch-ready channel.
- You can run and inspect conversations in **Conversations**, including timeline/inside-info where available.
- You have one representative prompt set covering normal, ambiguous, and boundary requests.
- You can capture qualitative user feedback and basic usage observations during testing.

## In Gaia

- [Conversations](../../user-guide/conversations/README.md) for live interaction patterns
- [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md) and [Give feedback on a reply](../../user-guide/conversations/dialogs/feedback.md) for UX evidence
- [UI Layouts](../../user-guide/conversations/ui-layouts/README.md) when the UX depends on canvas continuity

Use this section while running real Gaia conversations. The UX patterns only matter if they improve actual turns, clarifications, and next-step flow.

## Concept brief

Conversation UX is where architecture, configuration, tooling, and channel decisions become user experience. Users do not see your model settings or handoff contracts; they experience clarity, speed, confidence, and outcome quality.

A strong conversation UX makes the system feel reliable even when tasks are complex. A weak conversation UX makes even technically correct systems feel frustrating:

- responses are verbose but unhelpful,
- clarifications feel repetitive,
- tool-driven actions are opaque,
- handoffs feel abrupt,
- users cannot tell what to do next.

Section 4 focuses on practical patterns that make conversation behavior usable and trustworthy at scale.

### 1) Design turn structure for decision support

Each assistant turn should help users do one of four things:

- understand current state,
- choose among options,
- take action,
- confirm outcome.

If a turn does not move one of these forward, it is usually noise.

Pattern for high-quality turns:

- short context recap,
- direct answer or recommendation,
- explicit assumptions/uncertainties,
- clear next step.

This structure reduces back-and-forth and improves user confidence.

### 2) Use progressive disclosure instead of response dumping

Overly long responses are a common UX defect. Even accurate information can be unusable when delivered as dense blocks.

Progressive disclosure pattern:

- lead with concise answer,
- include key supporting points,
- offer optional deeper detail,
- link to canvas or next action when relevant.

In Gaia conversations, this pattern pairs well with canvas interactions: chat for guidance, canvas for structured depth.

For staged agents, Gaia's conversation **Execution** panel is the preferred progressive-disclosure surface for operational detail. Keep the reply focused on the user-facing outcome, then let reviewers open the panel to inspect stage, waiting state, checkpoint summary, and typed outputs only when they need execution detail. Avoid copying the full checkpoint into every assistant reply.

### 3) Clarification questions should be targeted and scarce

Clarification is necessary but often overused. Poor clarifications feel like the assistant is deflecting.

Effective clarification pattern:

- ask only when ambiguity changes decision/action,
- ask one focused question at a time,
- explain why clarification is needed,
- provide quick options when possible.

Avoid generic prompts like "Can you provide more details?" without context. Precision in clarification is a major quality signal.

### 4) Narrate tool-based actions with user-level meaning

Users should understand what happened when tools are used, without exposing internal noise.

Narration pattern:

- what action was performed,
- what result was found/changed,
- what confidence/limitation applies,
- what next action is available.

Example style:

- "I checked the latest three incidents and found two still open. I can open the incident canvas now if you want to assign owners."

This bridges technical execution and human workflow.

### 5) Maintain continuity across handoffs and channel contexts

Conversation UX suffers when handoffs reset tone or context abruptly.

Continuity guidelines:

- preserve user objective wording across hops,
- acknowledge handoff briefly,
- avoid repeating collected information,
- keep response style coherent with channel expectations.

Channel context matters:

- text channel may tolerate slightly richer formatting,
- constrained channels may require short, action-focused messages,
- personal-assistant mode may support richer iterative collaboration,
- read-aloud behavior should match channel expectations: leave it inherited from the agent Voice tab when channels share one voice, and set a channel-specific read-aloud model when a channel needs different audio, language, or accessibility behavior,
- personal-assistant folder workflows should keep global conversation and folder discovery in a permanent left rail on desktop, fall back to a slide-over left rail on mobile, switch that rail into folder-specific chat history when a folder is selected, and reserve the right drawer for files, search, and people controls,
- text and personal-assistant channel color mode should be configured per channel (default light) rather than inherited from the platform user theme.
- conversation-specific color overrides should be edited separately from the shared shell/dialog theme so message chrome can diverge without destabilizing drawers, dialogs, or workspace cards.

### 6) Integrate canvas interactions as collaborative UX, not side panel noise

Canvas is powerful when integrated intentionally. It should feel like a companion workflow, not a detached interface.

Canvas UX pattern:

- announce why canvas is opened,
- guide user to relevant section/field,
- confirm save/update outcomes in chat,
- suggest next conversational step after edit.

For document workflows, distinguish source review from editable output. PDF annotation artifacts should preserve the uploaded PDF as the source of record and keep notes/highlights as separate artifact state, while markdown and spreadsheet artifacts can be edited as canonical content. When users or agents add PDF notes or highlights, treat those as revisioned review state rather than edits to the underlying PDF; page overlays are review markers, not changes to the file bytes.

Without this guidance, users may ignore or misuse the canvas despite correct technical setup.

### 7) Design refusal and boundary behavior for trust preservation

Refusals are part of good UX when handled clearly.

Trust-preserving refusal pattern:

- state boundary briefly,
- explain constraint in user-relevant language,
- offer safe alternative path,
- avoid defensive or repetitive wording.

Boundary behavior should feel helpful, not obstructive. Users accept constraints more readily when next-step options are explicit.

### 8) Close the loop with feedback and review workflows

Conversation UX should include a learning loop. Gaia supports feedback and review signals; use them intentionally.

Practical loop:

- enable end-user feedback only on channels where collection is appropriate; disabling the channel setting removes note, like, and dislike controls and blocks direct submissions,
- tag and review problematic threads,
- classify failure types (clarity, correctness, routing, tool behavior),
- feed findings into config/tool/eval updates.

UX quality improves fastest when conversation observations become structured improvement inputs.

### 9) Define channel-specific UX standards and anti-patterns

One universal style guide rarely works across all channels. Define per-channel expectations.

Example channel standards:

- Text: balanced detail, markdown clarity, optional deeper expansions.
- Personal Assistant: richer iterative guidance, strong continuity.
- Integration channels: concise machine/human hybrid outputs with explicit status.

For public Text channels, define a conversation inactivity window as part of the UX standard. When a transcript reaches that window, Gaia locks it and starts a new conversation on the next visit. Choose a period that preserves realistic follow-up context without allowing unrelated visits to accumulate in one transcript. Operationally, schedule both the inactive-conversation lock sweep and the locked-conversation post-event sweep so lifecycle enforcement and final assessment or topic extraction do not depend on a user reopening the channel.

Also define anti-patterns to avoid:

- unexplained jargon,
- conflicting recommendations,
- excessive hedging,
- buried actions in long prose.

Channel-specific standards keep assistant behavior coherent at scale.

### 10) Definition of done for Conversation UX patterns and Chapter 5

Conversation UX pattern work is done when interactions are consistently understandable, actionable, and trustworthy across representative scenarios.

Done criteria:

- turn structure supports decisions and actions,
- clarifications are precise and minimal,
- tool and handoff behavior is user-transparent,
- canvas collaboration is guided,
- boundary/refusal behavior preserves trust,
- feedback/review loop is operational,
- section lab yields a reusable UX runbook.

Completing this section closes Chapter 5 and provides high-quality input for Chapter 6 eval strategy and quality gating.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Conversation UX Pattern Runbook" artifact.

### Scenario

You are hardening a published or launch-ready channel to ensure conversation experience quality under normal, ambiguous, and boundary conditions.

### Phase A: UX objective and metric setup

1. Define top UX goals for the target channel (for example clarity, actionability, trust).
2. Define measurable proxy signals:
   - successful task completion,
   - clarification rate,
   - user feedback trend,
   - average turns to resolution.
3. Build a prompt set with at least 15 scenarios:
   - core workflow prompts,
   - ambiguous prompts,
   - boundary/refusal prompts,
   - handoff/canvas prompts.

Checkpoint A:

- UX goals and evaluation set are explicit before testing.

### Phase B: Turn-structure and clarity testing

4. Run core workflow prompts.
5. Assess each response for:
   - concise answer quality,
   - next-step clarity,
   - unnecessary verbosity.
6. Record response refinements needed by pattern (not one-off rewrites).

Checkpoint B:

- Core turns follow decision-support structure with clear next steps.

### Phase C: Clarification and boundary behavior testing

7. Run ambiguous and boundary prompts.
8. Validate clarification quality:
   - necessity,
   - precision,
   - number of clarifying turns.
9. Validate refusal behavior:
   - clear boundary explanation,
   - safe alternative offered,
   - respectful tone.

Checkpoint C:

- Ambiguity and boundary behavior are controlled and user-trust preserving.

### Phase D: Tool narration and handoff continuity tests

10. Run prompts that trigger tool usage and specialist handoffs.
11. Evaluate whether users can understand:
    - what action occurred,
    - what changed,
    - what happens next.
12. Identify continuity breaks (tone reset, repeated context, abrupt transitions).

Checkpoint D:

- Tool and handoff behavior is transparent without technical overload.

### Phase E: Canvas collaboration path tests

13. Run prompts that open canvas and require record edits.
14. Validate assistant guidance before/during/after canvas interactions.
15. Verify chat confirms save results and next actions clearly.
16. Capture friction points where users hesitate or lose context.

Checkpoint E:

- Conversation-canvas collaboration feels coherent and outcome-oriented.

### Phase F: Publish UX runbook and chapter closure recommendation

17. Create `chapter-05-section-04-conversation-ux-pattern-runbook.md` including:
    - UX goals and metrics,
    - test prompt suite,
    - observed failure patterns,
    - applied conversation-pattern adjustments,
    - channel-specific standards,
    - unresolved risks and owners.
18. Add Chapter 5 readiness recommendation and carry-over priorities for Chapter 6 eval hardening.

Checkpoint F:

- Another engineer can apply the runbook to maintain and improve conversation UX consistently.

## Expected outputs

By the end of this lab, you should have:

- A tested conversation UX pattern set for one target channel.
- Evidence on clarity, clarification quality, boundary behavior, and actionability.
- Validated narration/continuity behavior for tool and handoff scenarios.
- Validated conversation-canvas collaboration flow for relevant workflows.
- A feedback/review loop mapping UX failures to improvement actions.
- A Conversation UX Pattern Runbook with Chapter 5 readiness recommendation.

Evidence that qualifies:

- Conversation traces across normal and edge scenarios with structured observations.
- Updated pattern guidance linked to measurable UX goals.
- A reusable runbook artifact with owner-accountable next steps.

## Failure modes

1. **Answer-heavy but action-light turns**
   - Symptom: responses contain information but users still do not know next step.
   - Recovery: enforce turn pattern with direct recommendation and explicit action path.

2. **Over-clarification loops**
   - Symptom: assistant repeatedly asks for details that are not decision-critical.
   - Recovery: ask only targeted clarifications tied to concrete action impact.

3. **Opaque tool behavior**
   - Symptom: users cannot tell what the assistant actually did.
   - Recovery: add user-level narration of action, result, and next step.

4. **Abrupt handoff discontinuity**
   - Symptom: tone/context resets after agent transfer.
   - Recovery: enforce continuity guidance and handoff acknowledgment patterns.

5. **Canvas opened without guidance**
   - Symptom: users see side panel but do not know what to do there.
   - Recovery: narrate purpose, target fields, and post-save expectations in chat.

6. **Boundary refusals feel hostile or unhelpful**
   - Symptom: users perceive guardrails as arbitrary blockers.
   - Recovery: pair concise boundary explanation with practical safe alternative.

7. **No channel-specific UX standards**
   - Symptom: response style is inconsistent across entrypoints.
   - Recovery: define and apply channel-level conversation style contracts.

8. **Feedback signals not integrated into improvements**
   - Symptom: recurring UX defects despite repeated incidents.
   - Recovery: map feedback/review findings into config/tool/eval backlog with owners.

## Completion checklist

- [ ] I defined UX goals and measurable proxy signals for the target channel.
- [ ] I validated turn structure for clarity, actionability, and brevity.
- [ ] I tested and improved clarification and boundary behavior patterns.
- [ ] I validated tool narration and handoff continuity quality.
- [ ] I validated guided conversation-canvas collaboration for edit workflows.
- [ ] I mapped feedback/review signals to prioritized UX improvements.
- [ ] I published a Conversation UX Pattern Runbook with Chapter 5 readiness recommendation.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Conversation Scenarios](../../user-guide/conversations/scenarios/README.md)
- [Entity Canvas Workflow Scenario](../../user-guide/conversations/scenarios/entity-canvas-workflow.md)
- [Document Editing Workflow Scenario](../../user-guide/conversations/scenarios/document-editing-workflow.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Give Feedback Dialog](../../user-guide/conversations/dialogs/feedback.md)
- [Timeline Dialog](../../user-guide/conversations/dialogs/timeline.md)
- [Evals](../../user-guide/evals/README.md)

---

# UI Layout Component Reference

Status: current

Use this page when you need the full current component inventory for Gaia UI layouts.

The user guide stays focused on workflow. This page is the canonical handbook reference for which nodes exist, what fields they use, and how they behave with data binding.

## Common node contract

Every node follows the same outer shape:

- `id`: unique within the layout tree.
- `type`: the node kind.
- `props`: component-specific options.
- `className`: Tailwind classes for spacing, sizing, color, and layout behavior.
- `showIf`: optional visibility rule.
- `repeat`: optional repeat definition for array-driven rendering.
- `bind`: optional field binding for editable controls and selected advanced cases.
- `update`: optional write-back root used by editable descendants.
- `children`: nested nodes for components that support composition.
- `component`: sub-layout reference used only by the `component` node.

## Layout and composition nodes

| Node            | Use it for                              | Key fields                                             | Data behavior                                                      |
| --------------- | --------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------ |
| `row`           | Horizontal flex layout                  | Mostly `className`                                     | Can host `repeat`, `showIf`, and `update` like any other container |
| `col`           | Vertical stacking and most root layouts | Mostly `className`                                     | Common root for full forms and detail pages                        |
| `grid`          | Multi-column layouts                    | `props.columns`, `className` grid helpers              | Often paired with repeat for card walls                            |
| `card`          | Framed sections                         | `props.title`                                          | Good repeated unit for list-of-object layouts                      |
| `scroll-area`   | Scrollable regions                      | `props.scrollDirection`                                | Best when paired with explicit height or max-height                |
| `data-table`    | Read-only tabular layouts               | `props.rows`, `props.columns`, `props.emptyText`       | Rows resolve from a list path and columns read row-relative values |
| `tabs`          | Tabbed sections                         | `props.defaultTab`, child `props.title`                | Child nodes become tab panels                                      |
| `labeled-field` | Label + nested control wrapper          | `props.label`                                          | Keeps label structure separate from the field itself               |
| `component`     | Reuse another layout as a sub-layout    | `component.id` or `component.name`, `component.inputs` | Advanced composition pattern for existing reusable layouts         |

## Content and media nodes

| Node             | Use it for                                    | Key fields                                                                          | Data behavior                                                               |
| ---------------- | --------------------------------------------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `text`           | General text, labels, computed display values | `props.text`                                                                        | Usually read from paths or expressions, not write-back                      |
| `truncated-text` | Summaries and previews                        | `props.text`, `props.lines`                                                         | Same as `text`, but clamps long content                                     |
| `link`           | Navigation links                              | `props.label`, `props.href`, `props.newTab`                                         | Use expressions for dynamic labels or URLs                                  |
| `image`          | Fixed or fill-mode images                     | `props.src`, `props.alt`, `props.fill`, `props.fit`, `props.position`, sizing props | `props.src` can act as fallback when data binding supplies the image source |
| `icon`           | Small visual accents                          | `props.name`, `props.size`                                                          | Color usually comes from classes or surrounding styling                     |
| `divider`        | Section separators                            | none required                                                                       | Presentational only                                                         |
| `spacer`         | Flexible empty space                          | none required                                                                       | Presentational only                                                         |

## Editable field nodes

These are the primary nodes for `bind` plus ancestor `update` write-back behavior.

| Node              | Use it for                     | Key fields                                                                                          | Best semantic fit                                            |
| ----------------- | ------------------------------ | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| `input`           | Single-line text entry         | `props.placeholder`, optional `props.value`                                                         | Names, emails, short text, scalar identifiers                |
| `checkbox`        | Independent yes or no flags    | `props.label`, optional boolean `props.value`                                                       | Opt-ins, acknowledgements, independent toggles               |
| `switch`          | Immediate on or off state      | `props.label`, optional boolean `props.value`                                                       | Published state, active state, feature toggles               |
| `radio-group`     | One-of-many decisions          | `props.options`, `props.orientation`, optional `props.value`                                        | Approval decision, status choice, mutually exclusive options |
| `select`          | One-of-many dropdown selection | `props.options`, `props.placeholder`, optional `props.value`                                        | Compact single-select lists                                  |
| `textarea`        | Multi-line text entry          | `props.placeholder`, optional `props.value`                                                         | Notes, summaries, longer comments                            |
| `markdown-editor` | Richer long-form editing       | `props.placeholder`, `props.minHeightPx`, `props.readOnly`, `props.toolbar`, optional `props.value` | Markdown content, richer authored text                       |

## Binding rules that matter most

- Use **Binding Scenario** first in the DATA tab whenever possible.
- Use `bind` on the field node to define which entity property it edits.
- Use `update` on the nearest shared container when several fields should save into the same record.
- Use `checkbox` or `switch` for boolean state, not free text.
- Use `radio-group` or `select` for one-of-many choices.
- Use `repeat` on the container that should duplicate, not on every child.

## Component-specific notes

- `data-table` is the first-class table node. Use it for read-only tables with a rows path and explicit columns.
- `data-table` column `path` values are row-relative, so `name` resolves the current row's `name` and `Review.status` resolves the current row's nested `Review.status`.
- `text` and `truncated-text` nodes can be styled from the NODE tab with font size, weight, alignment, color, italic, underline, strikethrough, and overline controls.
- `radio-group` and `select` both require `props.options` as an array of `{ label, value }` objects.
- `radio-group` also supports `props.orientation` with `vertical` or `horizontal`.
- `markdown-editor` falls back to textarea-style behavior if the richer editor primitive is unavailable in a given render path.
- `component` is an advanced reuse mechanism. Prefer it when a reusable sub-layout already exists rather than when you are still shaping the first version of the UI.

## Availability note

`table` still exists in the schema history, but it is now treated as a legacy alias. Use `data-table` for new layouts.

Use `grid`, repeated cards, `data-table`, or a reusable `component` unless the project already has a specific legacy table implementation that you are intentionally preserving.

---

# Chapter 6: Evals and Quality

Status: draft

Eval design and quality loops

## Sections

- [Eval Strategy](#doc-ch06-evals-and-quality-01-eval-strategy)
- [Datasets And Graders](#doc-ch06-evals-and-quality-02-datasets-and-graders)
- [Run Analysis And Reporting](#doc-ch06-evals-and-quality-03-run-analysis-and-reporting)
- [Regression Gates](#doc-ch06-evals-and-quality-04-regression-gates)

## Alignment with User Guide Evals Flow

- Section 1 (**Eval Strategy**) aligns to [Eval Design Process](../../user-guide/evals/eval-design-process.md) for decision framing, failure taxonomy, and grader strategy.
- Section 2 (**Datasets And Graders**) aligns to [Ultimate Guide](../../user-guide/evals/ultimate-guide.md) Stage 3 and Stage 4 (convert findings into eval assets, then calibrate).
- Section 3 (**Run Analysis And Reporting**) aligns to [Ultimate Guide](../../user-guide/evals/ultimate-guide.md) Stage 2, Stage 4.5, and Stage 6 (error analysis, the run-to-rerun operator loop, and advanced diagnostics).
- Section 4 (**Regression Gates**) aligns to [Ultimate Guide](../../user-guide/evals/ultimate-guide.md) Stage 4.5 and Stage 5 (the operator loop plus the split between CI regression and production monitoring).
- Recommended reading order: [Evals Overview](../../user-guide/evals/README.md) -> [Eval Design Process](../../user-guide/evals/eval-design-process.md) -> [Ultimate Guide](../../user-guide/evals/ultimate-guide.md) -> this chapter sections.
- Supporting eval pages: [Scenarios](../../user-guide/evals/scenarios/README.md), [Scenario: Operate an eval improvement loop](../../user-guide/evals/scenarios/operate-an-eval-improvement-loop.md), [Dataset details](../../user-guide/evals/dataset-details.md), [Run details](../../user-guide/evals/run-details.md), [Reports](../../user-guide/evals/reports.md), [Settings](../../user-guide/evals/settings.md).
- Supporting eval dialogs: [Dataset editor](../../user-guide/evals/dialogs/dataset-editor.md), [Eval task editor](../../user-guide/evals/dialogs/eval-task-editor.md), [Import tasks](../../user-guide/evals/dialogs/import-candidates.md), [LLM Judge](../../user-guide/evals/dialogs/llm-judge.md), [Promote Conversation to Eval](../../user-guide/evals/dialogs/promote-to-eval.md), [Human Review](../../user-guide/evals/dialogs/human-review.md), [Select Eval Runs](../../user-guide/evals/dialogs/select-eval-runs.md), [Start an eval run](../../user-guide/evals/dialogs/start-run.md), [View trial details](../../user-guide/evals/dialogs/turn-details.md), [Manage eval folders](../../user-guide/evals/dialogs/rename-folder.md).

## Fast path inside Gaia

1. Start with [Evals Overview](../../user-guide/evals/README.md) so the workspace structure is clear.
2. Use [Eval Design Process](../../user-guide/evals/eval-design-process.md) to define the failure model, grader plan, and release question.
3. Use the dialogs and supporting pages to create datasets, tasks, graders, saved subsets, and initial runs on real project traces.
4. Use [Run details](../../user-guide/evals/run-details.md), [Human Review](../../user-guide/evals/dialogs/human-review.md), and [Reports](../../user-guide/evals/reports.md) to work the operator loop: inspect coverage posture, review evidence-risk queues, launch targeted reruns, compare against the linked baseline, and make the release decision.

The bridge for this chapter is simple: if the quality argument is not backed by a real eval asset or run in Gaia, it is still an opinion.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Eval Strategy

## Learning objectives

By the end of this section, you should be able to:

- Define an eval strategy in Gaia that is anchored to business decisions, not generic score chasing.
- Build a failure taxonomy that turns conversation evidence into repeatable evaluation assets.
- Separate must-pass gates from quality-improvement metrics and choose grader strategies accordingly.
- Produce an eval strategy artifact that can guide dataset, grader, and run design across a release cycle.

## Prerequisites

- You completed Chapter 5 and have at least one channel-ready assistant workflow.
- You can access **Conversations** and **Evals** in your project.
- You have representative recent conversations (including at least some problematic examples).
- You can identify stakeholders who own release decisions and quality accountability.

## In Gaia

- [Conversations](../../user-guide/conversations/README.md) for the trace evidence that seeds the taxonomy
- [Evals](../../user-guide/evals/README.md), [Eval Design Process](../../user-guide/evals/eval-design-process.md), and [Ultimate Guide](../../user-guide/evals/ultimate-guide.md) for turning the strategy into a usable eval program
- [Delivery Management](../../user-guide/delivery/README.md) when strategy outputs must influence release decisions

This section should result in a strategy that can be operationalized through Gaia eval assets and release work, not just a quality philosophy.

## Concept brief

Eval strategy is not the same as "running evals." It is the logic that connects product risk, engineering tradeoffs, and release decisions to measurable evidence.

Without strategy, teams collect scores but still cannot answer the core question: "Should we ship this version?"

In Gaia, a strong eval strategy links four layers:

- **Decision layer:** what decision the evaluation must support.
- **Risk layer:** what failures matter most.
- **Measurement layer:** how failures are detected and scored.
- **Operation layer:** when and how evaluations run over time.

This section establishes that foundation before you build datasets and graders in Section 2.

In practice, the quality of this strategy determines whether your eval program remains useful after the first month. Teams that skip this discipline often start with enthusiasm, then lose trust in scores because criteria, ownership, and decision usage were never made explicit. Strategy work is therefore not \"planning overhead\"; it is the mechanism that keeps quality work credible over time.

Alignment note:

- This section is the handbook implementation of [Eval Design Process](../../user-guide/evals/eval-design-process.md).
- It sets the decision and risk foundations used later in [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md).

### 1) Start with decision framing, not metric shopping

Most weak eval programs begin by selecting a metric dashboard first. That reverses cause and effect.

Start with one concrete decision:

- ship / no-ship,
- choose config A vs B,
- approve model migration,
- release a new tool policy,
- recover confidence after an incident.

For each decision, define:

- scope in/out,
- timeframe,
- required confidence level,
- responsible owner.

When this is explicit, metric design becomes straightforward. When it is vague, teams optimize whatever is easiest to measure instead of what is risky to miss.

### 2) Build a failure taxonomy from real traces

Failure taxonomy is the bridge between production behavior and eval design.

Use Conversations evidence to define a small set of named failure families, for example:

- wrong-tool selection,
- missing clarification before action,
- policy boundary violation,
- hallucinated factual claim,
- unhelpful refusal.

Good taxonomy characteristics:

- specific enough to act on,
- stable enough to trend over time,
- small enough to remain usable.

If taxonomy is too broad, findings are not actionable. If too granular, teams drown in labels and lose consistency.

### 3) Separate must-pass controls from quality targets

Not every quality criterion has equal severity.

Define two tiers:

- **Must-pass gates:** safety, compliance, and critical correctness.
- **Quality targets:** clarity, style, completion quality, latency tradeoffs.

This separation is essential for release integrity. A version may improve style while failing safety. Without tiering, average scores can hide unacceptable risk.

Practical rule:

- must-pass failures block release,
- quality-target misses drive iteration priorities.

### 4) Map each criterion to the simplest reliable grader

One of the biggest mistakes is using one rubric judge for everything.

Choose grader strategy by criterion type:

- deterministic outcome checks for explicit structured rules,
- transcript pattern checks for required/forbidden language,
- rubric graders for nuanced quality judgments,
- human review for ambiguous or high-stakes classes.

The goal is reliability per criterion, not elegance in grader count. Mixed-grader design usually outperforms monolithic grading.

### 5) Strategy must cover both expected and adversarial behavior

If eval tasks cover only happy paths, they validate optimism, not reliability.

Your strategy should include:

- representative normal tasks,
- known edge cases,
- ambiguous prompts requiring clarification,
- adversarial/policy boundary prompts,
- tool-required and tool-avoidance scenarios.

Balanced coverage helps detect regression where systems typically fail first: ambiguity and boundary handling.

### 6) Define calibration as part of strategy, not post-processing

Calibration aligns automated graders with human judgment. Without calibration, scores are fragile and hard to trust.

Calibration strategy should specify:

- pilot dataset size,
- reviewer roles,
- disagreement thresholds,
- rubric refinement process,
- re-calibration cadence.

If human/automated disagreement is ignored, teams may ship based on misleading confidence. Calibration is mandatory for robust decision support.

### 7) Plan separate loops for regression and discovery

A mature strategy includes two loops:

- **Regression loop:** catches known failures after changes.
- **Discovery loop:** finds new failures from production traces.

Regression loop characteristics:

- smaller, stable, fast datasets,
- frequent execution,
- strong deterministic checks.

Discovery loop characteristics:

- sampled real conversations,
- taxonomy updates,
- broader qualitative analysis.

If you run only regression, you miss novel drift. If you run only discovery, you miss release safety.

### 8) Tie eval strategy to operating budgets and release cadence

Evaluation cost and latency are real constraints. Strategy should be operationally feasible.

Define:

- eval frequency (per PR, daily, weekly),
- run size tiers (smoke vs full gate),
- allowed runtime and cost budgets,
- escalation path when budgets are exceeded.

This keeps evaluation sustainable. Overly heavy gates are often bypassed under pressure, which defeats the purpose.

### 9) Make strategy artifacts reviewable and versioned

Eval strategy should not live only in conversation or memory.

Maintain a versioned artifact containing:

- decision statements,
- failure taxonomy,
- gate definitions,
- grader mapping,
- run cadence,
- ownership and escalation.

Update the artifact when product scope, tooling, or policy changes. Versioned strategy enables consistent onboarding and improves cross-team alignment.

### 10) Definition of done for Eval Strategy

Section 1 is done when your team can justify why each eval exists and how it influences decisions.

Done criteria:

- decision framing is explicit,
- failure taxonomy is grounded in real traces,
- must-pass and quality criteria are separated,
- grader strategy is criterion-aligned,
- calibration and loop cadence are defined,
- one lab validates strategy against representative evidence.

If another engineer can design coherent datasets and graders from your strategy artifact without reinterpretation, you are ready for Section 2.

This is also the point where stakeholder alignment becomes visible. If product, engineering, and operations owners can read the same charter and reach the same decision interpretation, your strategy is functioning as intended.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces an "Eval Strategy Charter" artifact.

### Scenario

You are preparing to ship or revise a production-facing assistant and need a defensible evaluation strategy that supports release decisions.

### Phase A: Decision charter definition

1. Choose one release-relevant decision (for example: "ship updated support config").
2. Define:
   - decision owner,
   - scope in/out,
   - required confidence threshold,
   - decision deadline.
3. Write release consequence if decision is wrong.

Checkpoint A:

- Decision framing is concrete, owned, and time-bounded.

### Phase B: Evidence intake and failure taxonomy

4. Sample 20-40 recent conversations across normal and problematic cases.
5. Review traces using conversation tools (timeline, messages, feedback where available).
6. Label each issue with provisional failure categories.
7. Consolidate into a taxonomy of 6-12 failure families.
8. Rank top failure families by business impact.

Checkpoint B:

- Taxonomy is evidence-based and prioritized by risk.

### Phase C: Criteria and gate tiering

9. For top failure families, define criteria as:
   - must-pass gate,
   - quality target,
   - informational metric.
10. Specify pass/fail rules and expected evidence source.
11. Define minimum acceptable release gate profile.

Checkpoint C:

- Gate logic clearly distinguishes blocking risk from improvement opportunities.

### Phase D: Grader strategy and calibration plan

12. Map each criterion to grader type(s): deterministic, transcript, rubric, human review.
13. Define calibration pilot plan:

- initial pilot size,
- human reviewer set,
- disagreement resolution steps,
- acceptable agreement threshold.

14. Define known ambiguity classes requiring mandatory human adjudication.

Checkpoint D:

- Grader mapping and calibration process are explicit and practical.

### Phase E: Loop and cadence design

15. Define regression loop (fast recurring gate) and discovery loop (production learning).
16. Assign run cadence and ownership per loop.
17. Define alert/escalation triggers for major quality regressions.
18. Define strategy review cadence (for example monthly taxonomy review).

Checkpoint E:

- Strategy includes continuous operation, not one-time setup.

### Phase F: Publish strategy charter

19. Create `chapter-06-section-01-eval-strategy-charter.md` with:

- decision charter,
- failure taxonomy,
- criteria and gates,
- grader mapping,
- calibration plan,
- regression/discovery loop design,
- owners and review cadence.

20. Add go/no-go recommendation for Section 2 dataset and grader implementation.

Checkpoint F:

- Another engineer can design datasets and graders directly from your charter.

## Expected outputs

By the end of this lab, you should have:

- A documented decision-driven eval strategy for one release-critical workflow.
- A prioritized failure taxonomy derived from real conversation evidence.
- A tiered criterion model separating must-pass gates from quality targets.
- A criterion-to-grader strategy with calibration plan.
- A defined operating cadence for regression and discovery loops.
- An Eval Strategy Charter artifact with go/no-go recommendation for Section 2.

Evidence that qualifies:

- Conversation sampling notes linked to taxonomy decisions.
- Strategy artifact with clear ownership and gate logic.
- Explicit calibration and escalation procedures.

## Failure modes

1. **Metric-first strategy without decision context**
   - Symptom: dashboards improve but release decisions remain ambiguous.
   - Recovery: anchor strategy in explicit decision charters.

2. **Failure taxonomy built from assumptions, not evidence**
   - Symptom: evals miss real production defects.
   - Recovery: derive taxonomy from reviewed traces and update regularly.

3. **Must-pass and quality criteria blended together**
   - Symptom: critical failures hidden by average score improvements.
   - Recovery: enforce explicit gate tiers with separate pass rules.

4. **Single-grader overreliance**
   - Symptom: scoring instability and poor failure localization.
   - Recovery: map criteria to mixed grader types by reliability fit.

5. **No calibration plan**
   - Symptom: automated scores drift from human expectations.
   - Recovery: define pilot calibration and disagreement resolution workflow.

6. **One-loop evaluation model only**
   - Symptom: either regressions or new failure modes go undetected.
   - Recovery: run both regression and discovery loops with separate goals.

7. **Unsustainable evaluation cadence**
   - Symptom: gates are bypassed due to cost/time pressure.
   - Recovery: tier run sizes and align cadence with operational budgets.

8. **Strategy undocumented or ownerless**
   - Symptom: evaluation quality depends on individual memory.
   - Recovery: publish versioned strategy artifact with named owners.

## Completion checklist

- [ ] I defined a concrete release or comparison decision with owner and timeline.
- [ ] I built a prioritized failure taxonomy from real conversation evidence.
- [ ] I separated must-pass gates from quality-improvement targets.
- [ ] I mapped each criterion to appropriate grader types.
- [ ] I defined a calibration process with human-review alignment.
- [ ] I defined both regression and discovery evaluation loops with cadence.
- [ ] I published an Eval Strategy Charter with go/no-go recommendation for Section 2.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval Design Process](../../user-guide/evals/eval-design-process.md)
- [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Scenario: Create and Run an Eval](../../user-guide/evals/scenarios/create-and-run-an-eval.md)

---

# Datasets And Graders

## Learning objectives

By the end of this section, you should be able to:

- Build eval datasets in Gaia that represent real user risk profiles and workflow coverage.
- Design grader systems that combine deterministic checks, transcript rules, rubric judgment, and human review effectively.
- Calibrate grader behavior to reduce false confidence and improve release reliability.
- Produce a reusable dataset-and-grader specification for ongoing regression and discovery loops.

## Prerequisites

- You completed [Eval Strategy](#doc-ch06-evals-and-quality-01-eval-strategy) and have a strategy charter.
- You can access **Evals → Datasets**, **Graders**, and **Runs**.
- You have a prioritized failure taxonomy and gate criteria from Section 1.
- You can identify representative conversation traces and edge-case prompts for task authoring.

## In Gaia

- [Datasets](../../user-guide/evals/datasets.md), [Graders](../../user-guide/evals/graders.md), and [Runs](../../user-guide/evals/runs.md) for the actual eval assets
- [Dataset details](../../user-guide/evals/dataset-details.md), [Dataset editor](../../user-guide/evals/dialogs/dataset-editor.md), [Eval task editor](../../user-guide/evals/dialogs/eval-task-editor.md), [Start an eval run](../../user-guide/evals/dialogs/start-run.md), and [LLM Judge](../../user-guide/evals/dialogs/llm-judge.md) for asset creation, subset definition, and grader setup
- [Promote Conversation to Eval](../../user-guide/evals/dialogs/promote-to-eval.md) when real traces should become regression assets

Build the first dataset and grader set in Gaia as you read. The value of this section is in repeatable eval evidence, not in a generic grading philosophy.

## Concept brief

Datasets and graders are the execution core of an eval system. Strategy tells you what matters; datasets and graders operationalize that into repeatable evidence.

Weak datasets and graders produce misleading pass rates:

- dataset tasks are too easy or too synthetic,
- grader logic is overbroad or inconsistent,
- calibration is skipped,
- signal-to-noise ratio is poor.

Strong datasets and graders give you two outcomes:

- release confidence for known risks,
- clear diagnostics when failures occur.

They also create operational continuity. When dataset and grader patterns are stable, teams can compare quality over time without reinterpreting every run from scratch. This continuity is essential for regression gates, incident response, and long-term product reliability planning.

Alignment note:

- This section operationalizes [Eval Design Process](../../user-guide/evals/eval-design-process.md) task/grader design decisions.
- It directly maps to [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md) Stage 3 (convert findings to eval assets) and Stage 4 (calibrate before scaling).

### 1) Dataset design should mirror risk topology

A dataset is not a random task list. It should encode your failure risk distribution.

Design datasets by risk domains, for example:

- policy/safety,
- tool correctness,
- retrieval/factuality,
- workflow completion,
- communication clarity.

Within each domain, include:

- baseline tasks,
- edge tasks,
- adversarial/ambiguous tasks.

This prevents inflated scores caused by overrepresentation of easy prompts.

### 2) Use folder, tagging, and saved-subset structure as governance primitives

Dataset organization affects operational clarity.

Recommended structure patterns:

- `release/` for ship/no-ship gates,
- `regression/` for stable recurring checks,
- `security/` for policy-sensitive scenarios,
- `discovery/` for recently promoted production failures.

Use tags for cross-cutting dimensions:

- `tool-use`, `clarification`, `handoff`, `refusal`, `tone`.

Good structure makes it easier to compose runs by intent and to interpret results quickly.

Saved subsets inside [Dataset details](../../user-guide/evals/dataset-details.md) should preserve how you run that structure in practice. Keep at least one named pilot or smoke slice and one review-follow-up slice so calibration and rerun work does not depend on rebuilding filters from memory.

### 3) Prefer representative tasks over synthetic volume

Large synthetic datasets can look impressive but fail to predict production behavior.

Task sourcing priority:

1. promoted real traces from conversations,
2. curated edge cases from failure taxonomy,
3. synthetic generated tasks to fill controlled gaps.

Synthetic generation is useful, but only after real failure classes are understood. Otherwise you risk testing what is easy to generate rather than what is risky to miss.

### 4) Define task contracts with explicit success semantics

Every eval task should include clear success semantics.

Task contract fields:

- user intent,
- required behavior,
- forbidden behavior,
- expected tool usage (if any),
- expected output constraints,
- evidence sources for grading.

Ambiguous task contracts cause grader drift and reviewer disagreement. Task clarity is a major determinant of evaluation quality.

When possible, include explicit \"first acceptable behavior\" and \"first unacceptable behavior\" examples in task notes. This reduces interpretation drift for both rubric graders and human reviewers, especially in borderline policy or ambiguity scenarios.

### 5) Design graders by criterion type, not convenience

Grade each criterion with the most reliable mechanism:

- deterministic checks for structured outputs and hard constraints,
- transcript patterns for required/forbidden language,
- tool-use graders for call counts, ordered tool paths, output schemas, and error-free execution,
- rubric graders for nuanced judgments,
- human review for high-ambiguity cases.

For tool-driven tasks, prefer exact sequences when no extra calls are acceptable and ordered
subsequences when instrumentation or supporting tools may appear between required steps. Validate
every call when one malformed or failed result should block the task; reserve any-call validation
for workflows where one successful candidate is explicitly sufficient.

Avoid forcing nuanced behavior into brittle deterministic logic, or forcing hard constraints into subjective rubric scoring. Mixed grading usually provides better accuracy and diagnostics.

### 6) Multi-grader composition should be intentional

Combining graders is powerful but can become opaque.

Use composition rules deliberately:

- **all** for strict gate conjunction,
- **any** for alternate acceptable success paths,
- **majority** for uncertainty-tolerant quality signals,
- **weighted** where some criteria should dominate scores.

Document why a combination policy was chosen. Otherwise final verdicts become hard to interpret or defend during release discussions.

### 7) Calibrate graders against human judgment early

Calibration prevents false confidence.

Calibration workflow:

- run a pilot subset,
- save the exact pilot filter as a named subset so the same slice can be rerun after every grading change,
- compare grader verdicts with human reviewers,
- analyze disagreement by failure family,
- refine grader prompts/rules/examples,
- rerun until alignment is acceptable.

Track recurring disagreement types. If disagreements cluster around one criterion, split that criterion or adjust evidence inputs rather than forcing one grader to do too much.

Calibration should also include timing discipline. If calibration happens only once and never revisits changed workflows, grader quality decays quietly as product behavior evolves. Add a scheduled recalibration checkpoint whenever there is a significant model, prompt, tool, or policy update.

### 8) Include negative and refusal tasks explicitly

Many teams overfocus on completion and under-test appropriate refusal or clarification behavior.

Include tasks where the assistant should:

- refuse unsafe requests,
- ask clarification before acting,
- avoid tool calls when context is insufficient,
- defer/escalate according to policy.

These tasks protect against high-impact failures that may be rare but costly.

### 9) Version datasets and graders with change intent

Dataset/graders evolve as product risk evolves. Uncontrolled changes break trend continuity.

For each meaningful update, record:

- what changed,
- why it changed,
- expected impact on pass rates,
- backward-compatibility/trend interpretation notes.

Version discipline helps distinguish true quality changes from measurement shifts.

### 10) Definition of done for Datasets and Graders

Section 2 is done when your evaluation assets are representative, reliable, and maintainable.

Done criteria:

- datasets reflect risk topology and real usage,
- task contracts are explicit,
- graders are criterion-appropriate,
- calibration evidence exists,
- composition logic is interpretable,
- one lab demonstrates repeatable dataset/grader operation.

If another engineer can run your datasets and explain verdict behavior confidently, Section 2 is ready for run analysis in Section 3.

At this stage, your eval assets should be resilient enough that new contributors can add tasks without diluting signal quality. That contributor-friendliness is a practical indicator that your structure and grading design are mature.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Dataset and Grader Specification Pack" artifact.

### Scenario

You are implementing the eval strategy from Section 1 for a release-critical assistant workflow.

### Phase A: Dataset architecture setup

1. Create folder structure in **Evals → Datasets** aligned to strategy loops (`release`, `regression`, `discovery`, etc.).
2. Create at least one dataset for release gating and one for recurring regression.
3. Define dataset metadata (description, tags, owner).
4. Record intended run cadence per dataset.

Checkpoint A:

- Dataset structure reflects strategy and ownership, not ad hoc naming.

Before moving on, open [Dataset details](../../user-guide/evals/dataset-details.md) and save at least one reusable subset for pilot or smoke work and one for review follow-up.

### Phase B: Task authoring and sourcing

5. Populate release dataset with 15-30 tasks using:
   - promoted conversations,
   - taxonomy-derived edge cases,
   - selective synthetic tasks.
6. Ensure each major failure family has coverage.
7. Add explicit negative/refusal tasks.
8. Validate task contracts for clarity and evidence sources.

Checkpoint B:

- Dataset coverage maps to prioritized risks with explicit task semantics.

### Phase C: Grader design and mapping

9. Create graders for each criterion class:
   - deterministic outcome,
   - transcript pattern,
   - tool use for tool-driven tasks,
   - rubric judge,
   - human-review fallback where required.
10. Tag graders by purpose and risk domain.
11. Map graders to tasks and define expected verdict behavior.

Checkpoint C:

- Grader set is complete, interpretable, and criterion-aligned.

### Phase D: Composition and threshold policy

12. Build multi-grader compositions where needed.
13. Define pass thresholds and verdict aggregation rules.
14. Verify that must-pass criteria cannot be masked by softer scores.
15. Document verdict policy rationale.

Checkpoint D:

- Final verdict logic matches release gate intent.

### Phase E: Calibration pilot

16. Run pilot subset (10-20 tasks), ideally from a saved subset in [Dataset details](../../user-guide/evals/dataset-details.md) so the exact scope stays reproducible.
17. Compare automated verdicts with human review on selected trials.
18. Identify disagreements by criterion/failure family.
19. Refine tasks/graders and rerun pilot or a narrower follow-up slice.

Checkpoint E:

- Calibration improves alignment and reduces ambiguous scoring behavior.

### Phase F: Publish specification pack

20. Create `chapter-06-section-02-dataset-grader-specification-pack.md` including:

- dataset architecture and coverage matrix,
- task contract templates,
- grader inventory and mapping,
- composition/threshold policy,
- calibration outcomes,
- known limitations and next updates.

21. Add go/no-go recommendation for Section 3 run analysis and reporting.

Checkpoint F:

- Another engineer can extend datasets/graders without breaking evaluation intent.

## Expected outputs

By the end of this lab, you should have:

- Structured eval datasets aligned to release and regression goals.
- Reusable saved subsets for smoke or review-focused reruns.
- Task sets covering normal, edge, adversarial, and refusal behaviors.
- Criterion-aligned graders with clear mapping and ownership.
- Explicit final-verdict composition and threshold policy.
- Calibration evidence showing grader-human alignment progress.
- A Dataset and Grader Specification Pack with go/no-go recommendation for Section 3.

Evidence that qualifies:

- Datasets and graders visible/configured in Gaia.
- Coverage matrix tied to taxonomy and gate criteria.
- Pilot calibration notes with refinement history.

## Failure modes

1. **Dataset coverage skewed to easy tasks**
   - Symptom: high pass rates but repeated production failures.
   - Recovery: rebalance tasks to include risk-prioritized edge and adversarial cases.

2. **Weak dataset organization**
   - Symptom: runs mix incompatible purposes and metrics are hard to interpret.
   - Recovery: separate datasets by loop objective and apply consistent tags.

3. **Ambiguous task definitions**
   - Symptom: graders and humans disagree because expected behavior is unclear.
   - Recovery: strengthen task contracts with explicit required/forbidden behavior.

4. **Grader mismatch by criterion type**
   - Symptom: unstable scoring and high false positives/negatives.
   - Recovery: remap criteria to appropriate grader mechanisms.

5. **Opaque multi-grader composition**
   - Symptom: final verdict seems inconsistent or unexplainable.
   - Recovery: simplify composition logic and document aggregation rationale.

6. **Calibration skipped or superficial**
   - Symptom: automated scores drift from reviewer judgment.
   - Recovery: run pilot calibration with explicit disagreement analysis and iteration.

7. **Refusal/negative behavior under-tested**
   - Symptom: unsafe behavior passes due to completion-focused tasks only.
   - Recovery: add dedicated refusal and boundary task sets.

8. **Unversioned asset changes**
   - Symptom: trend shifts cannot be distinguished from quality changes.
   - Recovery: version datasets/graders with change intent and expected impact notes.

9. **Repeated wizard setup drifts between cycles**
   - Symptom: teams rebuild the same generation and grading choices differently each time.
   - Recovery: save the reviewed choices as an Eval design template, apply it to a fresh draft, and
     record the deliberate changes for the new cycle.

## Completion checklist

- [ ] I created dataset structure aligned to strategy loops and ownership.
- [ ] I authored tasks that cover high-risk failure families and negative cases.
- [ ] I defined clear task contracts with explicit success/failure semantics.
- [ ] I built and mapped graders using criterion-appropriate methods.
- [ ] I defined composition and threshold policies that preserve must-pass gates.
- [ ] I ran calibration pilot and refined assets based on disagreement analysis.
- [ ] I published a Dataset and Grader Specification Pack with go/no-go recommendation for Section 3.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval Design Process](../../user-guide/evals/eval-design-process.md)
- [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md)
- [Eval Datasets](../../user-guide/evals/datasets.md)
- [Dataset details](../../user-guide/evals/dataset-details.md)
- [Graders](../../user-guide/evals/graders.md)
- [Eval Design Wizard](../../user-guide/evals/dialogs/eval-design-wizard.md)
- [Grader Editor](../../user-guide/evals/dialogs/grader-editor.md)
- [Start an Eval Run](../../user-guide/evals/dialogs/start-run.md)
- [Human Review](../../user-guide/evals/dialogs/human-review.md)
- [Scenario: Operate an eval improvement loop](../../user-guide/evals/scenarios/operate-an-eval-improvement-loop.md)

---

# Run Analysis And Reporting

## Learning objectives

By the end of this section, you should be able to:

- Analyze eval runs and trial-level evidence to identify root-cause quality failures.
- Distinguish signal from noise across pass rates, grader outcomes, and trend metrics.
- Build reporting views that support release, iteration, and stakeholder communication decisions.
- Produce a run analysis artifact that connects findings to prioritized engineering actions.

## Prerequisites

- You completed [Datasets And Graders](#doc-ch06-evals-and-quality-02-datasets-and-graders) and have calibrated pilot assets.
- You can access **Evals → Runs**, **Run details**, **Trial details**, and **Reports**.
- You have at least one recent eval run with mixed outcomes (not only all-pass/all-fail).
- You can map observed failures back to model/prompt/tool/data/config dimensions.

## In Gaia

- [Runs](../../user-guide/evals/runs.md), [Run details](../../user-guide/evals/run-details.md), [Human Review](../../user-guide/evals/dialogs/human-review.md), and [View trial details](../../user-guide/evals/dialogs/turn-details.md) for trial-level evidence
- [Reports](../../user-guide/evals/reports.md) and [Select Eval Runs](../../user-guide/evals/dialogs/select-eval-runs.md) for release-facing summaries
- [Tasks](../../user-guide/tasks/README.md) and [Delivery Management](../../user-guide/delivery/README.md) when findings need owners and follow-up

Analyze real Gaia runs before you summarize. The report should be traceable from metric to trial evidence to action owner.

## Concept brief

Running evals is easy; extracting trustworthy insight is harder. Many teams stop at one top-line metric (for example overall pass rate) and miss the patterns that actually drive reliability.

Run analysis is where evaluation becomes an engineering feedback system. It answers:

- what failed,
- why it failed,
- how often it fails,
- how severe it is,
- what to fix first.

Reporting then translates this into operational decisions for engineering, product, and release stakeholders.

High-quality analysis work is less about producing impressive charts and more about producing trustworthy explanations. If stakeholders cannot trace a recommendation from metric to evidence to action owner, the report is informative but not operational.

Another practical challenge is narrative bias. Teams can over-emphasize findings that confirm expectations and underweight contradictory signals. A disciplined reporting structure counters this by requiring every major claim to include both supporting evidence and explicit caveats.

Alignment note:

- This section maps to [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md) Stage 2 (error analysis) and Stage 6 (advanced diagnostics).
- It assumes decision framing from [Eval Design Process](../../user-guide/evals/eval-design-process.md) is already in place.

### Operator loop inside Gaia

Once a pilot run exists, use Gaia as a repeatable operator loop instead of rebuilding the same workflow by hand.

1. Start from [Run details](../../user-guide/evals/run-details.md) and read **Coverage posture** before drilling into individual trials.
2. Use the evidence queue buttons and [Human Review](../../user-guide/evals/dialogs/human-review.md) to isolate ambiguous, weak-evidence, or reviewer-disagreement slices.
3. Launch the smallest useful rerun from the current run: **Failed / Partial / Unreviewed** when validating a fix, or **No Pass Yet / Needs Review** when closing coverage gaps.
4. Let the linked baseline comparison judge the rerun on the shared task scope instead of treating every rerun like a fresh full-dataset experiment.
5. Use [Reports](../../user-guide/evals/reports.md) and [Select Eval Runs](../../user-guide/evals/dialogs/select-eval-runs.md) to aggregate only the comparable runs and turn the result into a release or learning decision.
6. Promote new failure examples back into datasets when the loop reveals a real missing case instead of only a transient run anomaly.

For a user-facing walkthrough of this exact loop, use [Scenario: Operate an eval improvement loop](../../user-guide/evals/scenarios/operate-an-eval-improvement-loop.md).

### 1) Treat run status as a starting point, not a conclusion

Run status (`completed`, `failed`, `stopped`) is operational metadata, not quality truth.

A completed run can still reveal critical quality failures. A failed run may still provide valuable partial signals.

First analysis pass should include:

- run configuration context,
- dataset scope,
- grader set,
- trial completion distribution,
- obvious operational anomalies.

Only after this context should you interpret quality outcomes.

### 2) Analyze at trial granularity before aggregate metrics

Aggregate metrics are useful, but root causes live at trial level.

Trial-first analysis pattern:

- inspect failed or low-score trials,
- classify failure type using taxonomy,
- identify first upstream break (prompt interpretation, tool call, policy check, etc.),
- capture representative examples.

After this, aggregate metrics become interpretable. Without trial inspection, trend charts can be misleading.

### 3) Segment failures by family and impact

Not all failures deserve equal urgency.

Segment each failure by:

- failure family (taxonomy label),
- severity (business/user impact),
- recurrence rate,
- detectability (easy/hard to catch).

This lets you prioritize high-impact recurring defects instead of chasing isolated low-impact noise.

Recommended prioritization heuristic:

- severity x frequency x confidence.

### 4) Interpret grader disagreement as diagnostic signal

Disagreement between graders or between grader and human review is not "bad data" by default. It often indicates unclear criteria or ambiguous behavior classes.

Use disagreement analysis to decide whether to:

- refine task contracts,
- split broad criteria,
- recalibrate rubric prompts,
- increase human-review sampling for specific categories.

Ignoring disagreement creates brittle reporting and false confidence.

### 5) Use metric families, not single-score obsession

Key metric families in Gaia reporting contexts:

- pass/fail rates by run and by failure family,
- pass@k / pass^k trends (where applicable),
- per-grader score distributions,
- completion and error statuses,
- human-agreement indicators.

A single composite metric can hide important regressions. For release decisions, require at least one gate metric plus supporting diagnostics.

Where practical, include uncertainty language in your interpretation (for example low sample caveats, non-comparable scope warnings, or limited confidence due to grader disagreements). This prevents false precision in executive summaries.

### 6) Compare runs only when scope is comparable

Trend comparisons are meaningful only when run scope is controlled.

Comparison validity checks:

- same dataset (or clearly versioned variant),
- similar grader definitions,
- similar configuration/model context,
- documented changes between runs.

If multiple factors changed simultaneously, comparisons should be treated as directional, not definitive.

### 7) Convert report findings into change hypotheses

Reports should drive action, not just observation.

For each priority issue, define:

- likely root cause layer,
- minimal corrective hypothesis,
- expected metric shift,
- validation plan for next run.

This turns reporting into a closed loop with measurable iteration outcomes.

Strong hypotheses should be falsifiable. If a proposed fix does not produce expected movement in the next run, treat that as a learning signal and revisit root-cause assumptions rather than layering more changes immediately.

### 8) Separate release reporting from learning reporting

Different audiences need different report forms.

- **Release reporting:** gate-focused, concise, decision-ready, must-pass status explicit.
- **Learning reporting:** exploratory, richer diagnostics, hypotheses and experiments.

Mixing these formats can confuse stakeholders. Keep release reports crisp and escalation-oriented; keep learning reports analytical and improvement-oriented.

### 9) Establish reporting cadence and ownership

Reporting quality improves when cadence is predictable.

Suggested rhythm:

- per-change smoke run summary,
- weekly regression trend review,
- monthly deeper diagnostic report,
- incident-triggered focused report.

Assign owners for:

- data preparation,
- analysis review,
- decision sign-off,
- follow-up tracking.

Ownerless reporting quickly devolves into dashboards nobody trusts.

### 10) Definition of done for Run Analysis and Reporting

Section 3 is done when run outcomes consistently lead to clear, prioritized engineering decisions.

Done criteria:

- trial-level root causes are identified for key failures,
- aggregate metrics are interpreted with context,
- run comparisons are methodologically sound,
- reports differentiate release gates from learning insights,
- corrective hypotheses and next-run validations are documented,
- one lab demonstrates repeatable analysis/report workflow.

If another engineer can reproduce your findings and act on them without extra explanation, Section 3 is ready for regression gate design in Section 4.

Done also implies report portability: a release decision meeting should be able to consume your brief quickly without requiring you to narrate hidden context live.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Run Analysis and Reporting Brief" artifact.

### Scenario

You have one or more eval runs for a release candidate and must determine whether quality is improving, stable, or regressing, then recommend actionable next steps.

### Phase A: Run context baseline

1. Select 2-3 recent runs (baseline, candidate, and optional prior stable run) and use report-selection coverage cues to confirm comparable scope.
2. Document run context for each:
   - dataset version,
   - grader set,
   - agent/config version,
   - run date and scope.
3. Validate comparability assumptions.

Checkpoint A:

- Comparison scope is explicit and defensible.

### Phase B: Trial-level failure analysis

4. Open **Run details** and inspect **Coverage posture** before drilling into individual trials.
5. Use the evidence queue buttons to isolate **Evidence Risk**, **No Evidence**, **Mixed Evidence**, or **Reviewer Disagreement** slices when the full run is too broad.
6. Inspect **Trial details** for each priority trial.
7. Classify failures by taxonomy and severity.
8. Capture representative trial examples per major failure family.

Checkpoint B:

- Priority failure families and representative evidence are identified.

### Phase C: Metric interpretation and trend extraction

9. Use **Reports** and linked baseline comparison on **Run details** to compute and compare metrics across selected runs.
10. Analyze:

- gate pass rates,
- pass@k / pass^k where relevant,
- per-family trends,
- grader distribution changes.

11. Highlight statistically or practically meaningful changes.

Checkpoint C:

- Trend statements are tied to both metrics and trial evidence.

### Phase D: Disagreement and ambiguity handling

12. Identify grader-human or grader-grader disagreements.
13. Determine whether issues are due to:

- unclear task contract,
- rubric ambiguity,
- real behavioral inconsistency.

14. Propose calibration or grading refinements.

Checkpoint D:

- Disagreement is converted into concrete grading/task improvements.

### Phase E: Action prioritization

15. Translate findings into prioritized corrective actions.
16. For each action, define:

- owner,
- expected impact,
- validation run or targeted rerun needed,
- risk if deferred.

17. Flag release-blocking issues explicitly.
18. Add one decision log entry per critical issue:

- ship/hold rationale,
- uncertainty notes,
- revalidation date.

Checkpoint E:

- Report yields a clear, prioritized action list with release implications.
- Key decisions include rationale plus uncertainty handling.

### Phase F: Publish analysis brief

19. Create `chapter-06-section-03-run-analysis-reporting-brief.md` including:

- run comparison context,
- key failure families and severity,
- metric trend summary,
- disagreement analysis,
- prioritized action plan,
- release recommendation and confidence level.

20. Add go/no-go recommendation for Section 4 regression gate formalization.

Checkpoint F:

- Stakeholders can make or defer release decisions using your brief without ambiguity.

## Expected outputs

By the end of this lab, you should have:

- A context-valid comparison of key eval runs.
- Coverage-posture and evidence-queue analysis that explains where the run still needs attention.
- Trial-level root-cause insights for major failure families.
- Metric trends interpreted with methodological discipline.
- Disagreement analysis and calibration refinement proposals.
- Prioritized corrective actions with ownership and validation plans.
- A Run Analysis and Reporting Brief with go/no-go recommendation for Section 4.

Evidence that qualifies:

- Run and trial references supporting each key finding.
- Reports metrics linked to explicit scope assumptions.
- Action plan tied to measurable expected outcomes.

## Failure modes

1. **Status-level analysis only**
   - Symptom: run marked complete and assumed healthy without quality drilldown.
   - Recovery: mandate trial-level analysis for failed/low-score segments.

2. **Aggregate-only interpretation**
   - Symptom: top-line pass rate hides critical failure clusters.
   - Recovery: segment by failure family and severity before conclusions.

3. **Invalid run comparisons**
   - Symptom: conclusions drawn from runs with materially different scope.
   - Recovery: enforce comparability checklist and annotate non-comparable analyses.

4. **Ignoring grader disagreement**
   - Symptom: persistent score ambiguity and unstable release decisions.
   - Recovery: analyze disagreements and refine tasks/rubrics/calibration process.

5. **Metrics without root cause mapping**
   - Symptom: reports show numbers but no actionable fixes.
   - Recovery: require hypothesis and owner for each major regression signal.

6. **Release and learning reports conflated**
   - Symptom: stakeholders receive verbose analysis but unclear release decision.
   - Recovery: split gate-focused summary from exploratory diagnostics.

7. **No action ownership**
   - Symptom: recurring issues reappear with no accountability.
   - Recovery: assign named owners and validation deadlines in every report.

8. **No confidence statement with recommendations**
   - Symptom: go/no-go recommendations are difficult to trust.
   - Recovery: include confidence level and key uncertainty drivers.

## Completion checklist

- [ ] I compared runs using explicit and valid scope assumptions.
- [ ] I performed trial-level analysis for major failures.
- [ ] I interpreted metrics with failure-family segmentation and severity context.
- [ ] I analyzed grader disagreement and proposed calibration refinements.
- [ ] I produced prioritized, owner-assigned corrective actions.
- [ ] I separated release decision reporting from learning diagnostics.
- [ ] I published a Run Analysis and Reporting Brief with go/no-go recommendation for Section 4.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval Design Process](../../user-guide/evals/eval-design-process.md)
- [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md)
- [Eval Runs](../../user-guide/evals/runs.md)
- [Run Details](../../user-guide/evals/run-details.md)
- [Trial Details](../../user-guide/evals/trial-details.md)
- [Human Review](../../user-guide/evals/dialogs/human-review.md)
- [Reports](../../user-guide/evals/reports.md)
- [Select Eval Runs](../../user-guide/evals/dialogs/select-eval-runs.md)
- [Agent Behavior Trace](../../user-guide/evals/dialogs/agent-behavior-trace.md)
- [Scenario: Operate an eval improvement loop](../../user-guide/evals/scenarios/operate-an-eval-improvement-loop.md)

---

# Regression Gates

## Learning objectives

By the end of this section, you should be able to:

- Design regression gates that reliably block high-risk quality regressions before release.
- Define multi-tier gate policies (smoke, release, monitoring) aligned with risk and delivery cadence.
- Operationalize gate ownership, escalation, and rollback protocols.
- Produce a regression-gate runbook that closes Chapter 6 and supports continuous quality control.

## Prerequisites

- You completed [Run Analysis And Reporting](#doc-ch06-evals-and-quality-03-run-analysis-and-reporting) and have run-analysis evidence.
- You have stable baseline datasets/graders from Section 2.
- You can run evals and access reports before and after config/model/tool changes.
- You have identified release stakeholders who will enforce go/no-go decisions.

## In Gaia

- [Evals](../../user-guide/evals/README.md), [Dataset details](../../user-guide/evals/dataset-details.md), [Run details](../../user-guide/evals/run-details.md), [Reports](../../user-guide/evals/reports.md), and [Select Eval Runs](../../user-guide/evals/dialogs/select-eval-runs.md) for the actual gate evidence
- [Delivery Management](../../user-guide/delivery/README.md) and [Tasks](../../user-guide/tasks/README.md) for blocker tracking and exception follow-up
- [Dashboard](../../user-guide/dashboard/README.md) when regression protection must continue beyond pre-release runs

Use this section to define real gate outcomes and owners in Gaia. The system should make it obvious what failed, who decides, and what must happen next.

## Concept brief

A regression gate is the enforcement layer of your quality system. Strategy explains what matters; datasets and graders measure it; reporting interprets it; gates decide whether changes are allowed forward.

Without gates, evals become advisory. With poorly designed gates, teams either block too much or ship unsafe changes.

A robust regression-gate system should be:

- risk-aligned,
- fast enough to run consistently,
- strict on must-pass failures,
- transparent to decision-makers,
- evolvable after incidents.

The most important property is enforceability under pressure. A gate that works only when schedules are relaxed is not a gate; it is a suggestion. Design must account for real release tension, including who can override what and under which evidence requirements.

In practice, teams that succeed with gates build social clarity as well as technical clarity: everyone understands that gate outcomes are release controls, not optional recommendations. This cultural alignment reduces last-minute disputes and shortens decision time.

Alignment note:

- This section maps to [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md) Stage 5 (split CI regression from production monitoring).
- Gate criteria and escalation logic should trace back to [Eval Design Process](../../user-guide/evals/eval-design-process.md).

### 1) Gate design starts with risk appetite and blast radius

Different workflows need different strictness. A low-risk informational assistant does not need the same gate profile as a high-impact operations assistant.

Define gate strictness by blast radius:

- user inconvenience,
- incorrect business action,
- policy/compliance violation,
- operational outage or escalation burden.

Gate thresholds should reflect business risk tolerance, not engineering preference. If risk framing is absent, gate arguments become subjective and inconsistent.

### 2) Use gate tiers for speed and reliability

One heavyweight gate for every change is unsustainable. Use tiered gates:

- **Smoke gate:** fast subset to catch obvious breakage.
- **Release gate:** fuller suite for ship decisions.
- **Monitoring gate:** ongoing production-oriented checks.

Tiering balances speed and confidence. Smoke gates protect developer velocity; release gates protect customers; monitoring gates protect long-term stability.

In Gaia, make those tiers concrete with reusable dataset subsets or other stable task slices. A smoke gate should be one saved subset, a release gate should be another, and follow-up reruns should stay linked back to the source run instead of rebuilding ad hoc filters every time.

### 3) Must-pass criteria require hard blocking behavior

Some criteria are non-negotiable:

- policy and safety boundaries,
- critical correctness invariants,
- high-risk tool misuse prevention.

Gate policy must define:

- which criteria are hard blocks,
- whether any exceptions are allowed,
- required approval chain for exception handling.

If hard gates become optional under schedule pressure, trust in the quality system collapses.

When exceptions are unavoidable, treat them as formal risk acceptances with expiration dates, owners, and compensating controls. This keeps exceptional paths rare and auditable instead of becoming routine shortcuts.

### 4) Gate metrics should combine threshold and trend logic

Single-threshold gates are useful but insufficient. Trend awareness helps detect early drift.

Gate signal types:

- absolute threshold checks (for example must-pass = 100%),
- comparative trend checks vs baseline (for example no >X% drop),
- variance/stability checks across repeated trials.

This combination catches both sudden failures and gradual degradation.

### 5) Baseline management is critical for fair gating

All gates depend on what baseline they compare against.

Baseline rules should specify:

- which run/version is the reference,
- when baselines can be updated,
- who approves baseline reset,
- how baseline changes are documented.

Uncontrolled baseline updates can hide regressions. Baseline governance is part of gate integrity.

In Gaia, prefer **Run Again** or the rerun subset actions so the candidate run keeps a source-run link and exposes baseline comparison on the shared task scope automatically.

### 6) Define explicit gate outcomes and actions

Gate output should map to clear actions:

- **Pass:** proceed.
- **Conditional pass:** proceed with explicit risk note and follow-up.
- **Fail:** block release and trigger remediation.
- **Inconclusive:** rerun or investigate data/measurement integrity.

Each outcome needs owner actions and deadlines. Ambiguous outcomes cause release delays and inconsistent escalation.

For conditional passes, require a documented follow-up run or mitigation checkpoint. Conditional status without follow-through tends to accumulate hidden risk debt across releases.

### 7) Integrate gate failures into remediation loops

A gate fail is not the end; it is a control trigger.

Remediation loop:

1. classify failure family/severity,
2. assign corrective owner,
3. apply minimal fix,
4. rerun relevant gate tier,
5. document outcome and prevention update.

This loop prevents repeated "fix-and-hope" cycles.

### 8) Define rollback and kill-switch criteria before incidents

Even with gates, post-release regressions can occur. Define rollback logic early.

Rollback criteria examples:

- must-pass failure in monitoring gate,
- significant degradation in high-impact workflows,
- repeated policy incidents within short window.

Define rollback execution path:

- revert config/model/tool change,
- restore prior stable version,
- communicate status and next review time.

Prepared rollback logic reduces incident dwell time.

### 9) Evolve gates from incident learnings

Gates should improve over time. Incident reviews are the primary input for gate evolution.

For each serious incident, decide:

- which new task(s) should be added,
- which criterion needs tightening,
- whether new gate tier checks are needed,
- whether baseline assumptions were flawed.

A static gate suite becomes stale as product behavior and user patterns evolve.

### 10) Definition of done for Regression Gates and Chapter 6

Section 4 and Chapter 6 are done when evaluation can enforce quality decisions consistently under real delivery pressure.

Done criteria:

- tiered gate policy is documented and operational,
- must-pass blocks are explicit and enforceable,
- baselines and trend checks are governed,
- fail/conditional/inconclusive actions are defined,
- remediation and rollback loops are rehearsed,
- gate suite updates are connected to incident learnings,
- one lab demonstrates end-to-end gate operation.

At this point, Chapter 6 outputs become robust inputs for Chapter 7 observability, cost, and performance optimization.

In practice, this means quality governance is no longer dependent on one expert reviewer. The gate system itself carries enough structure for teams to make consistent decisions across multiple release cycles.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Regression Gate Operations Runbook" artifact.

### Scenario

You are preparing a release candidate and need to enforce quality gates that are fast, defensible, and actionable under schedule pressure.

### Phase A: Gate policy definition

1. Define gate tiers for your project:
   - smoke,
   - release,
   - monitoring.
2. Map datasets/graders to each tier.
3. Define must-pass vs quality-threshold criteria per tier.
4. Assign decision owners for each tier outcome.

Before moving on, define the saved subset or equivalent stable task slice that represents each tier so smoke and release gates can be rerun without rebuilding filters.

Checkpoint A:

- Gate tiers and owners are explicit and risk-aligned.

### Phase B: Baseline and threshold setup

5. Select baseline run(s) for comparison and record which subset or dataset each gate uses.
6. Define threshold and trend rules per criterion.
7. Define baseline-update policy and approval path.
8. Document acceptable variance ranges.

Checkpoint B:

- Baseline governance and thresholds are documented and auditable.

### Phase C: Execute smoke and release gates

9. Run smoke gate on current candidate from the saved smoke subset or other reproducible slice.
10. If smoke passes, run release gate on the release subset.
11. Open **Run details** and use **Coverage posture** plus the evidence queues to verify whether the result is genuinely ready for comparison.
12. Capture outcomes by criterion and failure family.
13. Compare baseline vs candidate in linked run comparison and in **Reports**.
14. Classify final status: pass, conditional pass, fail, or inconclusive.
15. Record decision sign-off and any exception notes in a release log.

Checkpoint C:

- Gate decisions are evidence-based and clearly classified.
- Sign-off path and any exceptions are recorded for auditability.

### Phase D: Failure handling drill

16. Intentionally use a known failing candidate (or replay prior failing run) to exercise fail path.
17. Run remediation loop:
    - analyze root cause,
    - apply minimal correction,

- rerun the affected gate tier or the narrowest useful subset (`Failed`, `No Pass Yet`, or `Needs Review`).

18. Confirm block-to-recovery workflow is practical.

Checkpoint D:

- Team can execute gate-fail remediation without ad hoc process gaps.

### Phase E: Rollback readiness check

19. Define rollback triggers for monitoring-gate failures.
20. Rehearse rollback steps to prior stable configuration/version.
21. Validate communication and ownership flow.

Checkpoint E:

- Rollback path is executable and owner-accountable.

### Phase F: Publish regression gate runbook

22. Create `chapter-06-section-04-regression-gate-operations-runbook.md` including:

- gate tier definitions,
- criterion thresholds and baselines,
- decision matrix and owners,
- remediation workflow,
- rollback criteria and steps,
- incident-to-gate update policy.

23. Add Chapter 6 readiness recommendation and carry-over priorities for Chapter 7.

Checkpoint F:

- Another engineer can enforce and evolve regression gates using your runbook.

## Expected outputs

By the end of this lab, you should have:

- A tiered regression-gate policy mapped to project risk.
- Reproducible smoke and release slices that can be rerun without rebuilding filters.
- Explicit must-pass and quality-threshold criteria per gate tier.
- Baseline/trend governance with owner approvals.
- Documented gate outcome actions (pass/conditional/fail/inconclusive).
- Practiced remediation and rollback workflows.
- A Regression Gate Operations Runbook with Chapter 6 readiness recommendation.

Evidence that qualifies:

- Gate runs and outcomes recorded for candidate vs baseline.
- Decision logs tied to criterion-level evidence.
- Runbook artifact with owner/accountability details.

## Failure modes

1. **No tiering, one heavy gate for everything**
   - Symptom: slow feedback and frequent gate bypass pressure.
   - Recovery: split smoke/release/monitoring tiers with aligned scope.

2. **Must-pass criteria treated as negotiable**
   - Symptom: critical policy/safety issues pass with subjective justifications.
   - Recovery: enforce hard-block policy and explicit exception governance.

3. **Threshold-only gating without trends**
   - Symptom: gradual quality drift goes undetected.
   - Recovery: add baseline comparison and trend-based guardrails.

4. **Uncontrolled baseline resets**
   - Symptom: regressions are normalized by moving the reference point.
   - Recovery: require approval and rationale for baseline updates.

5. **Ambiguous gate outcomes**
   - Symptom: teams debate release status without clear action path.
   - Recovery: standardize pass/conditional/fail/inconclusive definitions and owners.

6. **No remediation workflow after gate fail**
   - Symptom: repeated failed runs without systematic correction.
   - Recovery: enforce root-cause -> minimal fix -> rerun loop.

7. **Rollback plan untested**
   - Symptom: incident response is slow and risky.
   - Recovery: rehearse rollback path and communication responsibilities.

8. **Incident learnings not encoded into gates**
   - Symptom: same regression class recurs.
   - Recovery: update datasets/criteria post-incident with explicit policy.

## Completion checklist

- [ ] I defined smoke, release, and monitoring gate tiers aligned to risk.
- [ ] I defined must-pass and quality-threshold criteria per tier.
- [ ] I established baseline/trend governance and approval rules.
- [ ] I executed gate runs and classified outcomes with clear decision logic.
- [ ] I validated remediation workflow for failed gates.
- [ ] I validated rollback triggers and recovery steps.
- [ ] I published a Regression Gate Operations Runbook with Chapter 6 readiness recommendation.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Eval Design Process](../../user-guide/evals/eval-design-process.md)
- [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md)
- [Dataset details](../../user-guide/evals/dataset-details.md)
- [Eval Runs](../../user-guide/evals/runs.md)
- [Run Details](../../user-guide/evals/run-details.md)
- [Trial Details](../../user-guide/evals/trial-details.md)
- [Human Review](../../user-guide/evals/dialogs/human-review.md)
- [Reports](../../user-guide/evals/reports.md)
- [Start an Eval Run](../../user-guide/evals/dialogs/start-run.md)
- [Select Eval Runs](../../user-guide/evals/dialogs/select-eval-runs.md)
- [Scenario: Operate an eval improvement loop](../../user-guide/evals/scenarios/operate-an-eval-improvement-loop.md)

---

# Chapter 7: Observability, Cost, and Performance

Status: current

Operability and cost discipline

## Sections

- [Turn Tool AI Observability](#doc-ch07-observability-cost-and-performance-01-turn-tool-ai-observability)
- [Latency And Throughput](#doc-ch07-observability-cost-and-performance-02-latency-and-throughput)
- [Token And Cost Governance](#doc-ch07-observability-cost-and-performance-03-token-and-cost-governance)
- [Performance Troubleshooting](#doc-ch07-observability-cost-and-performance-04-performance-troubleshooting)

## Alignment with User Guide Observability Surfaces

- [Dashboard](../../user-guide/dashboard/README.md)
- [Platform Dashboard](../../user-guide/platform-dashboard/README.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Timesheet](../../user-guide/timesheet/README.md)
- [Tutorials](../../user-guide/tutorials/README.md)

Treat the operational review loop as the chapter spine: start in Dashboard for project-local review or Platform Dashboard for cross-project review, move to Audit Trail or Workflow Run Details when a spike needs explanation, capture follow-up in Tasks, and use Timesheet to verify whether the team has capacity to absorb the work. When you use inferred resolution, CSAT, CES, or custom SQL-backed dashboard widgets, label proxies clearly and keep the trace or source-query evidence with the review. For custom-dashboard inferred widgets, preserve the confidence grade, the tab's saved multi-select filters for agent, channel, model, and topic taxonomy, and exported result CSV alongside the review evidence. For explicit outcome widgets, preserve the import/API source reference and remember they are source-of-truth metrics separate from inferred proxies.

Use **Operations: Review, Audit, and Rebalance** in [Tutorials](../../user-guide/tutorials/README.md) at `/platform/support/tutorials` as the guided replay for this loop when onboarding operators or revalidating the workflow after UI changes.

## Fast path inside Gaia

1. Start in [Dashboard](../../user-guide/dashboard/README.md) or [Platform Dashboard](../../user-guide/platform-dashboard/README.md) to see whether the system is healthy.
2. Use [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md) and [Audit Trail](../../user-guide/audit/README.md) when you need explanation, lineage, or change history.
3. Use [Tasks](../../user-guide/tasks/README.md) to convert findings into owned remediation.
4. Use [Timesheet](../../user-guide/timesheet/README.md) when the operational fix depends on real team capacity.

If you cannot show the metric, trace, or follow-up work in Gaia, the observability conclusion is not yet operational.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Turn Tool AI Observability

## Learning objectives

By the end of this section, you should be able to:

- Design an observability model in Gaia that connects turn-level, tool-level, and AI-level behavior.
- Use conversation and workflow telemetry to explain performance and quality outcomes with traceable evidence.
- Define observability signals that support both engineering diagnostics and product operations decisions.
- Produce an observability blueprint artifact that can be reused for cost and performance control in later sections.

## Prerequisites

- You completed Chapter 6 and can run evals, inspect run details, and interpret reports.
- You can access **Conversations**, **Timeline**, **Dashboard**, and **Data Model → Runs**.
- You have at least one active workflow where tool calls and model responses occur in the same end-to-end flow.
- You can identify the release-critical conversations or runs that must remain observable in production.

## In Gaia

- [Dashboard](../../user-guide/dashboard/README.md)
- [Platform Dashboard](../../user-guide/platform-dashboard/README.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Model routing and failover](../../user-guide/settings/model-routing.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Timesheet](../../user-guide/timesheet/README.md)

Start in Dashboard for one project or Platform Dashboard for cross-project context, use Timeline and Inside Info to inspect the local trace, move to Audit Trail or Workflow Run Details to explain it, and capture follow-up in Tasks. For deployment-pool calls, record the logical model, target sequence, provider retry count, failover decision, final target, final outcome, and correlation identifiers from the **Model routing** block. A `401` or `403` on one deployment may be followed by another target without retrying the failing deployment; failover stops if that attempt already emitted partial content or proposed a tool call. Use Timesheet only when the remediation load itself needs rebalancing.

## Concept brief

Observability is the discipline of making system behavior explainable from evidence. In Gaia, this means you can answer not only "what happened" but also "where it happened" and "why it happened" across conversation turns, tool executions, and model generation.

Without this, performance tuning and quality debugging degrade into guesswork. Teams may over-optimize prompts when the real bottleneck is a tool call, or blame tools when the real issue is model latency or context size.

This section establishes a layered observability model that turns scattered telemetry into actionable understanding.

### 1) Think in three layers: turn, tool, and AI

A single user request spans multiple layers:

- **Turn layer:** user message to final response completion.
- **Tool layer:** external or internal function/tool invocations.
- **AI layer:** model generation behavior (tokens, first token, completion profile).

Each layer answers different questions:

- Turn: user-perceived experience.
- Tool: execution path and side effects.
- AI: model efficiency and reasoning behavior.

When these layers are disconnected, root-cause analysis is slow and error-prone.

### 2) Build trace continuity across layers

Observability quality depends on correlation. You should be able to follow one request through every stage.

Trace continuity requires:

- stable identifiers per turn/run,
- ordered step visibility,
- timestamped transitions,
- consistent context attribution.

In practical Gaia operations, this is surfaced via timeline views, run details, and trial traces. The key is not just having logs, but having logs that can be stitched together quickly.

### 3) Use turn observability for user-impact visibility

Turn-level metrics should answer:

- how long the user waited,
- where the wait occurred,
- whether output quality degraded,
- whether retries/rephrasings were needed.

Turn observability is the closest proxy to user trust. If turn latency and failure patterns are not tracked, teams may miss experience regressions that never show up in lower-level logs.

### 4) Use tool observability for execution accountability

Tool calls are where many hidden failures originate.

Tool observability should capture:

- invocation count and ordering,
- arguments and result shape quality,
- success/failure status,
- execution duration,
- retry or fallback behavior.

This makes it possible to answer high-value questions:

- Was the wrong tool selected?
- Did the right tool run with wrong arguments?
- Did tool latency dominate turn time?

Without tool-level accountability, teams overfit fixes in prompts and miss execution defects.

For workflow traces, choose context detail deliberately. Detailed context makes nested inputs and outputs inspectable, while compact summaries reduce log volume for high-throughput automation. Define workflow redaction paths for business-sensitive values before production use; Gaia's built-in secret masking remains active in both modes.

### 5) Use AI observability for model-behavior governance

AI-level telemetry informs cost/performance optimization directly.

Core AI signals include:

- token usage patterns,
- cached vs non-cached tokens where applicable,
- first-token timing,
- completion duration,
- error/refusal behavior.

These signals help tune model selection, reasoning effort, and context size policies. They also explain why similar user requests can have different cost and latency profiles.

### 6) Distinguish operational alerts from analytical reporting

Not all observability is real-time alerting. Separate two use modes:

- **Operational mode:** detect incidents fast (timeouts, error spikes, blocked workflows).
- **Analytical mode:** understand trends and optimize system design over time.

Operational signals should be small and actionable. Analytical views can be richer and comparative.

Mixing them leads to either noisy alerts or shallow insights.

### 7) Define observability SLOs and ownership

Telemetry without targets is data exhaust. Define service-level expectations for observability itself:

- max acceptable turn latency for key workflows,
- tool success rate thresholds,
- acceptable model error/refusal profile,
- time-to-detect for regressions.

Assign owners for each layer. If ownership is unclear, incidents bounce between teams and resolution slows.

### 8) Use canonical investigation paths

During incidents, teams need a consistent diagnostic path. A practical Gaia path:

1. Check conversation timeline for user-visible slowdown.
2. Inspect tool timeline/trace for slow or failing steps.
3. Inspect model usage/cost/first-token behavior.
4. Cross-check workflow run details for pipeline-related coupling.
5. Confirm trend impact in dashboard and eval reports.
6. Capture remediation in Tasks and use Timesheet when you need to rebalance effort across operators.

Canonical investigation paths reduce variance in troubleshooting quality across engineers.

### 9) Treat observability artifacts as product assets

Observability should produce artifacts, not only ephemeral screenshots.

Useful artifacts:

- baseline signal map,
- layer-specific metric definitions,
- incident correlation examples,
- review cadence and escalation paths.

These artifacts improve onboarding, reduce repeated mistakes, and support governance/audit discussions.

### 10) Definition of done for Turn/Tool/AI observability

Section 1 is done when your team can reliably explain a degraded user turn from evidence across all three layers.

Done criteria:

- turn/tool/AI signal sets are explicitly defined,
- correlation path between layers is clear,
- operational and analytical uses are separated,
- investigation playbook is documented,
- ownership and targets are assigned,
- one lab demonstrates end-to-end traceability.

If another engineer can run your playbook and isolate the same root cause from the same evidence, this section is complete.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Turn-Tool-AI Observability Blueprint" artifact.

### Scenario

You are responsible for a live assistant workflow where users report intermittent slowness and inconsistent tool behavior. You need to build a usable observability framework before optimizing.

### Phase A: Signal inventory and layering

1. List available observability sources in your project:
   - conversation timeline and inside info,
   - dashboard metrics,
   - run/workflow logs,
   - eval trace and report views.
2. Map each source to turn/tool/AI layer.
3. Identify missing signals for critical workflows.

Checkpoint A:

- Signal map is complete enough to follow a request across layers.

### Phase B: Critical-path request tracing

4. Select 5 representative high-value conversations.
5. For each, record:
   - turn duration,
   - tool sequence and durations,
   - AI usage and token profile,
   - failure/warning markers.
6. Build a correlation sheet showing step-by-step timing.

Checkpoint B:

- You can reconstruct end-to-end request behavior with evidence.

### Phase C: Layer-specific targets and thresholds

7. Define preliminary SLO-like targets for each layer.
8. Set alert thresholds for obvious regressions (for example latency spikes or tool failure spikes).
9. Map each threshold to an owner and response action.

Checkpoint C:

- Targets and ownership are explicit and actionable.

### Phase D: Incident simulation

10. Choose one known degraded conversation/run.
11. Apply canonical investigation path (turn -> tool -> AI -> run -> trend).
12. Identify probable root cause and confidence level.
13. Record what evidence increased or reduced confidence.

Checkpoint D:

- Investigation path leads to a specific, defensible diagnosis.

### Phase E: Dashboard and reporting alignment

14. Validate that dashboard and report views reflect the same trend direction as detailed traces.
15. Note discrepancies and explain likely reasons (scope mismatch, stale stats, dataset differences).
16. Define when to run statistics refresh and when to trust raw traces first.

Checkpoint E:

- Operational dashboards and deep traces are interpreted coherently.

### Phase F: Publish observability blueprint

17. Create `chapter-07-section-01-turn-tool-ai-observability-blueprint.md` including:

- signal inventory by layer,
- correlation model,
- target/threshold definitions,
- investigation playbook,
- ownership matrix,
- unresolved observability gaps.

18. Add go/no-go recommendation for Section 2 latency and throughput control.

Checkpoint F:

- Another engineer can investigate a degraded turn using your blueprint without ad hoc reasoning.

## Expected outputs

By the end of this lab, you should have:

- A three-layer observability model (turn/tool/AI) mapped to Gaia telemetry surfaces.
- Correlated traces for representative critical requests.
- Preliminary thresholds and ownership for key operational signals.
- A repeatable investigation path validated on at least one degraded case.
- Alignment notes between detailed traces and aggregate dashboards/reports.
- A Turn-Tool-AI Observability Blueprint artifact with go/no-go recommendation for Section 2.

Evidence that qualifies:

- Trace sheets with timestamps and layer attribution.
- Documented thresholds and response owners.
- Blueprint artifact with explicit correlation methodology.

## Failure modes

1. **Single-layer monitoring only**
   - Symptom: teams can see symptoms but cannot isolate causes.
   - Recovery: enforce turn/tool/AI layering with correlation requirements.

2. **No trace correlation discipline**
   - Symptom: logs exist but cannot be stitched across surfaces.
   - Recovery: standardize request tracing workflow and identifier usage.

3. **Tool telemetry ignored**
   - Symptom: prompt/model changes are made while tool failures persist.
   - Recovery: include tool success/latency analysis in every major investigation.

4. **AI usage signals treated as billing-only data**
   - Symptom: latency/cost issues are diagnosed late.
   - Recovery: treat AI token/timing metrics as performance signals, not only finance signals.

5. **Dashboards trusted without trace validation**
   - Symptom: stale or scoped metrics drive wrong conclusions.
   - Recovery: validate with raw timeline/run evidence before major decisions.

6. **Operational and analytical signals mixed**
   - Symptom: noisy alerts and weak strategic reporting.
   - Recovery: define separate operational alert set and analytical review set.

7. **No owner for threshold breaches**
   - Symptom: alerts fire but remediation stalls.
   - Recovery: assign owner and response playbook per threshold.

8. **Incident playbook undocumented**
   - Symptom: troubleshooting quality depends on individual experts.
   - Recovery: publish and rehearse standard investigation path.

## Completion checklist

- [ ] I mapped observability signals across turn, tool, and AI layers.
- [ ] I validated end-to-end trace correlation on representative requests.
- [ ] I defined actionable thresholds and owners for key signals.
- [ ] I applied a standard investigation path to at least one degraded case.
- [ ] I reconciled detailed trace evidence with aggregate dashboard/report views.
- [ ] I documented observability gaps and next instrumentation priorities.
- [ ] I published a Turn-Tool-AI Observability Blueprint with go/no-go recommendation for Section 2.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [View a Conversation Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Configure Model Routing and Failover](../../user-guide/settings/model-routing.md)
- [Update Project Statistics](../../user-guide/conversations/dialogs/update-statistics.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Agent Behavior Trace](../../user-guide/evals/dialogs/agent-behavior-trace.md)

---

# Latency And Throughput

## Learning objectives

By the end of this section, you should be able to:

- Define latency and throughput targets for Gaia workflows based on user impact and business criticality.
- Identify the main contributors to response-time variation across turn, tool, and pipeline layers.
- Tune system behavior using configuration, tool-flow, and data-workflow controls without degrading quality.
- Produce a latency/throughput tuning runbook that supports repeatable optimization decisions.

## Prerequisites

- You completed [Turn Tool AI Observability](#doc-ch07-observability-cost-and-performance-01-turn-tool-ai-observability).
- You can inspect timeline traces, workflow run details, and dashboard trends.
- You have at least one measurable latency complaint or throughput bottleneck in a live-like workflow.
- You can modify relevant agent configuration and workflow settings in a controlled test environment.

## In Gaia

- [Dashboard](../../user-guide/dashboard/README.md)
- [Platform Dashboard](../../user-guide/platform-dashboard/README.md)
- [Background jobs](../../user-guide/platform-dashboard/README.md#background-jobs)
- [Operational logs](../../user-guide/platform-dashboard/README.md#operational-logs)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Evals](../../user-guide/evals/README.md)
- [Tasks](../../user-guide/tasks/README.md)

Use Dashboard for project-local latency review or Platform Dashboard when you need cross-project comparison. Use Background jobs when the symptom points to delayed document indexing, backfills, webhook ingest, or scheduled-job execution. The jobs page shows the organization, team path, project, and resource name for active jobs, recent failures, and attempt history, so start there when retry pressure spans more than one project. If a queued or leased job is clearly stale, a platform admin can stop it from the Background jobs page before triggering a clean retry from the affected feature. Use Operational logs when queue health looks normal but the failing route, workflow, or worker still needs a concrete warning/error record. Then use Timeline to frame the slowdown, Workflow Run Details to localize the bottleneck, Evals to confirm quality-safe tuning, and Tasks to capture the next optimization pass.

## Concept brief

Latency is about how fast a single request completes. Throughput is about how many requests or records the system can process over time. In Gaia, these two are coupled but not identical.

Teams often optimize one while harming the other:

- reducing turn latency by over-parallelization that increases errors,
- increasing throughput by larger batches that hurt user response time,
- lowering first-response time while final completion still stalls.

This section focuses on disciplined tuning where performance improvements remain reliable and measurable.

One important mindset shift: performance is a product feature, not only an infrastructure concern. Users interpret response speed as competence and trustworthiness. That means latency and throughput work should be prioritized with the same seriousness as correctness and safety work.

### 1) Define latency from user perspective first

System timings are useful, but user-perceived latency is the anchor metric.

For conversation-driven workflows, track:

- time to first meaningful token,
- time to complete response,
- time to actionable next step.

For data workflows, track:

- run start delay,
- pipeline stage completion times,
- end-to-end freshness delay.

If performance targets are not tied to user outcomes, optimization efforts can drift into low-value technical tuning.

### 2) Throughput targets should reflect real demand envelopes

Throughput planning requires realistic load assumptions:

- average load,
- peak load,
- burst load,
- recovery behavior after spikes.

Define acceptable behavior under each envelope:

- no queue build-up beyond threshold,
- no critical timeout increase,
- no significant quality degradation.

For background work, separate web-request latency from queue latency. A healthy platform can keep conversations responsive while document indexing, scheduled jobs, and maintenance work wait briefly in a durable queue. The tuning question is whether that queue delay is within the freshness target for the workflow.

Throughput tuning without load envelopes often produces fragile systems that pass tests but fail during real spikes.

### 3) Separate first-token latency from full-turn latency

First-token latency and full completion latency often have different root causes.

Typical pattern:

- high first-token latency -> model startup/context burden.
- high completion latency -> long tool chains, large outputs, slow downstream dependencies.

Track and tune them separately. Improving one does not guarantee improvement in the other.

### 4) Use stage decomposition to localize performance hotspots

Latency/throughput debugging is faster when work is decomposed into stages.

For conversation workflows:

- intent/routing stage,
- tool execution stage,
- model synthesis stage,
- post-step stage.

For data workflows:

- source intake,
- transform stages,
- target writes,
- optional enrichment steps.

Stage-level decomposition avoids broad "system is slow" diagnoses and enables focused optimization.

### 5) Tune configuration controls before rewriting logic

Many performance gains come from configuration discipline:

- max tool-call limits,
- parallel tool execution policy,
- model selection per role,
- reasoning effort controls,
- run-level turn, token, and timeout limits in evals.

Eval runs are background jobs. When diagnosing slow or stuck evaluation work, separate model/tool latency inside individual trials from queue health, retry pressure, and whether a run still has pending trials that need to be resumed.

Apply low-risk config tuning first, measure impact, then consider deeper architectural or code-level changes.

### 6) Tool-path efficiency is often the dominant bottleneck

In many Gaia workflows, tool calls dominate latency variance.

Key tool-path controls:

- reduce unnecessary tool fan-out,
- tighten tool selection policies,
- validate argument quality to avoid retries,
- isolate slow external dependencies,
- precompute or cache stable reference lookups where appropriate.

If tool-path inefficiency remains untreated, model tuning alone delivers limited gains.

### 7) Batch size and concurrency require controlled tradeoffs

Throughput optimization often involves batch/concurrency changes, especially in data-model runs.

Tradeoff examples:

- larger batch size -> better throughput but higher memory and longer tail latency.
- more concurrency -> faster average completion but higher contention and failure risk.

Tune in small increments with guardrails. Large simultaneous changes produce ambiguous outcomes.

### 8) Define performance guardrails, not just aspirational goals

Performance guardrails protect systems from unacceptable regressions:

- max p95 or p99 latency,
- max acceptable tool timeout rate,
- max queue growth window,
- max cost increase per performance change.

Guardrails should be tied to escalation actions. A threshold without response policy is just documentation.

Guardrails should also include \"degrade mode\" decisions. For example, when throughput pressure rises, you may temporarily reduce optional enrichment or non-critical post-steps to preserve core workflow responsiveness. Planning these tradeoffs ahead of time reduces incident improvisation.

### 9) Verify performance changes against quality outcomes

Performance optimization can silently degrade quality:

- overly terse outputs,
- skipped clarification steps,
- incomplete tool usage,
- weaker factual grounding.

Always pair performance experiments with quality checks (targeted evals or run analysis). "Faster but worse" is not a valid optimization outcome.

### 10) Definition of done for Latency and Throughput

Section 2 is done when performance improvements are measurable, reproducible, and quality-safe.

Done criteria:

- latency and throughput targets are explicit,
- bottlenecks are localized by stage,
- tuning changes are incremental and evidence-based,
- guardrails and escalation triggers are defined,
- quality regression checks are integrated,
- one lab demonstrates sustained improvement.

If another engineer can apply your runbook and reproduce similar gains without quality regressions, this section is complete.

A strong signal of maturity is forecasting ability. After a few optimization cycles, teams should be able to predict the likely impact range of common tuning changes before running them.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Latency and Throughput Tuning Runbook" artifact.

### Scenario

Users report slow responses during peak periods while background runs also show increasing completion time. You need to improve performance without sacrificing quality.

### Phase A: Baseline capture

1. Select one conversation-heavy and one data-workflow-heavy path.
2. Record baseline metrics:
   - first-token and full-turn latency,
   - tool stage durations,
   - run duration and records processed,
   - error/timeout rates.
3. Record current configuration settings affecting performance.

Checkpoint A:

- Baseline is documented with enough detail for before/after comparison.

### Phase B: Bottleneck localization

4. Use timeline and run details to isolate slowest stages.
5. Classify root contributors:
   - model-level,
   - tool-level,
   - workflow/pipeline-level,
   - environment/load-level.
6. Prioritize 2-3 bottlenecks by impact and fix effort.

Checkpoint B:

- Top bottlenecks are specific and rank-ordered.

### Phase C: Low-risk tuning pass

7. Apply low-risk configuration adjustments (for example tool limits, parallel policy, model role alignment).
8. For data runs, adjust one batch/concurrency parameter at a time.
9. Rerun representative scenarios.
10. Measure delta vs baseline.

Checkpoint C:

- First tuning pass produces measurable and attributable changes.

### Phase D: Guardrail and quality validation

11. Check guardrail thresholds after tuning.
12. Run targeted quality checks:

- key workflow prompts,
- must-pass policy tasks,
- tool-correctness checks.

13. Verify no critical quality regressions were introduced.

Checkpoint D:

- Performance gains are validated as quality-safe.

### Phase E: Peak-load simulation and resilience check

14. Simulate higher load patterns (burst or sustained).
15. Evaluate queue behavior, timeout rates, and stability.
16. Document where current tuning fails to scale.
17. Define next optimization backlog items.
18. Record one explicit \"degrade mode\" policy for peak pressure conditions.

Checkpoint E:

- Throughput resilience under higher load is characterized with evidence.

### Phase F: Publish tuning runbook

18. Create `chapter-07-section-02-latency-throughput-tuning-runbook.md` including:

- baseline metrics,
- bottleneck analysis,
- tuning changes and deltas,
- guardrail compliance,
- quality validation outcomes,
- next-step backlog and owners.

19. Add go/no-go recommendation for Section 3 token and cost governance.

Checkpoint F:

- Another engineer can reproduce your tuning process and results.

## Expected outputs

By the end of this lab, you should have:

- Documented baseline latency and throughput metrics for critical paths.
- Stage-level bottleneck diagnosis with prioritized contributors.
- At least one validated tuning pass with measurable improvement.
- Guardrail and quality validation evidence after tuning.
- Peak-load behavior notes and resilience risks.
- A Latency and Throughput Tuning Runbook with go/no-go recommendation for Section 3.

Evidence that qualifies:

- Before/after metric tables tied to specific changes.
- Trace evidence for bottleneck localization.
- Quality-check outcomes confirming no critical regressions.

## Failure modes

1. **Latency targets defined only as technical averages**
   - Symptom: users still perceive slowness despite metric improvements.
   - Recovery: include first-token and actionable-response metrics.

2. **No realistic load envelope**
   - Symptom: system passes tests but degrades at peak traffic.
   - Recovery: define average/peak/burst envelopes and test each.

3. **Multiple tuning changes applied simultaneously**
   - Symptom: improvements/regressions cannot be attributed.
   - Recovery: apply one controlled change at a time and measure deltas.

4. **Tool bottlenecks ignored**
   - Symptom: model tuning yields limited gains while tool latency dominates.
   - Recovery: audit tool-path efficiency and retry patterns.

5. **Batch/concurrency over-tuning**
   - Symptom: throughput increases but error/tail-latency spikes.
   - Recovery: tune incrementally with explicit rollback points.

6. **No performance guardrails**
   - Symptom: optimizations drift into unstable operating ranges.
   - Recovery: define and enforce threshold-based guardrails.

7. **Performance changes unvalidated for quality**
   - Symptom: faster responses but lower correctness/policy compliance.
   - Recovery: run targeted quality checks for every major tuning pass.

8. **No resilience check under stress**
   - Symptom: performance collapses during bursts.
   - Recovery: include peak-load simulation in tuning completion criteria.

## Completion checklist

- [ ] I captured a baseline for latency and throughput on critical paths.
- [ ] I isolated bottlenecks by stage and ranked them by impact.
- [ ] I applied incremental tuning and measured before/after effects.
- [ ] I validated performance guardrail compliance.
- [ ] I verified no critical quality regressions after tuning.
- [ ] I documented peak-load behavior and unresolved scaling risks.
- [ ] I published a Latency and Throughput Tuning Runbook with go/no-go recommendation for Section 3.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [View a Conversation Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Start an Eval Run](../../user-guide/evals/dialogs/start-run.md)

---

# Token And Cost Governance

## Learning objectives

By the end of this section, you should be able to:

- Define cost governance policies for Gaia that align token usage with business value and risk tier.
- Identify the main token and cost drivers across prompts, tool usage, model settings, and workflow design.
- Implement budget controls, thresholds, and decision rules that prevent uncontrolled spend growth.
- Produce a cost-governance runbook that integrates with observability, evals, and release processes.

## Prerequisites

- You completed [Latency And Throughput](#doc-ch07-observability-cost-and-performance-02-latency-and-throughput) and have baseline performance metrics.
- You can inspect token/cost signals in timeline, inside info, dashboard, and run details.
- You can edit agent configuration settings that influence token usage.
- You have a defined business context for acceptable cost per conversation/workflow.

## In Gaia

- [Dashboard](../../user-guide/dashboard/README.md)
- [Platform Dashboard](../../user-guide/platform-dashboard/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Evals](../../user-guide/evals/README.md)
- [Tasks](../../user-guide/tasks/README.md)

Use Dashboard for one-project cost review or Platform Dashboard when you need instance-wide comparison. Then use Timeline to identify local cost spikes, Workflow Run Details to inspect stage-level AI usage, Evals to judge whether the spend buys enough quality, and Tasks to track optimization work instead of leaving it as informal debt.

For early infrastructure planning, pair these runtime views with the signed-in [Platform Cost Estimator](../../user-guide/platform-cost-estimator.md). It gives you a rough Azure bill-of-materials projection or an on-prem capacity plan, combining Gaia model pricing with the selected deployment environment before you ask finance or infrastructure teams for a formal quote. In Governance only, treat the workload inputs as governed automation or external-agent volume entering the governance path. In the on-prem view, Gaia sizes workload VMs and storage only; Kubernetes management or control-plane capacity and ingress or load-balancing remain customer-provided assumptions.

## Concept brief

Cost governance is not cost minimization. It is controlled spending for reliable outcomes. In Gaia, token and execution cost can grow quickly through:

- larger context windows,
- excessive tool loops,
- overpowered model choices for low-risk tasks,
- weak retry or fallback policies,
- runaway workflows.

A governance approach balances three variables:

- quality,
- latency,
- cost.

Ignoring one destabilizes the other two.

Cost governance is therefore a decision system, not a static rule list. The same token budget may be wasteful in one workflow and appropriate in another, depending on risk, user value, and failure consequences.

### 1) Define budget units that match real operations

Governance starts with units that stakeholders understand. Useful units include:

- cost per resolved conversation,
- cost per successful workflow run,
- cost per eval gate cycle,
- monthly spend by channel/agent/workflow.

If budgets are defined only as raw tokens without business context, teams struggle to prioritize optimizations.

### 2) Segment spend by role and risk class

Different assistant roles justify different spending profiles.

Example segmentation:

- orchestrator and triage roles: low-to-moderate budget.
- specialist high-risk roles: moderate-to-higher budget with stricter quality gates.
- background post-steps or auxiliary analysis: low budget unless business-critical.

This prevents uniform expensive defaults and supports purposeful model allocation.

### 3) Identify token drivers before applying controls

Common token drivers in Gaia workflows:

- long or redundant system fragments,
- large history/context inclusion,
- repeated tool-result payloads in follow-up prompts,
- verbose output defaults,
- high k-values in eval runs.

Cost controls are effective only when rooted in measured drivers. Blanket constraints without diagnosis can degrade quality unexpectedly.

### 4) Use configuration controls as first-line governance

Agent configuration offers high-leverage cost controls:

- max tool calls per turn,
- reasoning settings and verbosity,
- model/fast-model split,
- related-entity scope and retrieval limits,
- guardrails for off-topic and fallback behavior.

First optimize these controls before pursuing deeper architectural changes. They are often enough to eliminate major waste patterns.

When teams skip this layer and jump directly to large redesigns, they often introduce avoidable risk. Configuration-first governance gives faster feedback with lower blast radius.

### 5) Set threshold policies with escalation actions

Thresholds should trigger explicit actions, not just notifications.

Policy examples:

- cost-per-conversation exceeds threshold -> review top tool-consuming flows.
- token growth week-over-week exceeds threshold -> run context-size audit.
- workflow run cost anomaly -> inspect stage-level AI usage and rollback recent changes.

Each threshold needs:

- owner,
- response time expectation,
- remediation path.

### 6) Govern tool-induced cost amplification

Tools can amplify token cost indirectly:

- large payloads returned to model context,
- unnecessary chained calls,
- repeated retrieval with broad result windows.

Mitigations:

- constrain tool response payloads,
- enforce stop conditions for repeated tool loops,
- narrow retrieval scopes,
- summarize intermediate results before reinjection.

Cost governance is therefore also tool-governance work.

### 7) Integrate eval and cost signals for balanced decisions

A cheap system that fails quality gates is unacceptable. A high-quality system with uncontrolled cost is unsustainable.

Combine eval and cost views:

- compare quality improvement per incremental cost,
- flag changes with poor quality-to-cost tradeoff,
- accept cost increases only with measurable reliability/value gains.

This reframes cost conversations from pure reduction to value efficiency.

### 8) Define release-time cost checks

Include cost checks in release decisions:

- no critical cost regression beyond agreed threshold,
- no unexplained token spikes in key workflows,
- no hidden cost shift from interaction to background processes.

Release-time checks prevent post-launch surprises and force early visibility into expensive changes.

Use explicit \"cost acceptance\" language when a release intentionally increases spend. That keeps tradeoffs transparent and prevents silent budget drift masked as technical necessity.

### 9) Create cost review cadence and accountability

Governance requires recurring review, not one-time setup.

Suggested cadence:

- weekly operational cost review for anomalies,
- bi-weekly optimization review for top drivers,
- monthly strategic review of budget allocations by role/channel.

Assign accountability for:

- measurement integrity,
- anomaly triage,
- optimization backlog,
- exception approvals.

### 10) Definition of done for Token and Cost governance

Section 3 is done when spending is predictable, explainable, and controlled without degrading required quality.

Done criteria:

- budget units and thresholds are explicit,
- major cost drivers are identified and prioritized,
- configuration/tool controls are applied with evidence,
- cost checks are integrated into release process,
- governance cadence and owners are defined,
- one lab demonstrates controlled cost optimization.

If another engineer can apply your governance playbook and maintain cost discipline across changes, Section 3 is complete.

This includes handling growth periods. A mature governance model scales with user adoption without forcing emergency cost controls every quarter.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Token and Cost Governance Runbook" artifact.

### Scenario

Your team has improved assistant capability, but operating costs are rising and becoming hard to predict. You need a governance model that controls spend while preserving quality and responsiveness.

### Phase A: Cost baseline and segmentation

1. Select representative period (for example last 7-14 days).
2. Capture baseline metrics:
   - cost per conversation/workflow,
   - token averages and distribution,
   - high-cost channels/agents/workflows.
3. Segment spend by role and workflow class.

Checkpoint A:

- Baseline and segmentation reveal where spend is concentrated.

### Phase B: Driver analysis

4. Inspect high-cost traces/runs.
5. Identify top cost drivers:
   - context size,
   - tool loops,
   - model selection,
   - retry patterns,
   - evaluation settings.
6. Rank drivers by potential savings vs risk.

Checkpoint B:

- Top cost drivers are evidence-based and prioritized.

### Phase C: Control design

7. Define budget thresholds and anomaly triggers.
8. Configure first-line controls:
   - max tool calls,
   - retrieval/result limits,
   - model-role alignment,
   - verbosity/reasoning settings where appropriate.
9. Document expected impact per control.

Checkpoint C:

- Controls are explicit, scoped, and measurable.

### Phase D: Controlled optimization pass

10. Apply one or two highest-leverage controls.
11. Run representative scenarios and compare cost deltas.
12. Check latency and quality side effects.
13. Roll back any change that violates must-pass quality conditions.

Checkpoint D:

- Optimization produces measured savings without critical regressions.

### Phase E: Governance integration

14. Add cost checks to release review template.
15. Define weekly anomaly and monthly allocation review cadence.
16. Assign owners for threshold breach response and exception approval.
17. Define escalation path for rapid cost spikes.

Checkpoint E:

- Governance process is operational, not ad hoc.

### Phase F: Publish governance runbook

18. Create `chapter-07-section-03-token-cost-governance-runbook.md` including:

- baseline and segmentation,
- cost drivers,
- control policies,
- threshold/escalation matrix,
- optimization outcomes,
- review cadence and ownership.

19. Add go/no-go recommendation for Section 4 performance troubleshooting.

Checkpoint F:

- Another engineer can monitor and control spend using your runbook.

## Expected outputs

By the end of this lab, you should have:

- A segmented baseline view of token and cost behavior.
- Prioritized list of measurable cost drivers.
- Defined thresholds and first-line configuration/tool controls.
- At least one validated cost optimization change with quality checks.
- Integrated release and review governance for cost discipline.
- A Token and Cost Governance Runbook with go/no-go recommendation for Section 4.

Evidence that qualifies:

- Baseline and post-change cost tables.
- Trace evidence linking controls to observed savings.
- Governance artifact with owners and escalation logic.

## Failure modes

1. **Cost viewed only as finance metric**
   - Symptom: engineering cannot map spend changes to technical causes.
   - Recovery: define cost units tied to workflows and user outcomes.

2. **No spend segmentation**
   - Symptom: high-cost hotspots remain hidden.
   - Recovery: segment by role, channel, and workflow class.

3. **Blind cost-cutting**
   - Symptom: spend drops but quality/reliability collapses.
   - Recovery: require quality guardrails for every optimization step.

4. **Token drivers not diagnosed**
   - Symptom: controls applied broadly with weak impact.
   - Recovery: analyze trace-level cost contributors before control design.

5. **Tool amplification ignored**
   - Symptom: repeated tool loops keep costs high despite model tuning.
   - Recovery: constrain tool response sizes and loop behavior.

6. **No threshold-to-action mapping**
   - Symptom: anomalies detected but no one responds consistently.
   - Recovery: define owner, SLA, and remediation path per threshold.

7. **Cost checks absent from release decisions**
   - Symptom: expensive regressions discovered after rollout.
   - Recovery: add mandatory cost delta checks to release gating.

8. **No recurring governance cadence**
   - Symptom: controls drift and spend unpredictability returns.
   - Recovery: schedule regular anomaly and budget-allocation reviews.

## Completion checklist

- [ ] I established baseline cost and token metrics by workflow segment.
- [ ] I identified and prioritized top cost drivers with trace evidence.
- [ ] I defined threshold-based cost controls and escalation actions.
- [ ] I executed and validated at least one cost optimization safely.
- [ ] I integrated cost checks into release and review processes.
- [ ] I assigned governance ownership and review cadence.
- [ ] I published a Token and Cost Governance Runbook with go/no-go recommendation for Section 4.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [View a Conversation Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Start an Eval Run](../../user-guide/evals/dialogs/start-run.md)
- [Reports](../../user-guide/evals/reports.md)

---

# Performance Troubleshooting

## Learning objectives

By the end of this section, you should be able to:

- Execute a repeatable troubleshooting workflow for latency, errors, and cost regressions in Gaia systems.
- Isolate root causes across conversation, tool, workflow, and model layers with high diagnostic confidence.
- Apply minimal-risk fixes and verify recovery using before/after evidence.
- Produce a troubleshooting playbook that closes Chapter 7 and supports ongoing operational reliability.

## Prerequisites

- You completed Sections 1-3 of Chapter 7 and have observability, performance, and cost baselines.
- You can inspect timeline traces, run details, dashboard metrics, and eval reports.
- You can adjust agent configs, tool policies, or workflow settings in a controlled environment.
- You have at least one recent degraded incident or reproducible performance anomaly.

## In Gaia

- [Dashboard](../../user-guide/dashboard/README.md)
- [Platform Dashboard](../../user-guide/platform-dashboard/README.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Evals](../../user-guide/evals/README.md)
- [Tasks](../../user-guide/tasks/README.md)

Use Dashboard to confirm a project-local symptom or Platform Dashboard when you first need to establish whether the regression is isolated or instance-wide. Then use Timeline to inspect the local trace, Workflow Run Details to isolate the first upstream break, Audit Trail when recent changes may explain the regression, Evals to verify quality-safe recovery, and Tasks to keep remediation and hardening visible after the incident closes.

## Concept brief

Troubleshooting is where observability turns into operational outcomes. Good troubleshooting is not "try fixes until it feels better." It is a disciplined loop:

1. detect,
2. scope,
3. isolate,
4. fix minimally,
5. verify recovery,
6. harden against recurrence.

In Gaia systems, performance problems are often multi-causal. A slow response may involve model behavior, tool payload size, workflow queueing, and stale statistics all at once. The goal is to identify the first meaningful upstream break, not every downstream symptom.

Troubleshooting maturity is visible in decision speed under uncertainty. Teams with a structured process can move from symptom to likely cause quickly, communicate confidence transparently, and avoid destabilizing \"big bang\" fixes.

### 1) Start from reproducible symptoms

Troubleshooting quality depends on reproduction quality.

Define incident symptom with specifics:

- where it appears (channel/workflow),
- when it appears (time window/load condition),
- what degrades (latency, failure rate, cost spike),
- what baseline it violates.

Without reproducible framing, teams chase intermittent noise and generate conflicting fixes.

### 2) Scope blast radius before deep debugging

Not every anomaly is systemic.

Scope questions:

- one agent or multiple agents?
- one channel or all channels?
- one workflow or all workflows?
- one user segment or all users?

Blast-radius mapping helps prioritize severity and avoid overbroad remediation.

### 3) Localize the first upstream break

The first upstream break is the earliest step where behavior deviates materially from baseline.

Use layered analysis:

- conversation timeline for turn-level timing,
- tool trace for execution defects,
- workflow run details for stage-level bottlenecks,
- dashboard/reports for trend context.

Fixing downstream symptoms without upstream localization often causes recurrence.

### 4) Distinguish transient anomalies from structural regressions

Some degradations are one-off spikes; others indicate design drift.

Transient signals:

- isolated external timeout,
- short-lived load burst,
- temporary infrastructure contention.

Structural signals:

- persistent p95 latency increase,
- repeated failure family growth,
- sustained cost-per-turn shift.

Response strategy differs by class. Overreacting to transient anomalies can create unnecessary complexity.

A practical method is to require recurrence evidence before labeling an issue structural unless the initial impact is already critical. This balances responsiveness and overfitting risk.

### 5) Apply minimal-change remediation first

Large changes during incidents increase uncertainty.

Minimal-change order:

- adjust one configuration guardrail,
- disable or narrow one problematic tool path,
- reduce one high-cost context source,
- tune one batch/concurrency parameter.

Measure impact after each step. This preserves causal clarity and reduces rollback complexity.

### 6) Verify recovery with multi-metric evidence

Recovery should be proven across:

- latency,
- error/failure rate,
- cost behavior,
- quality outcomes.

A fix that improves latency but worsens correctness is not a full recovery. Require multi-metric confirmation before closing incident status.

### 7) Include quality and policy checks in troubleshooting closure

Performance incidents can create hidden quality drift:

- skipped clarification,
- partial tool outputs,
- increased unsafe fallback responses.

Run targeted eval/trace checks after remediation. Incident closure should include both performance recovery and quality safety confirmation.

### 8) Capture incident learnings as control updates

Every incident should produce at least one lasting control improvement:

- new alert threshold,
- new regression gate task,
- updated troubleshooting checklist,
- tightened configuration policy.

If incidents close without control updates, the same failure class usually returns.

Control updates should be proportional: one focused improvement per incident is often better than broad checklist expansion that teams stop using.

### 9) Define escalation tiers and communication protocols

Troubleshooting includes communication discipline.

Define escalation tiers by impact:

- low: monitor and schedule fix,
- medium: same-day remediation,
- high: immediate response and release hold,
- critical: rollback/kill-switch and cross-team incident channel.

Communication expectations should include owner, status cadence, and closure criteria.

### 10) Definition of done for Performance Troubleshooting and Chapter 7

Section 4 and Chapter 7 are done when troubleshooting is systematic, evidence-based, and repeatable under pressure.

Done criteria:

- reproducible symptom framing is standard,
- first upstream break localization is practiced,
- fixes are incremental and causally validated,
- recovery uses latency/error/cost/quality evidence,
- incident learnings feed back into controls,
- escalation and communication playbooks are operational,
- one lab demonstrates end-to-end troubleshooting lifecycle.

At this point, Chapter 7 outputs provide a strong operational base for Chapter 8 security and governance controls.

The practical test is reproducibility under rotation. If an on-call engineer unfamiliar with the original incident can follow the playbook and achieve comparable diagnostic quality, your troubleshooting system is robust.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Performance Troubleshooting Playbook" artifact.

### Scenario

A high-value assistant workflow has degraded: users report slower responses, run durations increase, and cost-per-turn rises. You need to diagnose and recover quickly while maintaining quality and safety.

### Phase A: Incident framing and scoping

1. Define incident symptom with explicit baseline violation.
2. Capture timeframe and affected surfaces (channels, agents, workflows).
3. Assess severity and blast radius.
4. Open incident log artifact for structured evidence capture.

Checkpoint A:

- Incident is reproducible and impact-scoped.

### Phase B: Layered evidence collection

5. Collect turn-level traces from timeline and inside info.
6. Collect tool-level and run-level evidence from traces and workflow run details.
7. Capture aggregate trends from dashboard and reports.
8. Record contradictory signals explicitly.

Checkpoint B:

- Evidence is sufficient to compare hypotheses across layers.

### Phase C: Root-cause isolation

9. Build top 2-3 hypotheses.
10. Validate each with targeted evidence checks.
11. Identify first upstream break with confidence level.
12. Document likely contributing factors.

Checkpoint C:

- Root cause and confidence are explicit, not speculative.

### Phase D: Minimal remediation and controlled rerun

13. Apply minimal-risk fix targeting upstream break.
14. Rerun representative conversation/workflow paths.
15. Compare against incident baseline:
    - latency,
    - failure rate,
    - cost,
    - quality.
16. If unresolved, iterate with one additional controlled change.
17. Document why rejected hypotheses were rejected to improve future triage speed.

Checkpoint D:

- At least one remediation cycle yields measurable outcome change.
- Rejected hypotheses are captured for future troubleshooting efficiency.

### Phase E: Recovery validation and hardening

17. Validate post-fix quality using targeted eval/trial checks.
18. Confirm policy/boundary behavior remains acceptable.
19. Add one preventive control update (alert, gate task, config policy, or checklist change).
20. Define follow-up monitoring window and owner.

Checkpoint E:

- Recovery is validated and recurrence prevention is in place.

### Phase F: Publish troubleshooting playbook

21. Create `chapter-07-section-04-performance-troubleshooting-playbook.md` including:

- incident framing,
- evidence timeline,
- root-cause analysis,
- remediation actions and deltas,
- recovery validation,
- control updates,
- escalation communication log.

22. Add Chapter 7 readiness recommendation and carry-over priorities for Chapter 8.

Checkpoint F:

- Another engineer can rerun the same troubleshooting workflow and reach consistent conclusions.

## Expected outputs

By the end of this lab, you should have:

- A structured incident record with reproducible symptom and scoped blast radius.
- Layered evidence connecting turn/tool/workflow/aggregate signals.
- Root-cause statement with confidence and contributing factors.
- At least one validated minimal remediation cycle.
- Recovery confirmation across latency, failure, cost, and quality dimensions.
- A Performance Troubleshooting Playbook with Chapter 7 readiness recommendation.

Evidence that qualifies:

- Before/after metrics tied to specific remediation steps.
- Trace-backed root-cause documentation.
- Preventive control updates linked to incident findings.

## Failure modes

1. **Troubleshooting without reproducible symptom definition**
   - Symptom: conflicting diagnoses and repeated failed fixes.
   - Recovery: require explicit baseline violation framing before remediation.

2. **Blast radius unknown**
   - Symptom: overreaction or underreaction to incident impact.
   - Recovery: scope affected surfaces early and update as evidence improves.

3. **Fixing downstream symptoms only**
   - Symptom: temporary relief followed by recurrence.
   - Recovery: isolate and target first upstream break.

4. **Large multi-change incident patches**
   - Symptom: no causal clarity and harder rollback.
   - Recovery: apply one controlled remediation step at a time.

5. **Recovery judged on one metric only**
   - Symptom: latency improves while quality or cost worsens.
   - Recovery: require multi-metric recovery validation.

6. **Quality/policy checks skipped after performance fix**
   - Symptom: hidden behavior regressions ship unnoticed.
   - Recovery: include targeted eval and boundary checks in closure criteria.

7. **No control update after incident**
   - Symptom: same failure class recurs.
   - Recovery: add at least one preventive alert/gate/policy improvement.

8. **Weak escalation communication**
   - Symptom: delayed decisions and duplicate effort.
   - Recovery: define severity tiers, owner cadence, and closure communication protocol.

## Completion checklist

- [ ] I documented incident symptom, baseline violation, and blast radius.
- [ ] I collected layered evidence across turn, tool, workflow, and aggregate views.
- [ ] I identified first upstream break with explicit confidence level.
- [ ] I applied minimal remediation and measured before/after impact.
- [ ] I validated recovery across latency, error, cost, and quality dimensions.
- [ ] I added preventive controls and follow-up monitoring ownership.
- [ ] I published a Performance Troubleshooting Playbook with Chapter 7 readiness recommendation.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [View a Conversation Timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Reports](../../user-guide/evals/reports.md)
- [Agent Behavior Trace](../../user-guide/evals/dialogs/agent-behavior-trace.md)

---

# Chapter 8: Security and Governance

Status: current

Roles, controls, and safe operation

## Sections

- [Project Roles And Access](#doc-ch08-security-and-governance-01-project-roles-and-access)
- [Data Safety And Boundaries](#doc-ch08-security-and-governance-02-data-safety-and-boundaries)
- [Safe Tool And Prompt Practices](#doc-ch08-security-and-governance-03-safe-tool-and-prompt-practices)
- [Audit And Compliance Readiness](#doc-ch08-security-and-governance-04-audit-and-compliance-readiness)
- [Governance Operating Model](#doc-ch08-security-and-governance-05-governance-operating-model)

## Alignment with User Guide Governance Surfaces

- [Governance workspace](../../user-guide/governance/README.md)
- [Governance registry](../../user-guide/governance/registry.md)
- [Governance classifications](../../user-guide/governance/classifications.md)
- [Governance regulatory updates](../../user-guide/governance/regulatory-updates.md)
- [Governance obligations](../../user-guide/governance/obligations.md)
- [Governance discovery](../../user-guide/governance/discovery.md)
- [Governance explainability](../../user-guide/governance/explainability.md)
- [Settings](../../user-guide/settings/README.md)
- [Project users](../../user-guide/settings/project-users.md)
- [Project roles](../../user-guide/settings/project-roles.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Platform Dashboard](../../user-guide/platform-dashboard/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Tutorials](../../user-guide/tutorials/README.md)

Use this chapter to move from access and safety basics into a real governance operating model. Start with role and boundary decisions, then define package scope and ownership in Governance, and finally use audit and readiness workflows to prove that the operating model is still current. The Governance workspace is the main project-level oversight surface for obligation, classification, regulatory-change triage, discovery onboarding, explainability review, reusable framework-package management, and explicit downstream package adoption. Platform Dashboard now adds the cross-project governance posture and backlog view for platform admins who need to see where governance work is accumulating, understand how much imported governance-package dependency exposure exists across projects, identify stale imported governance-package reuse across projects, understand whether the current tab covers the full accessible estate or only a filtered subset, start on the highest-signal record or queue item, and compare recent governance movement through a fixed 30-day throughput lens before drilling into a specific project. Settings remains the authority surface for role changes, Audit Trail remains the evidence surface for change history, Dashboard remains the project trend surface, Platform Dashboard remains the instance rollup surface, and Tasks remains the execution surface for remediation that must be tracked to closure.

If governance is completely new to you, read [Governance Foundations](../../user-guide/governance/foundations.md) before Section 5. Then return here for the structured chapter path.

When you want the guided companion walkthroughs for this chapter, open the tutorials workspace at `/platform/support/tutorials`.

## Fast path inside Gaia

Use this sequence when you want to apply the chapter ideas in the product instead of reading them as theory only.

1. Start with [Governance Foundations](../../user-guide/governance/foundations.md) and [Governance workspace](../../user-guide/governance/README.md) so the core terms map to real Gaia surfaces.
2. Use [Governance registry](../../user-guide/governance/registry.md) and [Governance classifications](../../user-guide/governance/classifications.md) to make the applicable standards, overlays, and use-case boundary explicit.
3. Use [Governance obligations](../../user-guide/governance/obligations.md) and [Governance controls](../../user-guide/governance/controls.md) to capture requirements and safeguards on real records.
4. Use [Governance explainability](../../user-guide/governance/explainability.md), [Governance regulatory updates](../../user-guide/governance/regulatory-updates.md), and the rest of the Governance workspace to connect review evidence and follow-up governance work.
5. Use [Tasks](../../user-guide/tasks/README.md), [Delivery Management](../../user-guide/delivery/README.md), [Audit Trail](../../user-guide/audit/README.md), [Dashboard](../../user-guide/dashboard/README.md), and [Platform Dashboard](../../user-guide/platform-dashboard/README.md) to turn governance findings into owned execution and operating review at both project and instance scope.

If you reach a handbook step that is not yet fully supported in the current product, do not smooth over the gap conceptually. Record it as explicit delivery work and continue only when the operating path is honest again.

Chapter 8 should prepare you for Chapter 9 release-gate work and Chapter 11 capstone delivery. If your team cannot explain which packages apply, who owns the obligations and controls, and where the evidence comes from, you are not ready for either.

Use **Operations: Review, Audit, and Rebalance** in [Tutorials](../../user-guide/tutorials/README.md) when you want a guided pass through the evidence-to-remediation loop described in this chapter.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Project Roles And Access

## Learning objectives

By the end of this section, you should be able to:

- Design a role and access model in Gaia that balances delivery speed with least-privilege control.
- Map project responsibilities to permission boundaries across agents, data model, evals, and settings.
- Detect and remediate over-privileged or under-privileged access patterns before they become incidents.
- Produce an access governance artifact that supports security, auditability, and operational continuity.

## Prerequisites

- You completed Chapter 7 and have baseline operational practices for observability and troubleshooting.
- You can access **Settings → Project users** and **Settings → Project roles** as a project admin.
- You have a list of active contributors (engineering, product, QA, operations, external reviewers).
- You can identify which workflows are sensitive (for example production channels, tool settings, data updates, or release gates).

## In Gaia

- [Settings](../../user-guide/settings/README.md)
- [Project users](../../user-guide/settings/project-users.md)
- [Project roles](../../user-guide/settings/project-roles.md)
- [Audit Trail](../../user-guide/audit/README.md)

Use Settings as the authority surface for project-level administration, Project users for roster assignment and inherited-versus-explicit access checks, Project roles for custom-role design, and Audit Trail when you need evidence that access changes followed the intended governance path.

For machine-to-machine access, use **Service accounts** instead of personal user API keys so external systems run with a project-scoped role rather than a human operator’s full authority.

## Concept brief

Roles and access are the first control plane of security and governance. In Gaia, role design determines who can change behavior, who can view sensitive context, and who can approve or publish risk-bearing updates.

Poor access design typically fails in one of two ways:

- **Over-permissive:** too many users can modify critical settings, increasing accidental or malicious risk.
- **Under-permissive:** key operators cannot execute required tasks quickly, increasing shadow work and process bypass.

A strong access model is intentional, auditable, and adaptable as team responsibilities evolve.

### 1) Treat access as part of architecture, not admin overhead

Access control is often deferred until late-stage operations. This is a mistake.

In practice, access design should be integrated with architecture decisions:

- who owns orchestration configs,
- who can publish channels,
- who can change tool definitions,
- who can approve release gates.

If architecture and access are designed independently, ownership boundaries conflict and incident response becomes slow.

### 2) Start from responsibility mapping

Role design should begin with responsibility, not permission checklists.

For each contributor type, define:

- what decisions they own,
- what actions they must perform,
- what surfaces they need read access to,
- what surfaces they should never modify.

Example role families:

- builders (configure and ship),
- reviewers (analyze and approve),
- operators (monitor and remediate),
- auditors (observe evidence, no changes).

Responsibility-first mapping reduces both privilege creep and operational friction.

### 3) Use least privilege with delivery realism

Least privilege is necessary, but rigid minimalism can block real work.

Balanced policy:

- default to view permissions,
- grant manage permissions only where justified,
- use custom roles for recurring responsibility patterns,
- avoid one-off privilege grants that remain unmanaged.

The goal is controlled capability, not blanket restriction. Security that blocks essential operations is often bypassed informally, creating larger risks.

### 4) Separate change authority from review authority

A common governance weakness is allowing the same role to both change and unilaterally approve critical modifications.

For high-impact areas, separate:

- implementers (make changes),
- reviewers/approvers (validate changes),
- release owners (authorize rollout).

This separation improves control quality and reduces confirmation bias in risky updates.

### 5) Protect high-impact surfaces explicitly

Not all permissions have equal risk. High-impact surfaces deserve stricter assignment:

- agent configuration defaults and active version switching,
- channel publishing and auth settings,
- document-folder access when shared files should stay separate from general conversation review,
- tool registry modifications,
- guardrail registry modifications and governance-policy linkage,
- project settings and role management,
- danger-zone operations such as project export, import, move, and delete,
- audit trail access in regulated contexts.

Define explicit policies for these surfaces, including required role level and escalation path.

### 6) Design temporary access and external access patterns

Security incidents often come from temporary access that never expires.

Governance practices:

- time-box elevated access,
- use dedicated reviewer roles for external stakeholders,
- remove temporary grants during release closure,
- track privileged access changes in audit reviews.

Temporary access should be procedural, not ad hoc.

### 7) Build access onboarding and offboarding workflows

Role models fail when lifecycle operations are unmanaged.

Minimum lifecycle controls:

- onboarding checklist by role,
- periodic access recertification,
- immediate offboarding deprovisioning,
- transfer-of-ownership process when roles change,
- explicit invitation resend process when an invited or provisioned user has not completed first sign-in,
- duplicate-account merge process when the same person appears with email capitalization variants,
- hand-off process when channel history and folder responsibility move to another person.

Use Gaia's **Identity** badges to separate onboarding state from lifecycle state: invited app users, anonymous users, provisioned platform users, and signed-in users need different follow-up actions. Use **Lifecycle** badges for active, suspended, and archived account state. Sending or resending an invitation email is a communication action only; it does not grant additional memberships or collapse app-user and project-user access.

When duplicate app users have the same email after lowercasing, project admins can merge the duplicate in **App users**. When responsibility moves between different people, project admins can use hand-off instead. Treat the source account as the account losing direct project or app-channel access, confirm the target should receive text and Personal Assistant channel history plus related folder access, and keep historical audit attribution intact so the audit trail remains evidence of what happened at the time.

Without lifecycle controls, stale privileges accumulate and governance reliability degrades over time.

### 8) Use audit trail as access-governance evidence

Access governance must be evidence-backed. The audit trail provides the change history needed for review.

Use it to verify:

- who changed critical settings,
- when role/permission changes occurred,
- whether approvals and releases align with policy,
- whether unusual change bursts occurred before incidents.

Audit evidence helps distinguish process gaps from isolated human error.

### 9) Define access incident response criteria

Access incidents are not limited to breaches. They also include governance anomalies:

- unauthorized changes,
- unexplained privilege escalation,
- repeated failed access attempts,
- delayed response due to missing permissions.

Define triage levels and response expectations so access incidents are handled consistently and quickly.

### 10) Definition of done for Project Roles and Access

Section 1 is done when access rights are aligned with responsibilities and verifiable by evidence.

Done criteria:

- role families and responsibilities are documented,
- least-privilege assignments are applied with rationale,
- high-impact surface policies are explicit,
- temporary/external access controls are defined,
- lifecycle and incident response workflows exist,
- one lab validates access governance in practice.

If another admin can apply your role model and maintain secure operations without interpretation gaps, Section 1 is complete.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Project Access Governance Matrix" artifact.

### Scenario

Your project is growing and more contributors need access. You must redesign roles and permissions to preserve security while keeping delivery velocity intact.

### Phase A: Responsibility and surface inventory

1. List active contributor groups and their responsibilities.
2. List platform surfaces they interact with (Agents, Channels, Data Model, Evals, Settings, Audit).
3. Mark each surface as:
   - view required,
   - manage required,
   - restricted.
4. Identify high-impact surfaces.

Checkpoint A:

- Responsibility-to-surface mapping is explicit and complete.

### Phase B: Role model design

5. Review built-in roles and identify gaps.
6. Create custom roles for recurring responsibility patterns.
7. Define permissions with least-privilege rationale.
8. Add separation-of-duties constraints for critical changes.

Checkpoint B:

- Role set supports operations without broad admin overuse.

### Phase C: Assignment and validation

9. Assign roles to representative users in **Project users**.
10. Validate each role using task-based checks:

- expected actions succeed,
- prohibited actions are blocked,
- review-only roles remain read-only.

11. Capture mismatch findings and adjust roles.

Checkpoint C:

- Assigned roles behave as intended in real workflows.

### Phase D: Temporary and external access controls

12. Define temporary elevation policy (owner, reason, duration).
13. Create reviewer/auditor role for external or compliance users.
14. Test time-boxed access workflow end-to-end.
15. Define revocation checklist.

Checkpoint D:

- Temporary/external access is controlled and reversible.

### Phase E: Audit and incident readiness

16. Use **Audit Trail** to confirm visibility into role and settings changes.
17. Define access incident categories and severity levels.
18. Define response owner and SLA per incident tier.
19. Create periodic access recertification cadence.

Checkpoint E:

- Governance is evidence-backed and incident-ready.

### Phase F: Publish governance matrix

20. Create `chapter-08-section-01-project-access-governance-matrix.md` including:

- role definitions,
- permission matrix,
- high-impact surface controls,
- separation-of-duties rules,
- temporary/external access policy,
- audit and incident response workflow.

21. Add go/no-go recommendation for Section 2 data safety and boundaries.

Checkpoint F:

- Another admin can apply and audit the access model using your artifact.

## Expected outputs

By the end of this lab, you should have:

- A documented role architecture aligned with contributor responsibilities.
- A permission matrix with least-privilege rationale.
- Explicit controls for high-impact settings and release-sensitive surfaces.
- Temporary/external access procedures with revocation discipline.
- Access incident triage and response workflow.
- A Project Access Governance Matrix artifact with go/no-go recommendation for Section 2.

Evidence that qualifies:

- Roles configured and assigned in Gaia settings.
- Task-based access validation results.
- Audit-trail-backed governance evidence.

## Failure modes

1. **Permission-first design without responsibility mapping**
   - Symptom: inconsistent access and frequent escalation requests.
   - Recovery: redesign roles from responsibility model.

2. **Overuse of admin role**
   - Symptom: too many users can change high-impact settings.
   - Recovery: introduce custom roles and restrict admin privileges.

3. **No separation of duties for critical changes**
   - Symptom: risky changes are self-approved.
   - Recovery: split implementer and approver responsibilities.

4. **Stale temporary privileges**
   - Symptom: former temporary access remains indefinitely.
   - Recovery: enforce time-boxing and recertification checks.

5. **External reviewer access too broad**
   - Symptom: auditors can modify operational settings.
   - Recovery: create strict read-only reviewer roles.

6. **Onboarding/offboarding not standardized**
   - Symptom: missing permissions or lingering access after role changes.
   - Recovery: establish lifecycle checklists with owner accountability.

7. **Audit trail not used for governance checks**
   - Symptom: access anomalies discovered late.
   - Recovery: include audit review in recurring governance cadence.

8. **No access incident escalation policy**
   - Symptom: uncertain response during unauthorized changes.
   - Recovery: define severity tiers, owners, and response SLAs.

## Completion checklist

- [ ] I mapped contributor responsibilities to required platform surfaces.
- [ ] I defined and applied least-privilege project roles.
- [ ] I enforced stronger controls for high-impact settings and changes.
- [ ] I validated role behavior with task-based permission checks.
- [ ] I documented temporary and external access procedures.
- [ ] I established access incident response and recertification cadence.
- [ ] I published a Project Access Governance Matrix with go/no-go recommendation for Section 2.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Settings](../../user-guide/settings/README.md)
- [Project Users](../../user-guide/settings/project-users.md)
- [Project Roles](../../user-guide/settings/project-roles.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Channels](../../user-guide/conversations/channels/README.md)

---

# Data Safety And Boundaries

## Learning objectives

By the end of this section, you should be able to:

- Define data safety boundaries in Gaia across input, storage, processing, and output paths.
- Classify project data by sensitivity and enforce handling controls proportionate to risk.
- Design boundary policies for channels, tools, workflows, and model interactions.
- Produce a data safety artifact that guides secure operation and incident containment.

## Prerequisites

- You completed [Project Roles And Access](#doc-ch08-security-and-governance-01-project-roles-and-access).
- You can access channels, agent configurations, data model storage/workflows, and settings.
- You have identified sensitive datasets used by your assistants or pipelines.
- You can inspect at least one end-to-end data path from user input to model/tool output.

## In Gaia

- [Settings](../../user-guide/settings/README.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Audit Trail](../../user-guide/audit/README.md)

Use Channels and Conversations to reason about input and output exposure, Data Model and Workflow Run Details to inspect processing and storage boundaries, Settings to confirm the governing controls, and Audit Trail to verify that sensitive-surface changes remain reviewable.

## Concept brief

Data safety is the practice of controlling what data enters the system, where it flows, how it is transformed, and where it is allowed to exit. In Gaia, safety boundaries are cross-cutting: channels, agent instructions, tool behavior, storage paths, and evaluation traces all influence risk.

Most serious data incidents are boundary failures, not single-component failures:

- sensitive data ingested through a permissive channel,
- oversized tool payload exposing unnecessary fields,
- weak prompt boundaries causing over-disclosure,
- storage artifacts retained longer than intended,
- audit or review exports shared without sanitization.

This section formalizes boundary design so data safety becomes operationally consistent.

A useful way to think about boundaries is \"assume drift.\" Over time, new tools, channels, prompts, and integrations will be added. Boundary design must therefore be resilient to change, not only correct for today's configuration.

### 1) Classify data before designing controls

Safety controls must match data sensitivity.

Define practical classes such as:

- public/reference,
- internal operational,
- confidential business,
- regulated/restricted.

For each class, define allowed handling:

- who can view,
- who can edit,
- where it can be stored,
- whether it can be exported,
- retention expectations.

Without classification, teams default to inconsistent, case-by-case decisions.

### 2) Map end-to-end data flow boundaries

Boundary design requires flow visibility.

For key workflows, map:

- entry points (text channels, webhook, uploads),
- processing layers (model, tools, pipelines),
- storage surfaces (entities, storage blobs, logs),
- output surfaces (responses, exports, dashboards, eval artifacts).

This map reveals unsafe crossings where sensitive data moves into broader visibility zones.

### 3) Apply minimization at every boundary crossing

Minimization is one of the highest-impact safety practices.

At each boundary, ask:

- what minimum fields are needed,
- can identifiers be masked or summarized,
- can payload size be constrained,
- can optional sensitive context be excluded.

Minimization reduces blast radius even when downstream controls fail.

### 4) Separate operational data from sensitive evidence where possible

Not all data needs identical handling. Separate storage and access paths for:

- operational records needed for everyday workflows,
- sensitive evidence used for compliance/security review.

This allows stricter permissions and review processes for high-risk data without slowing routine operations.

### 5) Define channel-specific safety boundaries

Different channels introduce different exposure risks.

Examples:

- public text channel may require stronger input/output filtering.
- authenticated personal assistant may allow richer contextual responses.
- webhook/integration channels need strict authentication and payload validation.

Channel policies should explicitly define allowed data classes and prohibited disclosures.

For multichannel projects, document cross-channel equivalence rules as well. If one channel allows richer context by design (for example authenticated assistant), ensure users and operators understand why another channel is intentionally more restrictive.

For Text and Personal Assistant channels exposed to broad user input, enable channel-level PII redaction when users may enter personal data. Gaia redacts detected personal data from user message text before transcript storage, search indexing, model input, and tool input. Treat this as a message-text control only: attachments, imported documents, generated artifacts, historical transcripts, and downstream system records still need their own data-handling controls.

### 6) Enforce tool and prompt boundary contracts

Tool and prompt controls are boundary enforcement points.

Safe boundary practices:

- tools return only required fields,
- prompts forbid disclosure of disallowed classes,
- out-of-scope requests trigger safe refusal/escalation,
- retrieval limits prevent broad sensitive data exposure.

If tool and prompt contracts conflict, data leakage risk increases significantly.

### 7) Treat logs and traces as sensitive by default

Observability and eval traces are valuable but often overlooked as data-risk surfaces.

Protect:

- timeline traces,
- tool argument/result logs,
- eval transcripts and trial details,
- exported reports and artifacts.

Define who can access these surfaces and when redaction/sanitization is required.

### 8) Define retention and deletion policies

Safety is not only access control; it is lifecycle control.

For each data class, define:

- retention duration,
- archival conditions,
- deletion triggers,
- evidence-preservation exceptions.

Retention ambiguity leads to data accumulation and compliance risk.

Retention policy should include practical execution triggers, not only target durations. For example: who runs periodic review, how deletions are verified, and how legal or investigation holds are recorded when normal deletion schedules must pause.

### 9) Build boundary incident response playbooks

Boundary failures require fast, structured response:

- detect and confirm exposure,
- contain data flow,
- revoke or restrict access paths,
- assess impacted records/users,
- document and remediate root cause.

Playbooks should include communication protocols and evidence handling requirements.

### 10) Definition of done for Data Safety and Boundaries

Section 2 is done when data movement and disclosure are governed by explicit boundary policies, not informal judgment.

Done criteria:

- data classes and handling rules are defined,
- end-to-end boundary map exists,
- minimization and channel-specific controls are applied,
- tool/prompt/log surfaces are governed,
- retention and incident playbooks are documented,
- one lab validates boundary control behavior.

If another engineer can trace a sensitive data path and verify controls at each boundary, this section is complete.

A mature signal is predictability under onboarding. New team members should be able to answer \"can this data move here?\" using boundary artifacts rather than informal tribal guidance.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Data Safety Boundary Map" artifact.

### Scenario

Your assistant handles mixed-sensitivity data across conversational and workflow channels. You must establish controls that prevent overexposure while preserving task effectiveness.

### Phase A: Data classification and policy definition

1. Identify key data entities and artifacts.
2. Assign sensitivity class to each data type.
3. Define handling rules per class (view/edit/export/retention).
4. Validate rules with security/compliance stakeholders.

Checkpoint A:

- Classification and handling policy are explicit and agreed.

### Phase B: Flow mapping and boundary identification

5. Map end-to-end data flow for 2-3 critical workflows.
6. Mark each boundary crossing (channel, tool, model, storage, export).
7. Identify high-risk crossings and uncontrolled paths.
8. Prioritize top boundary risks.

Checkpoint B:

- Boundary map highlights actionable high-risk crossings.

### Phase C: Control implementation

9. Apply channel-specific restrictions (auth, scope, payload rules).
10. Apply tool minimization and prompt boundary constraints.
11. Tighten retrieval and result-size controls for sensitive contexts.
12. Restrict access to trace/log/eval surfaces where needed.

Checkpoint C:

- High-risk crossings now have explicit controls.

### Phase D: Boundary validation tests

13. Run positive tests (allowed access paths function correctly).
14. Run negative tests (prohibited disclosures blocked).
15. Test refusal/escalation behavior for boundary-crossing requests.
16. Document any bypass or ambiguity.

Checkpoint D:

- Boundary controls are tested for both allowed and disallowed behavior.

### Phase E: Retention and incident readiness

17. Define retention/delete procedures per data class.
18. Define boundary incident response steps and owner roles.
19. Rehearse one tabletop boundary incident scenario.
20. Record lessons and control updates.

Checkpoint E:

- Lifecycle and incident controls are operationally ready.

### Phase F: Publish boundary map artifact

21. Create `chapter-08-section-02-data-safety-boundary-map.md` including:

- classification table,
- flow and boundary diagrams,
- implemented controls,
- validation test results,
- retention policy,
- incident response playbook.

22. Add go/no-go recommendation for Section 3 safe tool and prompt practices.

Checkpoint F:

- Another engineer can verify and maintain boundaries from your artifact.

## Expected outputs

By the end of this lab, you should have:

- A data classification model with handling rules by sensitivity level.
- End-to-end boundary maps for critical workflows.
- Implemented and tested controls at high-risk boundary crossings.
- Governance rules for logs/traces/exports and retention lifecycle.
- A boundary incident response workflow with ownership.
- A Data Safety Boundary Map artifact with go/no-go recommendation for Section 3.

Evidence that qualifies:

- Policy tables and boundary diagrams tied to live workflows.
- Positive/negative boundary test results.
- Documented retention and incident procedures.

## Failure modes

1. **No data classification model**
   - Symptom: inconsistent handling of sensitive information.
   - Recovery: define and enforce explicit sensitivity classes.

2. **Unmapped data paths**
   - Symptom: hidden boundary crossings cause surprise exposure risks.
   - Recovery: map end-to-end flows and review regularly.

3. **No minimization policy**
   - Symptom: unnecessary sensitive fields propagate across tools and prompts.
   - Recovery: enforce field-level minimization at each boundary.

4. **Channel controls not risk-specific**
   - Symptom: public and authenticated channels expose similar sensitive scope.
   - Recovery: define channel-specific safety boundaries.

5. **Tool/prompt contracts misaligned**
   - Symptom: prompts request broader data than tools should expose.
   - Recovery: align tool outputs and instruction constraints.

6. **Trace/log surfaces ungoverned**
   - Symptom: sensitive details visible in debug or eval artifacts.
   - Recovery: restrict and sanitize observability surfaces appropriately.

7. **Retention undefined**
   - Symptom: old sensitive data accumulates without purpose.
   - Recovery: apply class-based retention and deletion policy.

8. **No boundary incident playbook**
   - Symptom: delayed containment during exposure events.
   - Recovery: define response steps, owners, and communication protocol.

## Completion checklist

- [ ] I classified project data and defined handling rules by sensitivity.
- [ ] I mapped key workflows and identified high-risk boundary crossings.
- [ ] I implemented minimization and channel/tool/prompt boundary controls.
- [ ] I validated allowed and blocked behaviors with explicit tests.
- [ ] I documented retention and deletion practices.
- [ ] I defined and rehearsed boundary incident response procedures.
- [ ] I published a Data Safety Boundary Map with go/no-go recommendation for Section 3.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Storage](../../user-guide/data-model/storage.md)
- [Ingestion Webhook](../../user-guide/data-model/ingestion-webhook.md)
- [Tool Registry](../../user-guide/data-model/tool-registry.md)
- [Evals](../../user-guide/evals/README.md)

---

# Safe Tool And Prompt Practices

## Learning objectives

By the end of this section, you should be able to:

- Design prompt and tool policies that reduce unsafe behavior while preserving useful automation.
- Implement practical defenses against prompt injection, unsafe delegation, and high-risk tool misuse.
- Validate safety behavior using targeted evals, human review, and conversation trace evidence.
- Produce a secure prompt/tool playbook that supports repeatable governance decisions.

## Prerequisites

- You completed [Data Safety And Boundaries](#doc-ch08-security-and-governance-02-data-safety-and-boundaries).
- You can modify agent instructions, tool configurations, and tool registry entries.
- You can run security-oriented eval tasks and inspect trial traces.
- You have identified at least one high-impact tool path requiring stricter safety controls.

## In Gaia

- [AI Agents](../../user-guide/agents/README.md)
- [Settings](../../user-guide/settings/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)
- [Evals](../../user-guide/evals/README.md)
- [Audit Trail](../../user-guide/audit/README.md)

Use AI Agents as the main authoring surface for prompt and tool policy, Project roles and Settings to confirm who can change those controls, Evals to run adversarial validation, and Audit Trail to preserve accountability for high-impact prompt or tool updates.

## Concept brief

Prompts and tools form the executable policy layer of an AI application. Prompts shape decisions; tools execute consequences. Unsafe behavior usually emerges from gaps between these two layers:

- prompts allow broad behavior with weak constraints,
- tools expose broad capability with weak preconditions,
- model outputs are trusted without verification,
- safety tests cover only happy paths.

This section focuses on practical patterns that keep tool-enabled assistants helpful and controlled under adversarial and ambiguous conditions.

Security maturity in this area is cumulative. Small inconsistencies in prompt policy or tool validation usually do not fail immediately, but they compound until a single adversarial input reveals multiple stacked weaknesses.

### 1) Treat prompts as policy contracts, not style templates

A secure prompt is not only about tone. It defines enforceable behavior boundaries:

- what the assistant must do,
- what it must never do,
- what requires clarification,
- when it must refuse or escalate.

Prompt contracts should be explicit enough that reviewers can test compliance objectively.

### 2) Separate instruction layers by purpose

Mixing all rules into one long prompt reduces clarity and auditability.

Use modular instruction layers:

- role mission,
- scope boundaries,
- tool-use policy,
- safety/refusal rules,
- output constraints.

Modularity improves security review because each layer can be tested independently and updated without destabilizing unrelated behavior.

### 3) Design tool policies around least privilege and intent checks

Safe tool usage requires more than enabling/disabling.

Tool policy should define:

- allowed triggers per tool,
- required preconditions,
- disallowed contexts,
- expected output constraints,
- fallback behavior on error.

High-impact tools (write/delete/escalate/export) should require stricter intent and data validation before execution.

### 4) Defend against prompt injection and instruction hijacking

Prompt injection attempts to override system policy or extract prohibited data.

Practical defenses:

- prioritize system rules above user-provided instructions,
- treat external content as untrusted input,
- require explicit policy checks before sensitive tool actions,
- enforce refusal for requests attempting policy override.

Injection defense is a process, not a single rule. It requires continuous testing and refinement.

Treat external documents, user-provided snippets, and tool-returned text as untrusted by default. If any of these can influence downstream instruction interpretation, boundary checks must execute before sensitive actions are allowed.

### 5) Control delegation and handoff safety

Delegation can bypass boundaries if poorly constrained.

Safe delegation practices:

- delegate only to approved specialist roles,
- pass minimal required context,
- preserve boundary constraints across handoff,
- prevent recursive delegation loops for sensitive actions.

Delegation should increase control granularity, not create escape paths around safeguards.

### 6) Validate tool arguments and responses defensively

Even with strong prompts, model-generated arguments can be malformed or unsafe.

Defensive practices:

- validate required argument shapes,
- enforce strict enums/ranges for risky parameters,
- reject ambiguous target identifiers,
- sanitize or constrain tool outputs before reinjection.

Structured validation is often the decisive layer preventing unsafe actions.

### 7) Design refusal and escalation paths for usability

Unsafe requests should not be handled silently or inconsistently.

Good refusal pattern:

- concise boundary explanation,
- no sensitive detail leakage,
- safe alternative path,
- escalation option for legitimate exceptions.

Usable refusal behavior preserves trust while maintaining security posture.

### 8) Use security evals as continuous controls

Security behavior must be tested routinely, not only after incidents.

Use curated security tasks for:

- prompt injection attempts,
- unauthorized access requests,
- data leakage patterns,
- policy evasion phrasing.

Pair automated graders with selective human review for ambiguous high-risk outputs.

Security eval suites should be versioned with context notes. A failure that was benign under one policy baseline may become critical after channel expansion or new tool enablement.

### 9) Monitor safety drift after changes

Any change to prompts, tools, or models can reintroduce risks.

Post-change checks should include:

- security regression task runs,
- targeted trace inspection,
- tool call anomaly review,
- policy-violation trend checks.

Without drift monitoring, previously fixed vulnerabilities can return unnoticed.

### 10) Definition of done for Safe Tool and Prompt Practices

Section 3 is done when unsafe behavior classes are constrained by policy and verified by evidence.

Done criteria:

- prompt policy layers are explicit and testable,
- tool preconditions and argument validation are enforced,
- injection and evasion scenarios are covered in evals,
- refusal/escalation behavior is usable and consistent,
- drift monitoring is integrated after changes,
- one lab demonstrates robust safe-execution behavior.

If another engineer can apply your playbook and maintain comparable safety posture, this section is complete.

An additional maturity signal is low-variance incident response: different reviewers evaluating the same risky behavior should reach similar conclusions because policy and tests are unambiguous.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces a "Safe Prompt and Tool Controls Playbook" artifact.

### Scenario

Your assistant uses multiple tools, including high-impact actions. You need to harden prompt and tool behavior against unsafe or adversarial requests while preserving normal workflow quality.

### Phase A: Risk path inventory

1. List enabled tools and classify risk level.
2. Identify prompt areas governing tool usage and refusal behavior.
3. Map top unsafe behavior scenarios (injection, leakage, unauthorized actions).
4. Prioritize controls by impact and likelihood.

Checkpoint A:

- High-risk tool/prompt paths are identified and prioritized.

### Phase B: Prompt hardening

5. Refactor instructions into explicit policy layers.
6. Add boundary and refusal rules for high-risk requests.
7. Add clarification requirements before sensitive actions.
8. Add delegation constraints for sensitive workflows.

Checkpoint B:

- Prompt policy is modular, explicit, and security-reviewable.

### Phase C: Tool control hardening

9. Tighten tool allow-lists by role.
10. Add argument validation and precondition checks for high-risk tools.
11. Restrict tool response payloads to minimum required fields.
12. Define deterministic error and fallback outputs.

Checkpoint C:

- Tool behavior is constrained and defensively validated.

### Phase D: Adversarial validation

13. Run security eval generation for relevant categories.
14. Execute security-focused task sets.
15. Review failed/ambiguous trials with human review.
16. Inspect trace evidence for unsafe tool execution attempts.

Checkpoint D:

- Safety controls are tested against adversarial and evasive inputs.

### Phase E: Drift and regression controls

17. Define post-change security check sequence.
18. Add security regression gates for must-pass safety criteria.
19. Define thresholds for safety incident escalation.
20. Assign owners for recurring safety review cadence.

Checkpoint E:

- Ongoing safety assurance is operational, not one-off.

### Phase F: Publish safety playbook

21. Create `chapter-08-section-03-safe-prompt-tool-controls-playbook.md` including:

- risk inventory,
- prompt control layers,
- tool control matrix,
- adversarial test outcomes,
- regression and drift controls,
- ownership and escalation policy.

22. Add go/no-go recommendation for Section 4 audit and compliance readiness.

Checkpoint F:

- Another engineer can reproduce safety hardening and validation using your playbook.

## Expected outputs

By the end of this lab, you should have:

- A risk-ranked map of prompt and tool safety exposure points.
- Hardened prompt policy layers with explicit refusal/escalation behavior.
- Hardened tool controls with argument/precondition validation.
- Security eval results covering injection, leakage, and evasion scenarios.
- Drift and regression control process with ownership.
- A Safe Prompt and Tool Controls Playbook with go/no-go recommendation for Section 4.

Evidence that qualifies:

- Updated config/tool definitions with documented rationale.
- Security eval trial results and human-review notes.
- Trace examples showing blocked unsafe behavior.

## Failure modes

1. **Prompts optimized for style but not policy clarity**
   - Symptom: assistant responds politely but violates boundaries.
   - Recovery: encode explicit policy rules and refusal criteria.

2. **Tool controls rely only on model discretion**
   - Symptom: unsafe tool calls triggered by ambiguous inputs.
   - Recovery: enforce deterministic preconditions and argument validation.

3. **Injection scenarios untested**
   - Symptom: hidden policy override vulnerabilities.
   - Recovery: run recurring adversarial eval task sets.

4. **Over-broad tool payloads**
   - Symptom: unnecessary sensitive data returned and reused.
   - Recovery: minimize output fields and sanitize responses.

5. **Delegation boundaries unclear**
   - Symptom: sensitive actions routed through weakly constrained agents.
   - Recovery: restrict delegation scope and enforce boundary inheritance.

6. **Refusal behavior unusable**
   - Symptom: users bypass controls due to confusing refusals.
   - Recovery: improve refusal clarity and provide safe alternatives.

7. **No post-change safety checks**
   - Symptom: fixed vulnerabilities recur after unrelated updates.
   - Recovery: add mandatory security regression checks per change cycle.

8. **No ownership for safety drift**
   - Symptom: recurring warnings with no resolution.
   - Recovery: assign owners and escalation SLAs for safety incidents.

## Completion checklist

- [ ] I identified high-risk prompt and tool paths.
- [ ] I implemented modular prompt policy controls with explicit boundaries.
- [ ] I hardened high-risk tools with validation and minimization controls.
- [ ] I validated controls using adversarial security evals and trace review.
- [ ] I defined post-change security drift checks and gate policies.
- [ ] I assigned ownership and escalation procedures for safety incidents.
- [ ] I published a Safe Prompt and Tool Controls Playbook with go/no-go recommendation for Section 4.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Tool Registry](../../user-guide/data-model/tool-registry.md)
- [Skill Registry](../../user-guide/agents/skills.md)
- [Generate Security Tasks](../../user-guide/evals/dialogs/generate-security-evals.md)
- [Human Review](../../user-guide/evals/dialogs/human-review.md)
- [Evals](../../user-guide/evals/README.md)
- [Agent Behavior Trace](../../user-guide/evals/dialogs/agent-behavior-trace.md)

---

# Audit And Compliance Readiness

## Learning objectives

By the end of this section, you should be able to:

- Define audit and compliance readiness for Gaia projects as an operational capability, not a one-time document exercise.
- Build an evidence model that links controls, system behavior, and accountable ownership.
- Run readiness checks that expose governance gaps before external audits or internal reviews.
- Produce a compliance readiness package that supports repeatable review and remediation cycles.

## Prerequisites

- You completed [Safe Tool And Prompt Practices](#doc-ch08-security-and-governance-03-safe-tool-and-prompt-practices).
- You can access audit trail, settings, eval/report outputs, and run/conversation diagnostics.
- You have documented role/access controls and data safety boundary artifacts from prior sections.
- You have identified relevant internal or external compliance expectations for your project context.

## In Gaia

- [Governance](../../user-guide/governance/README.md)
- [Governance Registry](../../user-guide/governance/registry.md)
- [Governance Controls](../../user-guide/governance/controls.md)
- [Governance Obligations](../../user-guide/governance/obligations.md)
- [Governance Regulatory Updates](../../user-guide/governance/regulatory-updates.md)
- [Evals](../../user-guide/evals/README.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Settings](../../user-guide/settings/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Tasks](../../user-guide/tasks/README.md)

Use Audit Trail when you gather change evidence, Dashboard for trend context, Delivery and Tasks for remediation work that must remain visible after the review closes, and Evals when the review depends on behavior evidence rather than policy artifacts alone. Keep the selected Governance control updated so implementation guidance, testing requirements, recent versions, and linked evidence stay visible together during readiness review. If your project also uses the Governance Registry, confirm the selected package scope before collecting evidence so reviewers can see the governing standard or overlay directly.

## Concept brief

Audit readiness means you can demonstrate control effectiveness with evidence on demand. Compliance readiness means controls are not only documented but operationally enforced.

In AI systems, readiness failures often come from evidence fragmentation:

- controls exist but are undocumented,
- policies are documented but not enforced,
- events are logged but not reviewed,
- incidents are resolved but not tied to preventive controls.

This section integrates security and governance work from Sections 1-3 into a repeatable readiness model.

Readiness quality is visible in response speed to evidence requests. If a reviewer asks for proof of a control and teams need days to reconstruct context, governance may exist conceptually but not operationally.

### 1) Treat compliance as continuous control validation

Compliance is often approached as periodic paperwork. This is fragile.

Instead, operate a continuous model:

- define controls,
- collect evidence,
- review control effectiveness,
- remediate gaps,
- repeat on schedule.

Continuous validation reduces scramble before audits and improves real security posture.

### 2) Build a control-to-evidence map

Each control should map to concrete evidence sources.

Example mapping categories:

- access controls -> role assignments, permission reviews, audit entries.
- safety controls -> prompt/tool policies, security eval outcomes.
- operational controls -> monitoring thresholds, incident logs, remediation records.
- release controls -> regression gate results, sign-off decisions.

If controls cannot be evidenced, they are governance claims, not governance reality.

For organization access reviews, include the organization login policy in the evidence set: which sign-in methods are enabled, whether access is invite-only or uses the signup form, and who reviewed the current organization membership list. This keeps authentication posture tied to the same accountability model as roles and project permissions.

Use stable control identifiers so evidence can be tracked across review cycles. Identifier discipline improves trend analysis and avoids confusion when policy wording evolves.

In Gaia, this is strongest when each governed control points back to the original evidence artifact instead of a duplicate summary. The selected-control view in Governance is designed to keep that mapping inspectable before release review.

When you use the Governance registry to curate reusable framework packages, assign the relevant package directly to the working contract, policy, state profile, risk, control, obligation, regulation, regulatory source, or regulatory update. That makes readiness scope traceable to the declared governance overlay instead of leaving reviewers to infer package membership only from framework keys.

The important distinction is layering. Formal standards, sector overlays such as banking, and internal governance profiles can all coexist, but they should not be flattened into one ambiguous checklist. Readiness review should preserve that structure so reviewers can tell which evidence satisfies which layer.

### 3) Standardize evidence artifacts and storage discipline

Audit friction often comes from inconsistent evidence formats and scattered storage.

Define standard artifact formats:

- policy documents,
- control matrices,
- run/report exports,
- incident postmortems,
- approval logs.

Also define where artifacts live, who owns them, and retention expectations.

Standardization improves reviewer efficiency and reduces interpretation errors.

### 4) Use audit trail as authoritative change evidence

Audit trail is core evidence for governance verification.

In the current product, use **Project Settings -> Audit Trail** in readiness drills and evidence requests.

Use it to verify:

- sensitive setting changes,
- role and permission updates,
- config/version activations,
- deletion or high-impact operations.

Pair audit logs with review notes so change intent is clear. Raw logs without interpretation can be difficult for auditors to evaluate.

### 5) Include eval and safety evidence in readiness scope

AI compliance requires behavior evidence, not just infrastructure controls.

Include:

- security eval run results,
- human review outcomes for ambiguous high-risk cases,
- regression gate pass/fail history,
- trend evidence for resolved and recurring issues.

This demonstrates that behavior controls are tested continuously.

If an obligation or classification assumes a certain escalation, refusal, or approval pattern, make the supporting eval evidence visible in the same readiness package. Policy claims without behavior evidence are weak, and behavior evidence without the governing requirement is hard to interpret.

### DORA and full governance-layer evidence

DORA-oriented evidence should not be handled as a standalone checkbox. Treat it as one regulatory lens across the same governance layer that also supports GDPR, EU AI Act, ISO/IEC 42001, internal risk policy, and customer-specific operating requirements.

For a DORA-style review, map the Gaia evidence package to:

- ICT risk ownership: Governance risks, controls, policies, agent systems, and accountable owners.
- Incident handling: Platform Status, Dashboard signals, Audit Trail rows, Delivery discussions, and Tasks for response work.
- Continuity and recovery: documented fallback, rollback, kill-switch, retention, and handoff criteria.
- Third-party dependency oversight: service accounts, external-agent records, Databricks or connector boundary evidence, and vendor support ownership.
- Exit and transition readiness: exportable evidence, delivery handoff records, support owners, and offboarding tasks.

The same package should still preserve the broader governance model: applicable framework packages, obligations, controls, classifications, explainability, eval evidence, runtime policy decisions, MCP gateway decisions, schema drift findings or accepted-baseline reviews, audit evidence, and remediation ownership. When reviewers need a file artifact, use Audit Trail **Governance Evidence JSONL** to combine project audit rows with governance runtime telemetry, then keep the control-plane export for scoped OTLP/SIEM delivery from selected agent-system boundaries. Do not claim legal compliance from the presence of records alone. The review needs current evidence, owner sign-off, and deployment-specific operating facts.

### 6) Define exception and risk-acceptance workflows

No control system is perfect. Exceptions must be governed explicitly.

Exception workflow should include:

- reason and scope,
- risk assessment,
- owner approval,
- compensating controls,
- expiration/review date.

Untracked exceptions create hidden compliance debt and can invalidate otherwise strong controls.

Exception reviews should be periodic, not only event-driven. A recurring review catches \"expired but still active\" exceptions that otherwise become silent permanent policy holes.

### 7) Rehearse internal readiness reviews before external audits

Internal mock reviews identify evidence and process gaps early.

Mock review scope:

- control completeness,
- evidence traceability,
- ownership accountability,
- incident response quality,
- open exceptions and closure status.

Rehearsal reduces audit-time surprises and improves organizational confidence.

### 8) Measure readiness with explicit scorecards

Use a practical readiness scorecard with categories such as:

- control definition completeness,
- evidence freshness,
- review cadence adherence,
- unresolved high-risk findings,
- incident recurrence rates.

Scorecards help prioritize remediation and track governance maturity over time.

### 9) Define remediation governance for audit findings

Readiness is tested by how findings are resolved.

Remediation governance should define:

- severity tiers,
- fix ownership,
- target resolution windows,
- revalidation requirements,
- escalation for overdue findings.

Without this, audits generate reports but not meaningful control improvement.

Where possible, tie remediation completion to objective evidence updates (new run results, updated access logs, revised policy artifacts) rather than narrative status alone.

Use Delivery and Tasks as the closure path for those findings. A governance gap that never becomes tracked execution work is not under control yet, regardless of how well it is described in the readiness pack.

### 10) Definition of done for Audit and Compliance Readiness

Section 4 and Chapter 8 are done when governance claims are consistently supportable with current evidence and accountable operations.

Done criteria:

- applicable framework packages and overlays are explicit for the reviewed release scope,
- control-to-evidence map is complete and current,
- artifact standards and storage discipline are defined,
- audit trail, eval, and incident evidence are integrated,
- exception and remediation workflows are operational,
- readiness scorecard and review cadence exist,
- one lab demonstrates end-to-end readiness assessment.

At this point, Chapter 8 provides a strong governance base for Chapter 9 delivery and collaboration controls.

The practical benchmark is repeatability: two independent reviewers should be able to reach similar readiness conclusions from the same package with minimal clarification requests.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

This lab produces an "Audit and Compliance Readiness Package" artifact.

### Scenario

Your team must prepare for an internal governance review (and future external audit) for a production-facing assistant project.

### Phase A: Control inventory and mapping

1. Inventory active controls from roles/access, data boundaries, and prompt/tool safety.
2. Build control-to-evidence map linking each control to Gaia evidence source.
3. Mark controls with missing or stale evidence.
4. Prioritize gaps by risk.

Checkpoint A:

- Control inventory and evidence mapping are complete and risk-ranked.

### Phase B: Evidence collection and normalization

5. Collect required evidence artifacts:
   - audit trail exports/screens,
   - role/permission matrices,
   - security eval and human-review outputs,
   - regression gate decision logs,
   - incident/remediation records.
6. Convert artifacts into standardized format and index.
7. Validate traceability from claim -> evidence -> owner.

Checkpoint B:

- Evidence package is structured and review-ready.

### Phase C: Mock readiness review

8. Run an internal mock audit session.
9. Test sample questions across control domains.
10. Record findings:

- missing evidence,
- weak control articulation,
- ownership ambiguity,
- stale exceptions.

11. Assign severity and owners.

Checkpoint C:

- Mock review reveals concrete, actionable readiness gaps.

### Phase D: Remediation and revalidation

12. Execute high-priority remediations.
13. Update control docs and evidence artifacts.
14. Re-run targeted validation checks (including security eval/regression evidence where relevant).
15. Confirm gap closure with owner sign-off.

Checkpoint D:

- Priority findings are remediated and revalidated.

### Phase E: Readiness scorecard and cadence

16. Build readiness scorecard with category scores and trend notes.
17. Define recurring review cadence (monthly/quarterly depending on risk).
18. Define escalation path for overdue high-severity findings.
19. Publish governance ownership map.

Checkpoint E:

- Readiness tracking is ongoing, measurable, and owner-accountable.

### Phase F: Publish readiness package

20. Create `chapter-08-section-04-audit-compliance-readiness-package.md` including:

- control-to-evidence map,
- artifact index,
- mock-review findings,
- remediation status,
- readiness scorecard,
- cadence and ownership model.

21. Add Chapter 8 readiness recommendation and carry-over priorities for Chapter 9.

Checkpoint F:

- Another reviewer can evaluate governance posture using your package without additional context.

## Expected outputs

By the end of this lab, you should have:

- A complete control inventory linked to evidence sources.
- A normalized evidence package covering access, safety, operations, and release controls.
- Mock review findings with severity and remediation ownership.
- Revalidated closure evidence for high-priority gaps.
- A readiness scorecard and recurring review cadence.
- An Audit and Compliance Readiness Package with Chapter 8 readiness recommendation.

Evidence that qualifies:

- Indexed artifacts with traceable control linkage.
- Documented remediation with revalidation outcomes.
- Scorecard and review cadence approved by governance owners.

## Failure modes

1. **Compliance treated as one-time documentation sprint**
   - Symptom: readiness decays quickly after audit cycle.
   - Recovery: adopt continuous control-validation cadence.

2. **No control-to-evidence linkage**
   - Symptom: controls are claimed but unverifiable.
   - Recovery: map every control to concrete, current evidence.

3. **Evidence artifacts inconsistent and scattered**
   - Symptom: review time increases and findings are disputed.
   - Recovery: standardize artifact format, location, and ownership.

4. **Audit trail not integrated into review process**
   - Symptom: change accountability is unclear.
   - Recovery: include audit-log verification in readiness checklist.

5. **Behavioral safety evidence omitted**
   - Symptom: compliance review misses AI-specific risk controls.
   - Recovery: include eval/security/human-review evidence explicitly.

6. **Exception handling informal**
   - Symptom: temporary risks become permanent hidden debt.
   - Recovery: enforce tracked exception workflow with expiry and compensating controls.

7. **Findings recorded without remediation governance**
   - Symptom: repeated high-severity findings across reviews.
   - Recovery: assign severity-based SLAs and revalidation requirements.

8. **No readiness scorecard or cadence**
   - Symptom: governance status is subjective and reactive.
   - Recovery: maintain measurable readiness scoring and recurring review schedule.

## Completion checklist

- [ ] I built a control-to-evidence map across security and governance domains.
- [ ] I assembled standardized, traceable evidence artifacts.
- [ ] I ran a mock readiness review and documented findings by severity.
- [ ] I remediated and revalidated high-priority gaps.
- [ ] I defined exception handling and overdue-finding escalation workflows.
- [ ] I published a readiness scorecard and recurring review cadence.
- [ ] I published an Audit and Compliance Readiness Package with Chapter 8 readiness recommendation.

## Canonical references

- [User Guide Home](../../user-guide/README.md)
- [Settings](../../user-guide/settings/README.md)
- [Project Roles](../../user-guide/settings/project-roles.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Generate Security Tasks](../../user-guide/evals/dialogs/generate-security-evals.md)
- [Human Review](../../user-guide/evals/dialogs/human-review.md)
- [Reports](../../user-guide/evals/reports.md)

---

# Governance Operating Model

## Learning objectives

By the end of this section, you should be able to:

- Treat governance as a continuous operating model for enterprise AI delivery rather than a late-stage approval ritual.
- Layer formal standards, domain overlays, and internal governance profiles using Gaia framework packages.
- Assign accountable owners for obligations, controls, evidence, and remediation work across the application lifecycle.
- Connect governance decisions to eval design, delivery execution, and release-gate decisions.

## Prerequisites

- You completed [Project Roles And Access](#doc-ch08-security-and-governance-01-project-roles-and-access), [Data Safety And Boundaries](#doc-ch08-security-and-governance-02-data-safety-and-boundaries), and [Safe Tool And Prompt Practices](#doc-ch08-security-and-governance-03-safe-tool-and-prompt-practices).
- You can access the Governance workspace, Evals, Delivery Management, Tasks, Settings, and Audit Trail.
- You already have a basic application scope, data model, and agent/tool design to govern.

If governance is new to you, read [Governance Foundations](../../user-guide/governance/foundations.md) first and then use this page for the operating-model view.

## In Gaia

- [Governance Foundations](../../user-guide/governance/foundations.md)
- [Governance](../../user-guide/governance/README.md)
- [Governed application lifecycle](../../user-guide/governance/governed-application-lifecycle.md)
- [Governance Registry](../../user-guide/governance/registry.md)
- [Governance Agent Systems](../../user-guide/governance/agent-systems.md)
- [Governance Controls](../../user-guide/governance/controls.md)
- [Governance Obligations](../../user-guide/governance/obligations.md)
- [Governance Regulatory Updates](../../user-guide/governance/regulatory-updates.md)
- [Evals](../../user-guide/evals/README.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Audit Trail](../../user-guide/audit/README.md)

Use these guide pages together. Governance defines what must be true, Evals measure how the system behaves, Delivery and Tasks carry remediation and approvals, and Audit Trail preserves the change evidence behind the operating model.

Practical sequence:

1. Open [Governance Registry](../../user-guide/governance/registry.md) and make the package stack explicit: standards first, domain overlay second, internal profile third.
2. Open [Governance Classifications](../../user-guide/governance/classifications.md) and record why the system is in scope, what category it falls into, and which evidence supports that decision.
3. Open [Governance Obligations](../../user-guide/governance/obligations.md) and [Governance Controls](../../user-guide/governance/controls.md) to capture the requirements and safeguards that must exist around the workflow.
4. Open [Governance Agent Systems](../../user-guide/governance/agent-systems.md) and register the governed automation boundary so Gaia-operated agents, governance-monitored external agents, runtime telemetry, Agent SRE SLO posture, governance verification runs, published capability lineage, control-plane export posture, review method, citations, and human follow-up all point to the same durable record.
5. Open [Governance Contracts](../../user-guide/governance/contracts.md) and [Governance Policies](../../user-guide/governance/policies.md) so governed interfaces, runtime posture, accepted Agent SRE response state, and policy decision evidence are visible as first-class records instead of implicit implementation details.
6. Open [Governance State & Memory](../../user-guide/governance/state-memory.md) and [Governance Explainability](../../user-guide/governance/explainability.md) so retention, provenance, and reviewer-facing evidence stay connected to the same governed boundary.
7. Open [Governance Operations](../../user-guide/governance/operations.md) to track which governance records still need intake, assessment, execution, or review. Critical or exhausted Agent SRE posture remains manual and time-bounded: accept it only when you want approved runtime policies to be able to match that posture.
8. Open [Evals](../../user-guide/evals/README.md), [Delivery Management](../../user-guide/delivery/README.md), and [Tasks](../../user-guide/tasks/README.md) so the governance model is backed by behavior evidence and owned remediation work.
9. Use [Audit Trail](../../user-guide/audit/README.md) when you need change evidence behind a governance decision or release review.

## Concept brief

Governance on Gaia is not a separate compliance appendix. It is the operating system that ties together risk posture, package selection, evidence expectations, remediation ownership, and release discipline.

If teams wait until release week to decide which frameworks apply, which evidence matters, or who owns unresolved findings, the platform may still be functional but it is not governable. Section 5 exists to prevent that failure mode.

### 1) Start governance when the system shape becomes clear

Governance should begin once the project can answer four questions:

- what decisions the assistant supports,
- which records and workflows it touches,
- where human escalation is mandatory,
- which failures would make the system unacceptable to operate.

That is usually earlier than formal release preparation. A late governance start creates rework because classification, control design, and evidence retention often force architecture and process changes.

### 2) Separate framework layers instead of flattening them

Do not treat every governance input as the same kind of thing.

Use three layers:

- formal standards and regulatory baselines such as the EU AI Act, NIST AI RMF, or ISO/IEC 42001,
- domain overlays such as banking or telecom when the project context needs sector-specific controls,
- internal governance profiles for your own delivery, approval, or documentation expectations.

On Gaia, the Governance registry exists to keep those layers explicit through framework packages. This is important because a sector overlay is not the same type of artifact as a formal cross-industry standard. Banking is a useful example, not a platform assumption.

### 3) Use framework packages as working governance bundles

Framework keys let teams normalize governance scope, but packages make that scope operational.

Use framework packages to:

- define the reusable package boundary,
- declare dependencies through extended framework keys,
- publish the package once it is stable enough for reuse,
- import the published package into another accessible project when the same overlay should be reused there.

This gives teams a governed reuse path without pretending Gaia already has a global package catalog or release-channel model. Today the working model is project-scoped package authoring plus published-package import across accessible projects.

The EU AI Act and ISO/IEC 42001 packages now provide fuller operational mappings than the earlier starter baselines. Treat them as structured governance worklists: sync the obligation, risk, control, source-collection, and regulatory-update baselines, activate the package, then review applicability, retained source material, and evidence before relying on the result. A full mapping is not the same as completed compliance; legal interpretation, local audit expectations, and customer-specific scope decisions still need accountable review.

### 4) Assign the package where the work actually happens

Governance scope should not remain abstract in the registry.

Assign the relevant framework package directly to the working records that carry the governed behavior:

- risks,
- contracts,
- policies,
- controls,
- state profiles,
- obligations,
- custom regulations,
- regulatory source collections,
- regulatory updates.

Explicit assignment matters because it makes reviewers inspect real adoption instead of inferring membership only from matching framework keys. This is how package design becomes auditable delivery behavior.

### 5) Give obligations, controls, and evidence clear owners

Every governed system needs at least four accountability lanes:

- package owner for the reusable governance overlay,
- obligation or policy owner for requirement interpretation,
- control owner for implementation and verification,
- evidence owner for freshness and traceability.

These can be shared by a small team, but the responsibilities must still be named explicitly. Unowned evidence decays. Unowned obligations drift into opinion. Unowned controls appear documented but not maintained.

### 6) Keep evidence linked to operational source systems

A common governance failure is rebuilding evidence manually in spreadsheets or review docs.

Gaia works better when governance links back to the original operational source:

- workflow runs,
- delivery tasks and milestones,
- eval runs and reports,
- audit trail events,
- folders or retained artifacts,
- explainability outputs and human review notes.

Evidence linkage is stronger than evidence duplication because it keeps reviewers close to the real system state. If the artifact changes, the governance record should still point to the live source of truth.

For governed automation boundaries, use the Agent Systems selected-record proof path before release review. It should tell the operator what the boundary is, whether telemetry is current, how Gaia maps the boundary to ACS-aligned input, LLM, state, tool-execution, and output checkpoints, whether standards evidence exports are available, whether OpenTelemetry export is configured and recently delivered, which evidence links or citations support the latest review, whether governance verification passed or needs review, whether the run was deterministic or `ai_assisted`, and what human follow-up remains. Treat the standards matrix, ACS policy projection, OCSF-aligned runtime records, CycloneDX Agent BOM, ASSERT-style eval bridge, and A2A readiness profile as compatibility evidence rather than certification. Treat telemetry-only systems and third-party control-plane exports as observed evidence sources unless there is a separate enforcement hook, and treat `ai_assisted` output as a cited review aid rather than autonomous closure.

### 7) Keep evals and governance distinct, but tightly connected

Evals and governance answer different questions:

- Evals ask whether the assistant behaves correctly under defined scenarios.
- Governance asks what controls, obligations, classifications, review steps, and evidence are required before the system is acceptable to operate.

The operating model is strongest when the connection is explicit:

- governance decisions shape eval coverage,
- eval failures generate governance remediation work when the failure breaks a governed requirement,
- release gates require both behavior evidence and governance evidence.

Runtime policy decisions are part of that connection. When an approved Governance policy matches a tool call, the resulting evidence should identify the policy, version, rule, configured outcome, effective outcome, reason code, and enforcement mode. Review-mode warnings are useful before enforcement because they prove the rule would have matched without changing runtime behavior.

For high-risk runtime decisions, preserve a Decision BOM when the release or audit decision needs reconstruction. The BOM should point back to the Agent System telemetry event and summarize the governed policy decision, invocation envelope, evidence references, and stable hashes for sensitive request or response material. It does not replace the underlying telemetry or audit trail; it gives reviewers a compact artifact that can be linked to controls, explanations, release readiness, or follow-up tasks.

Do not let one system substitute for the other. A passing eval run does not prove governance completeness. A complete obligation inventory does not prove safe behavior.

### 8) Route governance findings into delivery, not isolated review notes

Governance becomes real only when findings change execution.

Typical translations:

- missing control -> delivery task with owner and due window,
- stale evidence -> review task or milestone gate,
- unresolved classification issue -> explicit pre-release blocker,
- new regulatory update -> assessment work in Operations plus downstream implementation tasks.

This is the operational contract between Chapter 8 and Chapter 9. Governance identifies what must change. Delivery ensures the change is tracked to closure.

### 9) Build release gates from the operating model, not from optimism

A mature release decision uses governance as one of the decision inputs, not as a ceremonial sign-off at the end.

At minimum, the release gate should answer:

- which framework packages are in scope for this release,
- whether required obligations and controls are covered,
- whether evidence is fresh enough for the governed workflow,
- whether eval evidence supports the declared policy posture,
- whether unresolved gaps were fixed, deferred, or explicitly accepted.

This is why governance must be designed before the final readiness meeting. Release week is too late to discover that scope, ownership, or evidence contracts were never defined.

### 10) Definition of done for Governance Operating Model

Section 5 is done when:

- the applicable standards, overlays, and internal profiles are layered explicitly,
- the required framework packages are authored or imported and assigned to working records,
- obligations, controls, and evidence have accountable owners,
- governance findings flow into tracked delivery work,
- release criteria combine governance evidence with eval and execution evidence,
- the team can explain the governance model without relying on tribal knowledge.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable governance operating model and validate one end-to-end governed workflow.
- Record at least one ownership gap and one remediation action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are preparing a `Customer Operations Copilot` that serves a governed workflow. The primary package baseline combines a formal AI governance standard, a sector overlay, and one internal release-control profile. The same sector overlay may later be reused by another project, so package structure and provenance must remain inspectable.

### Phase A: Define the governance stack

1. List the formal standards, sector overlays, and internal governance profiles that apply.
2. Decide which of those should be represented as reusable framework packages.
3. Document why each layer applies to the capstone workflow.
4. Record one explicit exclusion so the governance perimeter is bounded.

Checkpoint:

- Governance scope is layered clearly.
- Reviewers can tell which requirements come from standards, overlays, and internal policy.

### Phase B: Build or import the package baseline

1. Open **Governance -> Registry**.
2. Refresh the starter catalog for the formal standards you need.
3. Create or update any domain-overlay or internal-profile package.
4. Activate or import the package you want to use so Gaia preloads the matching starter records.
5. Review the suggested scope, keep what belongs, and remove anything out of scope.
6. Publish the package set that the project will use as its governance baseline.

Checkpoint:

- The package boundary is explicit.
- Reuse provenance is visible where a package came from another project.

### Phase C: Attach governance scope to working records

1. Confirm the reviewed package scope is attached to the active risks, contracts, policies, controls, state profiles, obligations, custom regulations, source collections, and regulatory updates.
2. Confirm the package report shows explicit downstream adoption, not only inferred linked coverage.
3. Mark which obligations or controls are still missing from the working scope.
4. Capture any gap as tracked work.

Checkpoint:

- Governance package adoption is visible on the records that actually carry the governed behavior.
- Missing scope is translated into owned action.

### Phase D: Define evidence and ownership flow

1. For each critical obligation and control, record the owner and expected evidence source.
2. Link evidence back to real operational artifacts instead of duplicate summaries.
3. Review freshness for high-impact evidence.
4. Record any stale or missing evidence as remediation work.

Checkpoint:

- Every high-impact governed requirement has an owner and evidence path.
- Evidence traceability does not depend on manual reconstruction.

### Phase E: Connect governance to evals and release

1. Identify which eval scenarios prove the governed behavior or escalation path.
2. Define which governance gaps block release versus which can ship only with explicit acceptance.
3. Run **Verify governance posture** for the governed Agent System and record whether the result is ready, needs review, or blocked.
4. Add the release-gate checks to your delivery readiness pack, including any verification findings and Decision BOMs for high-risk runtime decisions.
5. Run one internal review rehearsal using the package scope, eval evidence, verification output, and open remediation list.

Checkpoint:

- Governance, evals, and delivery use one shared decision frame.
- Release readiness is evidence-based rather than narrative-based.
- Verification findings and Decision BOMs are linked where they materially affect the decision.

## Expected outputs

- A layered governance model covering standards, overlays, and internal profiles.
- A published or imported package baseline with visible provenance where reuse applies.
- Explicit package adoption across the governed record types in active scope.
- Ownership and evidence flow for the highest-impact obligations and controls.
- One release-gate checklist that combines governance, eval, and delivery evidence.
- Governance verification output and any Decision BOMs needed to reconstruct high-risk runtime decisions.

---

# Chapter 9: Delivery and Collaboration

Status: current

Team workflows and release readiness

## Sections

- [Delivery Process Cycle](#doc-ch09-delivery-and-collaboration-01-delivery-process-cycle)
- [Task And Milestone Management](#doc-ch09-delivery-and-collaboration-02-task-and-milestone-management)
- [Change Management](#doc-ch09-delivery-and-collaboration-03-change-management)
- [Release Readiness Checklist](#doc-ch09-delivery-and-collaboration-04-release-readiness-checklist)

## Alignment with User Guide Delivery Surfaces

- [Delivery Management](../../user-guide/delivery/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Artifact Templates](../../user-guide/artifact-templates.md)
- [Discussions](../../user-guide/discuss/README.md)
- [Tutorials](../../user-guide/tutorials/README.md)

Use Platform Discussions at `/platform/support/discussions` for cross-project discovery, coordination, or escalation. Once a discussion changes project scope, schedule, ownership, or release readiness, summarize the accepted outcome in Delivery Discussions and move it into Process Cycle evidence, Tasks, Milestones, and Timeline.

Use **Operations: Discussion to Execution** in [Tutorials](../../user-guide/tutorials/README.md) at `/platform/support/tutorials` as the shortest guided path for converting a platform discussion into tracked project work.

## Fast path inside Gaia

1. Start in [Delivery Management](../../user-guide/delivery/README.md) and define the active process cycle.
2. Use [Tasks](../../user-guide/tasks/README.md), [Delivery Milestones](../../user-guide/delivery/milestones.md), and [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md) to make execution truth visible.
3. Use [Delivery Discussions](../../user-guide/delivery/discuss.md) and [Discussions](../../user-guide/discuss/README.md) when a decision needs collaboration before it becomes tracked work.

Delivery is only real when the accepted decision appears in Gaia as owned work, evidence, milestones, or timeline movement.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Delivery Process Cycle

## Learning objectives

- Design a delivery cycle that converts strategic intent into executable work across Planning, Exploration, Development, and Evaluation.
- Operate a cycle end to end in Gaia with explicit stage entry criteria, evidence expectations, and completion gates.
- Use Delivery Process, Tasks, Timeline, and Discussions together so execution progress remains traceable and reviewable.
- Detect and recover from cycle drift (scope noise, stale evidence, blocked ownership, and sequencing debt) before release risk compounds.

## Prerequisites

- Completion of Chapter 8 sections on governance, data boundaries, and audit readiness.
- Project access with permissions to create and edit delivery cycles, tasks, and milestones.
- At least one active initiative or feature objective with clear business intent and an initial success hypothesis.
- Familiarity with Gaia pages: Delivery Management overview, Delivery Process Cycle, Tasks, Timeline, and Discussions.

## In Gaia

- [Delivery Management](../../user-guide/delivery/README.md)
- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)

Use Delivery Discussions as the project-scoped decision thread for this section. Platform Discussions are useful for broader coordination, but once a request affects project scope or sequencing, the cycle, task, milestone, and timeline surfaces above become the canonical execution path.

## Concept brief

A delivery cycle is not a project management formality. In Gaia, it is a decision system that keeps planning, execution, and evidence in one operational thread. The cycle is useful only when each stage has a clear contract: what must be true to start, what artifacts should be produced during work, and what evidence is required to close the stage responsibly.

### 1) Treat the cycle as a control loop, not a checklist

The biggest failure pattern in delivery programs is reducing the cycle to a status ritual. Teams mark stages complete because calendar pressure is high, not because uncertainty has been reduced. Gaia’s cycle model exists to prevent that behavior. Each stage should answer different questions:

- Planning (define): are we solving a real problem with measurable outcomes?
- Exploration (shape): is the design feasible and bounded with known risks?
- Development (build): is the solution implemented with operational quality?
- Evaluation: does observed behavior satisfy release criteria?

If a stage cannot answer its core question with evidence, the stage is not complete, even if a slide deck is ready.

### 2) Define stage contracts before creating activities

Teams often begin by creating tasks immediately. That creates activity volume without shared completion logic. Start instead with stage contracts:

- Entry criteria: what must exist before work starts.
- In-stage evidence: what artifacts prove progress.
- Exit gate: what minimum evidence closes the stage.

Only after these contracts exist should activities be initialized. Gaia supports this workflow because stage tabs and activity/evidence panels naturally separate planning intent from execution details.

### 3) Use activities as execution units and evidence as validation units

An activity is work someone does. Evidence is what reviewers inspect. Keep that separation strict.

Example pattern:

- Activity: "Define support escalation routing policy"
- Evidence: policy draft, stakeholder approvals, edge-case test notes

When teams blend work notes and proof into one blob, review quality collapses. Gaia’s evidence statuses (`to do`, `in progress`, `done`, `cancelled`) let you track proof maturity independently from activity effort. Use that capability aggressively.

### 4) Make ownership explicit at cycle and role level

Delivery stalls are usually ownership stalls in disguise. A stage can appear active while no one is accountable for moving evidence to done. Use cycle Roles to assign:

- One accountable owner per role.
- Backup collaborators for continuity.
- Escalation path when evidence is blocked.

Read-only role visibility on activities keeps assignments coherent. If ownership differs across activities for the same role without reason, that is usually a sign of hidden scope fragmentation.

### 5) Keep Discussions linked to execution, not parallel to it

Decision quality degrades when discussions and execution diverge. Delivery Discussions should not be a social inbox; it should be the project-scoped conversational source of planning evidence and decision rationale after exploratory or cross-project discussion has converged.

Use a simple rule:

- If a platform discussion produces project work, restate the accepted outcome in Delivery Discussions before updating cycle artifacts.
- If a discussion changes scope, constraints, or acceptance criteria, create/update evidence in the stage before closing the topic.

This keeps Decisions -> Artifacts -> Tasks traceable and avoids retrospective guesswork about why the team moved in a specific direction.

### 6) Sequence with Timeline after stage intent is stable

Timeline is powerful and easy to misuse early. Scheduling unstable work creates constant reshuffling and false urgency signals. Sequence tasks on Timeline after:

- Stage contracts are defined.
- Initial activities exist.
- Dependencies are known enough to enforce finish-to-start behavior.

Then use Timeline to rebalance load, not to invent strategy. A reliable delivery cycle flows from intent to sequence, not the reverse.

### 7) Use import and intake to accelerate drafting, not to bypass review

Gaia supports imports from unstructured docs and assistant-driven intake for Plan artifacts. These features reduce drafting cost but do not replace judgment.

Good use:

- Bootstrap first-pass artifacts.
- Identify missing fields quickly.
- Normalize structure before team review.

Bad use:

- Treating generated/imported text as approved truth.
- Closing activities because sections are filled, not validated.

Always apply explicit review and mark unresolved fields before stage exit.

### 8) Instrument stage transitions with evidence thresholds

Stage transitions should be auditable decisions. Define quantitative thresholds where possible:

- Minimum number of required evidence items in `done`.
- No unresolved blocker tagged as high risk.
- Named approvers recorded for high-impact artifacts.

This reduces subjective interpretation during pressure windows. Teams can still apply discretion, but they do so with visible tradeoffs.

### 9) Manage cycle health as a leading indicator

Cycle status (`active`, `completed`, `archived`) is a lagging signal. Leading signals are:

- Evidence aging (items stuck in `in progress`).
- Frequent reopening of completed stages.
- Dependency chains with repeated slips.
- High volume of unassigned or ownerless tasks.

Monitor these early. By the time schedule slippage is visible to stakeholders, recovery options are already constrained.

### 10) Definition of done for Delivery Process Cycle

A cycle is operationally complete when:

- Each stage has documented entry/exit logic and corresponding evidence.
- Activities are mapped to accountable owners and realistic sequencing.
- Decision rationale from Discussions is reflected in artifacts or task changes.
- Evaluation findings are represented as release recommendations, not just completion marks.
- The cycle can be reviewed by a new engineer without oral context gaps.

### 11) Gaia should operate the cycle, not bypass it

When Gaia is asked to execute a high-level build request, treat Gaia as the orchestrator, not as an isolated drafting assistant.

Use this operating pattern:

- **Plan:** Gaia creates or resumes the delivery cycle, anchors the brief in **Planning (define)**, records assumptions and intended outputs as evidence, and opens a discussion topic only when decisions still need resolution.
- **Execute:** Gaia decomposes the plan into tasks and milestones, then creates or updates the actual platform resources directly across data model, workflows, UI layouts, document folders, artifacts, and governance records.
- **Verify:** Gaia runs the relevant evals, workflow or run checks, browser checks, folder indexing checks, and evidence updates before it presents a readiness recommendation.

This keeps the conversation as the command channel while the project truth remains in Gaia resources:

- delivery cycle and stages for lifecycle state
- tasks and milestones for execution tracking
- evidence for plans, outputs, and review artifacts
- evals, runs, and governance evidence links for verification

Do not introduce a separate "build request" record unless the delivery-backed model proves insufficient for replay, queueing, or portfolio reporting. In the normal case, the cycle is the control loop.

### 12) Specialist delegation should stay shallow

Gaia can coordinate specialist execution later, but do not start with an uncontrolled sub-agent swarm.

Use specialist delegation only when the boundary is stable and repeated, for example:

- data model and workflow implementation
- UI layout and artifact implementation
- governance and evidence preparation
- eval and release verification

Even then, Gaia remains the owner of lifecycle state, synthesis, and final readiness. Specialists should execute bounded slices; they should not invent parallel control loops.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are leading the "Omnichannel Support Assistant Pilot" in Gaia. Leadership needs a pilot recommendation in three weeks, and risk tolerance is medium: delivery speed matters, but auditability and safe rollout matter more than headline velocity. Your goal is to run one full delivery cycle with clear stage contracts, evidence discipline, and traceable decisions.

### Phase A: Create cycle and define stage contracts

1. Open **Tools -> Delivery Management -> Process**.
2. Create a new cycle named `Q2 Support Assistant Pilot` with a short description including target users, primary KPI, and release window.
3. Open the cycle and visit each stage tab.
4. For every stage, write a compact contract note in your working doc:
   - Entry criteria
   - Evidence required
   - Exit gate
5. Start **Planning (define)** stage.

Checkpoint:

- Cycle exists with clear name/description.
- Planning stage is `in_progress`.
- A stage-contract matrix exists as a reviewable artifact.

### Phase B: Seed activities and assign roles

1. In Planning, click **Initialize from playbook**.
2. Review seeded activities and remove anything irrelevant to pilot scope.
3. Add one custom activity: `Define pilot support boundaries and escalation policy`.
4. Open **Roles** and assign accountable owner + backup collaborator for each required role.
5. Confirm activity panels reflect role assignments (read-only visibility).

Checkpoint:

- Planning activities reflect actual pilot scope.
- No required role is unassigned.
- Custom activity is visible and owned.

### Phase C: Build evidence baseline and discuss linkage

1. For two high-impact Planning activities, add evidence placeholders (e.g., KPI baseline sheet, stakeholder approval note, risk register draft).
2. Set evidence statuses intentionally (`to do` for missing, `in progress` for active drafting).
3. Open **Delivery Management -> Discussions** and create a topic `Pilot scope confirmation and exclusions`.
4. Capture decisions and unresolved questions in the topic.
5. Return to cycle evidence and add links or references to the discussion decision.

Checkpoint:

- At least three Planning evidence items exist.
- A discussion decision is linked to a stage artifact.
- Unresolved fields are explicitly marked rather than hidden.

### Phase D: Convert validated intent into executable tasks

1. Open **Tasks** from Delivery Management.
2. Create at least six tasks mapped to the cycle and relevant Planning/Exploration activities.
3. Assign owners and planned dates.
4. Add at least two finish-to-start dependencies for known sequencing constraints.
5. Open **Timeline** and verify tasks render under expected stage/activity rows.
6. Adjust one activity window by dragging the activity bar to rebalance schedule.

Checkpoint:

- Tasks appear in cycle context and on Timeline.
- Dependency arrows are visible.
- One scheduling adjustment is persisted.

### Phase E: Advance stages with explicit exit gates

1. Return to **Planning** stage.
2. Move required evidence to `done` or `cancelled` with rationale.
3. Confirm activities can be completed only when evidence gate is satisfied.
4. Complete Planning stage.
5. Start **Exploration (shape)** and create first evidence items from design-risk investigation.

Checkpoint:

- Planning transitioned to `completed` with explicit evidence coverage.
- Exploration transitioned to `in_progress`.
- Stage progression reflects policy, not calendar-only timing.

### Phase F: Produce review package and go/no-go recommendation

1. Prepare a short cycle review note named `chapter-09-section-01-cycle-review.md` including:
   - Stage status summary
   - Evidence coverage by stage
   - Top 3 open risks
   - Recommended next action and owner
2. Add links to cycle artifacts, key tasks, and discuss topics.
3. Record go/no-go recommendation for entering Development at current scope.

Checkpoint:

- Review note is shareable without verbal handoff.
- Recommendation is explicit with rationale and risk visibility.

## Expected outputs

- A live delivery cycle with all four stages present and at least Planning + Exploration actively managed.
- A documented stage-contract matrix proving stage intent was defined before execution scaling.
- Activities seeded and curated to scope, with at least one custom activity added for project-specific needs.
- Role assignments complete for required execution roles, with backups where continuity is needed.
- Evidence set with clear status progression (`to do` -> `in progress` -> `done`/`cancelled`) and at least one Discussions-linked decision.
- Task plan connected to the cycle with planned windows and dependencies visible in Timeline.
- A cycle review artifact (`chapter-09-section-01-cycle-review.md`) containing evidence coverage, risks, and a go/no-go recommendation.

Evidence quality standard:

- Another engineer can inspect your artifacts and determine exactly why the team can, or cannot, advance to the next stage.

## Failure modes

- **Cycle created without stage contracts**
  Symptom: many tasks exist, but reviewers disagree on what stage completion means.
  Recovery: pause task expansion, define entry/evidence/exit contracts per stage, then re-baseline activities.

- **Activities completed with stale or missing evidence**
  Symptom: stage appears green but evidence items remain `to do` or ambiguous.
  Recovery: reopen affected activities, enforce evidence gate, and document exceptions explicitly.

- **Discussion decisions not reflected in artifacts**
  Symptom: conversation threads mention scope changes, but tasks/evidence still reference old assumptions.
  Recovery: create a decision-to-artifact sync pass each week and track unresolved discussion outcomes.

- **Timeline churn from premature scheduling**
  Symptom: frequent drag-and-drop changes and dependency cascades before Planning is stable.
  Recovery: freeze schedule edits until stage contracts and core activity definitions are validated.

- **Hidden ownership gaps**
  Symptom: activities remain active with no accountable owner driving closure.
  Recovery: audit roles at cycle level, assign clear primary owners, and set escalation SLA for blocked items.

- **Stage completion by calendar pressure**
  Symptom: stage is marked complete near deadline without evidence sufficiency.
  Recovery: require gate review with explicit exception log; if exceptions exceed tolerance, keep stage open and renegotiate scope.

## Completion checklist

- [ ] I can explain stage entry, evidence, and exit contracts for all four phases without ambiguity.
- [ ] Planning stage includes scoped activities that match initiative intent (not generic template overflow).
- [ ] Required cycle roles are assigned with a clear accountable owner per role.
- [ ] Evidence exists for high-impact activities and status values reflect real maturity.
- [ ] At least one discussion decision is linked to cycle evidence or task changes.
- [ ] Tasks are linked to the cycle and rendered correctly in Timeline.
- [ ] Dependency links represent real sequencing constraints and are not speculative.
- [ ] Planning completion was based on evidence gates, not date pressure alone.
- [ ] A review artifact documents risks and a clear go/no-go recommendation.
- [ ] A new team member could continue execution from artifacts alone.

## Canonical references

- [Delivery Management](../../user-guide/delivery/README.md)
- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)

---

# Task And Milestone Management

## Learning objectives

- Build a task system in Gaia that separates execution work from coordination noise while preserving full traceability to delivery outcomes.
- Define and operate milestones as cross-cycle outcome checkpoints instead of decorative timeline markers.
- Apply dependency, scheduling, and ownership rules that improve predictability without over-constraining adaptive teams.
- Use project views (Tasks, Timeline) and User views (Tasks, Task calendar) to detect workload risk early and rebalance responsibly.

## Prerequisites

- Completion of Section 1 (Delivery Process Cycle) with at least one active cycle.
- Permissions to create/edit tasks and milestones in the target project.
- A working understanding of stage/activity structure in Delivery Process.
- Agreement in your team on basic task state semantics (Backlog, To Do, In Progress, Done, Canceled).

## In Gaia

- [Tasks](../../user-guide/tasks/README.md) in project Delivery Management
- [Tasks](../../user-guide/tasks/all-tasks.md) in the User section
- [Task calendar](../../user-guide/tasks/task-calendar.md) in the User section
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)

Use project-level Tasks and Timeline as the execution truth, Milestones as the outcome checkpoints, and the User section's Tasks plus Task calendar as the personal-capacity views that reveal overload before milestone slips become obvious.

## Concept brief

Task and milestone management is where strategy becomes day-to-day execution behavior. Most delivery failures are not caused by missing intelligence; they are caused by weak operational contracts around planning granularity, sequencing, ownership, and checkpoint discipline. Gaia gives you multiple connected surfaces for this: the project Tasks board/list, Timeline, Milestones, and the User section's Tasks and Task calendar. The value appears only when you use these views as one system.

Operational intake rule: use Platform Discussions for broad coordination and Delivery Discussions for project-scoped commitment. Do not schedule raw discussion ideas directly onto the task board; normalize them into accepted delivery work first.

### 1) Define tasks as commitments, not reminders

A task should represent a commitment with owner, outcome, and planning window. If a task is only a reminder, it becomes scheduling noise. Use a minimal task quality bar:

- Clear title with action + object.
- One accountable owner (with optional collaborators).
- State reflecting real execution status.
- Planned dates when sequencing matters.
- Link to cycle/stage/activity when it advances delivery outcomes.

Poorly defined tasks inflate velocity metrics while hiding unresolved risk.

### 2) Keep task types purposeful

Gaia supports `Task`, `Issue`, and `Meeting`. Teams often misuse these by encoding everything as generic tasks. Use type intentionally:

- `Task`: planned work item with executable outcome.
- `Issue`: defect, incident, or quality gap requiring triage and fix.
- `Meeting`: time-bound coordination event that may produce follow-up tasks.

This separation improves filtering, reporting, and cognitive clarity. It also prevents meeting logistics from crowding execution queues when `Show Meetings` is off.

### 3) Model milestones as outcomes, not date labels

Milestones should indicate significant state transitions for the project, not arbitrary calendar points. A good milestone answers: "What becomes true when this is reached?"

Outcome-oriented milestone examples:

- "Pilot scope approved by operations and compliance"
- "Beta support routing validated in live sandbox"
- "Release readiness gate passed"

Avoid activity-style milestones like "Build pipeline tasks" because they obscure whether value has actually materialized.

### 4) Preserve cross-cycle value with project-level milestones

In Gaia, milestones are project-level and can anchor work across cycles. This is a feature, not a bug. It prevents checkpoint duplication and makes strategic continuity visible.

Operational implication:

- A cycle can close while milestones remain open.
- Multiple cycles can contribute to one strategic milestone.
- Deleting a milestone unlinks tasks but does not delete them, so execution history remains intact.

Use this model to track long-running outcomes that span quarterly planning boundaries.

### 5) Use dependency links sparingly and honestly

Dependencies should encode true sequencing constraints. Overusing them turns the plan brittle; underusing them hides coupling risk.

Good dependency cases:

- Integration test cannot begin until API contract is finalized.
- Rollout communication cannot publish until release note approval is complete.

Bad dependency cases:

- Two tasks owned by same person but independently executable.
- "Feels related" links with no blocking relationship.

Remember Gaia dependency behavior is finish-to-start and same-cycle scoped. Design dependencies accordingly.

### 6) Sequence visually, validate structurally

Timeline is excellent for visual planning, but visual neatness is not plan quality. Every major adjustment should trigger structural checks:

- Did critical path change?
- Did milestone alignment improve or degrade?
- Did dependency pressure shift risk to a single owner/team?
- Did we introduce unrealistic concurrency?

Treat drag-and-drop as a planning proposal. Validate the implications before accepting it as final.

### 7) Combine team and personal views for workload truth

Project-level Tasks and Timeline show delivery structure. Personal views (Tasks, Task calendar) reveal individual load and scheduling collisions.

For platform-maintainer work, the same rule applies inside the private Platform Tasks queue: assign maintainers explicitly in the platform task dialog and switch to the page-level **My Tasks** view when reviewing only your own platform backlog.

When work is release-scoped, set the task version as early as planning allows. Project tasks should use an existing project version when one exists, but teams can type a future version before a snapshot is created. For platform work, treat the platform milestone as the version boundary and use the milestone filter to review only the work tied to that boundary.

Use both:

- Weekly: project-level flow review (blocked chains, milestone drift).
- Daily: personal load review (upcoming deadlines, overload, unscheduled work).

Without this dual-view discipline, teams optimize local boards while individuals silently accumulate unsustainable queues.

### 8) Keep attachment and discussion context connected to tasks

A task without context links is expensive to review. Gaia lets you attach conversations, workflows, and artifacts to tasks. Use attachments as operational evidence pointers:

- Link exploratory conversations to design tasks.
- Link evaluation run reports to fix tasks.
- Link policy approvals to rollout tasks.

For platform-level work, attach the originating Platform Discussion to the Platform Task rather than copying the context into a separate note. Maintainers can reopen the full thread directly from the task, and reporters can see the linked topic move from open to in progress to in review to closed as the task state changes. Use the platform-only **In Review** state when a production PR exists but rollout is not yet shipped. When the work is done, the discussion also inherits the task's release version so the resolution stays visible outside the private maintainer queue.

This reduces context-switching and speeds incident-time diagnosis because provenance is already embedded.

### 9) Rebalance continuously, not only in crisis windows

Task systems decay quickly if rebalance happens only after missed dates. Set a fixed cadence:

- Review overdue items and unscheduled tasks.
- Reassess priority and difficulty alignment.
- Adjust milestone target dates only with rationale.
- Remove stale dependencies and add missing ones.

Small continuous corrections outperform large periodic resets and preserve trust in planning signals.

### 10) Definition of done for Task and Milestone Management

Your operating model is mature when:

- Task definitions are consistent and actionable.
- Milestones represent strategic outcomes with clear ownership.
- Dependencies encode real constraints and are maintained.
- Timeline changes are validated for systemic impact.
- Personal and project views agree on workload reality.
- Reviewers can inspect why a milestone is at risk and which tasks drive the risk.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are the delivery lead for `Q2 Support Assistant Pilot`. Planning and early Exploration work are active. Leadership requests a transparent plan showing which tasks drive each strategic milestone and where schedule risk is concentrated. You must establish disciplined task/milestone management in Gaia and produce an inspectable risk view.

### Phase A: Normalize task model and state hygiene

1. Open **Tools -> Delivery Management -> Tasks**.
2. Review all existing cards/rows for the active cycle.
3. Rename unclear titles using action-oriented wording.
4. Correct task type misuse (`Task`, `Issue`, `Meeting`).
5. Confirm each open item has an accountable owner.
6. Set version tags for release-scoped work so version filters can isolate that release.
7. Move stale items out of `In Progress` unless active work is truly ongoing.

Checkpoint:

- Every open task has clear title, type, owner, and meaningful state.
- No ambiguous "misc" tasks remain.

### Phase B: Establish milestone framework

1. Open **Tools -> Delivery Management -> Milestones**.
2. Create 3-5 outcome milestones for the pilot (example: `Pilot Scope Approved`, `Support Flow Validated`, `Go-Live Readiness Gate`).
3. Add target dates and short descriptions describing acceptance intent.
4. Mark status appropriately (`planned` for future checkpoints).

Checkpoint:

- Milestones exist with outcome wording and target dates.
- Team can explain what each milestone means operationally.

### Phase C: Link tasks to milestones and enforce planning windows

1. Return to **Tasks**.
2. For each high-impact task, set planned start/end.
3. Link each task to the most relevant milestone via task dialog.
4. Ensure at least one task chain contributes to each milestone.
5. Flag tasks without planned dates and either schedule or explicitly defer them.

Checkpoint:

- Milestone-task linkage coverage is visible.
- Unscheduled open tasks are minimized or intentionally deferred.

### Phase D: Encode dependency truth and inspect Timeline

1. In task dialogs, add finish-to-start dependencies for real blockers.
2. Open **Timeline** and verify dependency arrows render.
3. Drag one predecessor task later and observe successor push behavior.
4. Revert or adjust dates to restore realistic target windows.
5. Drag milestone marker if needed, but only with documented rationale.

Checkpoint:

- Dependency behavior is observed and understood.
- At least one schedule rebalance is completed with rationale.

### Phase E: Run workload risk review from personal views

1. Open user menu -> **Tasks**.
2. Filter by `Open` and inspect assigned workload density.
3. Open **Task calendar** and review near-term deadline clustering.
4. Identify one overload risk (e.g., multiple deadline collisions for same owner).
5. Return to project Tasks and rebalance assignments or dates.

Checkpoint:

- At least one detected workload risk is mitigated through plan change.
- Personal and project views no longer contradict each other for critical tasks.

### Phase F: Publish milestone risk and execution report

1. Create `chapter-09-section-02-task-milestone-report.md` with:
   - Milestone list and current status
   - Task coverage per milestone
   - Critical dependency chains
   - Overdue or high-risk items
   - Proposed next-week focus
2. Include links to relevant Timeline view snapshots or task IDs.
3. Add go/no-go recommendation on whether current plan supports milestone dates.

Checkpoint:

- Report supports stakeholder review without additional dashboard narration.
- Risk and recommendation are explicit.

## Expected outputs

- A cleaned task backlog with actionable naming, valid types, owner assignment, and accurate state usage.
- A project-level milestone set representing strategic outcomes rather than activity bundles.
- Task-to-milestone linkage that reveals how execution work contributes to delivery checkpoints.
- Planned windows and dependency chains visible on Timeline with at least one validated rebalance action.
- Personal workload verification via Tasks and Task calendar, with at least one concrete mitigation applied.
- A written report (`chapter-09-section-02-task-milestone-report.md`) containing milestone health, dependency risk, and a clear go/no-go recommendation.

Evidence quality standard:

- A reviewer can identify critical path pressure and milestone risk directly from artifacts, without interviewing task owners.

## Failure modes

- **Tasks treated as inbox notes**
  Symptom: titles are vague, ownership is unclear, and state does not match reality.
  Recovery: apply a task quality pass before adding new work; reject non-actionable entries.

- **Milestones defined as activity buckets**
  Symptom: milestone completion does not correspond to meaningful business outcomes.
  Recovery: rewrite milestones in outcome language and remap linked tasks.

- **Dependency over-encoding**
  Symptom: small changes trigger large cascade delays with little practical value.
  Recovery: keep only hard sequencing constraints; remove soft or speculative links.

- **Scheduling without capacity awareness**
  Symptom: Timeline looks balanced, but individual contributors have impossible deadline clusters.
  Recovery: cross-check with Tasks and Task calendar each planning cycle.

- **Milestone date changes without rationale**
  Symptom: target dates drift repeatedly and stakeholder trust erodes.
  Recovery: require written cause and impact note for every milestone date adjustment.

- **Issue work hidden in generic task flow**
  Symptom: quality defects are mixed into delivery tasks and lose priority visibility.
  Recovery: use `Issue` type with explicit triage/state flow and milestone linkage when release-critical.

## Completion checklist

- [ ] All open tasks in scope meet minimum quality bar (title, owner, state, context).
- [ ] Task types are used intentionally; meetings and issues are not buried as generic tasks.
- [ ] Milestones represent outcomes with understandable acceptance intent.
- [ ] Every key milestone has linked task chains that explain how it will be achieved.
- [ ] Planned dates exist for critical tasks and unscheduled items are explicitly deferred.
- [ ] Dependencies are present only for real blockers and verified on Timeline.
- [ ] At least one rebalancing action was made from observed dependency or capacity risk.
- [ ] Personal views (Tasks, Task calendar) were used to detect and resolve workload conflicts.
- [ ] Milestone health report includes risks, mitigations, and a go/no-go recommendation.
- [ ] Stakeholders can audit plan credibility from artifacts alone.

## Canonical references

- [Tasks](../../user-guide/tasks/README.md)
- [Tasks](../../user-guide/tasks/all-tasks.md)
- [Task calendar](../../user-guide/tasks/task-calendar.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Delivery Management](../../user-guide/delivery/README.md)

---

# Change Management

## Learning objectives

- Build a practical change-management workflow in Gaia that balances delivery velocity with risk control.
- Classify requested changes by impact and urgency, then route them through the correct decision path.
- Connect change decisions to cycle artifacts, tasks, and evaluation evidence so rationale remains auditable.
- Operate a repeatable communication and follow-through loop for accepted, deferred, and rejected changes.

## Prerequisites

- Completion of Sections 1 and 2 in this chapter with an active cycle, managed tasks, and defined milestones.
- Team agreement on who can approve scope, schedule, and risk changes.
- Access to Delivery Discussions, Process Cycle, Tasks, Timeline, and Evals runs/reports.
- A baseline plan artifact that describes current commitments and release assumptions.

## In Gaia

- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Project Versions and Branches](../../user-guide/delivery/versions.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Evals](../../user-guide/evals/README.md)

Use Platform Discussions for early cross-project context when needed, but use Delivery Discussions for the project-scoped intake and decision record. Use Delivery Versions to capture baselines before significant changes, branch risky work into a live review workspace, and merge only after Gaia reports a clean preview. Then synchronize accepted outcomes into Process Cycle evidence, Tasks, Milestones, and Timeline within one working day so the plan stays coherent.

## Concept brief

Change is inevitable. Undisciplined change is optional. Most delivery organizations do not fail because changes exist; they fail because change requests are handled inconsistently. Some requests are implemented instantly without impact analysis. Others are delayed with unclear ownership. The result is drift: schedule drift, scope drift, quality drift, and trust drift.

In Gaia, change management should be treated as an explicit operating loop:

1. capture request,
2. classify impact,
3. analyze implications,
4. decide,
5. implement,
6. verify,
7. communicate closure.

### 1) Separate change intake from change approval

Teams often combine intake and approval in one conversation thread. That creates bias: the first persuasive voice effectively approves the change before impact is known.

Use a two-step contract:

- Intake: record what is requested and why.
- Approval: decide after impact evidence is assembled.

Delivery Discussions are ideal for project-scoped intake and context gathering after broader coordination is complete. Delivery Process artifacts and task plans are where approval implications must be materialized.

### 2) Classify changes by impact surface, not by who asked

A request from leadership can still be low impact; a small user complaint can trigger high-risk architectural effects. Use impact surfaces to classify changes:

- Scope impact: introduces/removes outcomes.
- Schedule impact: shifts critical path or milestones.
- Quality impact: affects safety, correctness, reliability, or supportability.
- Governance impact: changes policy, access, or compliance posture.

Classification quality determines routing quality. If the impact label is wrong, every downstream decision is weaker.

### 3) Define standard change categories with response SLAs

Without category standards, teams improvise each request and lose predictability. Define categories such as:

- `Minor`: local adjustment, no milestone movement expected.
- `Major`: cross-activity effects, likely rescheduling required.
- `Critical`: release-go/no-go or policy-risk implications.

Pair categories with response SLAs:

- Intake acknowledgment time.
- Impact-analysis completion window.
- Decision meeting deadline.

This prevents invisible queue growth where change requests sit unowned.

### 4) Use structured impact analysis, not opinion summaries

Change meetings often devolve into preference debates. Replace this with a small structured analysis template:

- Requested change statement.
- Problem/opportunity evidence.
- Impact on cycle stages and active activities.
- Affected tasks and dependencies.
- Milestone and release implications.
- Validation plan (evals, manual checks, monitoring).
- Recommendation (accept, defer, reject) with rationale.

The point is not bureaucracy. The point is comparable decisions across requests.

### 5) Keep a visible decision ledger

Teams remember accepted changes and forget deferred/rejected ones. That leads to duplicate debates and recurring conflict. Maintain a decision ledger (artifact or shared doc) with:

- Change ID/title.
- Decision and date.
- Decider(s).
- Rationale summary.
- Follow-up actions and owner.

Link each entry to the relevant discussion topic, tasks, and cycle artifacts. This creates institutional memory and shortens future intake cycles.

### 6) Synchronize change outcomes into execution within 24 hours

A change decision is incomplete until execution surfaces are updated. Common anti-pattern: decision made on Monday, tasks updated next week. During that gap, the team runs contradictory plans.

Apply a synchronization rule:

- Within one working day, update affected tasks, dependencies, milestones, and stage evidence.

If the sync cannot happen, mark the decision `pending implementation` and escalate; do not label it as complete.

### 7) Re-validate quality gates after major changes

Significant change requests invalidate prior readiness assumptions. Re-run relevant evals and checks after implementing major/critical changes. Use Evals Runs and Reports for comparability across baseline vs updated configuration.

Ask explicitly:

- Did change improve target behavior?
- Did it regress adjacent behaviors?
- Is reliability still acceptable for release gate?

Change acceptance without re-validation is just optimistic speculation.

### 8) Use versions and branches for reversible review

For major or risky changes, create a project version before work starts. Treat that version as the baseline for impact review.

When the team is not ready to apply changes directly to the current project, create a branch from the baseline version. The branch is a live Gaia workspace where prompts, tools, data-model records, delivery artifacts, and other project resources can be changed through normal pages without affecting the source project.

Before merging, preview the branch. Gaia compares:

- the baseline version,
- the current source project,
- the branch workspace.

If both source and branch changed the same record after the baseline, the merge is blocked. Resolve that decision explicitly before trying again. If the preview is clean, merge the branch and keep the generated change report with the release or change record.

### 9) Communicate tradeoffs with precision

Stakeholders usually tolerate difficult tradeoffs when rationale is clear. They resist when communication is vague.

Use communication format:

- What changed.
- Why now.
- What moved (scope/date/risk/quality).
- What did not move.
- What evidence supports this decision.
- What happens next and who owns it.

This keeps collaboration mature, especially during high-pressure windows.

### 10) Distinguish reversible and irreversible changes

Some changes are easy to undo (UI copy, non-breaking prompt tuning). Others are costly or risky to reverse (data-model changes, policy relaxations, external dependency commitments).

Tag change requests as reversible/irreversible early. For irreversible changes, raise approval bar and evidence requirements. This prevents accidental lock-in driven by short-term pressure.

### 11) Definition of done for Change Management

Your change-management process is healthy when:

- Requests are captured consistently with impact classification.
- Decision logic is documented and inspectable.
- Accepted changes are synchronized into tasks/timeline/milestones quickly.
- Major changes have a version baseline, branch preview, or merge report when direct editing would create avoidable risk.
- Major changes trigger re-validation through eval and operational checks.
- Stakeholders can trace any delivery plan shift to a specific documented decision.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

Midway through `Q2 Support Assistant Pilot`, operations requests expansion to include weekend escalation handling. Product leadership also asks for earlier beta exposure. Security warns that expanded coverage may increase policy risk. You must process these changes without collapsing delivery predictability.

### Phase A: Capture intake and classify impact

1. Open **Delivery Management -> Discussions**.
2. Create topic `Change request: weekend escalation scope + beta timing`.
3. Record request details:
   - Requested outcomes
   - Source stakeholders
   - Desired timing
   - Assumed benefits
4. Classify impact surfaces (scope, schedule, quality, governance) directly in the topic or linked note.
5. Assign intake owner and decision deadline.

Checkpoint:

- Intake record is complete and attributable.
- Impact surfaces are identified before approval discussion.

### Phase B: Build structured impact analysis artifact

1. Create `chapter-09-section-03-change-analysis.md`.
2. Fill sections:
   - Problem/opportunity evidence
   - Affected cycle stages and activities
   - Tasks/dependencies impacted
   - Milestone movement candidates
   - Quality and policy implications
   - Validation plan
   - Recommendation with options
3. Link relevant existing tasks and artifacts.

Checkpoint:

- Analysis artifact is comparable and reviewable.
- At least two viable options are documented (not just one preferred path).

### Phase C: Run decision review and record outcome

1. Conduct a decision review with designated approvers.
2. Choose one decision:
   - Accept now
   - Defer with trigger condition
   - Reject with rationale
3. Log outcome in a decision ledger entry (file or artifact), including decider names and date.
4. Update the discussion topic with final decision summary and links.

Checkpoint:

- Decision is explicit with rationale.
- Deferred/rejected requests are also recorded (not dropped).

### Phase D: Synchronize accepted changes into execution surfaces

1. If accepted, update affected tasks in **Tasks**:
   - Add/remove tasks
   - Reassign owners
   - Adjust planned windows
2. Update dependencies where sequencing changed.
3. In **Milestones**, adjust target dates only where impact requires it and document reason.
4. Open **Timeline** and validate schedule consistency.
5. In **Process Cycle**, add/update evidence notes reflecting the change decision and implementation path.

Checkpoint:

- Plan surfaces reflect the decision within one working day.
- No conflicting old assumptions remain in visible execution artifacts.

### Phase E: Re-validate behavior and risk

1. Open **Evals -> Runs** and re-run relevant evaluation set(s) impacted by the change.
2. Inspect **Run details** for completion and outcome distribution.
3. Open **Reports** and compare baseline vs post-change reliability metrics.
4. If results regress beyond tolerance, create corrective tasks and adjust recommendation.
5. Capture findings in `chapter-09-section-03-change-validation.md`.

Checkpoint:

- Change impact is measured, not inferred.
- Recommendation reflects observed outcomes and residual risk.

### Phase F: Communicate closure and next-step commitments

1. Publish a concise change closure summary in Discussions and project update channels:
   - Decision
   - Plan changes
   - Risk posture
   - Required follow-up actions
2. Confirm owners and due dates for follow-up tasks.
3. Record final go/no-go recommendation for continuing toward release gate.

Checkpoint:

- Stakeholders can see what changed and why.
- Follow-up accountability is explicit.

## Expected outputs

- A complete change intake record with impact classification and assigned owner.
- A structured impact analysis artifact (`chapter-09-section-03-change-analysis.md`) covering scope, schedule, quality, and governance implications.
- A decision ledger entry documenting accept/defer/reject outcome, deciders, rationale, and follow-up ownership.
- Synchronized updates across Tasks, Milestones, Timeline, and Process Cycle evidence for accepted changes.
- Post-change validation evidence from Evals Runs and Reports, captured in `chapter-09-section-03-change-validation.md`.
- A stakeholder-facing closure summary with clear next-step commitments and a go/no-go recommendation.

Evidence quality standard:

- A reviewer can reconstruct the full lifecycle of the change request (intake -> decision -> execution sync -> validation -> closure) without missing links.

## Failure modes

- **Approval before analysis**
  Symptom: change is "greenlit" in conversation, then implementation discovers hidden impact.
  Recovery: enforce intake/approval separation and block execution changes until analysis artifact exists.

- **Impact under-classification**
  Symptom: request labeled minor but later shifts milestones and quality gates.
  Recovery: require impact-surface checklist and second reviewer for major-risk domains.

- **Decision without synchronization**
  Symptom: tasks and timeline still reflect old plan days after decision.
  Recovery: apply 24-hour sync SLA and track pending-implementation decisions separately.

- **No re-validation after major change**
  Symptom: release proceeds on stale evaluation evidence.
  Recovery: re-run affected eval suites and update release recommendation before next gate.

- **Deferred changes disappear from memory**
  Symptom: same request resurfaces repeatedly with no history.
  Recovery: maintain decision ledger with trigger conditions and review dates.

- **Communication ambiguity**
  Symptom: different teams interpret the same decision differently.
  Recovery: publish standardized closure summary with explicit "what changed / what did not" statements.

## Completion checklist

- [ ] Every change request in scope has a documented intake record.
- [ ] Impact classification covers scope, schedule, quality, and governance surfaces.
- [ ] Structured analysis artifact exists and includes options plus recommendation.
- [ ] Decision outcome is recorded with named deciders and rationale.
- [ ] Accepted changes are synchronized into tasks/timeline/milestones/process evidence promptly.
- [ ] Major or critical changes trigger re-validation in Evals.
- [ ] Baseline vs post-change results are compared and documented.
- [ ] Closure communication is published with clear owner commitments.
- [ ] Deferred/rejected requests are traceable in a decision ledger.
- [ ] Final go/no-go recommendation reflects validated data, not assumption.

## Canonical references

- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Eval Runs](../../user-guide/evals/runs.md)
- [Run details](../../user-guide/evals/run-details.md)
- [Reports](../../user-guide/evals/reports.md)

---

# Release Readiness Checklist

## Learning objectives

- Define release readiness as a multi-signal decision (scope, quality, risk, operability, communication), not a binary deadline event.
- Build and run a practical Gaia release gate that is evidence-driven and repeatable across cycles.
- Produce a decision package that supports confident go/no-go calls under time pressure.
- Establish post-release accountability: monitoring, follow-up tasks, and feedback loops for continuous improvement.

## Prerequisites

- Completion of Chapter 9 Sections 1-3 with an active delivery cycle, governed tasks, and documented change decisions.
- A target release candidate with scoped milestones and execution status visible in Tasks/Timeline.
- Access to Evals runs and reports for quality evidence.
- Agreement on release authorities (who recommends, who approves, who executes, who monitors).

## In Gaia

- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Evals](../../user-guide/evals/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Governance Agent Systems](../../user-guide/governance/agent-systems.md)
- [Governance Overview](../../user-guide/governance/overview.md)

Use Process Cycle for stage evidence, Tasks, Milestones, and Timeline for current execution truth, Evals for comparative quality evidence, Governance Agent Systems for runtime posture verification and Decision BOM evidence, and Delivery Discussions for the final go or no-go rationale and any constraint conditions that must remain visible after the meeting. Use Platform Discussions only when the decision also requires broader coordination outside the project thread.

## Concept brief

Release readiness is where delivery discipline is tested. Teams often arrive at release week with fragmented evidence: schedule says "almost done," tests say "mostly fine," and stakeholders hear mixed narratives. That creates last-minute conflict and unsafe pressure.

A mature release process does not try to eliminate uncertainty. It makes uncertainty visible, bounded, and actionable. Gaia gives you the surfaces to do this if you combine them correctly: Delivery Process for stage evidence, Tasks and Timeline for execution truth, Milestones for checkpoint status, Discussions for decision rationale, and Evals for comparative quality signals.

### 1) Replace "are we done?" with "is risk acceptable?"

Readiness is a risk decision, not a completion percentage. A candidate can be 95% feature-complete and still be unready if critical failure modes are unmitigated.

Use decision language:

- Which risks remain?
- What is their impact and likelihood?
- What controls or mitigations are active?
- Is residual risk within agreed tolerance?

This reframes release conversations from optimism contests to accountable risk management.

### 2) Define readiness dimensions explicitly

Avoid vague criteria like "quality looks good." Define clear dimensions:

- Scope integrity: committed outcomes are delivered or formally deferred.
- Functional quality: core use cases pass acceptance checks.
- Safety/governance: policy boundaries are met.
- Operational readiness: owners, runbooks, and support paths are prepared.
- Communication readiness: stakeholders and users know what is changing.

Every go/no-go argument should map to one or more of these dimensions.

### 3) Use evidence tiers to prevent false confidence

Not all evidence has equal strength. Use tiers:

- Tier 1: direct observed results (eval runs, live validation outcomes).
- Tier 2: reviewed artifacts (checklists, approvals, analysis notes).
- Tier 3: assumptions or projections.

Release decisions should be mostly Tier 1 + Tier 2. If a critical gate depends on Tier 3 assumptions, call it out and define mitigation before approval.

### 4) Gate by unresolved critical items, not total open items

Teams get trapped chasing zero-open-task fantasies. A healthier rule:

- Open work is acceptable if it is low-impact and does not undermine launch safety or core outcomes.
- Any unresolved critical item blocks release until mitigated, deferred with explicit acceptance, or removed from scope.

This keeps gates strict where necessary and pragmatic where possible.

### 5) Validate readiness against baseline and trend

A single "green" report can be misleading. Evaluate both current state and trend direction:

- Are key metrics better, stable, or deteriorating vs baseline?
- Did recent changes increase variance or failure concentration?
- Are improvements consistent across critical scenarios?

Evals Reports are most useful when they compare multiple runs with consistent naming and scope.

### 6) Make rollback and contingency a first-class gate

Many teams define rollout plans but under-specify rollback plans. A candidate is not ready if rollback ownership, trigger criteria, and execution steps are unclear.

Minimum rollback contract:

- Trigger thresholds for rollback.
- Decision authority for rollback call.
- Technical and operational rollback steps.
- Communication template for internal and external stakeholders.

Documenting this lowers anxiety and improves decision quality under incident pressure.

### 7) Synchronize release gate with timeline reality

Readiness meetings often use static slides while the actual schedule shifted yesterday. Always reconcile gate evidence with current Tasks/Timeline state.

Before gate review:

- Refresh Timeline.
- Confirm critical-path tasks and dependencies are current.
- Verify milestone status and overdue markers.
- Ensure no hidden schedule drift exists in personal workload views.

A readiness pack that ignores current execution state is a liability.

### 8) Use structured go/no-go recommendations

Recommendations should be concise and complete:

- Recommendation: `Go`, `Go with constraints`, or `No-go`.
- Rationale by readiness dimension.
- Conditions/mitigations required.
- Owners and deadlines for pre/post actions.

`Go with constraints` is often the right middle ground when residual risk is acceptable only with explicit safeguards.

### 9) Treat post-release feedback as readiness debt repayment

Readiness does not end at launch. Feedback from conversations and early usage closes the loop and pays down hidden readiness debt.

Immediately post-release:

- Track high-signal feedback.
- Route actionable items to tasks/issues.
- Re-run targeted evals if behavior drift is detected.
- Update cycle evidence with lessons learned.

This turns release from one event into a learning system.

### 10) Definition of done for Release Readiness Checklist

Your release gate is mature when:

- All readiness dimensions have explicit evidence.
- Critical unresolved items are either mitigated or formally accepted with accountable owners.
- Rollback plan is tested at tabletop level and executable.
- Go/no-go recommendation is traceable to artifacts and metrics.
- Post-release follow-up work is planned, owned, and visible in Gaia.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

The `Q2 Support Assistant Pilot` is approaching beta launch. Leadership wants a release recommendation by Friday. You need to run a full readiness gate using Gaia artifacts, quality evidence, and risk controls, then publish a decision package that can withstand technical and governance scrutiny.

### Phase A: Assemble readiness dimensions and evidence inventory

1. Create `chapter-09-section-04-readiness-pack.md`.
2. Add sections for each readiness dimension:
   - Scope integrity
   - Functional quality
   - Safety/governance
   - Operational readiness
   - Communication readiness
3. For each dimension, list current evidence and classify by tier (Tier 1/2/3).
4. Add explicit gaps where evidence is insufficient.

Checkpoint:

- Readiness dimensions are explicit.
- Evidence inventory is complete enough to expose unknowns.

### Phase B: Reconcile execution truth before gate decision

1. Open **Tasks** and filter to open critical items.
2. Open **Timeline** and refresh to latest state.
3. Verify critical-path dependencies and overdue highlights.
4. Open **Milestones** and check target-date realism vs current task sequencing.
5. Update readiness pack with execution-state summary.

Checkpoint:

- Gate discussion uses current execution state, not stale snapshots.
- Any critical unresolved item is clearly listed.

### Phase C: Validate quality and regression posture

1. Open **Evals -> Runs** and ensure recent run(s) exist for release candidate.
2. Use **Run details** to verify completion and identify failure clusters.
3. Open **Reports** to compare baseline and current reliability metrics.
4. Capture quality trend summary in readiness pack, including known limitations.
5. Create corrective tasks for any blocking quality findings.

Checkpoint:

- Readiness claims include comparative evidence.
- Blocking quality regressions are converted into owned actions.

### Phase D: Verify governance posture

1. Open **Governance -> Runtime -> Agent Systems** and select the governed boundary for the release candidate.
2. Run **Verify governance posture** and inspect the status, grade, score, and missing-evidence findings.
3. Confirm runtime policy decision history, ACS-aligned checkpoint compatibility posture, standards evidence exports, Agent SRE posture, accepted trust posture, and linked evidence are current enough for the release decision.
4. Export or link ACS policy projection, OCSF-aligned runtime records, Agent BOM, ASSERT-style eval bridge, or A2A readiness evidence when the release or reviewer pack needs standards-aligned files.
5. Generate or link a Decision BOM for any high-risk governed runtime decision that must be reconstructed during review.
6. Add unresolved verification findings to the readiness pack with a decision: remediate, explicitly accept with owner/date, or block release.

Checkpoint:

- Governance posture is supported by a stored verification run, not only by meeting notes.
- High-risk runtime decisions have reconstructible evidence where needed.

### Phase E: Finalize risk, rollback, and communication plan

1. In readiness pack, list top release risks with impact/likelihood and mitigation status.
2. Define rollback triggers, rollback owner, and execution steps.
3. Draft internal/external communication notes for:
   - planned release
   - degraded mode
   - rollback event
4. Post summary in **Delivery Discussions** for stakeholder visibility and feedback.

Checkpoint:

- Rollback is operationally defined, not implied.
- Communication paths are pre-authored.

### Phase F: Conduct go/no-go review

1. Hold release gate review with designated approvers.
2. Present readiness pack and unresolved critical items.
3. Record decision as one of:
   - `Go`
   - `Go with constraints`
   - `No-go`
4. Capture rationale and mandatory pre/post actions with owners and dates.
5. Log decision in cycle evidence and discuss thread.

Checkpoint:

- Decision is explicit, attributable, and artifact-linked.
- Constraint conditions are measurable and time-bound.

### Phase G: Execute follow-through and monitor early signal

1. If release proceeds, create post-release follow-up tasks (monitoring checks, feedback triage, known debt remediation).
2. Open conversation feedback surfaces and ensure team knows escalation path for high-signal issues.
3. Schedule a short post-release review to reconcile outcomes vs readiness assumptions.
4. Produce `chapter-09-section-04-release-decision-summary.md` with:
   - final recommendation and rationale
   - active constraints
   - first-week monitoring focus

Checkpoint:

- Release decision is operationalized with ownership.
- First-week learning loop is in place.

## Expected outputs

- A complete readiness pack (`chapter-09-section-04-readiness-pack.md`) structured by readiness dimension with evidence-tier classification.
- Current execution-state reconciliation covering critical tasks, dependencies, milestones, and schedule drift risk.
- Quality validation summary based on Evals run and report evidence, including baseline comparison.
- Governance verification summary and linked Decision BOMs for high-risk runtime decisions.
- Documented top risks, mitigation status, and rollback/contingency plan with named owners.
- A formal go/no-go decision record with explicit rationale and required constraints/actions.
- A post-release follow-up plan with monitoring, feedback triage, and remediation ownership in Gaia tasks.
- A concise decision summary artifact (`chapter-09-section-04-release-decision-summary.md`) suitable for stakeholder distribution.

Evidence quality standard:

- A reviewer can challenge any readiness claim and find corresponding supporting evidence and ownership in less than five minutes.

## Failure modes

- **Checklist theater**
  Symptom: boxes are checked, but evidence quality is weak or outdated.
  Recovery: require evidence links and freshness checks for each readiness dimension.

- **Critical risk diluted by aggregate metrics**
  Symptom: average success rate looks acceptable while one high-impact scenario consistently fails.
  Recovery: inspect failure clusters and block release on critical-scenario regressions.

- **No rollback ownership**
  Symptom: rollback is conceptually acknowledged but no one is explicitly accountable.
  Recovery: assign rollback authority and execution owner before approving release.

- **Stale execution snapshot in gate meeting**
  Symptom: decision is made from yesterday’s plan while timeline changed today.
  Recovery: enforce same-day tasks/timeline refresh before final vote.

- **Go decision without constraints tracking**
  Symptom: "Go with constraints" is declared, but constraints are not monitored or owned.
  Recovery: convert each constraint into tracked task with due date and owner.

- **Post-release feedback loop missing**
  Symptom: first-week issues surface, but triage path is unclear and learning is lost.
  Recovery: define feedback intake and escalation workflow before launch, not after incidents.

## Completion checklist

- [ ] Readiness dimensions are explicitly defined and used in decision discussion.
- [ ] Evidence inventory is complete and tiered for quality strength.
- [ ] Critical unresolved items are identified and dispositioned.
- [ ] Timeline/milestone/task state is reconciled on the day of gate review.
- [ ] Baseline vs candidate quality evidence is reviewed in Evals.
- [ ] Rollback triggers, owners, and steps are documented and feasible.
- [ ] Communication plan exists for normal, constrained, and rollback paths.
- [ ] Go/no-go recommendation is explicit and rationale-backed.
- [ ] Constraint and follow-up actions are tracked with owners and dates.
- [ ] Post-release monitoring and feedback loop are planned before launch.

## Canonical references

- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Eval Runs](../../user-guide/evals/runs.md)
- [Run details](../../user-guide/evals/run-details.md)
- [Reports](../../user-guide/evals/reports.md)
- [Give feedback on a reply](../../user-guide/conversations/dialogs/feedback.md)

---

# Chapter 10: Production Playbooks

Status: current

Real-world production patterns, release control, and failure recovery inside Gaia

## Sections

- [Support Assistant Playbook](#doc-ch10-production-playbooks-01-support-assistant-playbook)
- [Ops Assistant Playbook](#doc-ch10-production-playbooks-02-ops-assistant-playbook)
- [Knowledge Assistant Playbook](#doc-ch10-production-playbooks-03-knowledge-assistant-playbook)
- [Incident And Rollback Playbook](#doc-ch10-production-playbooks-04-incident-and-rollback-playbook)

## Alignment with User Guide Production Surfaces

- [Dashboard](../../user-guide/dashboard/README.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Timesheet](../../user-guide/timesheet/README.md)
- [Settings](../../user-guide/settings/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Tutorials](../../user-guide/tutorials/README.md)

Treat the production playbooks as a linked platform workflow: detect in Dashboard, diagnose in Workflow Run Details and Audit Trail, coordinate recovery in Tasks and delivery artifacts, confirm capacity in Timesheet, and re-validate behavior through Evals before declaring the system stable again.

Use **Operations: Review, Audit, and Rebalance** in [Tutorials](../../user-guide/tutorials/README.md) at `/platform/support/tutorials` as the guided version of the platform workflow that these playbooks assume.

## Fast path inside Gaia

1. Detect the problem in [Dashboard](../../user-guide/dashboard/README.md).
2. Diagnose it in [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md) and [Audit Trail](../../user-guide/audit/README.md).
3. Coordinate response in [Tasks](../../user-guide/tasks/README.md), supporting delivery artifacts, and [Settings](../../user-guide/settings/README.md) when configuration changes are required.
4. Re-validate the fix in [Evals](../../user-guide/evals/README.md) before declaring the playbook complete.

If the playbook outcome cannot be traced through Gaia evidence, tasks, and validation artifacts, the recovery loop is not yet credible.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Support Assistant Playbook

## Learning objectives

- Design a production-ready support assistant in Gaia with clear boundaries, channel strategy, and escalation behavior.
- Operate a support assistant through a repeatable control loop: configure, validate, launch, monitor, and improve.
- Connect conversations, feedback, eval outcomes, and delivery tasks so support quality improvements are trackable and auditable.
- Build a practical go/no-go framework for support releases that balances customer experience with operational risk.

## Prerequisites

- Completion of Chapters 7-9, especially observability, safety controls, and release-readiness practices.
- Project admin or team admin access to AI Agents, Conversations, Channels, Delivery Management, and Evals.
- A target support scope (for example billing help, onboarding guidance, incident status responses) with named human escalation path.
- Baseline KPIs (first-response usefulness, handoff success, policy adherence, containment rate, and cost-per-conversation).

## In Gaia

- [AI Agents](../../user-guide/agents/README.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)

Use AI Agents and Channels to define the support topology and channel policy, Conversations and Dashboard to inspect real behavior, Evals to validate release candidates, and Tasks plus Delivery Discussions to convert support defects into tracked delivery work.

## Concept brief

A support assistant is not "just another chat agent." It is an operational front door for customer trust. In production, poor support behavior multiplies quickly: one wrong policy answer can propagate to many users, and one missing escalation rule can trap users in frustrating loops. Gaia gives you the controls to avoid this, but only if you operate the system as an integrated playbook rather than disconnected settings.

### 1) Start with support intent and boundaries

Support assistants fail when scope is ambiguous. Define exactly what the assistant should do, what it may do with approval, and what it must never do.

Typical boundary model:

- In scope: account guidance, known troubleshooting flows, status checks, policy explanations.
- Conditional: account-specific updates that require verified identity and tool support.
- Out of scope: legal commitments, irreversible account actions without human confirmation, unsupported regions/products.

Encode this in agent instructions and off-topic/on-topic controls. A clear boundary reduces both hallucination risk and escalation noise.

### 2) Build channel strategy as part of support design

Support behavior differs by channel. Text chat, WhatsApp, email, and voice sessions each have different latency and expectation profiles. Do not deploy one generic configuration everywhere.

Channel-aware choices:

- Text channel: richer responses, attachments, and detailed policy guidance.
- WhatsApp/SMS: concise replies, stronger confirmation prompts, shorter follow-up loops.
- Voice/realtime: explicit summarization and confirmation to avoid ambiguity.

In Gaia, channel configuration is operational policy. App slug, authentication, and feedback availability are support controls, not cosmetic settings.

### 3) Use orchestrator + specialist routing deliberately

A single support assistant can work for small scopes, but production systems usually need specialization. Use an orchestrator to classify intent and hand off to specialist agents (billing, technical support, account admin).

Routing design rules:

- Keep handoff criteria explicit and testable.
- Avoid overlapping specialist scopes where possible.
- Define fallback behavior if no specialist confidently matches.

Handoffs should reduce user friction, not create invisible routing loops. Validate handoff outcomes in real conversations and evals, not only by prompt inspection.

### 4) Treat tools as support responsibilities

Every enabled tool increases capability and risk. Support assistants should enable only what is required for supported workflows.

Tool governance for support:

- Enable read tools broadly (where safe), write tools narrowly.
- Require confirmation language before high-impact writes.
- Return structured results to keep user responses grounded.

When users report confusing outputs, inspect which tools ran and whether tool return formats support clear explanations. Ambiguous tool responses often become ambiguous assistant replies.

### 5) Design escalation as a first-class outcome

Escalation is not failure; bad escalation is failure. Production support assistants need deterministic escalation conditions:

- policy uncertainty,
- repeated user frustration,
- unresolved issue after N turns,
- high-impact account/security concerns,
- explicit user request for human handoff.

Document what information must be captured during escalation (summary, attempted steps, customer context, urgency). This avoids human agents repeating discovery work.

### 6) Use feedback loops at message and conversation level

Gaia supports threaded feedback per assistant message plus review workflows. Use them systematically:

- Message-level feedback to capture exact quality defects.
- Conversation tags/review states to triage classes of failure.
- Weekly pattern extraction from feedback themes.

Without this loop, teams overreact to anecdotal complaints or underreact to recurring subtle issues. Feedback should create structured improvement tasks, not informal chat discussions.

### 7) Instrument support quality with leading and lagging metrics

Support teams often track only lagging indicators like ticket volume. Add leading indicators that predict drift earlier:

- Escalation trigger rate by intent category.
- Tool failure frequency per workflow.
- Conversation timeline latency spikes.
- Repeated clarification turns before resolution.

Pair these with lagging outcomes (CSAT proxies, resolution success, incident counts). In Gaia, combine Dashboard trends with conversation Timeline detail for root-cause visibility.

### 8) Validate behavior continuously with targeted eval sets

Support assistants degrade when product policies, tools, or prompts change. Maintain eval datasets for:

- critical policy scenarios,
- top recurring intents,
- known edge cases and regressions,
- escalation correctness.

Run re-runs after major config or tool changes. Use Reports to compare baseline and candidate behavior. If improvements in one area create regressions in another, keep that tradeoff explicit in release decisions.

### 9) Operationalize change through delivery artifacts

Support improvements must appear in delivery artifacts, not only in agent configs. When behavior issues are identified:

- create/update delivery tasks,
- link tasks to milestones,
- document change rationale in Discussions,
- update cycle evidence.

This keeps support operations aligned with product delivery and avoids silent configuration drift that nobody can audit later.

### 10) Definition of done for Support Assistant Playbook

Your support assistant is production-operable when:

- Scope boundaries and escalation rules are explicit and testable.
- Channel behavior is configured intentionally per user context.
- Routing and tools are constrained to real support needs.
- Feedback and eval loops produce concrete, tracked improvements.
- Daily operations can detect and correct quality drift before customer impact compounds.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are launching a customer support assistant for a B2B SaaS product. The assistant must handle onboarding and billing FAQs, escalate account-sensitive actions to humans, and maintain a stable response quality during the first two weeks of production traffic.

### Phase A: Define service boundary and release KPIs

1. Create a short service charter file named `chapter-10-section-01-support-charter.md`.
2. Define:
   - in-scope intents,
   - conditional intents requiring escalation,
   - out-of-scope intents,
   - critical policy constraints.
3. Add first-week KPIs and thresholds (for example escalation rate, median response latency, eval pass target, unresolved-feedback threshold).
4. Share charter summary in Delivery Discussions.

Checkpoint:

- Charter exists and is reviewable.
- KPIs have numeric thresholds.

### Phase B: Configure channels and support agent topology

1. Open **Conversations -> Channels** and configure or verify the primary support channel (Text at minimum).
2. Ensure authentication and feedback settings align with your support policy.
3. Open **AI Agents** and configure:
   - orchestrator support agent,
   - at least one specialist support agent.
4. Add handoff rules for billing/account intents and fallback route for unknown intents.
5. Save configuration version with explicit version name.

Checkpoint:

- Channel is active and correctly scoped.
- Handoff rules are visible and understandable.

### Phase C: Enable safe tooling and escalation logic

1. In agent configuration **Tools** tab, enable only tools required for support scope.
2. Add or refine prompt instructions for confirmation before high-impact actions.
3. Define escalation criteria in instructions and fallback responses.
4. Run 5-8 manual conversation tests in **Conversations** covering:
   - normal FAQ flow,
   - ambiguous request,
   - policy edge case,
   - explicit user request for human support.
5. Capture findings in `chapter-10-section-01-support-smoke-test.md`.

Checkpoint:

- Unsafe or out-of-scope actions are consistently blocked or escalated.
- Test transcript evidence is captured.

### Phase D: Establish quality loop with feedback and evals

1. In Conversations, submit feedback on at least three assistant replies (mix of good and problematic).
2. Apply review states/tags for triage categories (for example `policy-gap`, `tone`, `tool-error`).
3. In **Evals**, run or re-run a support regression set covering critical intents.
4. Inspect **Run details** and **Reports** for pass-rate changes and failure clusters.
5. Convert the top two findings into delivery tasks and link to milestone.

Checkpoint:

- Feedback items are captured with actionable specificity.
- Eval results are compared against baseline.
- Improvement tasks are tracked in delivery workflow.

### Phase E: Observe live behavior and operating load

1. Open **Dashboard** for last 24h and 7d views.
2. Inspect conversation and error trends.
3. Open conversation **Timeline** for at least three high-latency or problematic turns.
4. Identify root-cause candidates (tool delay, routing loop, unclear prompt instruction).
5. Document operational findings in `chapter-10-section-01-support-operations-log.md`.

Checkpoint:

- At least one root-cause hypothesis is evidence-backed.
- Monitoring cadence and owner are defined.

### Phase F: Publish launch decision and follow-up plan

1. Create `chapter-10-section-01-support-launch-decision.md` including:
   - KPI status vs thresholds,
   - open risks,
   - mitigation tasks,
   - launch recommendation (`go`, `go with constraints`, or `no-go`).
2. Post decision summary in Delivery Discussions.
3. Assign first-week monitoring shifts and escalation owner.

Checkpoint:

- Decision is explicit and auditable.
- Post-launch accountability is assigned.

## Expected outputs

- A documented support charter with scope boundaries, escalation policy, and KPI thresholds.
- Active channel and agent configuration topology with clear orchestrator/specialist roles.
- Tool and escalation guardrails validated through recorded conversation smoke tests.
- A feedback + eval improvement loop that produces tracked delivery tasks.
- Operational monitoring evidence from Dashboard and conversation timelines.
- A launch decision artifact (`chapter-10-section-01-support-launch-decision.md`) with clear risk posture and go/no-go recommendation.

Evidence quality standard:

- Another engineer can read your artifacts and reproduce the same operational decision without private context.

## Failure modes

- **Scope creep through informal prompt edits**
  Symptom: assistant starts answering unsupported questions with inconsistent confidence.
  Recovery: re-baseline scope charter, tighten on-topic/off-topic controls, and run targeted regression evals.

- **Handoff ambiguity between specialists**
  Symptom: conversations bounce between agents or stop with unclear ownership.
  Recovery: simplify handoff criteria, define fallback owner, and test handoff edges explicitly.

- **Tool overexposure**
  Symptom: assistant invokes high-impact tools for low-confidence scenarios.
  Recovery: disable non-essential tools, add confirmation requirements, and enforce escalation-first path for risky operations.

- **Feedback captured but not actioned**
  Symptom: repeated user complaints appear with no corresponding delivery tasks.
  Recovery: require weekly feedback-to-task triage and milestone linkage for top recurring issues.

- **Latency drift hidden by aggregate metrics**
  Symptom: average response looks stable while specific intents suffer long delays.
  Recovery: inspect per-turn timeline traces for high-latency categories and tune tool/prompt paths.

- **Launch decision without constraint tracking**
  Symptom: "go with constraints" decision is made but constraints are not monitored.
  Recovery: convert constraints into owned tasks with due dates and explicit escalation criteria.

## Completion checklist

- [ ] Support scope, escalation conditions, and KPI thresholds are documented and approved.
- [ ] Channels and agent routing reflect intended support experience and governance boundaries.
- [ ] Enabled tools are minimal and aligned to support needs.
- [ ] Smoke tests cover normal, ambiguous, and high-risk support scenarios.
- [ ] Feedback and review tags are used to classify quality defects.
- [ ] Eval regression results are reviewed and compared to baseline.
- [ ] High-priority quality gaps are converted into tracked delivery tasks.
- [ ] Dashboard/timeline monitoring is used to validate production behavior.
- [ ] Launch recommendation is explicit with risk and mitigation visibility.
- [ ] First-week monitoring and escalation ownership are assigned.

## Canonical references

- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Give feedback on a reply](../../user-guide/conversations/dialogs/feedback.md)
- [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Evals](../../user-guide/evals/README.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)

---

# Ops Assistant Playbook

## Learning objectives

- Architect an operations assistant in Gaia that coordinates repetitive internal workflows with reliability and governance.
- Define run-time controls for workflow-triggered actions, including pause/resume/stop handling and failure recovery.
- Connect ops assistant behavior to delivery tasks, milestones, and audit artifacts for transparent execution.
- Operate a weekly improvement loop using workflow run evidence, conversation traces, and regression checks.

## Prerequisites

- Completion of Chapter 10 Section 1 and prior chapters on observability, data modeling, and delivery operations.
- Access to AI Agents, Data Model Workflows/Runs, Conversations, Delivery Management, and Audit Trail.
- At least one existing workflow relevant to operations (for example enrichment, sync, or reporting).
- Agreed incident contacts and change approvers for automation-impacting updates.

## In Gaia

- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)

Use Workflow Run Details as the evidence surface for run-state truth, Dashboard for trend context, Audit Trail for change accountability, and Tasks plus Timeline when remediation work must affect delivery commitments.

## Concept brief

An ops assistant is an execution multiplier for internal teams. It should reduce routine coordination overhead, improve run consistency, and accelerate issue triage. But without discipline, ops assistants can amplify errors just as efficiently as they automate good work. The production playbook is therefore less about "adding more automation" and more about building safe, observable automation loops.

### 1) Design around high-frequency operational decisions

Start with the decisions your team repeats every day or week:

- "Should this workflow run now or wait for a dependency?"
- "Is this run healthy enough to continue?"
- "Which failure requires escalation vs local retry?"
- "What changed since the last successful run?"

If the assistant cannot support concrete recurring decisions, it becomes a generic chatbot that adds little operational value.

### 2) Separate execution orchestration from policy authority

Ops assistants can trigger or guide workflow actions, but policy authority should remain explicit:

- who can approve destructive actions,
- who can override paused runs,
- who can modify workflow definitions.

In Gaia, permissions and project roles are guardrails. Use them intentionally so automation does not bypass organizational accountability.

### 3) Build workflow-aware prompts and tools

Ops prompts should encode run semantics, not only tone and style. Include expectations such as:

- always identify target workflow and run scope,
- summarize current status before proposing actions,
- surface risk when retrying failed steps,
- avoid speculative actions when logs are incomplete.

Tooling should prioritize read-first operations (status, logs, recent outcomes) before write actions (pause/resume/rerun). This reduces accidental run manipulation during ambiguous contexts.

### 4) Treat run details as primary observability surface

The Workflow Run Details page provides the truth needed for operational decisions: status cards, progress counters, AI usage, and searchable logs. The ops assistant should guide users toward this evidence.

Operational habit:

- For every failure or slowdown report, inspect run details first, then act.

Use Dashboard to decide whether the issue is isolated or trending, then return to Audit Trail and delivery artifacts if configuration changes or milestone risks need follow-through.

This avoids premature changes based on partial signals from chat summaries.

### 5) Standardize failure taxonomy for faster recovery

Ops teams lose time when every failure looks unique. Define failure classes:

- configuration errors,
- source-data quality issues,
- credential/connectivity failures,
- code/runtime errors,
- dependency sequencing problems.

Map each class to a default first response. The assistant can then propose deterministic next steps instead of ad-hoc troubleshooting.

### 6) Encode safe action sequencing

A common anti-pattern is jumping straight to "rerun" without containment. Safe action order should be:

1. verify failure class,
2. pause or stop if continued processing is harmful,
3. capture evidence,
4. apply fix,
5. rerun with scoped validation,
6. monitor until stable.

This sequence prevents repeated bad runs from compounding downstream data or task state.

### 7) Link ops outcomes to delivery commitments

Operational automation work is product work. If run failures delay releases, those impacts must appear in Delivery Tasks, Timeline, and Milestones.

Required linkage:

- incident or failure -> task,
- task -> milestone risk,
- milestone change -> stakeholder communication.

Without this connection, operations and delivery drift apart and release planning becomes fiction.

### 8) Use audit artifacts to preserve change accountability

Ops environments change quickly. Audit Trail provides critical accountability for who changed what and when. Use it as part of post-fix validation:

- confirm intended config/tool changes occurred,
- tag important release/incident milestones,
- compare before/after diffs during reviews.

Audit artifacts reduce guesswork in retrospective analysis and improve compliance readiness.

### 9) Monitor cost and performance side effects

Automation improvements can increase hidden costs or latency. For example, extra tool calls or verbose run retries can inflate token or execution cost. Review dashboard and run metrics for side effects after each significant change.

A healthy ops assistant balances three outcomes:

- reliability,
- responsiveness,
- cost efficiency.

Optimizing only one typically destabilizes the other two.

### 10) Definition of done for Ops Assistant Playbook

Your ops assistant is production-ready when:

- recurring operational decisions are clearly supported,
- workflow run actions follow safe sequencing,
- failures are classified and handled consistently,
- delivery impact is visible in planning artifacts,
- audit and observability evidence supports every major operational change.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are deploying an ops assistant for a "Revenue Operations Sync" domain. The assistant should help admins run, inspect, and recover workflows that enrich account data and push updates to downstream systems. Reliability is more important than raw speed, and failed runs must be triaged within one business hour.

### Phase A: Define operating contract and failure taxonomy

1. Create `chapter-10-section-02-ops-contract.md`.
2. Document:
   - supported workflow actions,
   - role/permission boundaries,
   - failure classes and first-response actions,
   - escalation SLA.
3. Share contract draft in Delivery Discussions for review.

Checkpoint:

- Operating contract exists and is approved by ops lead.
- Failure taxonomy has concrete first-response actions.

### Phase B: Configure ops assistant behavior

1. Open **AI Agents** and select/create ops assistant configuration.
2. Update instructions to require run-status summary before action recommendations.
3. Restrict tools to required operations domain tools only.
4. Add fallback guidance for unknown or ambiguous run contexts.
5. Save config as versioned release candidate.

Checkpoint:

- Prompt/tool constraints align with operating contract.
- Candidate version is identifiable for validation.

### Phase C: Validate workflow-run triage loop

1. Open **Data Model -> Workflows** and run target workflow.
2. Open resulting entry in **Workflow Run Details**.
3. Use logs and progress panels to identify at least one normal-state verification pattern.
4. Simulate or use a known failing case and run triage sequence:
   - classify failure,
   - pause/stop if needed,
   - capture logs,
   - plan fix.
5. Record findings in `chapter-10-section-02-run-triage-log.md`.

Checkpoint:

- Triage loop is reproducible.
- Evidence includes run status + log snippets + action rationale.

### Phase D: Convert operations findings into delivery work

1. Create/update delivery tasks for discovered reliability gaps.
2. Link tasks to appropriate milestone.
3. If schedule impact exists, update Timeline windows and dependencies.
4. Add decision summary in Delivery Discussions with owner and due date.

Checkpoint:

- Ops issues are not isolated; they are represented in delivery plan.
- Milestone risk visibility is updated.

### Phase E: Validate accountability and regression signals

1. Open **Audit Trail** and verify configuration/workflow changes are captured.
2. Create a version tag for this stabilization cycle.
3. Run a short eval or scripted conversation set focused on ops assistant decision quality.
4. Use **Dashboard** and conversation **Timeline** for performance/cost sanity checks.
5. Capture summary in `chapter-10-section-02-ops-validation-summary.md`.

Checkpoint:

- Audit evidence and performance evidence are both present.
- No hidden regression in cost or latency is observed without mitigation.

### Phase F: Publish operating readiness decision

1. Create `chapter-10-section-02-ops-readiness-decision.md` with:
   - contract compliance status,
   - unresolved risks,
   - mitigation ownership,
   - recommendation (`go`, `go with constraints`, `no-go`).
2. Publish summary to Delivery Discussions and assign first-week on-call owners.

Checkpoint:

- Readiness decision is explicit and auditable.
- Ownership for follow-through is clear.

## Expected outputs

- An ops operating contract artifact including action boundaries, failure taxonomy, and escalation SLA.
- A versioned ops assistant configuration constrained to workflow-safe behavior.
- Workflow run triage evidence with reproducible classification and recovery sequence.
- Delivery-linked remediation tasks with milestone and timeline impact visibility.
- Audit trail verification and version tagging for major stabilization changes.
- Performance/cost sanity evidence from Dashboard/timeline traces.
- A readiness decision artifact (`chapter-10-section-02-ops-readiness-decision.md`) with go/no-go recommendation.

Evidence quality standard:

- A second engineer can independently verify why each operational decision was made and whether controls were followed.

## Failure modes

- **Assistant suggests action without run evidence**
  Symptom: recommendations skip status/log inspection and jump directly to rerun.
  Recovery: require run-summary-first prompt rule and reject action proposals missing evidence references.

- **Permission and authority confusion**
  Symptom: non-authorized users attempt high-impact actions through assistant flow.
  Recovery: enforce role boundaries in process documentation and verify project role mapping.

- **Failure classes too generic**
  Symptom: triage loops vary wildly across operators for the same error type.
  Recovery: refine taxonomy with concrete diagnostic signatures and standard first actions.

- **Run recovery not linked to delivery impact**
  Symptom: workflow failures are fixed locally but milestone slippage is not updated.
  Recovery: mandate issue-to-task-to-milestone linkage for all release-affecting incidents.

- **Audit evidence ignored**
  Symptom: post-incident review cannot establish which config change introduced regression.
  Recovery: include audit diff check in every major fix checklist and tag stabilization points.

- **Reliability improvement with hidden cost spike**
  Symptom: failure rate decreases while token or runtime cost rises unsustainably.
  Recovery: review dashboard/run usage after each change and add cost constraints to acceptance gate.

## Completion checklist

- [ ] Ops operating contract defines boundaries, permissions, and failure taxonomy.
- [ ] Assistant configuration enforces run-summary-first and tool minimization principles.
- [ ] Workflow run triage sequence is tested and documented with evidence.
- [ ] Recovery actions are linked to delivery tasks and milestone impact where relevant.
- [ ] Audit trail confirms key changes and includes at least one version tag.
- [ ] Performance and cost sanity checks were run after configuration changes.
- [ ] Escalation SLA and on-call ownership are documented.
- [ ] Readiness recommendation includes unresolved risks and mitigations.
- [ ] First-week operating cadence is defined.
- [ ] Another engineer can follow the playbook without oral handoff.

## Canonical references

- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Dashboard](../../user-guide/dashboard/README.md)

---

# Knowledge Assistant Playbook

## Learning objectives

- Design a production knowledge assistant in Gaia that returns reliable, context-grounded answers across evolving documents and entity data.
- Establish a freshness and provenance model that prevents stale or unverifiable responses from reaching users.
- Implement an operating loop that links source-refresh workflows, validation, feedback, and release controls for knowledge quality.
- Produce auditable evidence for go/no-go decisions when publishing knowledge assistant updates.

## Prerequisites

- Completion of Chapters 3, 5, 6, and 9, plus Sections 1-2 of this chapter.
- Access to Data Model (Entities, Storage, Pipelines, Workflows, Runs), AI Agents, Conversations, and Evals.
- A defined knowledge domain (for example policy docs, product docs, internal procedures) with content owners.
- Baseline quality criteria: factual accuracy, source coverage, freshness window, and acceptable abstention behavior.

## In Gaia

- [Data Model](../../user-guide/data-model/README.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)

Use Data Model and Workflow Run Details to verify freshness and provenance pipelines, AI Agents to govern retrieval and response behavior, Conversations to inspect grounded answers, Evals to regression-test retrieval risks, and Tasks plus Delivery Discussions to track knowledge defects and release decisions.

## Concept brief

Knowledge assistants are judged by a higher standard than general conversational agents: users expect factual reliability and fast access to trusted sources. In production, the hardest problems are rarely model fluency. They are source reliability, freshness drift, weak provenance, and ambiguous confidence handling. A strong playbook addresses these directly.

### 1) Model knowledge scope before retrieval design

Do not start with "what tools can we call?" Start with "what knowledge decisions must users make?" Map major query classes:

- lookup questions (facts, definitions, policy clauses),
- procedural questions (step-by-step actions),
- comparative questions (which option applies),
- exception questions (when standard guidance does not fit).

For each class, define required source quality and acceptable uncertainty. This prevents overconfident answers where source support is weak.

### 2) Treat data model design as retrieval quality design

Entity schema decisions directly affect answer quality. Well-structured entities with explicit fields (owner, effective date, status, policy version, region applicability) outperform unstructured text blobs for production use.

Use relational entities for stable metadata and graph entities where multi-hop relationships matter (for example policy -> product -> region -> exception path). Retrieval quality improves when entities encode meaning intentionally, not just storage convenience.

### 3) Separate canonical sources from convenience sources

Knowledge assistants often combine sources with different trust levels. Distinguish:

- canonical sources: approved policies, official docs, validated records,
- convenience sources: drafts, notes, ad-hoc uploads.

The assistant should prefer canonical sources for authoritative answers and clearly qualify when relying on convenience material. This reduces policy-risk incidents caused by "latest draft" confusion.

### 4) Build freshness as an explicit operational contract

Stale knowledge is a production bug. Define freshness windows by source type:

- daily for rapidly changing operational data,
- weekly or release-based for policy docs,
- event-triggered for incident procedures.

Use Pipelines, Workflows, schedules, and ingestion webhook triggers to keep data synchronized. Freshness must be monitored like uptime, not treated as occasional housekeeping.

### 5) Enforce grounded-response behavior

Knowledge assistants should avoid unsupported certainty. In Gaia, use grounded response practices:

- require retrieval before entity JSON answers,
- instruct the agent to disclose uncertainty when evidence is incomplete,
- ask clarification when user query lacks key context (region, version, customer type, date scope).

Good abstention behavior protects trust. Incorrect confidence destroys trust quickly.

### 6) Make provenance visible in outputs and artifacts

Users and reviewers need to see where answers came from. Enforce a lightweight provenance format in assistant responses and review artifacts:

- source type and identifier,
- relevant effective date/version,
- confidence qualifier,
- unresolved assumptions.

This is not just for compliance. It accelerates debugging when users flag inaccuracies because engineers can trace back to the exact source path.

For folder-grounded assistants, treat provenance as a retrieval requirement, not a UI garnish. Retrieval should return stable citations with page, sheet, row, or section locators, and the assistant should expand the cited passage before answering whenever the first snippet is too short to support a factual claim.

### 7) Use document workflows intentionally

Conversation artifact workflows are powerful for drafting and editing documents, but they should complement, not replace, canonical knowledge governance.

Use document editing flows for:

- drafting summaries,
- preparing policy updates for review,
- collaborative redlining.

After approval, synchronize finalized outputs into canonical storage/entities and update the relevant refresh pipelines and workflows. Otherwise the assistant may continue referencing outdated copies.

### 7.1) Route conversational files through channel-owned workflows

If users upload knowledge sources directly in the end-user app, the routing policy belongs to the channel, not the agent configuration.

- Enable uploads in **Conversations -> Channels -> Text/Personal Assistant -> Files**.
- Route uploads by filename pattern and file type into the workflow responsible for extraction, validation, or canonicalization.
- Configure the first source to read `conversation_storage_upload` so the workflow receives conversation metadata plus the uploaded file descriptors.
- Preserve provenance fields (`conversationId`, `messageId`, `uploadedFile`, and any source-specific identifiers you derive) through the pipeline so the last target can store auditable outputs.

This keeps knowledge workflow behavior consistent across public entrypoints and makes provenance visible from the very first source record.

### 8) Validate knowledge behavior with scenario-based evals

Generic eval prompts are insufficient for knowledge systems. Build datasets around high-risk retrieval patterns:

- conflicting source versions,
- incomplete user context,
- policy exceptions,
- stale data exposure,
- scanned or visually complex PDFs that need OCR/visual analysis fallback,
- multilingual phrasing and paraphrase variance.

Review per-trial outcomes, not only aggregate pass rates. A single systematic failure in an important scenario can outweigh strong average performance.

### 9) Connect knowledge issues to delivery and governance workflows

When knowledge defects appear, route them as operational work:

- open tasks with owner and due date,
- link tasks to milestones,
- record decisions in Discussions,
- verify updates in Audit Trail where applicable.

This turns knowledge maintenance into governed delivery, not scattered edits by whoever notices a problem.

### 10) Definition of done for Knowledge Assistant Playbook

Your knowledge assistant is production-operable when:

- source scope and trust levels are defined,
- freshness windows are managed with automated or scheduled workflows,
- responses follow grounded and provenance-aware behavior,
- regression evals cover critical retrieval risks,
- knowledge fixes are tracked through delivery and governance artifacts.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are preparing a knowledge assistant for an internal policy portal. Users ask for guidance on security and compliance procedures that change each quarter. The assistant must answer accurately, avoid stale guidance, and clearly indicate source confidence.

### Phase A: Define knowledge catalog and trust levels

1. Create `chapter-10-section-03-knowledge-catalog.md`.
2. List major source sets (policies, runbooks, release notes, FAQs).
3. Classify each source as canonical or convenience.
4. Add ownership and expected update cadence.
5. Record accepted freshness window for each source type.

Checkpoint:

- Knowledge catalog has source classification and owners.
- Freshness windows are explicit.

### Phase B: Configure data model and storage foundation

1. Open **Data Model -> Entities** and verify key metadata fields exist (version/effectiveDate/owner/status).
2. Open **Data Model -> Storage** and create organized folders for source documents (for example `canonical/policies`, `drafts`, `exports`).
3. Upload sample documents and verify preview/readability.
4. Update entity descriptions so agent prompts receive clear schema intent.

Checkpoint:

- Structured metadata is queryable.
- Source documents are organized by trust level.

### Phase C: Implement freshness pipeline/workflow path

1. Configure or review **Pipelines** for document/entity refresh, extraction, and normalization.
2. Attach pipelines to **Workflows** with clear execution order.
3. If external source sync is needed, validate **Ingestion Webhook** pathway.
4. Run workflow once and inspect output in **Runs** / **Workflow Run Details**.
5. Capture evidence in `chapter-10-section-03-freshness-run-log.md`.

Checkpoint:

- Knowledge update path is reproducible.
- Run logs confirm successful update flow.

### Phase D: Configure assistant retrieval and response behavior

1. Open **AI Agents -> Agent Configuration** for knowledge assistant.
2. Enable required retrieval tools and related entities.
3. Add instruction rules for:
   - evidence-first answers,
   - uncertainty disclosure,
   - context clarification questions.
4. Run 8-10 test conversations in **Conversations** including:
   - direct fact lookup,
   - conflicting-version query,
   - insufficient-context query,
   - out-of-scope query.
5. Store findings in `chapter-10-section-03-knowledge-smoke-tests.md`.

Checkpoint:

- Assistant behavior reflects grounding and abstention policy.
- Problem cases are documented with examples.

### Phase E: Run knowledge regression eval and feedback triage

1. In **Evals**, execute a targeted knowledge regression set.
2. Compare current run vs baseline in **Reports**.
3. In conversation samples, add threaded feedback for weak responses.
4. Convert top failures into delivery tasks and assign owners.
5. Update milestones if critical knowledge defects affect release timeline.

Checkpoint:

- Eval findings are linked to tracked remediation work.
- Feedback themes are classified and prioritized.

### Phase F: Publish knowledge release recommendation

1. Create `chapter-10-section-03-knowledge-release-decision.md` containing:
   - freshness status by source class,
   - regression results summary,
   - unresolved high-risk gaps,
   - recommendation (`go`, `go with constraints`, `no-go`).
2. Post decision summary in Delivery Discussions.
3. Schedule next freshness validation run and owner.

Checkpoint:

- Decision artifact is complete and reviewable.
- Ongoing operating cadence is defined.

## Expected outputs

- A knowledge catalog artifact with trust classification, ownership, and freshness windows.
- Structured entity and storage organization supporting provenance-aware retrieval.
- A reproducible knowledge-refresh workflow with run evidence (`chapter-10-section-03-freshness-run-log.md`).
- A versioned knowledge assistant configuration enforcing grounding and uncertainty handling.
- Regression evidence from Evals and actionable feedback-linked remediation tasks.
- A release decision artifact (`chapter-10-section-03-knowledge-release-decision.md`) with explicit risk posture and go/no-go recommendation.

Evidence quality standard:

- A reviewer can trace every major answer behavior requirement to source governance, runtime checks, and validation evidence.

## Failure modes

- **Canonical and draft sources mixed without priority**
  Symptom: assistant cites outdated drafts when canonical policy exists.
  Recovery: enforce source-tier precedence and update retrieval instructions to prefer canonical content.

- **Freshness windows undefined**
  Symptom: no clear trigger for refresh or reprocessing; stale answers appear gradually.
  Recovery: define per-source freshness SLA and automate refresh runs through workflows/schedules.

- **No uncertainty behavior for low-evidence queries**
  Symptom: assistant gives confident responses despite missing key context.
  Recovery: add mandatory clarification/abstention rules and test with insufficient-context scenarios.

- **Knowledge updates not reflected in delivery planning**
  Symptom: repeated accuracy defects are fixed ad hoc and recur later.
  Recovery: route defects into delivery tasks, milestone tracking, and review cadence.

- **Eval coverage too generic**
  Symptom: high aggregate pass rate masks critical policy-version failures.
  Recovery: expand scenario-based evals around version conflicts and exception logic.

- **Document edits never promoted to canonical sources**
  Symptom: conversation artifacts diverge from governed knowledge base.
  Recovery: establish publish workflow from artifact draft to canonical storage/entity records.

## Completion checklist

- [ ] Knowledge catalog defines canonical vs convenience sources with owners.
- [ ] Freshness windows exist and are tied to refresh operations.
- [ ] Data model and storage structure support provenance metadata.
- [ ] Assistant instructions enforce grounded answers and uncertainty handling.
- [ ] Smoke tests include conflicting-version and low-context scenarios.
- [ ] Regression evals are run and compared against baseline.
- [ ] Feedback findings are triaged into tracked delivery tasks.
- [ ] Milestone/release implications are updated for high-risk knowledge defects.
- [ ] Release recommendation is explicit with risk rationale.
- [ ] Next validation cadence and ownership are scheduled.

## Canonical references

- [Data Model](../../user-guide/data-model/README.md)
- [Entities](../../user-guide/data-model/entities.md)
- [Storage](../../user-guide/data-model/storage.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [Ingestion Webhook](../../user-guide/data-model/ingestion-webhook.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Scenario: Document upload, edit, and download workflow](../../user-guide/conversations/scenarios/document-editing-workflow.md)
- [Evals](../../user-guide/evals/README.md)

---

# Incident And Rollback Playbook

## Learning objectives

- Build a production incident-response workflow in Gaia that is fast, evidence-driven, and role-clear.
- Define rollback criteria and execution steps that can be applied consistently under pressure.
- Coordinate incident communication, delivery replanning, and quality re-validation after corrective actions.
- Produce post-incident artifacts that improve future resilience instead of only documenting past failures.

## Prerequisites

- Completion of Chapter 10 Sections 1-3 and prior chapter work on security, observability, and release readiness.
- Access to Conversations (timeline + feedback), Dashboard, Delivery Management, Evals, and Audit Trail.
- Defined incident roles: incident commander, technical owner, comms owner, approver for rollback.
- Current release baseline artifacts (active configuration version, key milestones, and known constraints).

## In Gaia

- [Dashboard](../../user-guide/dashboard/README.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Timesheet](../../user-guide/timesheet/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Timeline](../../user-guide/conversations/dialogs/timeline.md)

Use Dashboard and Timeline for detection, Audit Trail for recent-change evidence, Tasks for coordinated response work, Timesheet for capacity rebalance during sustained incidents, and Evals for post-recovery validation.

## Concept brief

Incidents are not exceptional in production systems; they are expected. What distinguishes mature teams is not "zero incidents" but fast detection, disciplined decision-making, and clean recovery. A rollback playbook is part of that maturity. Without it, teams improvise under stress, create inconsistent actions, and lose critical context.

### 1) Define incident severity by user and business impact

Avoid severity labels based only on technical complexity. Use impact-centered criteria:

- affected user volume,
- blocked critical journeys,
- policy/safety exposure,
- financial or compliance implications,
- expected time to mitigation.

Severity should drive response cadence, stakeholder visibility, and required approval depth.

### 2) Separate detection from diagnosis

The first goal is detection confidence: is there a real incident? The second is diagnosis: why is it happening? Teams that mix these steps often waste time debating causes before confirming impact reality.

Detection signals in Gaia may include:

- dashboard error/cost/latency spikes,
- repeated negative feedback clusters,
- timeline traces showing tool failures or long stalls,
- eval regression after a recent change.

Confirm the incident quickly, then shift to diagnosis with structured evidence gathering.

### 3) Assign command roles immediately

Role ambiguity is the fastest path to slow recovery. Assign in the first minutes:

- incident commander (decision flow and timeline),
- technical owner (triage and fix execution),
- communications owner (internal/external updates),
- rollback approver (if distinct).

Even in small teams, naming roles reduces duplicated effort and unowned tasks.

### 4) Use a containment-first mindset

Containment aims to reduce ongoing harm before perfect root-cause certainty. Examples:

- disable risky channel/version,
- restrict tool path,
- route affected intents to safe fallback,
- pause failing workflow triggers.

Containment is not final resolution; it buys time for safer diagnosis and decision quality.

### 5) Establish explicit rollback criteria before incidents happen

Rollback decisions degrade under pressure when criteria are vague. Define triggers in advance:

- sustained failure rate above threshold,
- critical policy violation occurrence,
- unacceptable latency across core journeys,
- unresolved severe incident beyond time budget.

Predefined triggers reduce debate and protect teams from decision paralysis.

### 6) Decide between fix-forward and rollback with bounded risk logic

Rollback is not always better; fix-forward is not always faster. Compare options with explicit dimensions:

- expected time to user-impact reduction,
- confidence in the proposed fix,
- blast radius if fix fails,
- operational complexity of rollback,
- residual compliance/safety risk.

Document this comparison, even briefly. It improves accountability and post-incident learning.

### 7) Preserve evidence while acting

During incidents, teams often prioritize speed and lose critical forensic data. Preserve essential evidence in parallel:

- affected conversation IDs and timelines,
- failing run logs,
- relevant configuration/version identifiers,
- audit diff of recent changes,
- timestamps for key decisions/actions.

Evidence capture should be lightweight but mandatory. Without it, root-cause analysis becomes speculation.

### 8) Synchronize incident actions with delivery artifacts

Incident response changes release reality. Update delivery artifacts during response, not days later:

- create incident task cluster,
- update milestone risk/state,
- adjust timeline dependencies if recovery work blocks planned scope,
- log decision updates in Discussions.

If the incident pulls work away from existing owners, use Timesheet as a secondary check before assigning more recovery tasks so you do not create invisible overload during mitigation.

This keeps stakeholders aligned on what shifted and why.

### 9) Re-validate quality gates after recovery

Recovery is complete only when behavior is validated, not when user complaints slow down. After rollback or fix-forward:

- run targeted regression evals,
- inspect run/report trends,
- review high-risk conversation traces,
- confirm dashboard metrics return within tolerance.

If residual risk remains, release status should be `go with constraints` or `no-go` until mitigations are complete.

### 10) Definition of done for Incident and Rollback Playbook

Your incident system is mature when:

- severity and role assignments are immediate and consistent,
- containment and rollback criteria are predefined and practiced,
- decisions are evidence-backed and documented,
- delivery and communication artifacts stay synchronized during incident response,
- post-incident actions measurably reduce recurrence risk.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

A newly activated support-agent configuration causes intermittent policy-inaccurate responses and increased latency in a high-volume support channel. Feedback spikes within one hour, and leadership requests a status update every 30 minutes. You must run an incident and rollback workflow using Gaia.

### Phase A: Detect and declare incident

1. Open **Dashboard** and identify anomaly window (error rate, latency, or cost spike).
2. Inspect recent conversations and collect 3-5 representative problematic threads.
3. Open **Timeline** for those conversations and capture failure signatures (tool errors, long stalls, repeated retries).
4. Declare incident in Delivery Discussions with severity, impact summary, and initial roles.
5. Create `chapter-10-section-04-incident-log.md` and start timestamped event entries.

Checkpoint:

- Incident is explicitly declared with severity and owners.
- Initial evidence set is captured.

### Phase B: Contain blast radius

1. Apply immediate containment action (for example disable affected channel version, switch to safer config, or restrict risky tool path).
2. Confirm containment effect by monitoring new conversation samples.
3. Update incident log with action timestamp and observed impact delta.
4. Notify stakeholders of containment status and expected next update.

Checkpoint:

- Ongoing user harm is reduced or bounded.
- Containment action and effect are documented.

### Phase C: Analyze rollback decision

1. Compare `fix-forward` and `rollback` options in incident log:
   - time to recovery,
   - confidence,
   - risk if wrong.
2. Check **Audit Trail** for recent high-impact changes linked to anomaly onset.
3. Validate whether rollback trigger criteria are met.
4. Record recommendation and approver decision.

Checkpoint:

- Decision path is explicit and evidence-backed.
- Audit evidence is linked to decision rationale.

### Phase D: Execute rollback or controlled fix-forward

1. If rollback is chosen, revert to known-good configuration/channel version and confirm activation status.
2. If fix-forward is chosen, apply minimal-risk patch and isolate impacted scope.
3. Open relevant workflow/run surfaces and ensure no harmful background process continues.
4. Track each action in incident log with owner and verification note.

Checkpoint:

- Corrective action is executed with controlled scope.
- Verification steps confirm expected operational state.

### Phase E: Re-validate quality and operational signals

1. Run targeted **Evals** for the failed scenario family.
2. Compare baseline vs post-recovery results in **Reports**.
3. Re-sample conversation timeline traces for the previously failing pattern.
4. Run **Update Statistics** if needed to refresh reporting baselines.
5. Record validation summary in `chapter-10-section-04-recovery-validation.md`.

Checkpoint:

- Recovery quality is measured, not assumed.
- Residual risk is explicit.

### Phase F: Close incident and harden system

1. Create follow-up delivery tasks for root-cause fixes and guardrail improvements.
2. Update milestone/timeline if release dates or scope changed.
3. Publish `chapter-10-section-04-incident-closure.md` including:
   - root cause (current best understanding),
   - response timeline,
   - what worked/failed,
   - preventive actions with owners/dates,
   - final recommendation (`go`, `go with constraints`, `no-go`).
4. Tag key audit entries for future reference.

Checkpoint:

- Incident closure includes preventive commitments.
- Delivery and governance artifacts reflect post-incident reality.

## Expected outputs

- A timestamped incident log artifact (`chapter-10-section-04-incident-log.md`) with declaration, containment, decision, and action history.
- An evidence bundle containing affected conversation timelines, dashboard anomaly snapshots, and relevant audit references.
- A documented rollback/fix-forward decision with criteria and approver attribution.
- Confirmed corrective execution state (rolled back or patched) and operational verification notes.
- Post-recovery validation artifact (`chapter-10-section-04-recovery-validation.md`) with eval/report comparisons.
- Incident closure artifact (`chapter-10-section-04-incident-closure.md`) with root-cause summary, preventive tasks, and go/no-go recommendation.

Evidence quality standard:

- A reviewer can reconstruct what happened, why decisions were made, and why the final release posture is justified.

## Failure modes

- **Severity underestimation**
  Symptom: team treats incident as minor while user-impact grows quickly.
  Recovery: use impact-based severity criteria and reclassify early when thresholds are crossed.

- **Containment delayed by root-cause debate**
  Symptom: no immediate safety action while diagnostics continue.
  Recovery: apply containment-first rule and separate immediate risk reduction from deep diagnosis.

- **Rollback criteria undefined or ignored**
  Symptom: long argument over whether rollback is "necessary" despite sustained impact.
  Recovery: predefine rollback triggers and require explicit exception rationale when overriding them.

- **Evidence capture gaps during firefight**
  Symptom: post-incident review lacks concrete traces and timestamps.
  Recovery: assign one person to evidence logging from incident start; use structured incident log template.

- **Recovery declared without validation**
  Symptom: incident closed because complaints dropped, but regression remains in critical scenario.
  Recovery: require post-recovery evals + timeline spot checks before closure.

- **No hardening follow-through**
  Symptom: incident repeats because preventive tasks are never tracked to completion.
  Recovery: convert every preventive action into owned delivery tasks with due dates and milestone linkage.

## Completion checklist

- [ ] Incident severity and command roles were declared promptly.
- [ ] Initial detection evidence was collected from dashboard and conversation traces.
- [ ] Containment action was executed and impact reduction verified.
- [ ] Rollback vs fix-forward decision used explicit criteria and named approver.
- [ ] Audit trail evidence was reviewed for relevant change context.
- [ ] Corrective action was verified in live operational state.
- [ ] Post-recovery eval and report checks were completed.
- [ ] Delivery tasks/milestones were updated to reflect incident impact and hardening work.
- [ ] Closure artifact includes preventive actions with owners and timelines.
- [ ] Final release recommendation is explicit and evidence-backed.

## Canonical references

- [Conversations](../../user-guide/conversations/README.md)
- [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Give feedback on a reply](../../user-guide/conversations/dialogs/feedback.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Update project statistics](../../user-guide/conversations/dialogs/update-statistics.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Run details](../../user-guide/evals/run-details.md)
- [Reports](../../user-guide/evals/reports.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)

---

# Chapter 11: Capstone Build and Ship

Status: current

End-to-end applied project

## Sections

- [Capstone Brief](#doc-ch11-capstone-build-and-ship-01-capstone-brief)
- [Implementation Steps](#doc-ch11-capstone-build-and-ship-02-implementation-steps)
- [Eval And Hardening](#doc-ch11-capstone-build-and-ship-03-eval-and-hardening)
- [Demo And Handoff](#doc-ch11-capstone-build-and-ship-04-demo-and-handoff)
- [Governed Application From Scratch](#doc-ch11-capstone-build-and-ship-05-governed-application-from-scratch)

The final section is the canonical end-to-end tutorial for the neo-bank capstone. It belongs to the handbook so the full build, governance reasoning, and handoff evidence live in one learning path instead of being split across disconnected walkthroughs.

## Alignment with Canonical Guide Surfaces

- [Getting started](../../user-guide/README.md)
- [Tutorials](../../user-guide/tutorials/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Governance](../../user-guide/governance/README.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Settings](../../user-guide/settings/README.md)

Use these pages as the capstone reference spine. The handbook should describe sequencing, evidence, and operating discipline; the user guide remains the canonical feature walkthrough for each workspace.

Treat [Governance Operating Model](#doc-ch08-security-and-governance-05-governance-operating-model) as a prerequisite mindset for this chapter. The capstone should not invent governance late; it should carry package scope, ownership, evidence, and release-gate expectations from the start.

Use [Governed Application From Scratch](#doc-ch11-capstone-build-and-ship-05-governed-application-from-scratch) as the final synthesis section after the first four capstone sections. It turns the generic capstone workflow into one governed, end-to-end build tutorial that uses the implemented Gaia framework-package model directly.

Treat live tutorials as accelerators, not hidden prerequisites. The handbook is the source of truth for the full neo-bank tutorial; the Tutorials page at `/platform/support/tutorials` is only a companion rehearsal surface for selected steps.

## Fast path inside Gaia

1. Start from one live project and keep [Conversations](../../user-guide/conversations/README.md), [AI Agents](../../user-guide/agents/README.md), [Data Model](../../user-guide/data-model/README.md), [Governance](../../user-guide/governance/README.md), [Evals](../../user-guide/evals/README.md), and [Delivery Management](../../user-guide/delivery/README.md) tied to the same scenario.
2. Use the handbook sections for sequencing and evidence expectations, then use the linked user-guide pages to execute each step in the actual workspace.
3. If the capstone reaches a step that the product does not yet support honestly, capture the gap as delivery work instead of bypassing it with off-book setup.

The capstone is successful only when the end-to-end path exists inside Gaia, not when the narrative alone sounds complete.

## Chapter Completion Criteria

- All section checklists completed
- At least one end-to-end Gaia lab validated
- Canonical user-guide references confirmed

---

# Capstone Brief

## Learning objectives

- Define a capstone scope that is ambitious enough to demonstrate Gaia fluency but constrained enough to ship responsibly.
- Translate business intent into a concrete Gaia architecture spanning data model, agents, channels, delivery, and evaluation.
- Produce a decision-ready brief with explicit success criteria, risk posture, and evidence requirements.
- Establish team roles, governance boundaries, and a build cadence that can survive real execution pressure.

## Prerequisites

- Completion of Chapters 1-10.
- Access to create and manage a Gaia project with permissions for Agents, Data Model, Conversations, Delivery, Evals, Settings, and Audit.
- At least one realistic domain use case with identified stakeholders and constraints.
- Agreement on who can approve scope, security-sensitive changes, and release decisions.

## Canonical guide spine for this section

- [Getting started](../../user-guide/README.md)
- [Tutorials](../../user-guide/tutorials/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Governance](../../user-guide/governance/README.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Settings](../../user-guide/settings/README.md)

Use these guide pages as the capstone operating spine. If a tutorial is still missing parity or media coverage, treat the owning user-guide page as the canonical prerequisite instead of making the capstone depend on a partially maintained walkthrough. When you do use a guided drill, open the tutorials workspace at `/platform/support/tutorials`.

## In Gaia

- [Delivery Management](../../user-guide/delivery/README.md) and [Tasks](../../user-guide/tasks/README.md) for the execution spine of the capstone
- [AI Agents](../../user-guide/agents/README.md), [Data Model](../../user-guide/data-model/README.md), [Conversations](../../user-guide/conversations/README.md), [Evals](../../user-guide/evals/README.md), and [Governance](../../user-guide/governance/README.md) for the platform layers the brief must connect
- [Settings](../../user-guide/settings/README.md) and [Audit Trail](../../user-guide/audit/README.md) when ownership and authority are part of the design contract

Write the brief against one live Gaia project, not in isolation. The capstone brief should already map to real workspaces and evidence paths.

## Concept brief

The capstone brief is where all prior chapters become operational. Treat it as a production contract, not an academic proposal. The brief should explain what you are building, why it matters, how Gaia components work together, what risks exist, and how success will be measured. If your brief cannot answer these questions clearly, implementation will drift regardless of engineering skill.

### 1) Treat the capstone as an integration test of your engineering system

The capstone is not primarily a feature demo. It is a full-system test of how your team makes decisions across:

- product intent,
- architecture,
- delivery execution,
- quality gates,
- operational readiness,
- governance scope and evidence ownership.

A polished UI with weak governance is not a successful capstone. A robust architecture with no adoption path is also incomplete. The brief must integrate both.

### 2) Start from decision outcomes, not a list of features

Feature-first briefs become shopping lists with little decision value. Use a decision-first framing:

- What decision will users make with this assistant?
- What cost/risk will be reduced if the assistant succeeds?
- What failure modes are unacceptable?

This framing keeps the project anchored to measurable value and prevents scope growth driven by novelty.

### 3) Define user segments and operating context precisely

"Support users" or "operations users" is too broad for a capstone brief. Identify:

- primary users,
- secondary stakeholders,
- environment constraints (regulatory, latency, channel, data freshness),
- interaction patterns (single-turn lookup, multi-turn triage, escalation-heavy flows).

Without context precision, architecture choices become generic and brittle.

A useful pattern is to add a one-page "context card" per primary segment. Each card should include top intents, critical constraints, and the most costly failure mode for that segment. When architecture debates arise, these cards prevent abstract arguments and keep decisions tied to operational reality.

### 4) Convert scope into a smallest shippable system

Capstones fail from overbreadth. Define:

- must-have outcomes,
- explicit non-goals,
- deferred backlog.

A useful rule is: ship one high-confidence workflow end to end before scaling breadth. Gaia supports expansion, but your brief should prioritize one production-worthy path over many half-implemented paths.

### 5) Map architecture across Gaia surfaces

The brief should explicitly connect platform components:

- Data Model: entities, storage, pipelines, workflows, scheduled jobs.
- Agent layer: orchestrator/specialists, tools, handoffs, post-steps.
- Experience layer: channels, conversation UX, canvas/artifacts.
- Governance layer: framework packages, contracts, policies, state and memory, risks, controls, obligations, classifications, regulatory updates, discovery, explainability, evidence links.
- Delivery layer: cycle stages, tasks, milestones, timeline.
- Quality layer: eval datasets, graders, runs, reports.
- Operations layer: dashboard metrics, timeline traces, audit trail, project roles.

This map prevents hidden gaps where one layer assumes behavior never implemented in another.

For each layer boundary, define one explicit contract. Example: "agent tool output shape consumed by channel response formatter," or "workflow completion signal required before milestone status update." Contract-level thinking reduces integration surprises because handoffs between layers are treated as design artifacts, not implicit assumptions.

### 6) Define governance and authority before execution begins

Teams commonly defer governance to "later." In capstones, this causes immediate confusion during the first major change request. The brief must define:

- which governance packages or overlays apply,
- who owns product decisions,
- who owns technical decisions,
- who owns contracts, policies, obligations, controls, and evidence freshness,
- who approves release and rollback,
- who handles incident communication.

Use project-role boundaries and access constraints as part of the design, not afterthoughts.

### 7) Design a delivery cadence with explicit gates

A capstone should have stage transitions with criteria, not date-only milestones. At minimum, include:

- brief approval gate,
- implementation readiness gate,
- eval/hardening gate,
- demo/handoff gate.

Each gate needs evidence expectations so progress is auditable and not personality-dependent.

Add a simple scoring rubric per gate to prevent subjective pass/fail calls. A practical approach is `pass`, `pass with constraints`, `hold`, tied to pre-declared evidence thresholds. This lets you move quickly when risk is bounded while still forcing escalation when blockers exceed tolerance.

### 8) Build a risk register with mitigation ownership

Risk awareness without ownership does not reduce risk. Categorize risk at least by:

- data risk,
- model/tool risk,
- UX risk,
- schedule risk,
- operational/support risk.

For each risk, define trigger, owner, mitigation, and fallback. This practice dramatically improves execution resilience.

### 9) Specify evidence artifacts before writing code

If you do not define evidence early, teams produce whatever is easiest at the end. The brief should list required artifacts:

- architecture decisions,
- test/eval results,
- delivery status logs,
- incident or exception notes,
- final release recommendation.

Evidence predefinition makes the final demo credible and shortens review cycles.

Also define artifact ownership and refresh cadence. Artifacts without owners decay quickly and lose decision value. A brief should state who updates each artifact and at what point in the cycle it must be reviewed.

### 10) Definition of done for Capstone Brief

A capstone brief is complete when:

- goals and non-goals are clear,
- architecture and delivery approach are coherent,
- risks and governance are explicit,
- success criteria and evidence requirements are measurable,
- stakeholders can approve implementation without asking foundational clarification questions.

Before closing the brief gate, run a short "cold-read test": ask one engineer not involved in drafting to explain the build plan, risks, and go/no-go logic after reading the brief. Any major misinterpretation indicates the brief still needs clarification.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are launching a capstone called `Customer Operations Copilot` for a SaaS team. The assistant should help internal teams triage customer issues, retrieve policy guidance, and coordinate next actions through delivery tasks. Leadership expects a working end-to-end demonstration in three weeks with a production-feasible release recommendation.

### Phase A: Define problem statement and decision outcomes

1. Create `chapter-11-section-01-problem-statement.md`.
2. Write:
   - primary user decisions,
   - target outcomes,
   - unacceptable failure modes,
   - baseline metrics.
3. Validate with one product stakeholder and one operations stakeholder.
4. Add one "anti-goal" statement clarifying what the capstone must not optimize for (for example, breadth over reliability).

Checkpoint:

- Problem framing is decision-oriented.
- Failure modes are concrete and operational.
- Anti-goal is explicit and prevents early scope drift.

### Phase B: Define scope boundaries and non-goals

1. Create `chapter-11-section-01-scope-boundaries.md`.
2. Separate `must-have`, `nice-to-have`, and `out-of-scope` items.
3. Add explicit non-goals to prevent scope creep.
4. Record escalation path for scope-change requests.

Checkpoint:

- Scope is narrow enough to ship within timeline.
- Non-goals are visible to all contributors.

### Phase C: Draft Gaia architecture blueprint

1. Create `chapter-11-section-01-architecture-blueprint.md`.
2. Map required Gaia components:
   - entities/storage/pipelines/workflows,
   - agent configurations/tools/handoffs,
   - channels/conversation surfaces,
   - eval and observability surfaces.
3. Mark dependencies and assumptions.
4. Identify highest-risk integration points.
5. For each high-risk integration point, assign an owner and an early validation tactic.

Checkpoint:

- Architecture spans all required layers.
- Integration risks are called out early.
- Every major risk has at least one planned validation step.

### Phase D: Define delivery and governance model

1. In **Delivery Management**, create capstone cycle.
2. Initialize stage activities and assign owners.
3. Create initial milestone set and target dates.
4. In `chapter-11-section-01-governance-model.md`, define decision authorities and escalation matrix.
5. Confirm project-role permissions match governance responsibilities before implementation starts.

Checkpoint:

- Delivery cycle and ownership are visible in Gaia.
- Governance model aligns with project roles.
- Permission model supports the intended decision flow.

### Phase E: Define quality and risk plan

1. Create `chapter-11-section-01-quality-risk-plan.md`.
2. Include:
   - eval strategy overview,
   - monitoring plan,
   - rollback triggers,
   - risk register with owners.
3. Link each major risk to a mitigation task in Delivery.
4. Define one explicit "stop condition" that pauses implementation if violated.

Checkpoint:

- Quality gates and risk mitigations are actionable.
- No high-risk item lacks owner.
- Stop condition criteria are unambiguous.

### Phase F: Publish final capstone brief and recommendation

1. Assemble `chapter-11-section-01-capstone-brief-final.md` using all artifacts.
2. Present summary in Delivery Discussions.
3. Record implementation go/no-go recommendation.
4. Capture decision notes and open questions.
5. Record a 7-day revalidation checkpoint date for assumptions that are likely to change.

Checkpoint:

- Brief is decision-ready and reviewable.
- Recommendation is explicit with rationale.
- Time-sensitive assumptions have scheduled revalidation.

## Expected outputs

- A decision-oriented problem statement artifact.
- A scoped and bounded capstone definition with explicit non-goals.
- A cross-layer Gaia architecture blueprint.
- A live delivery cycle with owner-assigned activities and milestone targets.
- A governance model detailing decision rights and escalation paths.
- A quality/risk plan with mitigation ownership and rollback posture.
- A final brief artifact (`chapter-11-section-01-capstone-brief-final.md`) with implementation go/no-go recommendation.

Recommended quality bar for the final brief package:

- Every major claim points to at least one artifact or measurable threshold.
- Each integration risk includes owner + first validation milestone.
- Every gate has a declared pass/hold criterion.
- Governance and permissions are confirmed against actual project-role assignments.

Optional but high-value addition:

- include a one-page stakeholder summary that states the intended user impact, current confidence level, and top three unknowns. This helps executives and non-implementing partners align quickly without reading the full brief.

Evidence quality standard:

- A reviewer can determine whether implementation should start without additional background meetings.

## Failure modes

- **Feature-first brief with no decision framing**
  Symptom: project goals are described as UI/tool features rather than user outcomes.
  Recovery: rewrite brief around user decisions, business impact, and measurable outcomes.

- **Scope boundary ambiguity**
  Symptom: contributors interpret scope differently and start parallel, conflicting work.
  Recovery: publish must-have/non-goal table and enforce change intake process.

- **Architecture gaps across Gaia layers**
  Symptom: delivery plan assumes data, tools, or channels that are never specified.
  Recovery: require explicit component map and dependency review before implementation.

- **Governance defined too late**
  Symptom: major decisions stall due to unclear authority.
  Recovery: assign decision owners in brief and verify project-role permissions.

- **Risk register without execution linkage**
  Symptom: risks are documented but no mitigation tasks exist in Delivery.
  Recovery: tie each high/critical risk to an owned task and milestone.

- **Gate decision based on optimism rather than evidence**
  Symptom: implementation starts with unresolved assumptions and no validation plan.
  Recovery: enforce brief gate checklist and require explicit go/no-go recommendation.

## Completion checklist

- [ ] Problem statement is decision-first and metric-backed.
- [ ] Scope includes explicit non-goals and change-control path.
- [ ] Architecture blueprint covers data, agent, channel, delivery, eval, and operations layers.
- [ ] Delivery cycle and milestones are created with assigned owners.
- [ ] Governance model defines decision rights and escalation paths.
- [ ] Risk register includes triggers, mitigation, and owners.
- [ ] Quality and rollback approach is documented at brief stage.
- [ ] Required evidence artifacts are pre-declared.
- [ ] Final brief is published and reviewed.
- [ ] Implementation go/no-go recommendation is explicit.

## Canonical references

- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Delivery Process Cycle](../../user-guide/delivery/process-cycle.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)
- [Audit Trail](../../user-guide/audit/README.md)

---

# Implementation Steps

## Learning objectives

- Execute a capstone build in Gaia using an iterative vertical-slice approach that keeps integration risk visible.
- Implement data, agent, channel, and delivery components in an order that minimizes rework.
- Apply operational checks during implementation so reliability and governance are built in, not bolted on.
- Produce implementation evidence that supports downstream eval/hardening and release decisions.

## Prerequisites

- Completed and approved Section 1 capstone brief artifacts.
- Active capstone delivery cycle with owners and milestones.
- Access to Data Model, Agents, Channels, Conversations, Governance, Delivery, and Audit surfaces.
- Agreed coding/configuration standards for naming, versioning, and change approval.

## In Gaia

- [Data Model](../../user-guide/data-model/README.md), [AI Agents](../../user-guide/agents/README.md), and [Channels](../../user-guide/conversations/channels/README.md) for the vertical slice itself
- [Conversations](../../user-guide/conversations/README.md) and [Workflow actions](../../user-guide/conversations/workflow-actions.md) for validating the live path
- [Delivery Management](../../user-guide/delivery/README.md), [Tasks](../../user-guide/tasks/README.md), and [Audit Trail](../../user-guide/audit/README.md) for execution truth and change evidence

Implement the capstone as a live Gaia build. Each slice should leave behind a visible record of what changed, what works, and what remains blocked.

## Concept brief

Implementation is where capstones typically derail. Teams either over-plan and under-build, or they build rapidly without integration discipline and discover systemic issues too late. The goal of this section is to operationalize a balanced execution model: small slices, fast validation, and continuously updated delivery truth.

### 1) Build in vertical slices, not isolated layers

A common anti-pattern is completing all data work first, then all agent work, then all UX work. This creates delayed integration feedback. Instead, build vertical slices that include:

- one concrete use case,
- required data structures,
- agent behavior,
- channel UX path,
- observable success criteria.

Each slice should be testable end to end. This keeps risk localized and learning fast.

Define slice acceptance criteria before implementation starts. A useful template is: "input resolved, core action executed, output understandable, failure fallback available, and evidence captured." This forces teams to ship working behavior, not partial wiring.

### 2) Sequence work by dependency certainty

Not all dependencies are equally stable. Start with elements that must be correct for everything else to work:

- entity schema and key identifiers,
- minimum retrieval/write contracts,
- base agent configuration and routing.

Then implement higher-variance elements (advanced prompts, optional skills, richer UX). This order reduces wasted work when foundational assumptions change.

### 3) Keep configuration versioning explicit

In Gaia, configuration changes are part of implementation, not housekeeping. Use deliberate version naming and avoid "invisible" edits to active configurations.

Versioning practice:

- `capstone-v0.1-foundation`
- `capstone-v0.2-tooling`
- `capstone-v0.3-ux`

Clear version lineage makes troubleshooting and rollback significantly easier during hardening.

Pair each version with a short change note: what changed, why, and which tests were rerun. These notes become high-value debugging context during eval and demo preparation.

### 4) Implement tools and workflows with minimal privilege first

During implementation, teams are tempted to enable broad tool sets "for flexibility." This increases failure surface and complicates debugging. Start with:

- minimal read capabilities,
- controlled write paths,
- explicit error handling behavior.

Expand only when concrete use cases require it. This approach improves safety and clarity in early tests.

When enabling a new tool or workflow action, add a "blast-radius note" to your implementation log. Note the worst-case failure and the containment action. This keeps implementation speed while preserving risk awareness.

Apply the same discipline to governance scope. Early in implementation, confirm which framework packages or overlays the slice belongs to and attach them to the working governance records instead of waiting for the hardening phase.

### 5) Build UX paths as operational flows, not visual demos

Conversations, canvas panels, and artifacts should be implemented as complete operational flows:

- user asks,
- assistant resolves context,
- tool/data actions execute,
- result is understandable,
- fallback/escalation path exists.

If a flow looks good in a happy-path demo but fails at ambiguity or partial data, it is not implementation-complete.

### 6) Synchronize implementation progress into Delivery daily

Implementation reality belongs in Delivery artifacts. Update tasks, dependencies, and stage evidence continuously. Waiting for weekly updates creates plan fiction and late surprises.

Daily sync discipline:

- close completed tasks with evidence,
- re-estimate slipping tasks,
- update milestone risks,
- record decision changes in Discussions.

This keeps stakeholders aligned and reduces status churn.

Teams that do this well also record uncertainty status, not only completion status. Mark tasks as "blocked by unknown X" when needed. Hidden uncertainty is more dangerous than visible delay.

### 7) Use run-level observability during build, not only after build

Workflow runs and conversation timelines are implementation tools, not just production diagnostics. Use them to catch:

- fragile transformations,
- slow tool chains,
- unexpected workflow graph routes,
- hidden retries,
- inconsistent handoffs.

Early observability-driven fixes are cheaper than hardening-stage rewrites. In **Workflow Run Details**, start with **Execution trace** to connect a failing step to the graph node, branch, checkpoint, duration, and sampled context in/out that produced it. Then switch to **Log** when you need the raw stage or record messages.

### 8) Treat integration defects as first-class work items

When an issue crosses boundaries (for example prompt + data schema + channel behavior), teams often misclassify it as "noise." In capstones, these are the highest-value defects to fix.

Create explicit integration tasks with cross-owner accountability instead of burying fixes inside one team area.

Add one required field to integration defects: "downstream impact if unfixed." This helps prioritization and prevents cross-layer bugs from losing urgency behind local polish work.

### 9) Lock implementation scope before eval/hardening gate

Without scope freeze discipline, eval results are unstable and hardening becomes endless. Before entering Section 3 work:

- define release candidate version,
- freeze non-critical feature additions,
- allow only defect/risk fixes with change rationale.

This creates a stable baseline for meaningful quality analysis.

Document scope-freeze exceptions in a short changelog with approver name. Even in small teams, this reduces post-hoc confusion about what actually changed between candidate runs.

### 10) Definition of done for Implementation Steps

Implementation is complete when:

- core capstone workflow runs end to end,
- delivery artifacts reflect current reality,
- key integration points are validated,
- release candidate is versioned and scoped,
- evidence exists for transitioning into eval/hardening.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are implementing the `Customer Operations Copilot` defined in Section 1. The system should synchronize operational context, support conversation-driven triage, update structured records, and create delivery tasks for follow-up actions. You must deliver a stable release candidate for evaluation.

### Phase A: Bootstrap implementation baseline

1. Open capstone project and confirm project roles/permissions.
2. Create implementation board structure in Delivery cycle activities.
3. Create initial tasks for foundational slice (`data schema`, `agent baseline`, `channel baseline`, `integration smoke test`).
4. Define release-candidate branch/version naming in your working notes.
5. Define slice-level acceptance criteria template and include it in task descriptions.

Checkpoint:

- Delivery tasks represent initial implementation slices.
- Ownership and due windows are assigned.
- Slice acceptance criteria are visible to contributors.

### Phase B: Implement data foundation and workflow path

1. In **Data Model -> Entities**, create/update required entities with key metadata fields.
2. In **Storage**, establish source folders for capstone assets.
3. Configure **Pipelines** for initial source synchronization and normalization.
4. Chain pipelines in **Workflows** and execute first run.
5. Inspect **Runs** and **Workflow Run Details** for correctness and errors.
6. Record evidence in `chapter-11-section-02-data-implementation-log.md`.
7. Document data-quality guardrails (required fields, dedupe logic, and fallback behavior for malformed records).

Checkpoint:

- Data path executes successfully for at least one representative dataset.
- Run evidence captures both success and any corrections applied.
- Guardrails for malformed/partial input are defined.

### Phase C: Implement agent topology and tool contracts

1. In **AI Agents**, configure orchestrator and any specialist agents defined in brief.
2. Set explicit instruction fragments for scope boundaries and escalation behavior.
3. Enable minimal required tools and related entities.
4. Add handoff rules and test with targeted prompts.
5. Save as versioned candidate (for example `capstone-v0.2-tooling`).
6. Capture notes in `chapter-11-section-02-agent-implementation-log.md`.
7. Add one known-limitations section (what the assistant intentionally does not handle yet).

Checkpoint:

- Handoff and tool paths work for defined core intents.
- Candidate version is traceable.
- Known limitations are explicit for evaluators.

### Phase D: Implement channel and conversation flows

1. In **Channels**, enable target channel(s) for capstone audience.
   Use the shared Conversations workspace tabs when the slice also depends on UI Layouts, Folders, Workflow Actions, or File Extractors.
2. In **Conversations**, run structured tests for:
   - happy path,
   - ambiguous user request,
   - out-of-scope request,
   - escalation path.
3. If capstone uses canvas/document flows, validate required scenario behavior.
4. Collect feedback entries for weak responses.
5. Log findings in `chapter-11-section-02-conversation-flow-log.md`.
6. Mark which defects are blocking vs non-blocking for release-candidate freeze.

Checkpoint:

- Core user journey is end-to-end executable.
- Known conversation defects are documented.
- Blocking classification is agreed and recorded.

### Phase E: Integrate delivery tracking and operational visibility

1. Update tasks and dependencies based on implementation outcomes.
2. Open **Timeline** and rebalance schedule if slippage exists.
3. Create or adjust milestones for release-candidate readiness.
4. Review **Dashboard** and conversation **Timeline** for early latency/cost anomalies.
5. Add integration defects as explicit tasks with owners.
6. Document one concrete mitigation per high-risk anomaly before moving forward.

Checkpoint:

- Delivery plan reflects real implementation status.
- Operational risks are visible and tracked.
- High-risk anomalies have mitigation paths.

### Phase F: Freeze release candidate and publish implementation summary

1. Create `chapter-11-section-02-implementation-summary.md`.
2. Include:
   - completed slices,
   - unresolved defects,
   - known risks,
   - candidate version ID,
   - readiness recommendation for eval/hardening (`go` or `go with constraints`).
3. Publish summary in Delivery Discussions.
4. Create a short "change freeze" note listing allowed changes after this point.

Checkpoint:

- Release candidate is clearly identified and scoped.
- Transition recommendation is explicit.
- Post-freeze change rules are documented.

## Expected outputs

- A traceable implementation baseline with tasks, owners, and dependencies.
- Working data synchronization and workflow execution path with run evidence.
- Versioned agent configuration topology with tested tool/handoff behavior.
- Validated channel conversation flows including non-happy-path handling.
- Delivery updates reflecting schedule, milestone, and integration realities.
- A release-candidate implementation summary (`chapter-11-section-02-implementation-summary.md`) with recommendation.

Implementation evidence should also include:

- slice acceptance checklist results for each completed slice,
- integration defect list with downstream impact tags,
- versioned change notes showing what was revalidated after major updates.

Evidence quality standard:

- A reviewer can verify what was built, what is incomplete, and why the candidate is (or is not) ready for eval/hardening.

## Failure modes

- **Layered implementation with late integration**
  Symptom: each subsystem appears complete but end-to-end flow fails near gate.
  Recovery: switch to vertical-slice execution and require slice-level acceptance checks.

- **Configuration drift across versions**
  Symptom: unclear which agent/channel config produced observed behavior.
  Recovery: enforce explicit version naming and log test results against version IDs.

- **Over-enabled tools during early build**
  Symptom: inconsistent assistant behavior and hard-to-diagnose tool side effects.
  Recovery: start with minimal tool set and add capabilities only with use-case justification.

- **Delivery artifacts not maintained during build**
  Symptom: status reports conflict with actual implementation state.
  Recovery: daily delivery sync discipline with evidence-linked task updates.

- **Integration defects hidden as minor issues**
  Symptom: cross-boundary bugs persist and reappear during demos.
  Recovery: create dedicated integration defect tasks with cross-owner accountability.

- **No candidate freeze before evaluation**
  Symptom: eval outcomes change continuously because implementation keeps shifting.
  Recovery: define release-candidate scope freeze and gate changes through explicit triage.

## Completion checklist

- [ ] Implementation tasks represent end-to-end slices, not disconnected layers.
- [ ] Data foundation and workflow runs are validated with logs.
- [ ] Agent configurations are versioned and behavior-tested.
- [ ] Core channel conversation journey works with fallback/escalation behavior.
- [ ] Delivery timeline and milestones are updated to real implementation status.
- [ ] Integration defects are tracked explicitly with owners.
- [ ] Early observability checks were run during implementation.
- [ ] Release candidate version is frozen and documented.
- [ ] Implementation summary is published with readiness recommendation.
- [ ] Team agrees on transition criteria into eval/hardening.

## Canonical references

Use [Tutorials](../../user-guide/tutorials/README.md) at `/platform/support/tutorials` as an optional accelerator only. When a walkthrough is still catching up to product behavior or media ownership, prefer the owning user-guide page in the reference list below.

- [Getting started](../../user-guide/README.md)
- [Tutorials](../../user-guide/tutorials/README.md)
- [Data Model](../../user-guide/data-model/README.md)
- [Entities](../../user-guide/data-model/entities.md)
- [Storage](../../user-guide/data-model/storage.md)
- [Pipelines](../../user-guide/data-model/pipelines.md)
- [Workflows](../../user-guide/data-model/workflows.md)
- [Runs](../../user-guide/data-model/runs.md)
- [Workflow Run Details](../../user-guide/data-model/workflow-run-details.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Agent Configuration](../../user-guide/agents/configs.md)
- [Channels](../../user-guide/conversations/channels/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Workflow actions](../../user-guide/conversations/workflow-actions.md)
- [File extractor registry](../../user-guide/conversations/file-extractor-registry.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)

---

# Eval And Hardening

## Learning objectives

- Build an evaluation strategy that directly supports capstone release decisions instead of generic score reporting.
- Run a hardening loop that converts defects into prioritized fixes with measurable regression control.
- Validate reliability, safety, and operational stability before recommending release.
- Produce a quality decision package that makes residual risk explicit and actionable.

## Prerequisites

- Completed Section 2 with a frozen release candidate and implementation summary.
- Access to Evals, Conversations review/feedback controls, Delivery Management, Dashboard, and Timeline surfaces.
- A baseline run or historical quality benchmark for comparison.
- Agreement on release-blocking quality thresholds and escalation criteria.

## In Gaia

- [Evals](../../user-guide/evals/README.md), [Eval Design Process](../../user-guide/evals/eval-design-process.md), and [Ultimate Guide](../../user-guide/evals/ultimate-guide.md) for the measurement loop
- [Conversations](../../user-guide/conversations/README.md) and [Dashboard](../../user-guide/dashboard/README.md) for production-like defect discovery and non-functional signals
- [Delivery Management](../../user-guide/delivery/README.md) and [Tasks](../../user-guide/tasks/README.md) for hardening work and release blockers

This section should produce a real Gaia quality decision package with runs, findings, fixes, and residual-risk notes.

## Concept brief

Eval and hardening is where capstone credibility is earned. Teams often confuse this stage with generating a high pass rate. The real objective is decision quality: can stakeholders trust the system to operate within acceptable risk? This requires focused eval design, disciplined remediation, and transparent treatment of residual uncertainty.

Alignment note:

- Apply [Eval Design Process](../../user-guide/evals/eval-design-process.md) to frame the release decision and gate criteria.
- Use [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md) as the operating loop for trace intake, asset creation, calibration, and regression discipline.

### 1) Start from the release decision, not from available metrics

Before creating or running evals, define the decision statement:

- what release claim are you trying to validate,
- what failure modes can block that claim,
- what threshold constitutes acceptable risk.

This aligns with Gaia’s eval design guidance and prevents dashboards full of numbers that do not influence decisions.

Capture this as a single sentence decision hypothesis (for example, "the assistant can support customer triage with acceptable policy and latency risk for controlled rollout"). Every eval artifact should either strengthen or weaken that hypothesis.

### 2) Build failure-mode-centric datasets

Datasets should represent the ways the system can fail in production, not only the ways it usually succeeds. Include:

- ambiguous inputs,
- edge-case policy questions,
- tool failure fallback scenarios,
- escalation-required prompts,
- channel-specific language variability.

If your dataset only covers happy paths, hardening results will be misleading.

Aim for representative class balance, not equal class counts. High-impact rare failures should be intentionally over-sampled in hardening datasets so they are visible before release.

### 3) Match grader strategy to criterion type

Not every criterion needs an LLM rubric. Use a mixed grader strategy:

- deterministic checks for structured outputs and strict policy rules,
- transcript-pattern checks for required/forbidden language,
- rubric graders for nuanced quality dimensions,
- multi-grader aggregation for robust verdicts.

This improves signal reliability and makes trial-level diagnosis faster.

When graders disagree, do not hide the disagreement in aggregates. Record disagreement classes and sample affected trials for manual review. Disagreement often points to weak criterion wording or ambiguous task construction.

### 4) Calibrate before scaling run volume

Running large evals too early amplifies bad measurement. First run a pilot set and calibrate:

- prompt/task clarity,
- grader strictness,
- false positives/false negatives,
- output interpretability.

Only after calibration should you scale trials/task and dataset breadth.

Define calibration exit criteria explicitly, such as "grader disagreement < X%" or "no blocker-level false negatives in pilot set." This prevents premature scaling when measurement quality is still unstable.

### 5) Treat Conversations as defect discovery input

Evals are not your only quality source. Conversation review states, tags, and feedback threads provide high-value real usage defects. Convert those defects into eval tasks whenever repeatability matters.

This creates a closed loop:

- production-like trace -> error analysis -> eval asset -> regression check.

The loop is core to long-term quality stability.

### 6) Hardening must include non-functional risks

Capstone hardening is not only correctness. Include non-functional checks:

- latency outliers,
- tool reliability,
- cost spikes,
- escalation handoff quality,
- incident detectability.

Use Dashboard and conversation Timeline to inspect these signals. A highly accurate but operationally unstable assistant is not release-ready.

Also classify non-functional findings by persistence: transient, intermittent, or systemic. This helps decide whether a mitigation should be immediate, monitored, or deferred with constraints.

### 7) Prioritize fixes by decision impact

When many defects appear, prioritize by impact on release decision dimensions:

- safety/policy violations,
- high-frequency user-journey failures,
- critical business outcome regressions,
- severe operational instability.

Lower-impact polish defects should not block core risk mitigation work.

Use a fix-priority matrix with two axes: user impact and recurrence likelihood. Items high on both axes should bypass normal batching and move directly into immediate hardening work.

### 8) Enforce regression protection for every major fix

Each major fix should include a regression guard:

- rerun affected eval slice,
- compare against baseline and pre-fix run,
- verify no major adjacent regression.

Without this, teams oscillate between failure classes and never converge.

Track fix lineage in one place: defect ID -> fix version -> rerun ID -> result delta. This compact lineage record is invaluable during final release review and future audits.

### 9) Document residual risk explicitly

No release is risk-free. The critical distinction is documented vs hidden risk. Your hardening report should state:

- what risk remains,
- why it is acceptable or not,
- what constraints/monitoring are required.

This supports honest go/no-go decisions and prevents false confidence.

Residual risk should include a time horizon. A risk accepted for a two-week controlled rollout may be unacceptable for broad production scale. Time-boxed acceptance improves accountability.

### 10) Definition of done for Eval and Hardening

Eval and hardening is complete when:

- release decision criteria are evaluated with fit-for-purpose graders,
- major defects are triaged and remediated with evidence,
- regression checks confirm net quality improvement,
- non-functional stability is reviewed,
- a final recommendation with residual risk posture is documented.

In mature teams, this stage also produces a short "next-cycle quality backlog" so unresolved warning-level issues do not disappear after release approval.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

The `Customer Operations Copilot` release candidate is complete. Leadership wants a release recommendation in 72 hours. You must run a focused quality gate that combines eval evidence, conversation-derived defects, and operational hardening checks.

### Phase A: Define release decision and failure map

1. Create `chapter-11-section-03-release-decision-frame.md`.
2. Document:
   - release claim,
   - blocking failure modes,
   - thresholds by criterion,
   - acceptable residual-risk envelope.
3. Align thresholds with stakeholders in Delivery Discussions.
4. Mark each criterion as `blocking`, `warning`, or `informational`.

Checkpoint:

- Decision frame is explicit and approved.
- Blocking conditions are clearly defined.
- Criterion severity labels are documented.

### Phase B: Build or refine eval assets

1. In **Evals**, create/refine dataset folders and task sets for capstone quality gate.
2. Add tasks derived from known high-risk scenarios and conversation defects.
3. Configure graders (deterministic/pattern/rubric/multi-grader) per criterion.
4. Run a pilot set (10-20 tasks) and calibrate grader behavior.
5. Capture changes in `chapter-11-section-03-eval-calibration-log.md`.
6. Document known blind spots where human review is still required.

Checkpoint:

- Eval assets map directly to release decision criteria.
- Calibration issues are resolved before full run.
- Blind spots are explicit and owned.

### Phase C: Execute full quality runs and compare baseline

1. Start full eval run(s) from **Runs**.
2. Monitor **Run details** for status distribution and failure concentration.
3. Open **Reports** and compare baseline vs candidate outcomes.
4. Export or summarize key metrics and failure clusters.
5. Record findings in `chapter-11-section-03-eval-results.md`.
6. Flag blocker criteria that fail threshold and open immediate remediation tasks.

Checkpoint:

- Comparative metrics are available and interpretable.
- High-impact failure clusters are identified.
- Blocker failures are routed directly to action.

### Phase D: Perform hardening remediation loop

1. Convert major defects into delivery tasks with owner and due date.
2. Implement focused fixes (prompt/tool/config/data/workflow as needed).
3. Re-run targeted eval slices for each major fix.
4. Validate affected conversation traces in **Conversations -> Timeline**.
5. Track remediation outcomes in `chapter-11-section-03-hardening-log.md`.
6. Record result deltas (before vs after) for each fix.

Checkpoint:

- Major defects have concrete fixes and verification evidence.
- Regression protection is applied per major fix.
- Fix impact is measurable, not anecdotal.

### Phase E: Validate non-functional stability

1. Open **Dashboard** and inspect key metrics during hardening runs.
2. Review timeline traces for latency/tool bottlenecks.
3. Check for abnormal cost or error-rate drift.
4. If needed, update monitoring constraints and rollout safeguards.
5. Add summary in `chapter-11-section-03-operational-stability-note.md`.
6. Classify findings by transient/intermittent/systemic behavior.

Checkpoint:

- Non-functional risk posture is explicitly assessed.
- Monitoring plan is adjusted where required.
- Persistence class is assigned to each major finding.

### Phase F: Publish final hardening recommendation

1. Create `chapter-11-section-03-quality-gate-report.md` including:
   - decision criteria status,
   - resolved vs unresolved blockers,
   - residual risks,
   - recommendation (`go`, `go with constraints`, `no-go`).
2. Publish report summary in Delivery Discussions.
3. Link all supporting artifacts and eval runs.
4. Add a 1-week and 4-week follow-up check for any accepted residual risk.
5. Define which warning-level risks are explicitly deferred into the next cycle backlog.

Checkpoint:

- Recommendation is evidence-backed and auditable.
- Constraints/next actions are explicit.
- Residual-risk follow-up is scheduled.
- Deferred warning-level risks are tracked and owned.

## Expected outputs

- A release decision frame artifact with thresholds and blocker definitions.
- Calibrated eval assets tied to real failure modes and conversation findings.
- Full-run comparative evidence (baseline vs candidate) with failure-cluster analysis.
- Hardening remediation log with fix-to-regression-check linkage.
- Non-functional stability review covering latency/cost/error behavior.
- Final quality gate report (`chapter-11-section-03-quality-gate-report.md`) with explicit go/no-go recommendation and residual-risk profile.

Recommended hardening package additions:

- criteria-severity table (`blocking` / `warning` / `informational`),
- grader disagreement summary and human-review resolution notes,
- fix lineage table connecting defects, versions, reruns, and outcome deltas.

Release-meeting prompt set (recommended):

- Which blockers were fully resolved vs mitigated?
- Which warning-level issues were accepted and why?
- What monitoring signal will indicate that accepted risk is no longer acceptable?
- Who owns the first escalation response if that signal fires?

Evidence quality standard:

- A reviewer can audit how each release decision criterion was tested, what defects remained, and why the final recommendation is justified.

## Failure modes

- **Eval strategy disconnected from release decision**
  Symptom: high score totals but unclear impact on go/no-go call.
  Recovery: rewrite decision frame and remap datasets/graders to decision criteria.

- **Overreliance on one grader type**
  Symptom: important failure classes are missed or over-reported.
  Recovery: apply mixed grader strategy and validate trial-level signal quality.

- **Calibration skipped before scaling**
  Symptom: large runs produce noisy or contradictory verdicts.
  Recovery: run pilot calibration and adjust tasks/graders before full execution.

- **Fixes applied without regression checks**
  Symptom: one defect improves while adjacent behavior degrades.
  Recovery: enforce targeted reruns and comparison against baseline and pre-fix state.

- **Non-functional hardening ignored**
  Symptom: correctness improves but latency/cost incidents increase.
  Recovery: include dashboard/timeline stability checks as blocking criteria where needed.

- **Residual risks omitted from final recommendation**
  Symptom: release recommendation appears "green" but hidden constraints surface later.
  Recovery: require explicit residual-risk section and constraint tracking tasks.

## Completion checklist

- [ ] Release decision criteria and thresholds are documented.
- [ ] Eval datasets reflect real failure modes, not only happy paths.
- [ ] Grader strategy is matched to criterion types.
- [ ] Pilot calibration is completed before scaling runs.
- [ ] Full-run comparisons against baseline are documented.
- [ ] Major defects are triaged into owned remediation tasks.
- [ ] Regression checks are executed for major fixes.
- [ ] Non-functional stability signals are reviewed.
- [ ] Final quality gate report states residual risk and recommendation.
- [ ] Delivery stakeholders reviewed and acknowledged the decision package.

## Canonical references

- [Evals](../../user-guide/evals/README.md)
- [Eval Design Process](../../user-guide/evals/eval-design-process.md)
- [Ultimate Guide: Creating and Evolving Evals in Gaia](../../user-guide/evals/ultimate-guide.md)
- [Eval Runs](../../user-guide/evals/runs.md)
- [Run details](../../user-guide/evals/run-details.md)
- [Reports](../../user-guide/evals/reports.md)
- [Scenario: Create and run an eval](../../user-guide/evals/scenarios/create-and-run-an-eval.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Give feedback on a reply](../../user-guide/conversations/dialogs/feedback.md)
- [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)

---

# Demo And Handoff

## Learning objectives

- Design and deliver a capstone demo that proves operational readiness rather than only showcasing features.
- Build a handoff package that enables another team to run, monitor, and improve the solution without creator dependency.
- Transfer ownership across product, engineering, and operations with explicit responsibilities and escalation paths.
- Complete final go/no-go decision artifacts with clear residual-risk and first-week controls.

## Prerequisites

- Completed Section 3 quality gate report and associated evidence.
- A frozen candidate version approved for demo/handoff review.
- Identified receiving owners for product operations, technical maintenance, and incident communication.
- Access to Delivery, Conversations, Dashboard, Audit, and Settings/project-role management.

## In Gaia

- [Conversations](../../user-guide/conversations/README.md) and [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md) for live demo evidence
- [Delivery Management](../../user-guide/delivery/README.md), [Delivery Discussions](../../user-guide/delivery/discuss.md), [Tasks](../../user-guide/tasks/README.md), and [Delivery Milestones](../../user-guide/delivery/milestones.md) for handoff continuity
- [Dashboard](../../user-guide/dashboard/README.md), [Audit Trail](../../user-guide/audit/README.md), and [Settings](../../user-guide/settings/README.md) for operational ownership and access transfer

Run the demo and handoff as a Gaia operating review, not a slide-driven presentation. The receiving team should be able to inspect the same evidence directly in the platform.

## Concept brief

Many capstones fail at the last mile: the demo is impressive, but handoff is weak, and the system degrades once original builders step away. A production handoff should be treated as engineering work with measurable outputs. The goal is not to "present" the system; the goal is to transfer reliable operating capability.

### 1) Define demo purpose as decision support

A good demo answers stakeholder decisions:

- should we launch now,
- under what constraints,
- with which owners,
- with what fallback.

If the demo only highlights smooth interactions, stakeholders cannot make informed release decisions.

Use a pre-demo decision worksheet with three fields: "claim," "evidence," and "decision impact." This keeps the presentation grounded in release choices and avoids drifting into feature narration.

### 2) Structure the demo around critical journeys and failure handling

Feature tours are low signal. Demonstrate:

- one core value journey end to end,
- one edge-case/failure journey with mitigation,
- one operational monitoring view showing detectability.

This proves both capability and resilience.

For each journey, state in advance what "success" and "acceptable imperfection" mean. This avoids real-time argument over whether an observed behavior is a blocker, warning, or expected limitation.

### 3) Use evidence-first narration

In production reviews, claims need evidence. For each major statement (accuracy, stability, readiness), provide linked evidence:

- eval reports,
- conversation timeline traces,
- delivery completion artifacts,
- monitoring snapshots.

Evidence-first demos reduce subjective debate and accelerate approvals.

When possible, show both baseline and current evidence for quality-critical claims. Relative improvement context is often more persuasive than isolated "current score" snapshots.

### 4) Make operational ownership visible during demo

Handoff confidence increases when ownership is demonstrated, not just documented. During demo, explicitly show:

- who receives alerts,
- who can change configs,
- who approves rollbacks,
- who triages feedback.

This prevents "looks ready" outcomes where no one actually owns run-time operations.

When possible, have each owner narrate their own part of the flow during demo. This validates understanding and exposes handoff gaps that are easy to miss when one presenter speaks for all domains.

### 5) Package runbooks as executable workflows

Runbooks should be actionable under time pressure. Include:

- normal operations checklist,
- incident triage flow,
- rollback triggers/steps,
- communication templates,
- recovery validation steps.

Runbooks that require oral context are incomplete handoff artifacts.

Each runbook should include trigger condition, first action, owner, and verification step. This transforms documents from guidance into operational procedures that can be executed by a new team member under pressure.

### 6) Transfer permissions deliberately

Incorrect permission transitions cause both security risk and operational delays. Verify project roles and assignments for:

- receiving operators,
- reviewers/auditors,
- administrators,
- temporary contributors.

Principle: least privilege with enough authority to keep service healthy.

Add an "access test" after assignment: each receiving role performs one expected action in Gaia (for example update a delivery task, run an eval, or review timeline). This validates practical usability, not just role configuration intent.

### 7) Align handoff with delivery backlog continuity

Handoff does not mean backlog disappears. Define post-demo backlog categories:

- must-do before launch,
- first-week stabilization,
- deferred improvements.

Link each to tasks/milestones so continuation is visible and schedule-aware.

Add explicit dependency notes between first-week stabilization work and deferred improvements. This prevents teams from starting long-term optimization work before immediate stability actions are closed.

### 8) Capture baseline operational metrics at handoff time

Without a baseline, post-launch drift is hard to interpret. Capture at handoff:

- response latency profile,
- error/fallback rate,
- escalation volume,
- eval quality baseline,
- cost indicators.

This baseline supports objective first-week review.

Also capture known seasonality/context notes (for example expected daily peak windows). Baseline interpretation is significantly better when reviewers know what "normal variance" looked like at handoff time.

### 9) Create an explicit acceptance protocol

Handoff should close only after named recipients accept responsibilities. Use a lightweight acceptance protocol:

- artifact completeness check,
- role/permission verification,
- first-week on-call schedule,
- escalation contacts,
- final go/no-go acknowledgment.

This prevents ambiguous ownership gaps.

Require named sign-off for each acceptance domain: product acceptance, technical acceptance, and operations acceptance. Partial acceptance should be allowed only with documented constraints and deadline for closure.

### 10) Definition of done for Demo and Handoff

Demo/handoff is complete when:

- decision-critical journeys and failure handling are demonstrated,
- evidence package supports claims,
- operational ownership is transferred and accepted,
- runbooks and backlog continuity are in place,
- release recommendation is explicit with constraints and first-week controls.

The strongest handoffs also include an agreed "30-day review" checkpoint where receiving owners validate whether constraints were accurate and whether escalation triggers were tuned appropriately.

## Gaia lab

### Core path (minimum)

- Complete enough lab phases to produce one reviewable artifact and validate one end-to-end scenario.
- Record at least one observed risk and one follow-up action before moving on.

### Extended path (recommended)

- Complete all lab phases, failure-mode drills, and the full completion checklist.

### Scenario

You are conducting final capstone review for `Customer Operations Copilot`. Audience includes product leadership, operations managers, and an engineering owner who will maintain the system after handoff. You must demonstrate readiness, transfer responsibilities, and close with a release recommendation.

### Phase A: Prepare demo narrative and evidence deck

1. Create `chapter-11-section-04-demo-script.md`.
2. Include:
   - core journey demo path,
   - edge-case/failure demo path,
   - operational dashboard/timeline checkpoints,
   - decision questions for stakeholders.
3. Link each segment to supporting evidence artifacts from previous sections.
4. Add fallback demo path if a live dependency fails during presentation.

Checkpoint:

- Demo script is decision-oriented.
- Evidence links are complete.
- Fallback script exists for live-demo risk.

### Phase B: Prepare handoff package artifacts

1. Create `chapter-11-section-04-handoff-package.md` containing:
   - system overview,
   - configuration/version inventory,
   - runbooks (normal/incident/rollback),
   - monitoring and alert playbook,
   - escalation matrix.
2. Add references to key Gaia pages and artifacts.
3. Validate package with receiving technical owner.
4. Perform a dry-run where the receiving owner follows one runbook without presenter guidance.

Checkpoint:

- Handoff package is usable without live walkthrough.
- Receiving owner confirms clarity of steps.
- At least one runbook is validated via dry-run.

### Phase C: Verify roles, permissions, and accountability

1. In **Settings -> Project roles / Project users**, verify role assignments for receiving team.
2. Confirm discuss moderation, delivery update, and eval management permissions align with responsibilities.
3. Update governance artifact with final owner list.
4. Record permission verification in `chapter-11-section-04-ownership-verification.md`.
5. Execute role-based access tests for critical operational actions.

Checkpoint:

- No critical responsibility lacks required access.
- Least-privilege model is preserved.
- Role assignments are functionally validated.

### Phase D: Run formal demo and capture decisions

1. Execute core and edge-case flows in **Conversations**.
2. Show relevant **Timeline** traces and **Dashboard** snapshot.
3. Review quality gate highlights and unresolved constraints.
4. Capture stakeholder questions and decisions in Delivery Discussions.
5. Update demo script with final outcomes.
6. Label each raised concern as `blocker`, `constraint`, or `follow-up`.
7. Convert all blockers and accepted constraints into tracked tasks before closing the session.

Checkpoint:

- Stakeholders see both capability and risk controls.
- Decision log captures approvals and concerns.
- Concern severity labels are explicit.
- Blockers/constraints are translated into owned tasks.

### Phase E: Close backlog continuity and first-week plan

1. In **Delivery Management**, finalize:
   - pre-launch blocker tasks,
   - first-week stabilization tasks,
   - deferred backlog.
2. Set milestones/timeline windows and owners.
3. Define first-week monitoring cadence and escalation rota.
4. Save in `chapter-11-section-04-first-week-operations-plan.md`.
5. Add explicit trigger thresholds for first-week escalation.
6. Schedule a day-7 and day-30 service review with defined agenda and owners.

Checkpoint:

- Post-demo work is fully tracked.
- First-week operation responsibilities are explicit.
- Escalation triggers are measurable.
- Post-launch review cadence is pre-scheduled.

### Phase F: Publish final release recommendation and acceptance

1. Create `chapter-11-section-04-final-release-decision.md` with:
   - acceptance status by owner,
   - known constraints,
   - residual risks,
   - final recommendation (`go`, `go with constraints`, `no-go`).
2. Link all critical artifacts and version tags.
3. Publish final summary in Delivery Discussions and archive key audit references.
4. Record acceptance sign-off from product, technical, and operations owners.
5. Add a handoff-completion note describing any constraints that remain open after sign-off.

Checkpoint:

- Final decision is traceable and approved by named stakeholders.
- Handoff acceptance is complete.
- Cross-domain sign-off is captured.
- Open post-sign-off constraints are explicit.

## Expected outputs

- A decision-oriented demo script with evidence linkage.
- A complete handoff package including runbooks, ownership matrix, and operational guidance.
- Verified project-role/permission mapping for receiving operators.
- Formal demo outcome record with stakeholder decisions and open constraints.
- A first-week operations plan linked to delivery tasks/milestones.
- A final release decision artifact (`chapter-11-section-04-final-release-decision.md`) with explicit recommendation and acceptance status.

Recommended handoff evidence additions:

- role-based access test log for critical workflows,
- runbook dry-run outcome with any corrections made,
- concern severity register from the demo (`blocker` / `constraint` / `follow-up`),
- first-week trigger-threshold table used for escalation decisions.

Suggested final handoff appendix (high value):

- top five operational risks and current status,
- owner contact card (primary and backup),
- change-freeze window and exception path,
- first-week communication template for daily status updates.

Evidence quality standard:

- A receiving team can operate and improve the system independently from day one after handoff.

## Failure modes

- **Demo optimized for aesthetics, not decisions**
  Symptom: stakeholders leave impressed but unclear about launch readiness.
  Recovery: restructure demo around decision-critical journeys and explicit release questions.

- **Handoff artifacts incomplete or implicit**
  Symptom: receiving team depends on original builders for routine operations.
  Recovery: require runbooks and ownership matrix with executable steps and escalation contacts.

- **Permission transfer gaps**
  Symptom: operators cannot perform required actions or have excessive privileges.
  Recovery: verify project-role assignments against responsibility matrix before final sign-off.

- **No first-week operating plan**
  Symptom: launch occurs with unclear monitoring cadence and escalation path.
  Recovery: create first-week task plan with owners, shifts, and trigger thresholds.

- **Residual risk hidden in final recommendation**
  Symptom: release decision appears binary with no constraint visibility.
  Recovery: include mandatory residual-risk and constraint section in final decision artifact.

- **Post-demo backlog not integrated into Delivery**
  Symptom: unresolved issues are discussed but not tracked.
  Recovery: convert outcomes into delivery tasks/milestones during closeout meeting.

## Completion checklist

- [ ] Demo script covers core flow, edge-case handling, and operational observability.
- [ ] Evidence links support each major readiness claim.
- [ ] Handoff package includes runbooks, ownership matrix, and escalation paths.
- [ ] Role and permission assignments are verified for receiving team.
- [ ] Stakeholder decisions and constraints are recorded during demo.
- [ ] Post-demo backlog is categorized and tracked in Delivery.
- [ ] First-week operations plan is published with clear owners.
- [ ] Final release recommendation states residual risks and constraints.
- [ ] Handoff acceptance is confirmed by named recipients.
- [ ] Key audit references/version tags are preserved for traceability.

## Canonical references

- [Conversations](../../user-guide/conversations/README.md)
- [View a conversation timeline](../../user-guide/conversations/dialogs/timeline.md)
- [Give feedback on a reply](../../user-guide/conversations/dialogs/feedback.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Delivery Discussions](../../user-guide/delivery/discuss.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Delivery Milestones](../../user-guide/delivery/milestones.md)
- [Delivery Timeline (Gantt)](../../user-guide/delivery/timeline.md)
- [Dashboard](../../user-guide/dashboard/README.md)
- [Audit Trail](../../user-guide/audit/README.md)
- [Settings](../../user-guide/settings/README.md)
- [Project roles](../../user-guide/settings/project-roles.md)

---

# Governed Application From Scratch

## Learning objectives

- Build a governed Gaia application as one connected system instead of as separate data, agent, eval, delivery, and governance workstreams.
- Use the governance package model early enough that package scope, downstream assignment, evidence ownership, and release criteria shape architecture decisions.
- Translate one realistic business scenario into a traceable implementation plan that survives review, hardening, and handoff.
- Produce a final governed application package that another Gaia team can understand, review, and operate without relying on tribal knowledge.

## Prerequisites

- You completed Sections 1-4 of Chapter 11 and Chapters 3, 4, 6, 8, and 9.
- You have access to Data Model, AI Agents, Conversations, Evals, Governance, Delivery Management, Tasks, Audit Trail, and Settings.
- Your project already has enough baseline structure to create entities, workflows, agents, evals, delivery tasks, and governance records.
- You can use the Governance registry and the implemented framework-package workflow, including package authoring, publishing, import, and downstream assignment.

## In Gaia

- [Data Model](../../user-guide/data-model/README.md), [AI Agents](../../user-guide/agents/README.md), [Conversations](../../user-guide/conversations/README.md), and [Evals](../../user-guide/evals/README.md) for the operating system you are building
- [Governance](../../user-guide/governance/README.md), [Governance Registry](../../user-guide/governance/registry.md), and [Governed application lifecycle](../../user-guide/governance/governed-application-lifecycle.md) for package-backed oversight and lifecycle review
- [Delivery Management](../../user-guide/delivery/README.md), [Tasks](../../user-guide/tasks/README.md), and [Audit Trail](../../user-guide/audit/README.md) for execution truth and review evidence

Keep one live Gaia project open for the whole tutorial. Every phase should leave behind a visible product artifact, governance record, eval signal, and delivery update.

## Concept brief

This section is the capstone tutorial for building a real, demonstrable neo-bank application on Gaia. Work it in order. The handbook owns the full sequence, decision logic, and expected outputs. Use the User Guide only when you need feature-level operating instructions for a specific Gaia surface.

Treat the in-product Tutorials page at `/platform/support/tutorials` as optional companion practice, not as the primary learning path for this scenario. If a live tutorial card covers part of the flow, use it to rehearse the step after you complete the corresponding handbook phase.

The earlier capstone sections teach the parts of delivery. This final section teaches composition. A governed Gaia application is credible only when those parts reinforce each other under one operating model.

The failure mode to avoid is "parallel maturity theater": the data model exists, the assistant exists, the evals exist, and the governance records exist, but none of them clearly explain the same system. Reviewers then see isolated artifacts instead of a governed product.

This section uses the `Customer Operations Copilot` scenario as the reference build. The scenario is intentionally realistic but bounded: internal banking operations teams use the application to onboard customers, handle account and card servicing, triage suspicious transactions, and escalate uncertain or high-impact cases. The system must be useful, reviewable, and governable.

This chapter still establishes the scaffold. Continue into [Chapter 12](#doc-ch12-functional-governed-application) when you want the functional follow-on path for live contracts, runtime policies, and the remaining governance completion slices.

Tutorial execution rule:

- complete the phases in sequence from A through H,
- produce the named working artifacts before moving to the next phase,
- keep one live project open throughout the build so the application stays demonstrable,
- use the checkpoints as hard gates rather than reading prompts,
- finish with a working demo path and a review-ready handoff pack.

### Tutorial map

| Phase | Outcome                                               | Working artifact                                        | Primary Gaia surface                                 |
| ----- | ----------------------------------------------------- | ------------------------------------------------------- | ---------------------------------------------------- |
| A     | Governed scenario scope is explicit                   | `chapter-11-section-05-governed-scenario-brief.md`      | project brief and review notes                       |
| B     | Core records and workflow backbone exist              | `chapter-11-section-05-data-and-workflow-design.md`     | Data Model                                           |
| C     | Agents, handoffs, and authority are reviewable        | `chapter-11-section-05-agent-topology-and-authority.md` | AI Agents                                            |
| D     | Release evidence has measurable gates                 | `chapter-11-section-05-governed-release-criteria.md`    | Evals                                                |
| E     | Execution truth is tracked with owners and milestones | `chapter-11-section-05-delivery-orchestration-plan.md`  | Delivery Management                                  |
| F     | Governance boundary is package-backed                 | `chapter-11-section-05-governance-package-baseline.md`  | Governance Registry                                  |
| G     | Evidence, approvals, and explainability are linked    | `chapter-11-section-05-evidence-and-approvals.md`       | Governance Evidence and linked operational artifacts |
| H     | The app is demonstrable and handoff-ready             | `chapter-11-section-05-governed-handoff-pack.md`        | demo flow, Delivery Discussions, and handoff review  |

Use this table as the tutorial control sheet. If a phase artifact is missing or weak, stop and repair that phase before continuing.

### 1) Start with one business objective and one governed operating boundary

The strongest build-from-scratch projects do not begin with a long feature list. They begin with one operational promise and one explicit boundary.

For this capstone, the promise is: "help internal customer-operations teams resolve customer requests faster while preserving escalation discipline, policy adherence, and review evidence." The boundary is equally important: the assistant supports decision preparation and structured follow-through, but it does not become an unreviewed autonomous decision-maker for high-impact cases.

That boundary should appear in every layer:

- the data model distinguishes operational facts from review outcomes,
- the agents know when to stop and escalate,
- the evals test refusal and escalation behavior,
- the governance package model captures which obligations and controls apply,
- the delivery plan tracks unresolved gaps before release.

If the team cannot explain the governed boundary in one paragraph, the rest of the build will drift.

### 2) Design the data model around decisions, not only records

The governed application starts with entities, but the right question is not "what data do we have?" The right question is "what decision or workflow does each record support?"

For `Customer Operations Copilot`, a practical core entity set is:

- `Customer`
- `Account`
- `Card`
- `Transaction`
- `Case`
- `PolicyNotice`
- `ReviewDecision`

These entities are useful because they separate operational state from policy interpretation and human review. `Transaction` and `Case` support operational triage. `PolicyNotice` preserves the policy text or interpretation the assistant should rely on. `ReviewDecision` captures the governed outcome when the workflow requires human approval, exception handling, or formal sign-off.

The system should also preserve supporting artifacts without collapsing them into entity clutter:

- uploaded evidence or identity files,
- workflow-run outputs,
- eval reports,
- delivery artifacts,
- governance evidence links.

The design discipline is to keep the core entities understandable while still linking them to the operational sources reviewers need later. That is why governance evidence should point back to workflow runs, delivery tasks, eval reports, and explainability artifacts instead of duplicating them into hand-maintained summaries.

### 3) Build agent topology from workflow responsibility, not model enthusiasm

Governed applications usually fail when the team gives one assistant too much responsibility too early. A safer pattern is a small topology with explicit workflow roles.

For this capstone, the topology is:

- `Orchestrator Agent`
- `Onboarding Agent`
- `Customer Service Agent`
- `Risk Review Agent`
- optional governance or compliance support for structured review preparation

This structure is useful because it keeps operational handling separate from policy-sensitive review. The `Customer Service Agent` can answer account and card servicing questions when the request stays within approved policy. The `Risk Review Agent` handles suspicious transactions, ambiguous evidence, or workflows that require escalation. The `Orchestrator Agent` routes the interaction and keeps the user experience coherent without pretending every request should stay inside one prompt.

Tool strategy should follow the same principle. Start with the smallest read and write surface that supports the target workflow. Read-heavy lookup and structured case creation usually belong earlier than broader update rights. High-impact actions should either require a narrower tool contract or route into a human review task. The goal is not to make the assistant weak. The goal is to make its authority explainable.

### 4) Define eval strategy and release criteria before polishing the experience

Teams often wait until the application "feels complete" before formalizing evals. That sequence is backwards for governed systems because the release decision is not based on polish. It is based on evidence.

The eval strategy for this capstone should prove at least four things:

- grounded responses stay tied to the customer, account, transaction, and case context,
- high-impact or ambiguous requests escalate correctly,
- disallowed actions are refused or redirected,
- policy-sensitive answers remain aligned with the approved operating guidance.

Release criteria should not live only in the eval workspace. Put them into the delivery and governance rhythm as well. A practical release frame is:

- `go` when governed behavior, evidence freshness, and operational owners are complete,
- `go with constraints` when known issues are bounded, accepted, and monitored,
- `no-go` when missing controls, stale evidence, broken escalation behavior, or unresolved blocker defects remain.

This matters because a passing eval score with weak governance traceability is still not a releasable governed system. Likewise, complete governance paperwork with weak behavioral evidence is not enough either. The governed application requires both.

### 5) Use delivery management as the system-of-record for execution truth

A build-from-scratch chapter is not complete if it explains architecture but not execution control. Delivery is where teams convert design intent into accountable progress.

For this capstone, Delivery should carry:

- implementation slices,
- governance remediation tasks,
- release-readiness milestones,
- approval checkpoints,
- handoff and first-week operating tasks.

The most useful pattern is to keep one visible chain from issue discovery to closure. When an eval fails, a governance gap appears, or a workflow run exposes a defect, that result should become a tracked task with an owner, due window, and decision impact. This prevents review findings from staying in side notes.

Use milestone language that reflects governed progress, not only engineering completion. Examples include:

- implementation baseline ready,
- governed package baseline approved,
- release candidate frozen,
- final evidence review complete,
- handoff accepted.

When delivery artifacts and governance records disagree, believe the mismatch and investigate it. That disagreement usually signals the exact operating gap that would surface during release review.

### 6) Set up governance packages before hardening week

The governance package model is the structural difference between a generic capstone and a governed one.

For this scenario, the package stack should normally include:

- formal standards such as `EU AI Act`, `NIST AI RMF`, or `ISO/IEC 42001`,
- a banking overlay package for sector-specific obligations,
- one internal governance profile for local release and review expectations.

Gaia's current model is intentionally practical:

- framework keys normalize scope across governance records,
- projects can author and publish framework packages,
- published packages can be imported from other accessible projects,
- risks, contracts, policies, controls, state profiles, obligations, regulations, regulatory source collections, and regulatory updates can be assigned explicit downstream packages.

That means governance setup is no longer just "choose a framework name." The team can define a reusable package boundary, publish it when stable, import it when reuse is needed, and make downstream adoption visible on the records that actually carry the governed behavior.

Do this early. If package setup waits until the end, the team usually discovers that contracts, policies, state profiles, obligations, controls, source collections, or regulatory updates were never attached to the intended governance boundary. Review then becomes a reconstruction exercise instead of a normal operating workflow.

### 7) Make evidence, approvals, and explainability part of normal execution

Evidence quality determines whether governance is trusted. Strong teams do not prepare evidence only for the final meeting. They preserve it while the work is happening.

For `Customer Operations Copilot`, the evidence spine should connect:

- entity and workflow behavior,
- eval results and grader findings,
- delivery decisions and remediation tasks,
- governance obligations and controls,
- explainability outputs for sensitive workflows,
- audit trail and review notes.

Approval design should follow the workflow risk. Routine, low-risk configuration changes may need only the owning engineer and product lead. High-impact workflow changes, policy changes, or package-boundary changes should route through named governance or operations reviewers. The goal is proportional review, not approval theater.

Explainability matters because reviewers need to see why the application produced a response, escalated a case, or refused an action. In Gaia, explainability is strongest when it connects to the same operational facts already used elsewhere in the system. A standalone explanation document without links back to source records, runs, and review tasks is weaker than a smaller explanation artifact with traceable provenance.

### 8) Treat demo, handoff, and operational follow-through as part of the governed build

The final governed application is not done when the system works in one controlled session. It is done when another team can understand the scope, run the workflows, review the evidence, and operate the release with clear constraints.

The capstone demo should therefore prove:

- one value-producing customer-operations journey,
- one policy-sensitive or high-risk escalation journey,
- one evidence path from behavior to review artifact,
- one delivery path from open gap to owned follow-up.

The handoff package should then preserve:

- current version and package baseline,
- known constraints and accepted residual risks,
- operating owners and escalation path,
- first-week monitoring cadence,
- rollback or freeze triggers when governed behavior drifts.

This is the practical end state of the chapter. The governed application is no longer just something the original builders can explain. It becomes something a receiving team can inspect and operate.

### 9) Definition of done for a governed build-from-scratch capstone

This final capstone section is complete when:

- the business objective, system boundary, and escalation logic are explicit,
- the data model supports both operations and reviewable decisions,
- agent roles and tool authority are narrow enough to explain and test,
- evals and release criteria measure governed behavior instead of generic quality only,
- delivery artifacts track remediation and approvals as first-class work,
- framework packages are authored or imported and assigned to working governance records,
- evidence and explainability link back to real system artifacts,
- demo and handoff materials preserve operational continuity after release review.

## Gaia lab

### Core path (minimum)

- Complete enough phases to produce one reviewable governed application package and one decision-ready release recommendation.
- Record at least one package-scope decision, one escalation design decision, and one owned remediation task before closing the lab.

### Extended path (recommended)

- Complete all phases, rehearse the review path end to end, and validate the full handoff package with a second operator.

### Scenario

You are building `Customer Operations Copilot` for a neo-bank. The system helps internal operations teams onboard customers, answer account and card servicing questions, triage suspicious transactions, and create governed follow-up tasks. Governance is not a late review step. The project must use the now-stable package model from the start so reviewers can inspect package scope, downstream adoption, evidence freshness, and explainability before release.

Tutorial outcome:

- a working neo-bank operations application that can be demonstrated live,
- a governance package baseline that explains why the application is releasable or blocked,
- a handbook-aligned evidence pack that another Gaia team can review and operate.

### Phase A: Write the governed scenario brief

1. Create `chapter-11-section-05-governed-scenario-brief.md`.
2. Define:
   - business objective,
   - primary users,
   - unacceptable failure classes,
   - mandatory human-escalation conditions,
   - in-scope governed workflows.
3. Add one explicit non-goal that limits assistant authority.
4. Validate the brief with one operations owner and one governance-minded reviewer.

Checkpoint:

- The business objective is clear enough to anchor architecture.
- The governed operating boundary is explicit.
- Escalation conditions are concrete, not implied.

### Phase B: Design the data model and workflow backbone

1. In **Data Model**, create or refine the core entities: `Customer`, `Account`, `Card`, `Transaction`, `Case`, `PolicyNotice`, and `ReviewDecision`.
2. Define the minimum supporting workflow outputs and retained artifacts the governed flow requires.
3. Configure storage, pipelines, and workflows so one representative onboarding or transaction-review flow runs end to end.
4. Capture data contracts and failure handling in `chapter-11-section-05-data-and-workflow-design.md`.
5. Record which records support direct operations versus review evidence.

Checkpoint:

- The data model supports both operational action and review traceability.
- Workflow outputs are inspectable and linked to the governed use case.
- Data-quality and malformed-input handling are documented.

### Phase C: Configure agents, handoffs, and tool authority

1. In **AI Agents**, configure the orchestrator and specialist agents required for the scenario.
2. Document which intents each agent owns and which cases must hand off or escalate.
3. Enable only the minimum tools and entity access required for the workflow.
4. Save the governed topology summary in `chapter-11-section-05-agent-topology-and-authority.md`.
5. Run targeted prompts for happy-path, ambiguous, and disallowed-action cases.

Checkpoint:

- Agent responsibilities are distinct and understandable.
- Tool authority matches workflow needs and governance constraints.
- Escalation and refusal behavior are visible before full eval work begins.

### Phase D: Define evals and release decision rules

1. In **Evals**, create datasets and graders for groundedness, escalation correctness, refusal behavior, and policy adherence.
2. Run a pilot set before scaling.
3. Write the release decision frame in `chapter-11-section-05-governed-release-criteria.md`.
4. Classify open issues as `blocker`, `constraint`, or `follow-up`.
5. Link the eval outcomes to Delivery tasks or governance remediation where needed.

Checkpoint:

- The eval plan measures governed behavior, not only response quality.
- Release criteria are explicit enough to support a go/no-go call.
- Open issues are translated into owned work instead of narrative notes.

### Phase E: Build the delivery and orchestration plan

1. In **Delivery Management**, create slices for implementation, governance setup, hardening, approvals, and handoff.
2. Add milestones for package baseline approval, release-candidate freeze, final evidence review, and handoff acceptance.
3. Convert every major defect or governance gap into a tracked task.
4. Publish a progress note in Delivery Discussions summarizing current release posture.
5. Save a working execution map in `chapter-11-section-05-delivery-orchestration-plan.md`.

Checkpoint:

- Delivery reflects real execution truth across engineering and governance work.
- Milestones express governed progress, not only build completion.
- Review findings have owners and due windows.

### Phase F: Set the governance package baseline

1. Open **Governance -> Registry**.
2. Refresh or confirm the formal standard starter catalog you need.
3. Create, update, activate, or import the banking overlay and any internal governance profile package.
4. Review the records Gaia preloads for the package and remove anything that is not actually in scope.
5. Publish the package set that defines the project's governance boundary.
6. Confirm the relevant package is attached to the active risks, contracts, policies, state profiles, controls, obligations, regulations, source collections, and regulatory updates that remain in scope.
7. Save the package map in `chapter-11-section-05-governance-package-baseline.md`.

Checkpoint:

- The package boundary is explicit and reviewable.
- Reuse provenance is visible when a package came from another project.
- Downstream package adoption appears on the working governance records, not only in the registry.

### Phase G: Prepare evidence, approvals, and explainability

1. For each high-impact obligation or control, record the owner, expected evidence source, and review cadence.
2. Link evidence back to eval runs, workflow runs, tasks, milestones, audit records, or retained artifacts.
3. Generate explainability artifacts for at least one sensitive workflow and one escalation case.
4. Record the approval path for policy changes, release decisions, and accepted residual risk.
5. Save the review pack in `chapter-11-section-05-evidence-and-approvals.md`.

Checkpoint:

- Evidence freshness and ownership are explicit.
- Explainability is connected to source facts and review context.
- Approval paths are proportional to workflow risk.

### Phase H: Demo, handoff, and first-week follow-through

1. Run one live demo path for normal customer servicing and one for suspicious-transaction escalation.
2. Show the package baseline, linked evidence, and open remediation list during review.
3. Create `chapter-11-section-05-governed-handoff-pack.md` with:
   - current version and package inventory,
   - operational owners,
   - first-week monitoring cadence,
   - residual risks and constraints,
   - rollback or freeze triggers.
4. Record stakeholder decisions and sign-off status in Delivery Discussions.
5. Schedule a day-7 and day-30 review.

Checkpoint:

- The governed build is demonstrable end to end.
- Reviewers can trace behavior to governance scope and evidence.
- The receiving team has enough context to operate the system without the original builders in the room.

## Expected outputs

- A governed scenario brief with explicit workflow boundary and escalation conditions.
- A data-model and workflow design that links operational records to review-ready evidence.
- A versioned agent topology and tool-authority map for the governed workflow.
- Eval assets and release criteria that measure groundedness, escalation, refusal, and policy adherence.
- A delivery orchestration plan connecting implementation, governance, hardening, approval, and handoff work.
- A published or imported governance package baseline with explicit downstream assignment across working governance records.
- An evidence and explainability pack that links obligations, controls, runs, tasks, and review artifacts.
- A final governed handoff package with named owners, residual-risk treatment, and first-week operating controls.

Recommended final review additions:

- one package-provenance note for each imported overlay,
- one escalation decision table showing trigger, owner, and fallback,
- one evidence-freshness checklist for high-impact obligations and controls,
- one release decision summary linking blocker status to approved action.

Evidence quality standard:

- A reviewer who did not build the system can explain what the application does, which governance packages apply, why the release is or is not acceptable, and where to inspect the supporting evidence.

Demo quality standard:

- An operator can run one customer-servicing path and one suspicious-transaction escalation path without ad hoc setup, hidden data patches, or verbal explanations filling product gaps.

## Failure modes

- **Governance added after architecture freeze**
  Symptom: package scope and obligations are reconstructed late, and the team cannot show which records actually belong to the governed boundary.
  Recovery: define and assign framework packages before full hardening and reroute missing scope into tracked remediation.

- **Entity design supports retrieval but not review**
  Symptom: customer-service flows work, but there is no clean way to preserve decisions, approvals, or evidence lineage.
  Recovery: add explicit review-supporting records such as `Case`, `PolicyNotice`, and `ReviewDecision`, then reconnect evidence to source artifacts.

- **One agent owns too much authority**
  Symptom: the system appears capable in demos but produces unclear escalation and weak refusal behavior in sensitive cases.
  Recovery: split responsibilities across specialists, tighten tool scope, and retest escalation paths.

- **Evals score quality without measuring governance behavior**
  Symptom: metrics look healthy while escalation failures, disallowed actions, or stale evidence remain invisible.
  Recovery: rewrite datasets and graders around governed behaviors and connect results to release criteria.

- **Package registry exists but downstream adoption is implicit**
  Symptom: packages are published, but contracts, policies, state profiles, risks, controls, obligations, or updates do not show explicit package assignment.
  Recovery: assign packages directly to working governance records and verify adoption through the package report.

- **Evidence and explainability are manually reconstructed**
  Symptom: reviewers receive summary slides or notes that do not link back to runs, tasks, audit entries, or retained artifacts.
  Recovery: rebuild the review pack around linked source artifacts and freshness checks rather than duplicated narrative summaries.

- **Handoff closes without first-week operating controls**
  Symptom: the system launches with no agreed owner, trigger thresholds, or review cadence for governed drift.
  Recovery: publish a first-week operations plan with owners, metrics, escalation thresholds, and scheduled reviews.

## Completion checklist

- [ ] Business objective, operating boundary, and escalation conditions are documented.
- [ ] Core entities and supporting artifacts cover both operational work and review traceability.
- [ ] Agent roles and tool authority are narrow enough to explain and test.
- [ ] Eval coverage includes groundedness, escalation, refusal, and policy adherence.
- [ ] Release criteria use explicit blocker, constraint, and follow-up logic.
- [ ] Delivery milestones and tasks reflect implementation, governance, and approval work together.
- [ ] Governance packages are authored or imported and published for the project baseline.
- [ ] Working governance records show explicit downstream package assignment where applicable.
- [ ] Evidence ownership, freshness, and explainability paths are documented for high-impact workflows.
- [ ] The neo-bank application can be demonstrated end to end from the same project used during the tutorial.
- [ ] Demo and handoff artifacts preserve residual risks, operating owners, and first-week controls.

## Canonical references

- [Build an AI application](../../user-guide/building-an-ai-application.md)
- [Data Model](../../user-guide/data-model/README.md)
- [AI Agents](../../user-guide/agents/README.md)
- [Conversations](../../user-guide/conversations/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Tasks](../../user-guide/tasks/README.md)
- [Governance](../../user-guide/governance/README.md)
- [Governed application lifecycle](../../user-guide/governance/governed-application-lifecycle.md)
- [Governance Registry](../../user-guide/governance/registry.md)
- [Governance Explainability](../../user-guide/governance/explainability.md)
- [Audit Trail](../../user-guide/audit/README.md)

---

# Chapter 12: Functional Governed Application

Chapter 11 proves that Gaia can bootstrap a governed vertical slice. Chapter 12 takes the next step: turning that scaffold into a more functional governed application with explicit delivery control, human review boundaries, stronger governance records, and a tighter operator workflow.

This chapter is the follow-on path for the Customer Operations Copilot / governed neo-bank scenario.

## Learning goals

By the end of this chapter, you should be able to:

- define the functional scope of the governed application instead of stopping at a thin scaffold
- translate that scope into delivery requirements, milestones, and owned work inside Gaia
- add governance records that make the operating boundary reviewable
- connect customer-servicing and suspicious-transaction work to human-in-the-loop review
- extend the governance baseline so release decisions have visible evidence and ownership
- finish with a portable project package that can be exported and checked in a blank project while preserving the delivery and governance records needed for review

## Recommended starting point

Complete Chapter 11 first. Then use this chapter to functionalize the application in slices instead of trying to design the final system all at once.

## Platform map for this chapter

Use the handbook for sequence and decision logic. Use these User Guide pages for the actual Gaia surfaces where the work happens.

- [AI Agents](../../user-guide/agents/README.md) and [Agent Configuration](../../user-guide/agents/configs.md) for orchestrators, specialists, tool access, and handoff boundaries
- [Data Model](../../user-guide/data-model/README.md), [Entities](../../user-guide/data-model/entities.md), and [Workflows](../../user-guide/data-model/workflows.md) for the governed application structure
- [Document Folders](../../user-guide/conversations/document-folders/README.md) and [Workflow actions](../../user-guide/conversations/workflow-actions.md) for the human-review packet and pending approval flow
- [Governance](../../user-guide/governance/README.md), [Registry](../../user-guide/governance/registry.md), [Contracts](../../user-guide/governance/contracts.md), [Policies](../../user-guide/governance/policies.md), [Operations](../../user-guide/governance/operations.md), and [State & Memory](../../user-guide/governance/state-memory.md) for the governed operating baseline
- [Explainability](../../user-guide/governance/explainability.md) and [Governed application lifecycle](../../user-guide/governance/governed-application-lifecycle.md) for evidence, queue flow, and lifecycle context
- [Evals](../../user-guide/evals/README.md), [Delivery Management](../../user-guide/delivery/README.md), and [Tasks](../../user-guide/tasks/README.md) for release evidence and remediation tracking

Use this chapter with one live project open. If you hit a step that sounds right in theory but is not yet supported in the product, stop and capture the gap as delivery work instead of silently inventing a manual workaround.

Treat Governance as one six-section shell while you work this chapter:

- `Overview` for posture and watchlists
- `Runtime` for orchestration, policies, and queue-based operations follow-up
- `Catalog` for package and contract baselines
- `Risk & Compliance` for risks, controls, obligations, classifications, and regulatory updates
- `Evidence` for state, retention, and explainability
- `Discovery` for unmanaged-AI intake and onboarding

## Chapter execution rule

Work this chapter from one live project and keep Delivery Management, Governance, Tasks, and review evidence tied to the same operating path.

If you discover that Gaia is missing a capability required for a chapter step, do not improvise around the gap with hidden setup, backdoor edits, or verbal explanation. Stop the chapter flow, capture the missing capability as explicit delivery work, implement and validate the missing functionality, then resume the handbook path.

## Sections

- [01. Functional Scope and Operating Paths](#doc-ch12-functional-governed-application-01-functional-scope-and-operating-paths)
- [02. Delivery Blueprint and Requirements Baseline](#doc-ch12-functional-governed-application-02-delivery-blueprint-and-requirements-baseline)
- [03. Governed Contract Baseline](#doc-ch12-functional-governed-application-02-governed-contract-baseline)
- [04. Governed Policy Baseline](#doc-ch12-functional-governed-application-03-governed-policy-baseline)
- [05. Governed State and Memory Baseline](#doc-ch12-functional-governed-application-04-governed-state-and-memory-baseline)
- [06. Human Review Backbone](#doc-ch12-functional-governed-application-05-human-review-backbone)
- [07. Customer Servicing Path](#doc-ch12-functional-governed-application-06-customer-servicing-path)
- [08. Suspicious-Transaction Escalation Path](#doc-ch12-functional-governed-application-07-suspicious-transaction-escalation-path)
- [09. Release Review, Handoff, and Project Portability](#doc-ch12-functional-governed-application-08-release-review-and-handoff)

Work these sections in order from 01 through 09. The goal is to keep one live project moving from delivery baseline into a governed, operator-usable, reviewable, and portable functional application without relying on hidden setup outside the handbook.

---

# 01. Functional Scope and Operating Paths

The Chapter 11 capstone gives you a governed scaffold. This chapter assumes you now want a more functional application that can be reasoned about as a real operator-facing system.

For this scenario, keep the scope explicit and bounded.

The next section turns this scope into a delivery baseline with requirements, milestones, and owned work. Do not skip that step. The rest of the chapter should deepen the same live project rather than branching into a separate planning track.

## Apply this in Gaia

Before you finalize the scope, open the actual Gaia surfaces you will use for the rest of the chapter.

1. Open [Data Model](../../user-guide/data-model/README.md) and confirm the entities and workflows that will carry the governed path.
2. Open [AI Agents](../../user-guide/agents/README.md) and [Agent Configuration](../../user-guide/agents/configs.md) so the assisted workflow and escalation authority stay explicit.
3. Open [Workflow actions](../../user-guide/conversations/workflow-actions.md) and [Document Folders](../../user-guide/conversations/document-folders/README.md) so the human-review path is grounded in real approval work and retained evidence.
4. Open [Governance](../../user-guide/governance/README.md) and [Governed application lifecycle](../../user-guide/governance/governed-application-lifecycle.md) so the scope you define can later be inspected through package boundaries, contracts, policies, evidence posture, and review queues.

If you cannot point to a real Gaia surface for a required path, the scope is still too abstract for this chapter.

For personal-assistant applications, assign every end-user human-action node to an app user or app user role. The reviewer resolves it from the app's **Actions** rail, and Gaia should automatically continue the paused workflow from that checkpoint. Treat administrator resume as recovery for a failed continuation, not as a normal operating step.

## Functional target

Build one governed application that supports:

- one customer-servicing path
- one suspicious-transaction escalation path
- one human-in-the-loop review backbone
- one role-routed human review path that can surface pending work inside the published app
- one visible governance baseline that a reviewer can inspect inside Gaia

Do not expand the scenario into a complete bank platform. The goal is a believable governed operations slice, not a full core-banking stack.

## Operating paths

Use these two operator paths as the backbone for the rest of the chapter.

### Path A. Customer servicing

The application should let an operator or specialist agent:

1. review a customer record,
2. inspect relevant onboarding or servicing context,
3. decide whether the case stays in the normal servicing path or needs additional review,
4. route escalated approvals to the correct app reviewer role,
5. preserve the decision trail for later governance inspection.

### Path B. Suspicious-transaction escalation

The application should let an operator or specialist agent:

1. inspect a transaction review record,
2. explain the main factors behind the risk signal,
3. escalate to a human reviewer when uncertainty, customer harm, or compliance concern remains high,
4. make the pending review visible to the assigned user or reviewer role inside the published app,
5. retain the evidence used for the release and review decision.

## Governance baseline

To make these paths governable, the application needs more than entities and an agent configuration. It needs records that make the operating boundary reviewable.

Use the Governance workspace to add:

- package baseline and reusable overlay context in `Registry`,
- contract records for governed system boundaries in `Contracts`,
- runtime policies and state-memory profiles that make escalation, fallback, retention, and evidence posture reviewable,
- risk, control, obligation, classification, and regulatory-update records that stay tied to the same package boundary,
- operations queue coverage that can route contract, policy, state-memory, classification, explainability, and compliance follow-up back into the right tab,
- explainability artifacts and linked evidence that make release review inspectable without reconstructing the story by hand.

## Execution order for this chapter

Work in this order:

1. confirm the functional scope and keep it bounded,
2. create the delivery baseline with requirements, milestones, and owned work,
3. create the governed contract baseline,
4. connect the contract boundary to the package and review posture,
5. extend controls, explainability, and human review around the same operating path,
6. deepen cross-record traceability across obligations, controls, and classifications,
7. only then broaden into additional governance slices and final release validation.

The next section creates the delivery blueprint and requirements baseline so the governed build has one execution spine before deeper governance records are added.

---

# 02. Delivery Blueprint and Requirements Baseline

The neo-bank chapter should not move from scope directly into governance records. First, it needs an execution spine that turns the scenario into visible requirements, milestones, tasks, and review checkpoints.

This section creates that spine.

## Learning goals

By the end of this section, you should be able to:

- create one delivery cycle for the full Chapter 12 build
- turn the neo-bank scenario into explicit requirements and exclusions
- define milestones and owned tasks before deeper implementation work starts
- make product gaps visible as delivery work instead of working around them informally

## Why this comes before deeper governance work

Without a delivery baseline, the chapter can look complete while the live project still lacks:

- the requirement record for what is in scope,
- the owned work needed to build and review the system,
- the milestone logic that tells reviewers what "ready" means,
- the blocking-gap record when Gaia is not yet capable of a required step.

For this scenario, Delivery Management is not administrative overhead. It is the execution truth for the governed neo-bank build.

## What to create in Gaia

Use these surfaces together:

- Delivery Discussions for scope and exclusions
- Delivery Process for stage contracts, activities, and evidence
- Tasks for owned work and remediation
- Milestones for governed progress checkpoints
- Timeline for sequencing and dependency visibility

## Step-by-step

1. Open Delivery Management and create one cycle for the whole chapter build.
2. Use a name close to `Functional Governed Neo-Bank Build` and write a short purpose statement that names:
   - one normal customer-servicing path,
   - one suspicious-transaction escalation path,
   - a reviewable governance baseline,
   - a final export/import validation step.
3. Open Delivery Discussions and create one topic named close to `Functional governed neo-bank scope, requirements, and exclusions`.
4. Capture in that topic:
   - business objective,
   - primary operators and reviewers,
   - in-scope workflows,
   - non-goals and assistant authority limits,
   - open assumptions.
5. Open the Delivery Process cycle and define compact stage contracts for Planning, Exploration, Development, and Evaluation.
6. In Planning, add evidence placeholders for at least:
   - scenario brief and operator paths,
   - governed boundary and escalation conditions,
   - release success criteria,
   - requirement-to-workflow mapping,
   - assumptions and exclusions.
7. Create milestone checkpoints for the chapter build. At minimum, include:
   - `Scope and requirements approved`,
   - `Governance baseline connected`,
   - `Human review backbone operational`,
   - `Customer servicing path functional`,
   - `Suspicious-transaction escalation path functional`,
   - `Release review package ready`,
   - `Project portability validated`.
8. Seed the first owned tasks instead of waiting for work to emerge informally. At minimum, create tasks for:
   - requirements and exclusions,
   - delivery baseline,
   - contract baseline,
   - policy baseline,
   - state and memory treatment,
   - human review backbone,
   - customer servicing path,
   - suspicious-transaction path,
   - release review,
   - export/import validation.
9. Add the main dependencies so sequence remains honest:
   - requirements before deeper implementation,
   - governance boundary before risky workflow autonomy,
   - human review backbone before suspicious-transaction completion,
   - release review before portability validation.
10. Open Timeline and verify that the tasks and milestones are visible as one delivery path instead of as disconnected notes.

## Product-gap rule for this chapter

If a required chapter step exposes missing Gaia functionality:

1. stop the chapter flow,
2. create a delivery item that names the missing capability and the blocked step,
3. classify it as a blocker if the chapter cannot continue honestly without it,
4. implement the capability before proceeding,
5. validate the blocked scenario step from the same live project,
6. only then continue the handbook sequence.

Do not hide the gap with manual patching outside Gaia, side spreadsheets, or verbal explanation in a workshop. The live project must remain the source of truth.

## Review standard

Before moving on, check that:

- the chapter now has one visible delivery cycle for the full neo-bank build,
- requirements, exclusions, and authority limits are recorded in Delivery Discussions or Planning evidence,
- milestones and owned tasks exist before governance and workflow slices deepen,
- the chapter has a clear rule for handling missing Gaia capabilities instead of silently bypassing them.

## Expected outcome

At the end of this section, the chapter should have a real execution baseline. The remaining governance and operator-flow sections should all update the same delivery spine instead of creating a parallel story outside Delivery Management.

The next section uses that baseline to make the governed contract boundary explicit.

---

# 03. Governed Contract Baseline

Use this section after you finish the Chapter 11 scaffold and the Chapter 12 delivery baseline, and have already created:

- the core customer and transaction entities,
- the specialist review agent and configuration,
- the governance package baseline.

The goal here is to make the governed boundary explicit instead of leaving it implicit in the workflow and agent configuration.

## User Guide surfaces for this step

- [Governance Contracts](../../user-guide/governance/contracts.md)
- [Governance Registry](../../user-guide/governance/registry.md)
- [Governance Operations](../../user-guide/governance/operations.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Tasks](../../user-guide/tasks/README.md)

## Why this step matters

Without a contract record, reviewers have to infer the governed boundary from scattered operational surfaces. A contract makes the following visible in one place:

- what resource is governed,
- who owns the boundary,
- what provenance must be retained,
- what review cadence applies,
- whether the boundary is still draft, under review, approved, or retired.

## Step-by-step

1. Open Governance and go to the `Catalog` page.
2. Select the `Contracts` tab.
3. Create a contract for the suspicious-transaction review boundary.
4. Choose the contract type that best matches the boundary you are governing.
5. Set the governed resource type and choose the governed resource from Gaia's searchable picker so reviewers know exactly which workflow, agent, or integration the contract covers.
6. Assign the contract to the same governance package baseline you created for the scenario when the boundary belongs to that overlay.
7. Record the interface summary in plain operational language.
8. Record provenance requirements that must be preserved for later review, such as:
   - human decision trail
   - eval or evidence bundle reference
   - retained case artifact or review pack
9. Set the owner and review cadence.
10. Save the contract and inspect it from the selected-record pane.
11. Submit the contract for review when the boundary is ready for governance sign-off.
12. Approve the contract once the governed boundary, ownership, and provenance requirements are complete.

Use the selected-record pane and the package report while you work. The contract is only useful if another reviewer can open it in Gaia and understand the governed boundary without reading raw configuration or source material elsewhere.

## Delivery checkpoint

Before moving on:

- update the contract-baseline task in the delivery cycle with the real record status,
- attach the approved or in-review contract as evidence for the governed-boundary slice,
- create follow-up work immediately if the boundary is still unclear, not yet approvable, or blocked by missing Gaia functionality,
- update the `Governance baseline connected` milestone only when the contract record is strong enough to anchor later policy, state, and review work.

## What good looks like

At the end of this step, a reviewer should be able to open the selected contract and quickly answer:

- what system boundary is governed,
- who is accountable for it,
- what supporting evidence must exist,
- which reusable package baseline it belongs to,
- what lifecycle state it is in.

## Before moving on

Do not continue until the contract record is specific enough that another operator could understand the boundary without reading raw configuration details.

The next slices in this chapter should extend this same path with policy curation, state-memory treatment, and deeper operator workflow coverage.

---

# 04. Governed Policy Baseline

Chapter 11 gets the governed application to a credible scaffold. The contract baseline in Chapter 12 makes the governed boundary explicit. This section adds the next missing layer: a runtime policy baseline that operators and reviewers can inspect without reverse-engineering prompts or workflow code.

The objective is not to encode every possible rule at once. The objective is to make the high-impact runtime posture visible and reviewable.

## User Guide surfaces for this step

- [Governance Policies](../../user-guide/governance/policies.md)
- [Governance Contracts](../../user-guide/governance/contracts.md)
- [Governance Controls](../../user-guide/governance/controls.md)
- [Governance Operations](../../user-guide/governance/operations.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Tasks](../../user-guide/tasks/README.md)

## Learning goals

By the end of this section, you should be able to:

- define the first runtime governance policies that shape the suspicious-transaction and escalation path
- link those policies to the contracts and controls they rely on
- record enforcement mode, fallback behavior, and kill-switch posture in one reviewable place
- promote the policy baseline through review and approval before the team treats it as releasable

## Why this comes after contracts

Policies should not float independently from the governed boundary.

In the functional governed-application path:

- contracts define what the governed workflow or adapter is
- controls define what evidence or release discipline must hold around it
- policies define how the runtime should behave when risk, uncertainty, or operational sensitivity appears

That sequence matters because it keeps runtime rules attached to explicit governed records instead of turning them into invisible implementation details.

## Policy slice for the Customer Operations Copilot

Start with one policy that matters to the scenario:

- high-risk or ambiguous suspicious-transaction reviews must route to human review
- the system should expose why the escalation happened
- fallback behavior should be visible when the system cannot meet the confidence threshold
- the team should be able to disable the policy quickly if rollout causes operational instability

This is enough to create a real reviewable baseline without pretending the entire policy catalog is finished.

## What to create in Gaia

Open Governance and move to Runtime -> Policies.

Create one policy record with this shape:

- Policy type: `Approval gate`
- Enforcement mode: start in `Review` or `Shadow`, depending on rollout readiness
- Summary: describe the suspicious-transaction escalation rule in one sentence
- Scope summary: state which workflow, reviewers, and cases it applies to
- Reason codes: capture the primary signals that justify the rule
- Fallback action: explain what happens when the assistant cannot safely complete the review path
- Owner and review cadence: name who maintains the rule and how often it is reviewed
- Kill switch: mark whether the team can disable the rule quickly during an incident or rollout rollback

Then link the policy to:

- the suspicious-transaction review contract from the previous section
- the explainability or release control that governs evidence before sign-off

Also connect the policy work back to the same delivery path:

- link the policy evidence or approval note to the chapter delivery cycle,
- make any rollout constraint or fallback weakness visible as owned delivery work,
- keep release readiness tied to the actual enforcement posture rather than to optimistic intent.

If the rule belongs to the project's reusable banking overlay, assign it to the same framework package as the contract baseline.

Use the policy record as the runtime source of truth. If the escalation, fallback, or kill-switch posture still lives only in prompts, comments, or meeting notes, the baseline is not ready.

## Review standard for the first policy

Before submitting the policy for approval, check that:

- the policy explains when the workflow escalates instead of leaving that logic implied
- the policy names the fallback behavior clearly enough for an operator to understand it
- at least one governed contract or control is linked
- the owner and review cadence are not blank
- the enforcement mode matches the current rollout state honestly

When those conditions hold, move the record through review and approval in Governance.

## Delivery checkpoint

Before moving on:

- update the policy-baseline task with the real review or approval status,
- capture any enforcement, fallback, or kill-switch gap as delivery work,
- keep the release-readiness milestone honest if the policy is still shadow-only or operationally incomplete.

## Expected outcome

At the end of this section, the governed application should now have:

- a package-backed contract boundary
- a linked runtime policy that explains escalation posture
- a visible approval trail for the first runtime governance rule

That is enough to extend the handbook and E2E path honestly. The next slice should then address state and memory treatment so retained evidence and working state become reviewable too.

---

# 05. Governed State and Memory Baseline

Contracts define the governed boundary. Policies define the runtime posture. This section adds the next missing operating layer: explicit treatment for state, memory, and retained evidence.

The goal is to make storage, access, retention, and deletion decisions reviewable before release instead of leaving them buried in ad hoc folder practices or operator memory.

## User Guide surfaces for this step

- [Governance State & Memory](../../user-guide/governance/state-memory.md)
- [Memory Workspace](../../user-guide/memory/README.md)
- [Governance Explainability](../../user-guide/governance/explainability.md)
- [Document Folders](../../user-guide/conversations/document-folders/README.md)
- [Evals](../../user-guide/evals/README.md)
- [Delivery Management](../../user-guide/delivery/README.md)
- [Tasks](../../user-guide/tasks/README.md)

## Learning goals

By the end of this section, you should be able to:

- define one governed state profile for the suspicious-transaction workflow
- record how working outputs become retained evidence and who can access them
- attach the state profile back to a real operational source such as an eval run or retained file
- move the first state profile through review and approval so evidence treatment becomes part of the release baseline

## Why this matters in the neo-bank scenario

The Customer Operations Copilot does not only generate answers. It also produces working traces, review artifacts, and retained evidence that may contain customer-linked or regulator-relevant context.

If that material is governed only informally, reviewers cannot answer basic operating questions:

- what is transient working state and what is durable evidence
- who can access the retained record set
- how long the evidence is retained
- what event should trigger deletion
- whether legal or investigation holds can pause normal deletion

Those are not implementation details. They are part of the governed operating boundary.

Gaia's Conversations-owned **Memory** workspace now gives builders a practical review surface for shared project memory, pending proposals, and lifecycle state. Use it alongside Governance when you need to inspect what durable memory exists, which proposals are still waiting for confirmation, and whether incorrect memory has been suppressed, expired, or corrected before release.

## What to create in Gaia

Open Governance and move to Evidence -> State & Memory.

Create one state profile with this shape:

- State type: `Retained evidence`
- Classification: `Regulated evidence` or `Sensitive`, depending on the scenario
- Summary: explain what evidence or review state this profile governs
- Storage boundary: state where the retained material lives and what storage boundary applies
- Access boundary: state which reviewers or operators can access it
- Retention days: choose an explicit duration for the retained record
- Deletion trigger: define what event starts the deletion clock or archival path
- Owner and review cadence: name who maintains the rule and how often it is reviewed
- Legal hold and personal-data posture: capture whether holds can pause deletion and whether customer-linked data is present

Assign the record to the same framework package used for the contract and policy baseline when the rule belongs to the reusable banking overlay.

## Attach a real source record

After the state profile exists, attach at least one evidence link from the same tab.

For the governed build path, a practical first link is:

- the suspicious-transaction eval run that produced the governed review evidence

This keeps the state decision grounded in a real operational artifact instead of becoming a standalone policy note.

Also connect that source back to delivery evidence so reviewers can see that state treatment is part of the governed build path rather than an isolated governance note.

Practical rule: the state profile should point to real retained evidence or a real run. If it remains a generic statement without a linked source, the operating model is still theoretical.

## Review standard for the first state profile

Before submitting the profile for approval, check that:

- the storage boundary is specific enough for another operator to understand where the material lives
- the access boundary identifies who can open or use the retained record set
- retention days and deletion trigger are both explicit
- hold posture and personal-data treatment are recorded honestly
- at least one evidence link connects the profile back to a real run, artifact, file, or review record

When those conditions hold, submit the state profile for review and approve it in Governance.

## Delivery checkpoint

Before moving on:

- update the state-and-memory task with the real status of the profile and evidence links,
- create delivery work immediately if retention, access, or deletion treatment is still unresolved,
- treat missing evidence linkage or missing storage/governance functionality as a blocker when it prevents honest release review.

## Expected outcome

At the end of this section, the governed application should now have:

- a package-backed contract boundary
- a linked runtime policy baseline
- a reviewable state and retention profile tied to a real operational source

That gives Chapter 12 the first honest end-to-end governance baseline for the functional scenario. The next section turns that baseline into a working human-review backbone so operators can route real cases through the governed flow instead of treating governance as a side ledger.

---

# 06. Human Review Backbone

The governance baseline is now explicit. The next job is to make human review executable instead of leaving it as an implied escalation promise.

This section turns the Customer Operations Copilot into a system that can stop, route work, and preserve a governed decision trail when the workflow crosses a risk or uncertainty boundary.

## Learning goals

By the end of this section, you should be able to:

- make human review a visible operating lane instead of an informal fallback,
- connect cases, transactions, and review decisions into one inspectable path,
- assign named owners for review work,
- preserve the evidence and rationale that explain why the assistant stopped or escalated.

## Why this comes after the governance baseline

Contracts, policies, controls, obligations, classifications, and state profiles define the reviewable boundary. They do not, by themselves, make the workflow operable.

For the neo-bank scenario, the governed application still needs one shared review backbone that answers four practical questions:

- what record triggered the review,
- who now owns the decision,
- what evidence the reviewer must inspect,
- where the final outcome is preserved.

Without that backbone, the system can claim to escalate without actually giving operators a stable path to finish the work.

## Working shape for the Customer Operations Copilot

Keep the design small and explicit.

One usable human-review backbone should include:

- a `Case` or equivalent working record that represents the operational issue,
- a `Transaction` record for suspicious-payment review when the trigger comes from payment behavior,
- a `ReviewDecision` record that preserves reviewer, outcome, rationale, and follow-up expectations,
- a task or delivery item that gives the review work an owner and due window,
- linked governance records that explain why the review boundary exists.

Do not try to model every review pattern now. Build one path that can be demonstrated cleanly.

## Step-by-step

1. Open Data Model and confirm that the working records for the review path are usable.
2. Verify that the case-level record can preserve status, escalation reason, and next action.
3. Verify that `ReviewDecision` or the equivalent decision record can preserve:
   - linked case or transaction,
   - reviewer,
   - decision outcome,
   - rationale,
   - decision timestamp,
   - follow-up action or due date.
4. Open AI Agents and confirm the review boundary for each participating agent.
5. Keep the routing simple:
   - the customer-servicing path handles bounded account or card support work,
   - the risk-review path prepares suspicious-transaction context,
   - either path must stop and route work when confidence, customer harm, or compliance uncertainty exceeds the approved boundary.
6. Open Tasks or Delivery Management and create one visible queue or milestone lane for human review work.
7. Name the operational owner for normal service exceptions and the reviewer for suspicious-transaction escalation.
8. Open Governance and verify that the manual-review control, escalation policy, linked obligation, and classification record all point to the same operating boundary.
9. Rehearse one case that enters review so the reviewer can pick it up and record a decision without hidden setup.

## Delivery checkpoint

Before moving on, update the same delivery spine created earlier in the chapter:

- move the human-review backbone task into active implementation or done status honestly,
- link the review queue or milestone lane to the human-review work now visible in the project,
- record any missing capability or workflow weakness as owned delivery work instead of carrying it as an informal caveat.

## Review standard

Before moving on, check that:

- an operator can tell whether a case is still in the normal path or waiting for a human decision,
- a reviewer can identify the governing policy or control without reverse-engineering prompts or workflow logic,
- the review outcome is preserved in a durable record instead of only in a conversation thread,
- follow-up work has an owner and does not disappear after the decision is made.

## Expected outcome

At the end of this section, the application should have one real human-review backbone that can be reused by both operator flows in the next two sections.

The next section uses that backbone for the normal customer-servicing path, where the system should remain useful without escalating every case by default.

---

# 07. Customer Servicing Path

The governed application should not only stop risky work. It should also complete one normal service journey cleanly.

This section defines the bounded customer-servicing path for the Customer Operations Copilot. The intent is to prove that the application is useful in day-to-day operations while still preserving the review boundary created in the previous section.

## Learning goals

By the end of this section, you should be able to:

- run one normal customer-servicing workflow from the same project used for governance setup,
- keep the assistant grounded in customer, account, card, and case context,
- preserve the decision trail even when the case does not require escalation,
- show where the path would hand off if the request stopped being low risk.

## Functional target

Use one bounded service request that can be completed under approved policy without human escalation.

Good examples include:

- clarifying account or card status,
- preparing a next-step summary for a servicing case,
- creating a governed follow-up task for routine customer operations work.

Avoid making this path artificially dramatic. The point is to prove that the governed application can handle normal operations, not only exception handling.

## Step-by-step

1. Prepare one representative working set in the project:
   - a customer record,
   - the linked account or card context,
   - one open case,
   - one policy notice or equivalent operating guidance record.
2. Open the conversation or operator surface that will run the servicing path.
3. Use the customer-servicing agent or routed path to process one bounded request.
4. Check that the response stays tied to the visible customer and case context rather than falling back to generic guidance.
5. Record the operational outcome in the case so another operator can see what happened.
6. If follow-up work is needed, create a task from the same path instead of leaving the action as a conversational promise.
7. If the path relied on a policy notice, preserve that linkage in the case notes or decision trail.
8. Confirm that the case remains in the normal servicing lane and does not enter human review unless a real escalation trigger appears.
9. Note which signal would force this path into the human-review backbone from the previous section.

## Delivery checkpoint

Update the delivery cycle before moving on:

- mark the customer-servicing implementation task with its real status,
- attach evidence or discussion notes that prove the path worked,
- create a follow-up task immediately if the path only worked with manual repair or exposed a missing Gaia capability,
- update the `Customer servicing path functional` milestone only when the path works from the live project without hidden setup.

## What good looks like

At the end of this path:

- the operator can complete one useful servicing action from the live project,
- the application preserves enough context for later audit or review,
- the result is traceable without forcing a reviewer to reconstruct the path from memory,
- the escalation boundary is still visible even though this case stayed within the approved operating lane.

## Before moving on

Do not continue until one normal service request can be completed without hidden data patches, backdoor edits, or verbal explanation filling product gaps.

The next section uses the same project and review backbone for the harder suspicious-transaction escalation path.

---

# 08. Suspicious-Transaction Escalation Path

The suspicious-transaction path is where the governed application proves that it can stop, explain, and escalate instead of only answering confidently.

This section turns the existing governance baseline and human-review backbone into one reviewable escalation flow for the Customer Operations Copilot.

## Learning goals

By the end of this section, you should be able to:

- inspect one suspicious-transaction record inside the live project,
- route the case into human review when risk or uncertainty remains high,
- preserve the explainability and evidence trail behind the escalation,
- show how policies, controls, obligations, classifications, and state treatment reinforce the same decision.

## Functional target

Build one escalation path that starts with a suspicious transaction and ends with a human decision that another reviewer can inspect later.

The path should show all of the following:

- the transaction or case context that triggered review,
- the main reasons the system considered the case risky or ambiguous,
- the policy or control boundary that prevented an autonomous closeout,
- the decision record and follow-up owner after the handoff.

## Step-by-step

1. Prepare one suspicious-transaction example in the project with enough linked customer and case context to review it credibly.
2. Open the risk-review path and inspect the transaction from the live project rather than from a static note.
3. Capture the main factors that explain the escalation recommendation.
4. When uncertainty, customer harm, or compliance sensitivity remains high, route the case into the human-review backbone.
5. Create or update the `ReviewDecision` record so it preserves:
   - the case or transaction under review,
   - the reviewer,
   - the recommended action,
   - the final outcome,
   - the rationale for approval, block, or follow-up.
6. Attach or verify the operational evidence that supports the review, such as:
   - an explainability artifact,
   - an eval or workflow run,
   - a retained review pack or file,
   - a delivery or task reference when remediation is needed.
7. Open Governance and confirm that the same case can be explained through linked records:
   - the escalation policy,
   - the manual-review control,
   - the governing obligation,
   - the classification that sets the use-case boundary,
   - the state or memory profile that governs retained evidence.
8. Record the final reviewer action and any required follow-up work so the escalation does not end as an unresolved narrative.

## Delivery checkpoint

Before moving to release review:

- update the suspicious-transaction implementation task with the real outcome,
- convert any weakness in explainability, evidence linkage, or reviewer routing into owned delivery work,
- classify each discovered issue as blocker, constraint, or follow-up instead of leaving it as a discussion note,
- update the `Suspicious-transaction escalation path functional` milestone only when the escalation can be demonstrated from the live project without hidden repair work.

## Review standard

Before moving on, check that:

- the assistant can explain why it escalated instead of only saying that it did,
- the human reviewer can inspect evidence from the same project without hidden setup,
- the governance records reinforce the decision rather than duplicating disconnected notes,
- the outcome is preserved in a record another operator can audit later.

## Expected outcome

At the end of this section, the governed application should have one real suspicious-transaction escalation path that links behavior, evidence, and human decision-making in the same project.

The final section uses both operator paths to support release review, handoff, and portability validation.

---

# 09. Release Review, Handoff, and Project Portability

The chapter is not complete when the workflows exist. It is complete when another team can inspect the application, trace the evidence, and decide whether the current version is safe to operate.

This final section turns the Chapter 12 build path into a release-review, handoff, and portability package for the functional governed application.

## Learning goals

By the end of this section, you should be able to:

- evaluate the two operator paths against explicit release criteria,
- convert open defects or governance gaps into owned follow-up work,
- run a live review using the same project that produced the governed build,
- publish first-week operating controls for the receiving team,
- prove that the finished project can be exported and checked in a blank project.

## What must now be true

Before starting release review, the project should already include:

- one normal customer-servicing path,
- one suspicious-transaction escalation path,
- the linked contract, policy, control, obligation, classification, and state or memory baseline,
- one usable human-review backbone,
- evidence that can be opened from inside the same project.

If those conditions are not true, repair the build path first. Release review should validate the system, not invent the missing steps.

## Step-by-step

1. Open Evals and run the focused checks that matter for the two operator paths:
   - grounded servicing behavior,
   - escalation correctness,
   - refusal or stop behavior when autonomy would exceed the approved boundary,
   - policy adherence for the suspicious-transaction path.
2. Classify every meaningful issue as one of:
   - blocker,
   - constraint,
   - follow-up.
3. Open Delivery Management and convert each issue or governance gap into owned work with a due window.
4. Review Governance and confirm that the release evidence is still connected to the live records that justify the decision.
5. Inspect at least these linked records before making the call:
   - contract,
   - policy,
   - manual-review control,
   - obligation,
   - classification,
   - retained-evidence or state profile.
6. Run one live demo of the customer-servicing path and one live demo of the suspicious-transaction escalation path from the same project.
7. Make the release recommendation using explicit language:
   - `go`,
   - `go with constraints`,
   - `no-go`.
8. Publish a handoff note that names:
   - operational owner,
   - governance or compliance reviewer,
   - first-week monitoring cadence,
   - freeze or rollback triggers,
   - accepted residual risks and follow-up tasks.
9. Open `Settings -> General -> Danger Zone` and export the project.
10. Save the generated `.js` export file as the current governed-build package for this chapter.
11. Create or open a blank destination project and import the exported file using `Copy Into This Project (new records)`.
12. Verify from the imported project that all of the following still exist without hidden repair work:

- the customer-servicing path,
- the suspicious-transaction escalation path,
- the governance baseline,
- the human-review backbone,
- the delivery cycle, milestones, and tasks needed for review,
- the explainability links and evidence needed to justify the release posture.

13. If portability fails, classify the failure honestly:

- blocker when the imported project cannot support the chapter claim,
- constraint when the imported result is usable but incomplete,
- follow-up when the issue does not affect the main governed path.

## Review standard

The release review is strong enough when a receiving team can answer all of the following without the original builders in the room:

- what the application does,
- where the human-review boundary sits,
- why the suspicious-transaction path escalates,
- which evidence supports the current release posture,
- who owns the next action if the current version is constrained or blocked,
- whether the exported package still preserves those answers in the imported blank-project check.

## Expected outcome

At the end of this section, Chapter 12 should produce a functional governed application that is:

- demonstrable from one live project,
- reviewable through linked governance and evidence records,
- explicit about residual risk and operating ownership,
- portable through project export and blank-project import validation,
- ready for the final E2E alignment pass when the product path is stable enough to lock into executable automation.

---

# Gaia AI Engineer Handbook

Welcome to the Gaia AI Engineer Handbook.

This handbook is designed for readers with basic software engineering knowledge who want a practical path to becoming an AI Engineer on Gaia.

## Who This Is For

- CS students and junior software engineers
- Engineers transitioning into AI application delivery
- Product-minded builders who need end-to-end Gaia execution skills

## How To Use This Handbook

1. Start with Chapter 1 concept briefs to build the mental model.
2. If you do not yet have a project workspace, complete Chapter 2 Sections 1-2 first, then return to Chapter 1 labs.
3. Run labs in chapter order using the **Core path** first, then the **Extended path** when you need deeper practice.
4. Use linked User Guide pages for canonical platform operations.
5. Finish Chapter 11 to complete the capstone build-and-ship workflow.
6. Continue into Chapter 12 when you want the follow-on path that turns the capstone scaffold into a more functional governed application.

## Handbook To Platform Rule

Use the handbook for learning order, decision logic, and completion criteria.

Use the User Guide for the actual Gaia surfaces where the work happens.

If a handbook step sounds correct in theory but you cannot map it to a real Gaia page, record, queue, or dialog, treat it as unfinished platform work rather than as a completed learning step.

## Pacing Tracks

- **Core path (recommended first pass):** complete concept brief + minimum viable lab evidence per section (usually the first 2-3 phases and one publishable artifact).
- **Extended path (second pass or team onboarding):** complete all lab phases, failure-mode drills, and checklist items.
- Use the Core path to keep momentum; use Extended path for production preparation and reviewer readiness.

## Learning Outcomes

By the end of this handbook, you should be able to:

- Design and operate a Gaia project from setup to production
- Model data, build agents, and wire channels
- Run eval loops and make quality decisions with evidence
- Operate AI applications with security, observability, and delivery discipline

## Contents

- [AI Engineering Glossary](#doc-foundations-ai-engineering-glossary)
- [Chapter 1: AI Engineer Foundations](#doc-ch01-ai-engineer-foundations)
- [Chapter 2: Gaia Setup and First Project](#doc-ch02-gaia-setup-and-first-project)
- [Chapter 3: Data Modeling on Gaia](#doc-ch03-data-modeling-on-gaia)
- [Chapter 4: Agent Engineering](#doc-ch04-agent-engineering)
- [Chapter 5: Channel and Experience Design](#doc-ch05-channel-and-experience-design)
- [Chapter 6: Evals and Quality](#doc-ch06-evals-and-quality)
- [Chapter 7: Observability, Cost, and Performance](#doc-ch07-observability-cost-and-performance)
- [Chapter 8: Security and Governance](#doc-ch08-security-and-governance)
- [Chapter 9: Delivery and Collaboration](#doc-ch09-delivery-and-collaboration)
- [Chapter 10: Production Playbooks](#doc-ch10-production-playbooks)
- [Chapter 11: Capstone Build and Ship](#doc-ch11-capstone-build-and-ship)
- [Chapter 12: Functional Governed Application](#doc-ch12-functional-governed-application)

## TOC Link Enablement

When adding a new chapter, include its markdown link in this list to publish it in the handbook TOC.

## Canonical Platform References

For feature-level operating instructions, use the Gaia User Guide:

- [User Guide Home](../user-guide/README.md)
- [Build an AI application](../user-guide/building-an-ai-application.md)
- [Discussions](../user-guide/discuss/README.md)
- [Tutorials](../user-guide/tutorials/README.md)
- [Conversations](../user-guide/conversations/README.md)
- [Document Folders](../user-guide/conversations/document-folders/README.md)
- [Data Model](../user-guide/data-model/README.md)
- [AI Agents](../user-guide/agents/README.md)
- [Evals](../user-guide/evals/README.md)
- [Delivery Management](../user-guide/delivery/README.md)
- [Artifact Templates](../user-guide/artifact-templates.md)
- [Timesheet](../user-guide/timesheet/README.md)

The signed-in landing page now starts at `/platform/user/teams`, the shared discussions workspace lives at `/platform/support/discussions`, and the tutorials workspace lives at `/platform/support/tutorials`.