The Agentic Project LoopOpen in Colab

The Agentic Project Loop

Building with a foundation model is still a project with a loop, but the slots are different from a training project. You are not searching a weight space with gradient descent; you are shaping what a fixed model sees and can do, then running it in a loop and containing its failures. The agentic spine:

task→context & tools→orchestration→evaluation→guardrails→operations feedback→(back to task)\text{task} \rightarrow \text{context \& tools} \rightarrow \text{orchestration} \rightarrow \text{evaluation} \rightarrow \text{guardrails} \rightarrow \text{operations feedback} \rightarrow (\text{back to task})

Agentic project loop — every technique changes one or two of these slots

feedback closes the loop1Task definition2Context & tools3Orchestration loop4Evaluation5Guardrails6Operations feedbackstage 1 / 6Task definition
Define the goal, environment, and success criteria; what the agent is for and when it should act.
click a numbered stage →

The recurring shape of a project on this track. A new method almost never touches all six stages — it modifies one or two. The question for anything you learn: which slot does it change, and what was breaking before? Click a stage to read it. The bold edge is the feedback that closes the loop back to the start.

As with the ML project loop, the discipline is the same: every technique modifies one or two slots, and you learn faster by naming which.

The slot test, agentic edition

Which slot does this change, and what was breaking before?

TechniqueSlot it changesWhat was breaking before
Retrieval / RAGcontextthe model answered from stale parametric memory and hallucinated specifics
Reranking retrieved chunkscontexttop-k by raw similarity buried the one relevant passage below near-duplicates
ReAct (reason + act)orchestrationa single forward pass couldn't gather the facts it needed mid-answer
Self-reflection / critiqueorchestrationfirst drafts shipped errors no one checked before returning
LLM-as-judge eval setevaluation"looks good to me" didn't catch regressions when a prompt changed
Output guardrail / schema validatorguardrailsfree-form output occasionally leaked PII or broke the caller's parser
Prompt versioning + tracingoperationsa prompt edit silently changed behaviour with no way to diff or roll back

The transfer test

Place one you may not have been taught: where does a tool-call allow-list (the agent may only call search and read, never delete) sit? It's a guardrails change — the model's tool-selection was fine, but the blast radius of a wrong call was unbounded. The notebook drills this.

One agent, all the way around: answer questions over a docs folder

The running example is a documentation Q&A agent: given a question and a small folder of docs, it must find the answer and cite the source. Minimal, but it exercises every slot.

1 · Task definition

State the goal and the success criteria before any prompt. Goal: answer a user's question from the docs and cite the doc it came from. Success: the answer is grounded (the cited doc actually contains it) and the agent declines when the docs don't cover it. Deciding when it should refuse is part of the task, not an afterthought. → deeper: Foundation Model Agents, Goal Creation Patterns.

2 · Context & tools

Design what the model sees and can do. Two tools — search(query) returns candidate doc ids, read(doc_id) returns the text — plus a prompt that tells the model how to emit a tool call. This is the agentic analogue of the ML data stage: it decides what raw material reaches the decision. → deeper: Agents & Tool Use, RAG, MCP & A2A protocols.

3 · Orchestration loop

Run the model in a plan → act → observe cycle: the model proposes a tool call, the runtime executes it, the result is fed back, and the loop repeats until the model emits a final answer. This is where reflection, multi-step planning, and multi-agent topologies live — the agentic analogue of ML optimization (a search, but at runtime instead of training time). Bound the loop, or it spins. → deeper: Tool Use & MCP, Reflection Patterns, Sub-agent orchestration.

4 · Evaluation

Judge the outcome (did it answer correctly and cite the right doc?) and the trajectory (did it call the right tools in a sensible order, without looping?). Offline eval sets, LLM-as-judge on traces, and pass@k all live here. → deeper: Evaluating Agents, LLM Evaluation, Agent metrics taxonomy, LLM-as-judge.

5 · Guardrails

Contain failure the eval can't prevent at runtime: validate the output schema, refuse when retrieval returns nothing, restrict the tool allow-list, and defend against prompt injection hiding in a retrieved doc. Tool output is untrusted input. → deeper: Safety, Registry & Adaptation, Guardrails & LLMOps Security, Prompt-injection attacks & defenses.

6 · Operations feedback

Ship it, then trace every run: token cost, latency, tool-error rates, and failed trajectories. Version the prompts so a change is diffable and reversible, sample live traces into the eval set, and let production failures redefine the task. The loop closes. → deeper: Deploying Agents, Observability & Prompt Management, Agent observability.

Parallels & differences vs the ML loop

The two loops rhyme, which is why learning one accelerates the other, but the departures are where agentic engineering earns its own name.

ML loopAgentic loopRelationship
datacontext & toolsboth decide what raw material reaches the decision
hypothesis space(the model is fixed)the biggest departure: you don't choose the function family; you prompt a frozen one
objective + optimizationorchestrationML searches a weight space at train time; an agent searches an action space at runtime
evaluationevaluationsame idea, but agents add trajectory correctness on top of outcome
deployment feedbackoperations feedbackboth close the loop from production back to the start

The one-line summary: an ML project moves the weights; an agentic project moves the context, the tools, and the loop around weights it cannot move.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
The Agentic Project LoopThe ML Project LoopAgent Protocols: MCP & A2ASub-Agent Types: Sequenti…Agentic Analysis: A Metri…LLM-as-a-Judge: Automated…Prompt Injection: Attack …Prompt Engineering & In-C…Agents & Tool UseFoundation Model Agents &…Goal Creation PatternsRetrieval-Augmented Gener…Tool Use, Function Callin…