LLM Pipeline Patterns: From Prompt Chaining to Autonomous Agents

Six composable patterns for building LLM systems - prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer, and autonomous agents - with the trade-offs for each.
Author

Benedict Thekkel

Published

May 20, 2026

A single prompt is a function call. A system is how you compose those calls. The useful patterns are few and they compose - most real apps are two or three of these stacked together.

Built out in the DeepLearning repo under LLM/LLM_Pipelines. Ordered by increasing autonomy: the more control you hand to the model, the more capable and the harder to test.

The six patterns

Pattern When to use Control flow Cost Fails when Notebook
Prompt chaining Task decomposes into fixed steps Static, linear Low A step’s output drifts 02
Routing Distinct input classes need distinct handling Classify then branch Low Misclassification 03
Parallelization Independent subtasks / voting Fan-out, gather Med Needs a good aggregator 04
Orchestrator-workers Subtasks unknown until runtime Planner spawns workers Med-High Planner over/under-decomposes 05
Evaluator-optimizer Quality bar + clear rubric Generate, critique, retry High No clear stop condition 06
Autonomous agents Open-ended, tool-using loops Model-driven loop Highest Loops, cost blowups 07

Rule of thumb: reach for the least autonomous pattern that solves the task. Every step up buys capability and costs you testability.

Prompt chaining

Fixed steps, each consuming the last. A gate between steps catches drift early.

flowchart LR
  I[Input] --> S1[Step 1]
  S1 --> G{Gate}
  G -- ok --> S2[Step 2]
  G -- fail --> X[Stop / fix]
  S2 --> S3[Step 3] --> O[Output]

Routing

Classify the input, then dispatch to a specialised prompt/model. Keeps each branch simple and lets you send cheap queries to cheap models.

flowchart LR
  I[Input] --> R{Router}
  R -- billing --> A[Billing prompt]
  R -- code --> B[Code model]
  R -- general --> C[General prompt]

Orchestrator-workers

When you can’t enumerate subtasks in advance: a planner decomposes at runtime and fans work to workers, then a synthesiser merges results.

flowchart TB
  I[Task] --> O[Orchestrator / planner]
  O --> W1[Worker]
  O --> W2[Worker]
  O --> W3[Worker]
  W1 --> S[Synthesise]
  W2 --> S
  W3 --> S
  S --> Out[Result]

Evaluator-optimizer

A generator proposes, an evaluator scores against a rubric, and the loop repeats until the bar is met or a budget runs out. Quality climbs fast for the first couple of rounds, then flattens - which is exactly why you cap iterations.

Autonomous agents

The model runs its own loop: think, call a tool, observe, repeat, until it decides it’s done. Maximum capability, maximum need for guardrails - step caps, cost budgets, and tool sandboxes.

flowchart LR
  G[Goal] --> T[Think]
  T --> A[Act / tool call]
  A --> O[Observe]
  O --> D{Done?}
  D -- no --> T
  D -- yes --> R[Result]

Latency vs autonomy

Rough p50 latency as you move up the autonomy ladder - each pattern adds model round-trips.

Where RAG fits

Retrieval is a stage you drop into any of these - most often inside a routing or orchestrator pattern. The companion post, Building a production RAG system, drills into that stage; here it’s one box in a bigger graph.

Everything above lives in LLM/LLM_Pipelines.

Back to top