Agentic Pipelines and LLM Applications
The hard parts are never the prompts.
In plain English
A pipeline is several of those jobs chained together so something complicated gets done end to end. Most of the work is making sure it is right every time, not just the first time you demo it.
The technical version: An agentic pipeline is several steps chained together, where each step can call a model, use tools, and decide what happens next. The difficult parts are not the prompts: they are retrieval quality, evaluation, retries, guardrails, cost, and knowing when a run has quietly gone wrong rather than loudly failed. A pipeline that fails loudly is an inconvenience. One that fails quietly is the expensive kind, and most do.
What you could actually do with this
Every document a client sends is read, the important parts pulled out, checked against what you already hold, and filed where it belongs.
The pile that takes a person all week stops being a pile, and nothing sits waiting in somebody's inbox.
A customer question gets answered from your own documents, with the source shown, rather than from a model's guess.
You can put it in front of customers, because when it is wrong you can see why and fix it.
Work that takes a specialist an hour gets done in a minute, with the specialist checking the result instead of producing it.
Your most expensive people stop doing your most repetitive work, and you keep their judgement where it matters.
The same process runs for ten clients or five hundred without adding anyone.
Growth stops meaning hiring, which is the difference between a business that scales and one that just gets busier.
What it is worth
The question is not what the software costs. It is what a person costs to do the same volume today, and what you are turning down because taking it on would mean hiring first. The second number is usually the larger one and it rarely gets counted.
What I actually deliver
The technical detail, for whoever on your side has to check it.
- Retrieval that actually retrieves: chunking, embeddings and hybrid search tuned against your data, not a tutorial default
- An evaluation harness, so pipeline changes are measured rather than argued about
- Step-level observability, because "the output is wrong" is useless without knowing which step turned
- Idempotency and safe retries, so a partial run cannot double-charge, double-publish or double-send
- Cost per run as a tracked number, not a surprise at the end of the month
- A written architecture your team can extend after I am gone
How it works
- 01
Map the steps and the failure modes
Every step gets a question: what does wrong look like here, and would we notice? The steps where the answer is "we would not notice" are where the work is.
- 02
Fix retrieval before touching prompts
Most bad LLM output is bad context. Prompt engineering on top of poor retrieval is polishing the wrong surface, and it is where most of the wasted effort in this field goes.
- 03
Build the evaluation set first
Real inputs with known good outputs. Without this, every later change is a vibe, and a model version bump becomes an unattributable regression.
- 04
Make every step safe to retry
Idempotency keys, durable state and explicit timeouts. A pipeline that cannot be safely re-run is a pipeline you will be afraid to operate.
- 05
Ship, watch, and tighten
Into production with instrumentation, then tune against what real inputs actually do, which is never what the test set predicted.
When this is the wrong call
If you have not decided what a correct output looks like, do not start. You cannot evaluate, so you cannot improve, and you will ship something that feels right to whoever built it. If your data is scattered across systems nobody trusts, the first project is the data, not the pipeline. And if a deterministic query would answer the question, use the query: a model is the most expensive way to do arithmetic.
Where I have done this
SmileShape.ai
An AI pipeline processing dental scans, built HIPAA-aware and SOC 2 ready from the first commit rather than bolted on before a review. 8-figure valuation in under a year, and 5 filed patents with me as a named inventor.
Further reading
- Securing LLM Apps: Prompt Injection, Data Leakage & Guardrails A practical deep dive on the top three LLM security risks - prompt injection, data leakage, and weak guardrails - with concrete mitigations you can ship this week.
- Should You Let AI Agents Touch Production? A CTO's Framework A CTO's decision framework for giving AI agents write access to production: blast radius, reversibility, permission scoping, sandboxing, audit logs, and kill switches.
- Scaling from Zero to $50M: The Infrastructure Decisions That Mattered A case-study reflection on the architecture, hiring, and cost decisions that actually mattered scaling a startup from zero to $50M - and the ones that didn't.
Questions
What is the difference between an agentic pipeline and an AI agent? +
An agent is one actor with tools and a goal. A pipeline is a sequence of steps, any of which may use a model or tools, with control flow between them. Pipelines are easier to reason about, cheaper to evaluate and far easier to debug, which is why a lot of work sold as an agent should be a pipeline.
Do we need a vector database? +
Often not. Plenty of production retrieval runs well on Postgres with pgvector, or SQLite with a vector extension and full-text search alongside it. A dedicated vector database is a reasonable choice at scale and a premature one at the start.
How do you evaluate something with no single right answer? +
With graded criteria rather than exact match: does it cite the right source, does it avoid a claim it cannot support, does it stay inside the format. Model-graded evaluation works when the rubric is specific, and it is worthless when the rubric is "is this good".
How do you keep the cost predictable? +
Cost per run measured from day one, cheaper models on the steps that do not need the expensive one, caching on anything repeated, and hard budgets that stop a run rather than warn about it afterwards.