Generative AI systems built for production, not for the demo
Custom LLM applications, RAG pipelines, copilots, and multi-agent workflows — designed against a written spec, evaluated on your own data, cost-modelled before launch, and shipped with the monitoring needed to keep them honest.
3+
Years building production software
12+
Projects shipped end to end
100%
Systems ship with an evaluation set
Built on the model and retrieval tooling teams actually run
Why GenAI pilots stall
The demo works. Production is a different problem.
Almost every stalled generative AI project I have looked at failed for the same handful of reasons, and none of them were about model quality.
I build the opposite: a scored evaluation set before launch, citations on every retrieved answer, a token cost model, tool permissions scoped per action, and approval gates on anything irreversible — so the system can be trusted with real work rather than demoed to a board.
The problems I get called in to fix:
- 01
No evaluation, so no way to tell if it got worse
Without a scored test set, every prompt change is a guess and every regression ships silently until a user complains.
- 02
Answers that cannot be traced to a source
If the system cannot cite where a fact came from, nobody in a regulated or high-stakes workflow can act on it.
- 03
Token cost nobody modelled
A pilot costing cents per run becomes a serious line item once it touches real volume, and by then the architecture assumes it.
- 04
Nothing wired into the systems of record
Output that lands in a chat window still leaves a human copying it into the CRM. The work was moved, not removed.
- 05
No boundary on what the model may do
Unscoped tool access and no approval gate. It works until the day it takes an action nobody wanted and nobody can undo.
1
Evaluation set before launch
100%
Retrieved answers carry citations
0
Unscoped tool permissions
What generative AI work covers
From use-case selection to a system you can operate
Strategy, build, evaluation, and the operational layer. Every stage produces something you own.
GenAI strategy & use-case selection
We find the one workflow where generative AI actually pays for itself, then write down what success looks like before anything is built. You get a scoped spec, a cost model, and an honest read on what the technology will not solve.
- Workflow audit and use-case shortlist
- Build-vs-buy and model selection
- Written spec with acceptance criteria
- Where a rule beats a model, stated plainly
Generative AI expertise
The layers where accuracy is won or lost
I stay on mainstream, well-maintained tooling so the system is still supportable in three years, by someone who is not me.
Chosen per task rather than per preference — the cheap model handles the easy path, the strong one handles what actually needs it.
What gets built
Generative AI systems people actually use twice
The difference between a system that gets adopted and one that gets demoed is usually in this list.
Retrieval over your own knowledge
Documents, tickets, policies, and records turned into an answerable corpus. Hybrid retrieval and re-ranking tuned against real questions, with every answer citing the passage it came from so a reader can verify it in one click.
- Ingestion and chunking per document type
- Hybrid keyword plus vector retrieval
- Re-ranking tuned on real questions
- Inline citations back to the source
Structured extraction and generation
Model output constrained to a schema your code can consume: extracted fields, classifications, or generated records that are validated before they are written anywhere, with low-confidence cases queued for review.
- Schema-constrained output
- Validation before any write
- Confidence thresholds and review queues
- Retry and repair on malformed output
In-product copilots
Assistants embedded where the work happens, with the same context and permissions as the user, streaming responses, and tool access scoped so it can draft anything but only execute what it is allowed to.
- Shares the user's context and permissions
- Streaming with cancellation
- Scoped tool execution
- Approval gate on irreversible steps
Evaluation harnesses
A scored test set built from your real cases, run on every prompt or model change, so quality is a number that moves rather than an opinion. Regressions are caught in CI instead of by a customer.
- Test set built from your real cases
- Scored runs on every change
- CI gate on quality regression
- Per-case failure inspection
Cost control and model routing
Per-feature token accounting, semantic caching for repeated questions, and routing so simple requests never reach the expensive model. Cost becomes a dial you can turn rather than a bill you discover.
- Per-feature token dashboards
- Semantic and exact-match caching
- Cheap-model-first routing
- Budget alerts and hard caps
Not sure whether your use case is ready for a model?
Most teams have a workflow in mind and no way to judge feasibility. Book a free 30-minute call: describe it, and you will get an honest read on what is technically possible with the data you already have, what it would cost per run, and whether a simpler approach wins.
Real systems. Real data.
Generative AI work I have shipped
Assistants, retrieval systems, and agent workflows running against real content. Problem, solution, and my role on each.
AI Chatbot Builder
Agencies wanted to embed branded AI assistants on client sites but lacked a platform to manage per-client knowledge bases and measure how the assistants actually performed.
Customer Support Agent
A support team answered the same routine requests by hand, re-reading the help centre and past tickets for every one. Anything that needed an actual change, such as a record updated or a status corrected, meant leaving the ticket and doing it in another system, and escalations reached the next tier as a raw transcript with no context attached.
AI Chatbot (Chainlit)
Users needed an intelligent conversational AI assistant that could understand natural language, maintain context across conversations, and provide relevant, helpful responses.
AI Recipe Generator
Home cooks and food enthusiasts needed an intelligent platform for discovering and generating recipes with AI assistance and chef expertise.
Where these systems run
Same architecture, different tolerance for a wrong answer
Retrieval, evaluation, and guardrails are constant. What changes is how much a mistake costs and who has to sign off.
Customer support & service
Drafted replies grounded in your own knowledge base, with escalation on anything the system is not confident about.
- Grounded, cited draft replies
- Intent classification and routing
- Escalation on low confidence
- Knowledge-base gap reporting
How GenAI builds run
Evaluate before you scale, cost it before you commit
The sequence is what keeps a pilot from becoming a sunk cost.
Use-case selection
We look at candidate workflows and pick the one with the clearest payback and the most tolerant failure mode. Some candidates get ruled out here, which is the point.
Deliverables
- Use-case shortlist
- Payback estimate
- Feasibility read
Data & feasibility review
What data exists, what state it is in, and whether it can actually support the answers you want. Most RAG disappointments are data problems discovered late.
Deliverables
- Data inventory
- Quality assessment
- Retrieval plan
Evaluation set first
Before building the system, we write the questions and the expected answers. That set defines 'working' and becomes the regression suite.
Deliverables
- Scored test set
- Acceptance thresholds
- Baseline scores
Prototype
A working slice against real data, scored on the evaluation set, so the first conversation about quality is about numbers rather than impressions.
Deliverables
- Working prototype
- Eval scores
- Cost per run
Harden & integrate
Guardrails, injection testing, PII handling, tool scoping, and integration into the systems where the output has to land.
Deliverables
- Guardrail suite
- Integration wiring
- Security review
Deploy with monitoring
Production deployment with tracing, token accounting, quality sampling, and alerting on cost and latency budgets.
Deliverables
- Production deploy
- Cost dashboards
- Quality sampling
Tune & extend
A tuning window against real traffic, then the next use case with the retrieval and evaluation infrastructure already in place.
Deliverables
- Tuning report
- Updated eval set
- Next use-case plan
Why work with me
GenAI systems that keep working after the launch post
Anyone can produce an impressive demo now. The engineering is in everything that makes it dependable on the two-hundredth run.
Evaluation before enthusiasm
A scored test set exists before the system does, so quality is measured rather than asserted and regressions are caught in CI.
Cost modelled up front
Token cost per run is estimated before architecture is committed, and instrumented per feature after launch.
Grounded and citable
Retrieved answers carry their sources. If a system cannot show its working, it will not be trusted with real decisions.
Scoped autonomy
Tool permissions per action and approval gates on anything irreversible. Autonomy is earned incrementally, not granted at launch.
12+
Projects shipped end to end
3+
Years in production software
100%
Systems ship with an eval set
0
Unscoped tool permissions
Frequently asked
Questions people ask before we start
Have a workflow you think a model could take over?
Describe it in a few lines. You will get an honest read on feasibility with the data you have, a rough cost per run, and whether a simpler approach would win.