<about />
About Garrett
AI Systems & Agent Engineer · Multi-agent orchestration · LLM safety · Bedrock infra
I build production AI systems — the kind that run unattended and still behave.
My focus is the engineering that makes LLMs trustworthy in production — grounding gates, drift-proof context, and human-in-the-loop commits: gates that stop a model from shipping a claim it can’t support, context systems that keep an agent’s rules from drifting, and human-in-the-loop tooling that lets an agent operate a real system without being trusted to act unsupervised.
At Re:Build Manufacturing I build AI infrastructure — Terraform, observability, and RAG pipelines (a build-vs-buy call cut a team from six engineers to two; replacing a slow embedding path gave a 20x speedup). Generic application work is commoditizing fast, while agent and LLM-infrastructure work is where demand is heading. The systems below are how I stay on the right side of that shift.
Beyond software, I invest in real estate and direct a music festival — garrettjohnlaw.com ties it together.
<case-studies />
AI systems, in production
Not demos. Three systems where LLMs do real work behind guardrails — running unattended, with the numbers below tracing to the source and runtime config behind them.
Production multi-agent system
A self-improving fleet of LLM agents that runs a sales pipeline
An autonomous lead-generation system of ~15 LLM agents (plus a few supporting Lambdas) on AWS Lambda + Bedrock that discovers prospects, drafts grounded outreach, and rewrites its own strategy every week.
~15
LLM agents (plus supporting Lambdas)
$100→$50
Monthly Bedrock cap cut (spend roughly halved)
Weekly
Self-rewritten strategy brief
Problem
A one-person company cannot afford to spend hours each week prospecting, and naive LLM outreach is worse than none: it hallucinates criticism of a prospect’s website and burns money re-drafting the same held leads on every run. The system had to be cheap, self-correcting, and safe to send unattended.
AI architecture
- Structured as three layers — Sensors (discovery, analytics, event-enrichment) collect from Brave Search, Google Maps, and GA4; Synthesizers (the weekly strategy optimizer, daily pulse, weekly report) read the history and write a strategy brief; Dispatchers (outreach, content, demo-dispatcher) act on leads.
- A grounding gate sits in front of every send: a Claude Haiku evidence-auditor classifies each draft against the research it was built from, surgically repairs the one offending clause once, and HOLDS the draft rather than sending an unsupported claim — four blocking checks (unsupported-defect, premise-contradiction, diagnostic-subject, unsolicited-criticism).
- The strategy optimizer reads ~90 days of conversion history and uses Bedrock tool-use to emit a fresh strategy brief every Sunday; discovery and outreach read that brief at startup, so scoring multipliers, email experiments, and volume self-adjust to what actually converted.
- Two model tiers keep cost down — Haiku for scoring and validation, Sonnet only for the writing that matters. Concurrency is held by an S3 compare-and-set run-lock (the account is below the Lambda reserved-concurrency floor). Ships via GitHub Actions OIDC to a leadgen/prod branch.
Measured result
The pipeline runs fully unattended Monday–Friday; the founder only touches warm replies. Diagnosing a self-contradicting drafter prompt (it rewarded confident specificity with no gate to the evidence) and replacing the blind regenerate-cascade with a single surgical repair fixed the dominant cause of held drafts. Cutting the monthly Bedrock cap from $100 to $50 (in config), plus a 72-hour re-draft dedup window, roughly halved observed outreach spend.
Production agent + context engineering
A 135-tool operations agent with one source of truth for its rules
A Bedrock Claude agent that runs a whole operation — bookkeeping across three entities, real-estate underwriting, CRM, calendar and domain workflows — through 135 tools, with a "doctrine" system that single-sources its domain rules across the agent prompt, the Claude Code skills, and the code itself.
135
Agent tools in one registry
~76%
Per-turn tool-token cut via on-demand discovery
1
Source of truth for domain rules
180s
Async worker beats the 30s gateway cap
Problem
The same domain rules (how to route an expense, when a receipt links to a transaction, which calendar an event belongs to) lived in three places at once — the agent’s system prompt, a dozen Claude Code SKILL.md files, and the code — and silently drifted apart. And a conversational agent that calls real tools cannot fit inside an API Gateway’s 30-second response window.
AI architecture
- A doctrine module defines each domain contract exactly once as a structured fragment. renderDoctrine([...]) composes the agent’s system prompt from those fragments at load time; a generator stamps the same fragments into the Claude Code skill docs between markers — so editing one file updates every surface that consumes the rule.
- The agent’s entire tool surface (135 read/write tools) is declared once in a shared tool-registry: read tools auto-execute and feed results back to the model; write tools are surfaced with full schemas but intercepted and staged as reviewable plan-ops that only commit after the human approves.
- Loading all 135 tool schemas every turn would cost ~27K tokens; instead the agent loads only a curated hot set and discovers the long tail on demand via a search_tools meta-tool (ranked over the registry, with a per-conversation activated set) — a ~76% per-turn cut. Anthropic’s native tool-search defer-loading is InvokeModel-only on Bedrock and this agent runs on the Converse API, so it’s a hand-rolled equivalent of their tool-search guidance.
- To beat the 30s gateway cap, /chat/send writes a "thinking" status row and self-invokes the Lambda asynchronously (InvocationType: Event); the real agent loop runs off the request path with a 180s timeout while the UI polls status. Up to 20 model turns per conversation.
- A receipt-to-transaction matcher runs daily: it links a paid receipt to its ledger transaction on an exact-amount, ±7-day, unique match (vendor keyword as tie-breaker), and the link is enqueued by the canonical writer itself — the contract is code-enforced, not documented and hoped for.
Measured result
Domain rules stopped drifting because there is now structurally one copy. The agent runs long, multi-tool reasoning turns without hitting the gateway timeout, and proposes writes that a human approves before they ever touch live records — the assistant operates the system without being trusted to act unsupervised.
Automated evaluation engine
An evaluation engine that grades any website against a fixed standard
The Full Score™ — a quality-grading engine that audits a live site across six weighted pillars, runs real browser-rendered checks, and turns the gaps into a prioritized plan.
6
Weighted quality pillars
7
Automated checks, static + rendered
A–F
Grade on a 0–100 score
Problem
Most website audits grade one dimension (speed, or SEO) and stop. Worse, static accessibility checkers are blind to the real failure modes — they can only compare text against a solid CSS background color, so light text laid over a busy hero image passes a static check and fails a human. Selling a website rebuild needs a defensible, repeatable standard, not a vibe.
AI architecture
- Six weighted pillars (Brand & Design, Message & Persuasion, Technical Foundation, Integrations & Workflow, Maintainability & Ownership, Conversion & Growth) are codified as the single source of truth; seven automated checks (accessibility, SEO, links, content, structured-data, performance, mobile) map onto them and roll up to a 0–100 score and an A–F grade.
- A rendered (real-browser) mode adds what static tools cannot see: it screenshots the page with and without text to measure true text-over-media WCAG contrast, click-tests CTAs to catch buttons that do nothing, renders at a 390px phone width to find overflow and sub-44px tap targets, and measures per-image weight against explicit standards.
- The output that matters is not the letter grade — it is the prioritized list of gaps. The grade is the diagnostic lens; the gap vocabulary is the deliverable.
- Wired into the sales motion: the same engine grades a prospect’s legacy site and the proposed rebuild side by side, so the pitch is a before/after scorecard rather than an opinion.
Measured result
A repeatable, code-enforced website standard that produces a client-facing scorecard on demand — catching the exact text-over-image contrast failures that pass every static checker, and converting an audit into a numbered to-do list a prospect can act on whether or not they ever hire.
How I build AI systems
Guardrails before output
An LLM that can ship is also one that can hallucinate. I gate every model output against evidence and fail closed — never send an unsupported claim.
One source of truth
Domain rules drift when they live in the prompt, the docs, and the code at once. I single-source contracts so the agent, the tooling, and the docs stay in sync.
Operate, don’t override
The best agents propose and let a human commit. Large tool surfaces with a review step beat full autonomy that nobody can trust.
Ship it to production
A demo proves nothing. I run these systems unattended on real infrastructure — Bedrock, Lambda, DynamoDB, Terraform, CI/CD — and measure what they do.
I’m comfortable architecting agent fleets, writing grounding gates, tuning Bedrock cost, and shipping it all to production. If you’re hiring for AI or agent engineering, I’d love to connect.
View my résumé