- .json` with its verdict and score. The cron-runnable pipeline reads that file before any irreversible step (publishing, submitting, deploying) and blocks if the receipt is missing or marks a failure. The judgment happens in the session where it belongs; the determinism happens in cron where it scales.
Without this, you get one of two bad outcomes: drop the subjective gate and let an unattended pipeline ship slop, or block the whole pipeline on a human and lose the automation. The receipt is the contract between the two layers, and it is the only way I have found to put a judgment call inside a job that runs with nobody watching.
---
## The Bottom Line
[Claude Code](https://docs.anthropic.com/en/docs/claude-code) isn't just a code generator. With the right systems, it becomes a quality-controlled collaborator.
The goal isn't trusting AI less. It's trusting evidence more, and building systems that make "should work" impossible to accept.
Start with dev docs. Add the gate system. Implement progressive disclosure. Each piece builds on the last.
The AI was always capable. We just needed guardrails that made evidence the only path forward.
---
`WORK WITH ME · SAME PROCESS, YOUR REPO`
> If your team re-checks AI output by hand before every deploy, you are paying twice for the same work. I build custom Claude Code pipelines, $2,000 to $5,000 with a written scope and a fixed quote, that wire these hooks, gates, and multi-agent workflows directly into your repo. [See how the engagement works](/services).
--- END POST ---
================================================================================
POST: I Have 73 Browser Tabs Open. ADHD Made Me a Better Architect.
================================================================================
URL: https://chudi.dev/blog/adhd-systems-architecture-engineering
Date: 2025-12-21T00:00:00.000Z
Tags: adhd, neurodivergent, distributed-systems, architecture, systems-design, ai
Pillar: neurodivergent
Reading Time: 16 min
Word Count: 3140
TL;DR: Five ADHD traits that look like liabilities in engineering, scattered focus, constant distraction, poor memory, chaos, novelty obsession, map directly onto five core systems architecture skills. This is not a reframe. It is the architecture.
Key Takeaways:
- ADHD pattern recognition is parallel processing: seeing multiple signal threads simultaneously instead of one component at a time
- The 47 background threads in an ADHD brain are distributed systems intuition in disguise, context switching, async, eventual consistency
- Novelty-seeking is involuntary technology scouting; in AI the field moves faster than deliberate study can track
- Working memory limits force abstraction-first design, you build systems that don't require remembering everything because you literally cannot
- Living with constant failure is chaos engineering training; recovery speed beats prevention when failure is inevitable
--- CONTENT ---
Five ADHD traits that look like liabilities in engineering map directly onto five systems architecture strengths: pattern recognition becomes AI architecture design, parallel processing becomes distributed systems intuition, novelty seeking becomes technology scouting, working memory limits become better abstractions, and chaos experience becomes fault tolerance design.
"How many browser tabs do you have open right now?"
My friend asked this during a video call. She'd spotted the visible browser window: 73 tabs. She seemed concerned.
I laughed. That was just one window. I had four more windows open. Three different browsers.
She thought this was chaos. I thought: this is just how my brain works, externalized. (For the productivity-system grounding this post extends, see [The ADHD Engineer Productivity System](/blog/adhd-engineer-productivity-system).)
Here's what took me years to understand. The neural pattern that makes linear processes feel impossible is the same pattern that makes architectural thinking feel inevitable. The 73 browser tabs are not a failure of discipline. They are a distributed state management system built by a brain that thinks in parallel.
I don't do linear. Never have. Give me a sequential process, step 1, step 2, step 3, and something in my brain just stops. But give me a complex system with 40 moving parts, and I'm locked in. I see the patterns. I see how a change in one subsystem ripples through three others. I see the fragility points that aren't obvious until you're holding the entire architecture in mind at once.
For years I thought this was a bug. Executive dysfunction, time blindness, task initiation paralysis on linear work, these are real ADHD things. They're documented. They're frustrating as hell. But in the last few years of building AI systems, I've realized something: the cognitive profile is not broken. It's differently shaped. And for architecture work, where everything is concurrent, interconnected, and constantly failing, differently shaped is often better. The day-to-day Claude workflow I built around this brain is its own piece: [Claude for ADHD](/blog/claude-code-adhd-workflows).
This is the map. Five ADHD traits that look like liabilities. Five engineering strengths they produce.
---
## The Problem First
Let me not skip the hard part.
ADHD looks bad on paper for engineering. Scattered focus means you jump topics mid-task. Working memory limits mean you forget what you just read. Constant distraction means deep work is genuinely harder. Time blindness means you underestimate implementation by 3x. Task initiation paralysis means starting a straightforward file edit can feel impossible on bad days.
The traditional career advice for ADHD in tech: compensate. Build checklists. Use reminders. Find an accountability partner. Basically: be neurotypical, but with more systems.
That advice is not wrong. The scaffolding matters. But it misses something. The same cognitive patterns generating those problems are also generating real architectural intuitions. They are the same trait, appearing as liability or strength depending on whether the work is sequential or concurrent.
Software engineering, particularly systems architecture and AI, skews heavily concurrent. That is the opening.
---
## How Does Pattern Recognition Become an Architecture Advantage?
Most brains process sequentially. Task A finishes, then Task B starts. There is a clean handoff.
My brain doesn't work that way. Multiple signal streams are active at once. I am not thinking about one subsystem, I am holding five subsystems in attention simultaneously, tracking how they interact. When something changes in one, I immediately see the cascade.
This is not faster. It is differently shaped. But for architecture work, where everything connects to everything, differently shaped is better.
In bug bounty automation, this shows up clearly. I am not thinking about the scanner separately from the testing agent separately from the validator. I am thinking about the whole flow: how scanner output shapes what the testing layer sees, how testing results inform validation strategy, how validation failures loop back to the scanner.
Neurotypical sequential processing sees this linearly:
1. Scanner runs
2. Results pass to tester
3. Tester validates
4. Validation reports back
Parallel processing sees all of that at once, plus: what happens if the scanner fails on a specific input type? How does the tester's failure cascade affect confidence scoring? What signals from the validator should reshape the scanner's behavior?
The ADHD pattern-matching brain doesn't reason through failure one step at a time. It sees the whole failure cascade simultaneously.
ADHD hyperfocus amplifies this. My hyperfocus isn't triggered by task difficulty. It's triggered by novelty and connection. Show me a complex system, a multi-agent architecture, a distributed pipeline, a design system with 60 components, and I disappear into it. Not into one component. Into the relationships between components. How information flows, where the fragility hides, what changes ripple where. A 6-hour hyperfocus session on "how should this system handle failure?" is worth months of linear documentation.
Neurotypical sequential thinkers often catch fragility only after production breaks it. Parallel thinkers catch it earlier, because they are already holding the whole failure space in view.
---
## Trait 2: Parallel Processing Becomes Distributed Systems Intuition
Here is what is actually running in my head during any given moment of "focus":
**Active thread (foreground):** The task I'm supposedly working on
**Background threads (sample):** That email I need to send (holding: recipient, vague topic, no body). A half-formed idea for improving the deployment pipeline. A code pattern from yesterday that might solve a different problem. Partial solution to a bug I abandoned three days ago. Song fragment on loop. Memory I need groceries.
Each of these is a partial computation held in memory, waiting for either relevant input that triggers continuation, or attentional resources becoming available.
Sound familiar? This is literally how concurrent systems work.
When I first studied distributed systems architecture, I had an immediate reaction: "Wait, this is just how I think."
| Distributed Systems | ADHD Brain |
|---|---|
| Concurrent processes | Multiple thoughts running simultaneously |
| Context switching | Constantly, involuntarily, all day |
| Event-driven architecture | Attention triggered by stimuli, not by plan |
| Eventually consistent | Information updates gradually, not atomically |
| Message queues | Every notification, idea, and task waiting for attention |
| Priority inversion | Important-but-boring tasks starved indefinitely |
| Backpressure | ADHD overwhelm and shutdown |
The analogy isn't perfect. But it's close enough that lived experience with my own brain translated directly into intuition for systems design.
The key insight about context switching: you cannot eliminate it, so you need to optimize for it. My brain switches context whether I want it to or not. So I learned to keep task state externalized so reloading is fast, break work into atomic chunks that can be completed between switches, and build systems that assume interruption rather than requiring continuous focus.
Those same principles make distributed systems resilient. Checkpoint state frequently so recovery is fast. Design for resumability at any point. Assume failure and design for graceful recovery. Engineers who have never experienced involuntary context switching often design systems that require continuous operation. They are surprised when things fail. I am never surprised.
---
## How Does Novelty Seeking Become Technology Scouting?
I should be finishing this post. Instead, I have 14 tabs open about a new AI framework I discovered twenty minutes ago.
People call this "shiny object syndrome." They say it like it's a problem.
In most fields, it probably is. In AI? It is how I stay ahead.
The ADHD brain doesn't regulate dopamine the way neurotypical brains do. Routine tasks don't generate the dopamine hit that motivates action. Novel stimuli do. So the brain continuously hunts for novelty, not because of undiscipline, but because that is how the motivation system works.
In a stable field, this is pure liability: constantly distracted by irrelevant new things, unable to build deep expertise that stable fields reward.
In AI, the field moves faster than any expert can track linearly. The tool you learn today might be deprecated in six months. The approach you master might be replaced by something ten times better next quarter. In this environment, constant exploration isn't a distraction from real work. It is real work.
When Claude Code launched, I was using it within hours. Not because I planned to, because my novelty-seeking brain couldn't resist. That early adoption led to systems that now form the core of my workflow. By the time most people learn a tool, novelty-seekers have a year of experience with it.
The reframe: what if you had a team member whose job was to constantly scan the landscape for new tools, techniques, and approaches? Someone who spent their time trying new things, evaluating them against current problems, and reporting back on what's worth adopting?
You'd call that person a Technology Scout. That is what ADHD novelty-seeking does for free.
Managing it is not about suppression. The drive is structural, not behavioral. What works:
**Scheduled exploration time.** Budget it explicitly. Without this, your brain will steal it from focused work.
**The commitment list.** Before fully pursuing a new shiny object, check what you have already committed to finish. If the list is too long, the new object goes on a "to explore later" list. Your brain still gets the promise of future novelty.
**The 48-hour rule.** If you are still thinking about a new tool 48 hours after first encountering it, it is worth deeper exploration. If you have forgotten about it, the dopamine hit was the point and the tool wasn't relevant.
**The problem match test.** Every exploration should start with: what existing problem would this solve? If you cannot name a specific problem within five minutes, queue it and move on.
---
## Trait 4: Working Memory Limits Become Better Abstractions
"Can you walk me through the data flow from user input to database?"
I stare at the architecture I designed. I know it works. I know it's good. I cannot, at this moment, remember how any of it fits together.
This happens constantly. I build complex systems, understand them deeply during construction, and then forget the details within days. Not the concepts. I remember those. But the specifics? Gone.
For years, this felt like a devastating failure. How can you be a good architect if you can't remember your own architecture?
Then I realized: the fact that I cannot remember the details is exactly why my architectures are good.
ADHD working memory works like RAM that is perpetually undersized. The brain holds fewer active items. Those items evict faster under stimulation. Rebuilding context after any interruption costs real time.
When you cannot hold all the details, you cannot work at the detail level. You are forced to work at a higher level of abstraction.
Here is what happens when I try to understand a complex system:
**Neurotypical approach (I imagine):** Read through the code. Hold the details in memory. Build understanding from details up.
**My approach (necessity):** Get overwhelmed by details immediately. Fall back to finding the conceptual structure. Understand at the abstraction level. Only dive into details when absolutely necessary.
I cannot hold "function A calls function B which modifies state C which triggers event D." My brain just refuses to maintain that chain.
But I can hold "the input layer validates and transforms, the processing layer applies business logic, the output layer formats and delivers." Abstract concepts stick. Concrete details evaporate.
This forced abstraction produces a specific architectural instinct. I design every system assuming I will forget how it works. Because I will. Within weeks. So my systems must be self-explanatory: clear naming that reveals purpose, obvious structure that shows organization, minimal hidden state that requires remembering, documentation close to the code it explains.
When you design for a forgetful creator, you design for everyone. If I can re-understand a system six weeks after building it without re-reading all the documentation, anyone can. The best compliment I get on architectures: "I could understand it without reading everything." That means the abstraction level is right. The structure itself communicates.
There is also a practical toolkit that makes abstraction-first thinking sustainable:
**Architecture Decision Records (ADRs).** One page maximum. Three sections: context, decision, consequences. Long ADRs don't get read. These serve double duty for ADHD architects, they're the documentation you'll read in six months when you've forgotten why the codebase looks the way it does, and the forcing function that makes you articulate choices while they're still clear.
**Interface-first development.** Design the API before you write the implementation. Not the HTTP endpoints, the function signatures, the data shapes, the contracts between components. Writing the interface first forces abstraction into the design process rather than retrofitting it after.
**Living documentation.** Keep architecture notes in the same repository as the code, in the same commit. One PR, one change. No separate documentation system to maintain.
---
## Trait 5: Chaos Experience Becomes Fault Tolerance Design
I missed three meetings this week. Forgot to send two critical emails. Lost my train of thought mid-sentence at least a dozen times.
This is a normal week.
Living with ADHD means living with constant failure. Not occasionally, not when you're tired. Constantly. Things fall through cracks. Systems break. Plans dissolve.
The interesting part? This makes me better at designing systems that handle failure.
By the time I entered tech, I had decades of experience with system failure. The system being my own brain. Things I have learned to expect: alarms won't be heard. Tasks without external forcing functions won't happen. Memory will fail at the worst possible moment. Attention will wander during critical operations. Plans will not survive contact with my brain.
This isn't pessimism. It's realism, earned through thousands of personal failures. And it translates directly to how I think about technical systems.
I start every system design with an assumption: this will fail. Not "might fail eventually" but "will fail, probably soon, maybe at the worst possible moment."
This assumption leads to questions that don't occur to optimistic designers:
- What happens when this service is unreachable?
- How do we recover if this operation partially completes?
- What state are we in if this process crashes mid-execution?
- How do we know when this component is actually working?
ADHD also builds a specific instinct for redundancy. Because I can't trust my brain, I build redundant systems everywhere. Multiple task-capture channels. Multiple communication paths. Multiple verification checkpoints. The failure rate is known; the architecture adapts to it. Same pattern in production systems: redundant services, multiple availability zones, replicated data stores, retry mechanisms with backoff.
The chaos-to-clarity pipeline I have run for my own brain is the same pipeline that makes technical systems resilient:
**Phase 1 (Chaos):** Everything is chaotic. Failures are constant and unpredictable.
**Phase 2 (Pattern Recognition):** Start noticing which failures repeat.
**Phase 3 (System Building):** Build systems to catch common failures.
**Phase 4 (Iteration):** Systems reveal new failure modes. Iterate.
You don't eliminate chaos. You build systems that process chaos into manageable outcomes.
---
## The Shadow Side: Executive Dysfunction Is Real
I need to be honest here. Everything above is true. And executive dysfunction is just as true.
I can see a complex system and understand exactly what needs to change. But actually executing that change across 10 files, with proper testing, documentation, and verification? That requires external scaffolding.
Time blindness means I underestimate how long implementation takes, consistently. Task initiation paralysis means starting work on straightforward tasks feels impossible on bad days. Emotional dysregulation means a minor setback can derail focus for hours. Working memory limits mean I forget critical constraints I learned last week and have to be reminded.
The pattern recognition doesn't fix these things. The architectural insight doesn't substitute for discipline.
What works:
External structure. Written systems. Deadlines with accountability. Collaborative review that forces step-by-step explanation (which sounds like a limitation but actually clarifies whether the architecture is solid or just elegant-sounding).
In practice, I pair architectural hyperfocus, seeing the whole multi-agent flow, with external scaffolding: detailed PRDs, written components, incremental testing gates. The ADHD strength drives the vision. Executive dysfunction scaffolding makes sure that vision becomes real.
Good architecture from an ADHD mind looks like: brilliant system insight plus external structure forcing execution.
---
## What Is the ADHD-to-Architecture Trait Map?
| ADHD Trait | The Liability It Looks Like | The Engineering Strength It Actually Is |
|---|---|---|
| Pattern recognition across unrelated domains | Easily distracted, can't focus on task at hand | Spots system fragility others miss; sees ripple effects before they happen |
| Parallel thread processing | Scattered attention, can't commit to one thing | Distributed systems intuition; async-first thinking; concurrency comes naturally |
| Novelty-seeking / dopamine hunting | Shiny object syndrome, abandons projects | Involuntary technology scouting; early adopter advantage in fast-moving fields |
| Working memory limits | Forgets details, can't follow complex instructions | Forces abstraction-first design; builds self-documenting systems; interfaces that don't require internal knowledge |
| Living with constant failure | Unreliable, drops balls, chaos-prone | Assume-failure mindset; redundancy instinct; recovery-over-prevention architecture |
| Hyperfocus on complex relationships | Can't focus on "normal" work | 6-hour deep-work sessions on system design; sees relationships other engineers miss |
---
## Practical Toolkit: What Actually Helps
Reading the trait map is one thing. Here is what actually ships.
**Externalize everything.** Working memory is like RAM, fast but volatile. Architecture diagrams, design docs, ADRs, context notes. These are not optional supplements. They are how the work gets done. The [dev docs workflow](/blog/claude-context-management-dev-docs) I use for AI context management is the same pattern applied to human context management.
**Context capture before switching.** The [research says it takes 23 minutes to fully refocus after a context switch](https://www.apa.org/topics/research/multitasking). You can cut that to 2-3 minutes with 30 seconds of writing before you switch: what you were doing, where you were, next step. Before you switch tasks, write one sentence about where you are. That is the minimum viable version.
**Exploration time with a commitment list.** Budget explicit exploration time so novelty-seeking doesn't steal from focused work. Before fully pursuing a new thing, audit your current commitments. More than three active items? New thing goes to the queue.
**Interface-first, then implementation.** Design the contracts before writing the code. Forces the right abstraction level and prevents detail-first thinking from creating unmaintainable structures.
**ADRs for every non-obvious decision.** You will forget why you made the call. Write it down before you do.
**Assume failure at every design step.** Ask "what happens when this breaks" before "what happens when this works." ADHD has already trained you to expect failure. Apply that expectation to technical systems explicitly.
---
## Related Reading
The cognitive traits behind this post also drive the productivity challenges that come with ADHD engineering work. For the execution system, energy-aware scheduling, context capture, AI as external processor, see [ADHD Productivity: The System I Built After GTD Failed Me](/blog/adhd-engineer-productivity-system).
---
*Reply in the comments or [email me directly](mailto:hello@chudi.dev). I read and respond to everything.*
--- END POST ---
================================================================================
POST: How to Test AI-Generated Code Before Shipping: 57 Bugs Caught
================================================================================
URL: https://chudi.dev/blog/ai-code-verification-evidence-based
Date: 2025-12-15T00:00:00.000Z
Tags: claude-code, ai, quality, verification, debugging
Pillar: ai-building
Reading Time: 9 min
Word Count: 1789
TL;DR: 'Should work.' The AI said it. I believed it. Six hours later, I was still debugging a fundamental error from line one. I trust AI completely. That's why I verify everything. The paradox makes sense once you've been burned enough. Forced evaluation mode achieves 84% compliance by requiring actual evidence before marking done.
Key Takeaways:
- Red flag phrases ('should work', 'probably fine', 'I'm confident') indicate confidence without evidence
- Three psychological traps: authority transfer, completion illusion, and optimism bias
- Forced evaluation protocol: evaluate each skill → activate every YES → then implement (84% compliance)
- Replace confidence claims with facts: 'Build completed: exit code 0', 'Tests passing: 47/47'
- Add verification hooks that reject red flag phrases and require evidence before task completion
--- CONTENT ---
"Should work." Six hours later, I was still debugging a fundamental error that had existed from line one. If your AI development workflow verification stops at a confident sentence from the model, that six hours is coming for you too, and it usually lands the week you can least afford it.
AI code verification means replacing model confidence with build output, test results, and concrete evidence before any task is treated as complete. That shift matters because AI-assisted development fails in predictable ways when proof is optional, not random ones. It is the same discipline behind my [Claude Code complete guide](/blog/claude-code-complete-guide), the implementation checks in [WebMCP for SvelteKit](/blog/webmcp-sveltekit-implementation), and the adversarial validation loop from [bug bounty false-positive reduction](/blog/bug-bounty-automation).
Evidence-based completion for AI code means blocking confidence phrases and requiring proof before any task is marked done. Not "should work"--actual build output. Not "looks good"--actual test results. The psychology is simple: confidence without evidence is gambling. Forced evaluation achieves 84% compliance because it makes evidence the only path forward. [Research on LLM reliability](https://arxiv.org/abs/2402.01680) confirms that AI-generated outputs require external verification to catch hallucinations and errors that models present with high confidence.
## Why Do We Skip Verification?
You skip verification because AI-generated code looks plausible and the model sounds confident, which triggers authority transfer, you treat the AI's certainty as evidence of correctness. Completion illusion makes the task feel done. Optimism bias means you want it to work. These three forces combine to make verification feel like unnecessary extra work.
The pattern is universal. You describe what you want. The AI generates code. It looks reasonable. You paste it in.
That moment of hesitation--the one where you could run the build, could write a test, could verify the output--gets skipped. The code looks right. The AI sounds confident. What could go wrong?
That specific shame of shipping broken code--the kind where you have to message the team "actually, there's an issue"--became my recurring experience.
I trust AI completely. That's why I verify everything.
The paradox makes sense once you've been burned enough times.
## What Makes "Should Work" Psychologically Dangerous?
"Should work" is dangerous because it substitutes confidence for evidence. The phrase triggers authority transfer, completion illusion, and optimism bias simultaneously, making you feel the task is finished when verification hasn't happened. You inherit the AI's certainty without inheriting any actual proof that the code functions correctly.
The phrase creates false confidence through three mechanisms:
### 1. Authority Transfer
The AI presents with confidence. We transfer that confidence to the code itself, as if certainty of delivery equals certainty of correctness.
### 2. Completion Illusion
"Should work" feels like a finished state. The task feels done. Moving to verification feels like extra work rather than essential work.
### 3. Optimism Bias
We want it to work. We've invested time. Verification risks discovering problems we'd rather not face.
I thought I was being thorough. Well, it's more like... I was being thorough at the wrong stage. Careful prompting, careless verification.
## What Phrases Trigger the Red Flag System?
The red flag system blocks phrases that express certainty without evidence: "Should work," "Probably fine," "I'm confident," "Looks good," "Seems correct," "That should do it," "We're good," "All set," and "It shouldn't cause issues." Each phrase indicates a completion claim unsupported by build output, test results, or any verifiable proof.
Here's the complete list that gets blocked:
Each of these phrases indicates a claim without evidence. They're not wrong to think--they're wrong to accept as completion.
## What Evidence Replaces Confidence Claims?
Specific, verifiable proof replaces confidence claims: build output showing exit code 0, test results listing pass counts like "47/47," screenshots at exact viewport widths, and Lighthouse scores with numeric values. These are facts, not feelings. Any completion claim must cite one of these evidence types before the task can be marked done.
The replacement is specific, verifiable proof:
### Build Evidence
```
Build completed successfully:
- Exit code: 0
- Duration: 9.51s
- Client bundle: 352KB
- No errors, 2 warnings (acceptable)
```
### Test Evidence
```
Tests passing: 47/47
- Unit tests: 32/32
- Integration tests: 15/15
- Coverage: 78%
```
### Visual Evidence
```
Screenshots captured:
- Mobile (375px): layout correct
- Tablet (768px): responsive breakpoint working
- Desktop (1440px): full layout verified
- Dark mode: all components themed
```
### Performance Evidence
```
Lighthouse scores:
- Performance: 94
- Accessibility: 98
- Best Practices: 100
- SEO: 100
Bundle size: 287KB (-3KB from previous)
```
That hollow confidence of claiming something works--replaced with facts that prove it.
## How Does Forced Evaluation Achieve 84% Compliance?
Forced evaluation achieves 84% compliance by requiring explicit commitment before implementation. You evaluate each skill as YES or NO with reasoning, activate every YES, then proceed. Writing "YES, need verification" creates accountability. Passive suggestions produced only 20% compliance because nothing required follow-through. The commitment mechanism is what converts intent into actual verification behavior.
Research on skill activation showed a stark difference, a pattern consistent with findings from the [NIST AI Risk Management Framework](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10) on the importance of structured validation in AI-assisted workflows:
- **Passive suggestions**: 20% actually followed
- **Forced evaluation**: 84% actually followed
The mechanism is a 3-step mandatory protocol:
### Step 1: EVALUATE
For each potentially relevant skill:
```
- master-debugging: YES - error pattern detected
- frontend-guidelines: NO - not UI work
- test-patterns: YES - need verification
```
### Step 2: ACTIVATE
For every YES answer:
```
Activating: master-debugging
Activating: test-patterns
```
### Step 3: IMPLEMENT
Only after evaluation and activation complete:
```
Proceeding with implementation...
```
The psychology works because evaluation creates commitment. Writing "YES - need verification" makes you accountable to the claim. Skipping feels like breaking a promise to yourself.
## What Are the 4 Pillars of Quality Gates?
The four pillars are State and Reactivity, Security and Validation, Integration Reality, and Failure Recovery. Every piece of verification maps to one of these categories. State checks confirm reactive patterns work. Security checks confirm inputs are sanitized. Integration checks confirm components are actually used. Failure checks confirm error boundaries and loading states exist.
Every verification maps to one of four pillars:
## How Does Self-Review Automation Work?
Self-review automation works by prompting the AI to critique its own output before marking anything complete. You ask it to review its architecture for issues, explain the end-to-end data flow, and predict how the solution could break in production. The AI is effective at finding problems in code, including its own, when explicitly asked.
The system includes prompts that make the AI review its own work:
### Primary Self-Review Prompts
1. "Review your own architecture for issues"
2. "Explain the end-to-end data flow"
3. "Predict how this could break in production"
### The Pattern
```
1. Generate solution
2. Self-review with prompts
3. Fix identified issues
4. Re-review
5. Only then mark complete
```
Self-review catches issues before they become bugs. The AI is good at finding problems in code--including its own code, when asked explicitly. [Anthropic's documentation](https://docs.anthropic.com) covers how to structure these review prompts for reliable results.
## What Happens When Verification Fails?
When verification fails, the system escalates through three levels. A soft block requests evidence when a red flag phrase appears. A hard block prevents completion when no evidence exists. A rollback trigger fires when critical functionality breaks after a completion is accepted. Each level makes cutting corners progressively harder, forcing evidence as the only path through.
The system handles failures through structured escalation:
### Level 1: Soft Block
Red flag phrase detected. Request clarification:
"You mentioned 'should work'. What specific evidence supports this? Please provide build output or test results."
### Level 2: Hard Block
Completion claimed without evidence. Block the completion:
"Task cannot be marked complete. Required: build output showing success OR test results passing."
### Level 3: Rollback Trigger
Critical functionality broken after completion:
"Verification failed post-completion. Initiating rollback to last known good state."
The escalation makes cutting corners progressively harder. Evidence is the only path through.
## FAQ: Implementing Verification Gates
**Why is 'should work' dangerous in AI development?**
It indicates a claim without evidence. The AI (or developer) is expressing confidence without verification. This confidence often masks untested assumptions, missing edge cases, or fundamental errors.
**What is forced evaluation mode?**
A mandatory 3-step protocol: evaluate each skill (YES/NO with reasoning), activate every YES, then implement. Research shows 84% compliance vs 20% with passive suggestions. The commitment mechanism creates follow-through.
**What phrases indicate unverified AI code?**
Red flags include: 'Should work', 'Probably fine', 'I'm confident', 'Looks good', 'Seems correct'. These all express certainty without evidence of testing, building, or verification.
**What evidence should replace confidence claims?**
Specific proof: 'Build completed: exit code 0', 'Tests passing: 47/47', 'Screenshot at 375px shows correct layout', 'Bundle size: 287KB'. Facts, not feelings.
**How do I implement verification gates for AI code?**
Add hooks that run after AI responses. Check for red flag phrases and reject them. Require build output, test results, or screenshots before marking tasks complete. Make evidence the only path forward.
---
## What 84% Compliance Actually Looks Like
The difference between passive suggestions and forced evaluation isn't subtle. Here's a real before/after from my workflow.
**Before: Passive verification (20% compliance)**
I'd finish a feature and the AI would say something like: "The authentication flow should work. I've added the JWT utility, updated the login endpoint, and configured the cookie settings."
I'd read that, nod, move on. Three hours later, a 500 error in production. The cookie wasn't being set correctly because the response headers weren't configured for httpOnly. The AI was confident. I trusted the confidence.
**After: Forced evaluation (84% compliance)**
Now the same session ends with:
```
Build evidence:
npm run build → exit code 0 (9.51s)
No TypeScript errors
Test evidence:
Auth flow test: PASS
Cookie setting test: PASS
JWT expiry test: PASS
Screenshots:
Login form: renders correctly at 375px
Session persistence: verified across refresh
```
The AI runs the build. Shows the output. Takes screenshots. States the exit code.
The 84% compliance number comes from sessions where I tracked whether I actually ran verification before marking tasks done. Forced evaluation--where the AI had to provide evidence before I'd accept completion--produced compliance in 84 out of 100 sessions. Passive suggestions produced it in about 20.
The remaining 16%? Those were genuine time-sensitive situations where I consciously chose to accept risk. That's fine. The point isn't 100% verification. It's making the default behavior evidence-based instead of confidence-based.
Every time I skipped verification and got lucky, it reinforced the wrong habit. Every time the build caught something I'd have shipped, it reinforced the right one. Eventually the new habit won.
I hated the extra friction at first. But I love the part where I stop spending evenings debugging production issues.
I thought the problem was AI accuracy. Well, it's more like... the problem was my verification laziness. The AI generates good code most of the time. But "most of the time" isn't good enough for production.
Maybe the goal isn't trusting AI less. Maybe it's trusting evidence more--and building systems that make "should work" impossible to accept.
If your team re-checks AI output by hand before it ships, you are paying twice for the same work: once for the AI to write it, once for a person to verify it wasn't lying. I build the verification gates that catch bad output before it reaches production, custom Claude Code pipelines, $2,000 to $5,000. [See how that engagement works](/services).
The verification gate templates, CLAUDE.md configuration, and agent routing rules from this system are also packaged as a DIY kit in the [Battle-Tested Builder Kit](/products).
---
## Related Reading
This is part of the [Complete Claude Code Guide](/blog/claude-code-complete-guide). Continue with:
- [Quality Control System](/blog/how-i-build-with-claude-code) - Two-gate enforcement that blocks implementation until gates pass
- [Context Management](/blog/claude-context-management-dev-docs) - Dev docs workflow that prevents context amnesia
- [Token Optimization](/blog/reduce-ai-token-usage-progressive-disclosure) - Save 60% with progressive disclosure
--- END POST ---
================================================================================
POST: Claude Code 'Compaction Failed': Why Context Loss Happens
================================================================================
URL: https://chudi.dev/blog/claude-context-management-dev-docs
Date: 2025-12-15T00:00:00.000Z
Tags: claude-code, ai, workflow, context, productivity
Pillar: ai-building
Reading Time: 16 min
Word Count: 3068
TL;DR: 'We already discussed this.' I said it. Claude didn't remember. Thirty minutes of context gone after compaction. Dev docs are three files that persist task state outside the conversation. Before compaction, run /update-dev-docs. After compaction, say 'continue' and Claude picks up exactly where you left off.
Key Takeaways:
- Context compaction loses file paths, decisions, and progress state. The 'we already discussed this' moment becomes your weekly reality
- Three dev doc files: plan.md (approved strategy), context.md (key files and decisions), tasks.md (checklist)
- Workflow: plan mode then /create-dev-docs then implement 1-2 sections then review then /update-dev-docs before compaction
- Automation hooks handle quality control: UserPromptSubmit (skill activation), Stop-BuildChecker (automatic builds)
- Use dev docs for tasks over 30 minutes or spanning multiple sessions. Skip for quick one-off fixes
--- CONTENT ---
If you have said "we already discussed this" to Claude Code more than once, you are paying for the same context-rebuild every session. This is the context management module of the [complete Claude Code guide](/blog/claude-code-complete-guide): the dev-docs system that keeps state across compaction so you stop paying that tax.
**Claude context management** is the practice of persisting task state outside the conversation so Claude can resume exactly where it left off after context compaction. The standard method is a three-file system: plan.md (approved strategy), context.md (current state and key decisions), and tasks.md (progress checklist). Before compaction, run /update-dev-docs to save state; after compaction, type "continue" and Claude reads the files automatically, no re-explaining required.
I said it. Claude didn't remember. Thirty minutes of context-building: file locations, architectural decisions, progress updates, gone after compaction. We were starting over.
I've rebuilt context after compaction 40+ times before I fixed it. Each re-explanation averaged 20 minutes. That's over 13 hours of pure re-work.
The dev docs workflow prevents context amnesia by persisting task state outside the conversation. Three files: plan.md, context.md, tasks.md. They capture everything Claude needs to continue exactly where it left off. After compaction, say "continue" and the AI reads the docs automatically. No re-explaining. No lost progress.
## What happens during Claude Code context compaction?
Claude Code context compaction summarizes older conversation history when the active context becomes too large. The session continues, but exact file paths, rejected approaches, implementation decisions, error history, and unfinished steps can disappear from the working context. Compaction is not the same as a model becoming less capable. The model is working from a compressed and incomplete representation of the earlier session.
| Information at risk | What failure looks like | Persistent location |
|---|---|---|
| Approved architecture | Claude proposes an approach you already rejected | plan.md |
| Current files and symbols | Claude searches for the same code again | context.md |
| Completed work | Previously finished steps are repeated | tasks.md |
| Known errors | Failed approaches are attempted again | context.md |
| Next action | The resumed session starts from a generic guess | tasks.md |
Context compaction is necessary. Conversations grow too long. The AI summarizes older messages to make room for new ones. [Claude's context window documentation](https://docs.anthropic.com/en/docs/about-claude/models/overview) details the token limits that make compaction inevitable on long tasks.
The problem is what gets lost:
- **File paths** you spent time locating
- **Decisions** you already made together
- **Progress state** on multi-step tasks
- **Specific errors** you already debugged
That specific frustration: "we already discussed this." The kind where you want to rage-quit and start fresh. For me it became a weekly experience.
Here is the compaction math. A 3-hour implementation session hits compaction at roughly 90-120 minutes. If you have no persistence system, you spend 20-30 minutes rebuilding context. That is 25-35% of your session time going backwards.
I write more documentation to write less documentation. [Anthropic's documentation](https://docs.anthropic.com) describes how context management and structured prompting directly affect output quality across sessions.
The paradox resolves when you realize re-explaining is the most expensive documentation of all.
## How do you prevent context loss from compaction?
Prevent context loss by saving task state outside the conversation before compaction occurs. Keep the approved approach in plan.md, current files and decisions in context.md, and completed plus remaining work in tasks.md. Update those files after material decisions, not only when the context window is already full. After compaction, reload the three files before continuing implementation.
## What Are the Three Dev Doc Files?
The three dev doc files are plan.md, context.md, and tasks.md. Each task gets its own directory containing all three. Plan.md holds the approved implementation strategy. Context.md tracks current state, key files, and blockers. Tasks.md is a granular checklist with completion status. Together they persist everything Claude needs to resume after compaction.
Every non-trivial task gets a directory:
```
~/dev/active/[task-name]/
├── [task-name]-plan.md
├── [task-name]-context.md
└── [task-name]-tasks.md
```
### plan.md: The Approved Blueprint
The implementation plan, approved before coding begins.
```markdown
# Feature: User Authentication
## Approach
JWT-based auth with refresh tokens. Store in httpOnly cookies.
## Files to Modify
- src/routes/api/auth/+server.ts (create)
- src/lib/auth/jwt.ts (create)
- src/hooks.server.ts (modify)
## Decisions Made
- Chose JWT over sessions for stateless scaling
- 15-minute access token, 7-day refresh token
- No third-party auth initially
```
This file doesn't change during implementation. It's the reference point.
### context.md: The Living State
Current progress, key findings, blockers. Updated frequently.
```markdown
# Authentication - Current Context
## Progress
- [x] JWT utility created
- [x] Login endpoint working
- [ ] Refresh token rotation
- [ ] Protected route middleware
## Key Files
- src/lib/auth/jwt.ts:45 - Token generation
- src/routes/api/auth/login/+server.ts - Working endpoint
## Current State
Stuck on refresh token rotation. The cookie isn't being
set correctly in the response. See error at line 78.
## Next Steps
1. Debug cookie setting in +server.ts
2. Test with browser dev tools
3. Implement rotation logic
```
This is what Claude reads after compaction. It knows exactly where you are.
### tasks.md: The Checklist
Granular work items with status.
```markdown
# Authentication Tasks
## Phase 1: Core Auth
- [x] Create JWT utility functions
- [x] Implement login endpoint
- [x] Add password hashing
- [ ] Implement refresh rotation
- [ ] Add logout endpoint
## Phase 2: Protected Routes
- [ ] Create auth middleware
- [ ] Protect /dashboard routes
- [ ] Add redirect to login
## Phase 3: UI
- [ ] Login form component
- [ ] Error handling
- [ ] Success redirect
```
Check items as you complete them. Claude sees progress at a glance.
## What's the Complete Dev Docs Workflow?
The complete workflow runs in seven steps: enter plan mode, review the plan thoroughly, interrupt before implementation begins, run /create-dev-docs, implement one or two sections at a time, run /update-dev-docs before compaction, then say 'continue' after compaction. Claude reads the saved docs automatically and resumes without any re-explaining.
Here's the step-by-step process:
### 1. Enter Plan Mode
Start with planning, not coding. Always.
```
You: "I need to add user authentication"
Claude: [Enters plan mode, explores codebase]
```
### 2. Review the Plan Thoroughly
Catch mistakes before implementation.
```
You: [Read the plan carefully]
You: "What about rate limiting on login attempts?"
Claude: [Updates plan]
```
### 3. Hit ESC Before Implementation
Interrupt Claude before it starts coding.
```
Claude: "I'll start by creating the JWT utility..."
You: [Press ESC]
```
This prevents "gun-blazing" implementation without documentation.
### 4. Run /create-dev-docs
Create the three files from the approved plan.
```
You: "/create-dev-docs"
Claude: [Creates task directory with plan.md, context.md, tasks.md]
```
### 5. Implement 1-2 Sections at a Time
Don't do everything at once. Review between sections.
```
You: "Let's implement Phase 1: Core Auth"
Claude: [Implements, updates tasks.md]
You: [Review the code]
You: "Good, continue to Phase 2"
```
### 6. Run /update-dev-docs Before Compaction
When you sense context is getting long:
```
You: "/update-dev-docs"
Claude: [Updates context.md with current state, next steps]
```
### 7. After Compaction, Say "Continue"
The magic moment:
```
[Context compacted]
You: "continue"
Claude: [Reads dev docs automatically, knows exactly where you are]
```
No re-explaining. No lost context. Just continuation.
### Claude Code compaction recovery checklist
1. Stop implementation when the session begins losing exact file or decision recall.
2. Update context.md with current files, decisions, errors, and unresolved questions.
3. Mark completed and remaining work in tasks.md.
4. Confirm that plan.md still represents the approved approach.
5. Start the resumed context by loading all three files.
6. Ask Claude to state the next bounded action before editing.
The same failure mode appears at larger scale in [a 36,000-line production codebase](/blog/claude-code-production-trading-bot), where context loading and pre-compaction handoffs became operating requirements. For long agentic tasks, pair the persistence system with [model and effort-tier selection based on workload](/blog/claude-fable-5-vs-opus-4-8).
## How Does Multi-Root Workspace Structure Help?
Multi-root workspace structure separates context by repository so frontend knowledge doesn't pollute backend work. Each repo holds its own claude.md for repo-specific instructions, a PROJECT_KNOWLEDGE.md for architecture details, and a TROUBLESHOOTING.md for recurring issues. Claude reads only the files relevant to your current work, keeping context lean and accurate.
For projects with multiple repos:
```
~/git/project/
├── CLAUDE.md # Root config (~100 lines)
├── dev/active/[task-name]/ # Dev docs location
├── frontend/
│ ├── claude.md # Repo-specific (50-100 lines)
│ ├── PROJECT_KNOWLEDGE.md # Architecture details
│ └── TROUBLESHOOTING.md # Common issues
└── backend/
└── ...
```
Each repo has its own context files. Claude reads the relevant ones based on which files you're working with.
Benefits:
- **Separation of concerns**: Frontend context doesn't pollute backend work
- **Reusable knowledge**: PROJECT_KNOWLEDGE.md persists across sessions
- **Troubleshooting history**: Common issues documented once, used forever
## What Do the 16 Automation Hooks Do?
The 16 automation hooks enforce workflow quality without manual intervention. UserPromptSubmit analyzes prompts and activates relevant skills. PreToolUse validates operations before they run. PostToolUse logs every file change. Stop-BuildChecker runs builds automatically on modified repos. Stop-ErrorReminder scans for missed error patterns. Together they catch problems throughout the session without any extra commands. Context-switching is especially costly with ADHD; the [Claude Code workflow for ADHD brains](/blog/claude-code-adhd-workflows) builds this persistence in from the start.
Hooks enforce the workflow without manual intervention:
The pipeline runs automatically. You just work. The system catches problems.
## When Should I Skip Dev Docs?
Skip dev docs for quick questions, simple single-line fixes, and one-off tasks that fit entirely in a single conversation. Use them for any task taking more than 30 minutes, work spanning multiple sessions, complex features touching multiple files, or anything you would hate to re-explain from scratch after a context compaction event.
Not every task needs the full workflow:
**Skip for:**
- Quick questions ("How do I X?")
- Simple fixes (typo, single-line change)
- One-off tasks that fit in one conversation
- Exploration without implementation
**Use for:**
- Any task taking more than 30 minutes
- Multi-session work
- Complex features with multiple files
- Anything you'd hate to re-explain
The overhead of creating dev docs pays off the first time you avoid context loss.
## Context Loss Cost: Before vs. After Dev Docs
These are my actual numbers from 6 months of Claude Code use on a multi-repo SvelteKit project with 16 hooks configured.
| Metric | Without Dev Docs | With Dev Docs |
|---|---|---|
| Time lost to context re-build per session | 20-30 min | 0-2 min |
| Sessions hitting compaction (per week) | 4-6 | 4-6 |
| Total weekly re-explanation overhead | 80-180 min | 0-12 min |
| Decision drift (re-debating settled choices) | Weekly | Rare |
| "Gun-blazing" implementation incidents | 2-3/week | 0-1/month |
| Setup cost | 0 | 30 min (one-time) |
The break-even is a single session with compaction. After that, dev docs are pure time savings. The 30-minute setup has paid back over 60 hours since I built it.
## How Do I Get Started?
Start with the minimum viable setup: create a dev/ directory in your project root, create template files for plan, context, and tasks, then add 'continue' handling to your CLAUDE.md so Claude knows to read dev docs on that command. The full setup with hooks and workspace structure adds a few more hours but pays off quickly.
### Minimum Viable Setup
1. **Create a dev/ directory** in your project root
2. **Create template files** for plan, context, tasks
3. **Add "continue" handling** to your CLAUDE.md:
```markdown
## On "continue" command
1. Check for dev/active/*/context.md
2. Read the most recent context file
3. Resume from documented state
```
### Full Setup
1. **Install the dev docs commands** (slash commands or aliases)
2. **Configure hooks** for automatic skill activation
3. **Set up build checking** on Stop events
4. **Create workspace structure** for multi-repo projects
The full system takes a few hours to configure. But it saves that time on every long task thereafter.
## FAQ: Context Management for Claude Code
The questions below cover context compaction, dev doc setup, and when to use the workflow. Short answers are here. The sections above have the full implementation detail.
**What is context compaction in Claude?**
When conversations get too long, Claude summarizes older messages to free up space. This 'compaction' loses details: specific decisions, file paths, progress state. The AI continues but forgets what you already discussed.
**What are dev docs in Claude Code workflow?**
Dev docs are three files created for each task: plan.md (approved implementation plan), context.md (key files, decisions, current state), and tasks.md (checklist of work items). They persist outside the conversation.
**How does the dev docs workflow prevent context loss?**
Before compaction, run /update-dev-docs to save current state. After compaction, say 'continue' and Claude reads the dev docs automatically, picking up exactly where you left off with full context.
**When should I use dev docs vs regular conversation?**
Use dev docs for any task taking more than 30 minutes or spanning multiple sessions. Skip for quick questions, simple fixes, or one-off tasks that fit in a single conversation.
**How do 16 automation hooks fit into this workflow?**
Hooks automate quality control throughout: UserPromptSubmit activates skills, PostToolUse tracks file edits, Stop-BuildChecker runs builds automatically, Stop-ErrorReminder checks for missed errors. They enforce the workflow without manual intervention.
---
## The Failure That Caused This
The dev docs system came from a 3-hour session that hit compaction and lost everything: JWT vs. session decision, token lifetimes, cookie configuration. Claude resumed with a completely different approach. I spent 45 minutes reconstructing decisions I had already made. That night I built the first version of the system.
The context amnesia problem came to a head on a specific afternoon.
I was 3 hours into implementing a multi-step authentication flow. We'd made specific decisions: JWT over sessions, 15-minute access tokens, httpOnly cookies, no third-party auth libraries. All of this was in conversation history.
Then context compaction hit.
I typed "continue" and Claude started implementing. A completely different approach. Sessions instead of JWT. localStorage instead of cookies. Not because it was ignoring my instructions. It genuinely didn't have them anymore.
I spent 45 minutes figuring out what had changed and re-explaining the decisions we'd already made. Some of those decisions were subtle. The 15-minute token lifetime had a specific reason tied to our security requirements. I had to reconstruct the reasoning from scratch.
That night I built the first version of the dev docs system. Three files, 30 minutes to set up. The next session I ran into compaction again. This time I typed "continue" and Claude picked up exactly where we were. The cookies decision. The token lifetime. The files already modified. All of it.
The 30-minute setup has paid off dozens of times since. Context compaction is still annoying. But it's no longer a crisis.
I thought I could just re-prompt after compaction. Well, it's more like... I thought my memory was the backup, when I needed an actual backup.
Maybe the goal isn't longer context windows. Maybe it's better context persistence: systems that make "we already discussed this" impossible to say. Retrieval-augmented approaches are another direction entirely; the [RAG research by Lewis et al.](https://arxiv.org/abs/2005.11401) explores how external knowledge retrieval can supplement limited context windows.
## Advanced Patterns: Context Files for Different Work Types
The three-file structure is the baseline. For complex or long-running projects, these variants handle situations the baseline doesn't cover.
### API Design Work
When working on API design (endpoints, data schemas, contracts), context.md needs different sections than implementation work.
Add a **current endpoint inventory**: list every endpoint you've defined or are working toward with method, path, and status (designed, implemented, tested). This prevents the duplication problem where you design an endpoint in session one and Claude proposes a slightly different version in session three because it doesn't know the first one exists.
Add a **schema decisions** section: capture data shape choices and the reason for them. "Using string for userId instead of number, consistent with Clerk's user IDs" takes five seconds to write and prevents an hour of confusion when Claude suggests integer IDs.
### UI and Component Work
For frontend work, tasks.md needs visual acceptance criteria, not just functional ones. Add a column: "Visual check: [what it should look like at mobile/desktop]." This forces you to define the visual bar before implementation, not after.
Also add a **design decisions** section to context.md: color tokens used, responsive breakpoints that apply to this component, and any interaction states (hover, focus, disabled) that need to be handled. These are the details that get lost between sessions and cause inconsistency.
### Multi-Session Bug Investigation
Bugs that take more than one session to diagnose need a different structure. Replace plan.md with **investigation.md**: a hypothesis log. List every hypothesis you've tested, what you tried, and what you learned. "Hypothesis: race condition in auth middleware. Tested with logs. Result: not the issue. Logs show auth completes before the error."
This prevents testing the same hypothesis twice across sessions. It is more common than it sounds when you're working on a complex bug over several days.
Keep context.md but focus it on the error signature: exact error message, stack trace, files involved, reproduction steps. Update it every session with what changed.
### Team Projects
When multiple people work with the same context files, context.md needs a contributor section: "Last updated by [name], [date], summary of changes." This isn't bureaucracy. It's the minimum information that prevents one person from undoing another's documented decisions.
Also agree on update frequency. Individual projects, you update whenever it's useful. Team projects, agree on a rule: "update before ending every session involving more than one file change." Shared files need shared maintenance norms.
The baseline three-file structure handles 80% of sessions. These patterns handle the other 20%: the long-running, multi-person, investigative, and visually-complex work where the baseline creates gaps.
---
## What Happens at 38,000 Lines: Context Collapse at Scale
At around 15,000 lines, my trading bot stopped fitting in Claude's context window in any useful way. The model started hallucinating function signatures from files it couldn't see and citing methods that had been renamed three refactors ago. The tutorials show you the 500-line project; nobody writes about what happens at 38,000 lines in production.
Context collapse doesn't announce itself. You notice it when Claude starts being confidently wrong about your own code: suggesting imports that don't exist, referencing variables from a module you restructured two weeks ago. The model isn't hallucinating in the general sense; it's confabulating from stale partial context. Deep call graphs make it worse: showing Claude the execution layer in isolation means it's reasoning about a system it can't actually see, and it fills the gaps plausibly, which is the dangerous part.
Three disciplines that held at scale, beyond the three-file baseline:
**Scoped entry points, not file dumps.** Enter via a narrow question with a named scope: "In `signal_processor.py`, `calculate_momentum` returns negative values on gap-up candles," with three files attached, maximum. If I can't describe the problem in one sentence with three files or fewer, I haven't isolated the problem yet. The constraint is a forcing function for clarity.
**Checkpoint commits as context anchors.** Every meaningful working state gets a commit message a model could read cold: `refactor: move position sizing to RiskManager, remove from ExecutionEngine`, never `fix thing`. Starting a new session, `git log --oneline -10` carries more semantic load than a paragraph of explanation.
**Architecture written before code.** If I were starting today: write the context map before the first module, and keep modules shallower than feels right. Deep call graphs are context expensive; flat modules with explicit interfaces are easier to scope into a session. The bot's most maintainable subsystem is the one kept under 400 lines with clean input/output contracts.
The honest limit: context discipline extends your range, it doesn't eliminate the problem. At 38,000 lines you start making architectural tradeoffs specifically to manage model context. Subsystems a human engineer might integrate stay separated partly because separation makes them easier to work on with Claude. That's a property of the workflow, not a bug, and knowing it's a property means you can design around it.
---
## Related Reading
This is part of the [Complete Claude Code Guide](/blog/claude-code-complete-guide). Continue with:
- [Quality Control System](/blog/how-i-build-with-claude-code) - Two-gate enforcement that blocks "should work"
- [Evidence-Based Verification](/blog/ai-code-verification-evidence-based) - Why confidence without proof fails
- [Token Optimization](/blog/reduce-ai-token-usage-progressive-disclosure) - Save 60% with progressive disclosure
- [Deploy Your Agent on DigitalOcean](/blog/deploy-python-agent-digitalocean) - VPS setup in 5 steps for $6/month
Want the full system? The [Battle-Tested Builder Kit](/products) includes the CLAUDE.md templates, context management rules, and multi-agent patterns from these guides.
---
`WORK WITH ME · CONTEXT DISCIPLINE, INSTALLED`
> The 3-file system here is the manual version, and every teammate has to remember to run it. I build custom Claude Code pipelines, $2,000 to $5,000, that wire context persistence, verification gates, and session discipline directly into your repo so the whole team gets it automatically, not by habit. [See how the engagement works](/services).
--- END POST ---
================================================================================
POST: My Two-Gate System for Claude Code Cut Errors 84%
================================================================================
URL: https://chudi.dev/blog/how-i-build-with-claude-code
Date: 2025-12-15T00:00:00.000Z
Tags: claude-code, ai, workflow, automation, quality-control
Pillar: ai-building
Reading Time: 12 min
Word Count: 2317
TL;DR: I shipped broken code three times in one week. Not edge cases, fundamental errors. Without me realizing it, I was trusting confidence over evidence. The two-gate system I built blocks all implementation until Gate 0 (meta-orchestration) and Gate 1 (skill activation) pass. Well, it's more like enforced honesty. 'Should work' is banned.
Key Takeaways:
- Gate 0 loads meta-orchestration and validates context budget (<75%) before any tools unlock
- Gate 1 activates relevant skills based on query patterns, ensuring the right tools are loaded
- Red flag phrases ('should work', 'probably fine') are blocked and require actual verification evidence
- Progressive disclosure saves 60% tokens by loading skill metadata first, full content on demand
- The system enforces quality through hooks, not willpower. Tools literally cannot execute until gates pass
--- CONTENT ---
I shipped broken code three times in one week. Not edge cases--fundamental errors that any test would have caught. The AI said "should work" and I believed it. The complete Claude Code workflow this two-gate system fits inside is in [Claude Code: A Complete Guide](/blog/claude-code-complete-guide); this post is the quality-gate slice of that workflow.
Building a quality control system for AI code generation means enforcing mandatory gates before implementation begins--loading relevant skills, validating context budget, and blocking rationalization phrases like "should work" that indicate unverified claims. The result is a two-gate system where tools literally cannot execute until quality checks pass. [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) provides the hooks infrastructure that makes gate enforcement possible at the session level.
## Why Did I Need Quality Gates for AI?
Quality gates for AI code generation became necessary because unverified confidence phrases like "should work" replaced actual verification. Without mandatory checks, AI generates code that appears correct but breaks in fundamental ways. The gate system enforces evidence-based completion by blocking implementation tools until context is validated and the right skills are loaded.
The problem wasn't the AI's capability. [Claude](https://www.anthropic.com) is remarkably good at generating code. The problem was my workflow--or lack of one.
I'd describe what I wanted. Claude would write it. I'd paste it in. Sometimes it worked. Sometimes I'd spend hours debugging issues that existed from the first line. Without me realizing it, I was trusting confidence over evidence.
That specific anxiety of deploying something you haven't tested--the kind where you refresh the page three times hoping the error goes away--became my default state.
Well, it's more like... I was using AI as a code generator when I needed it to be a quality-controlled collaborator.
## How Does the Two-Gate System Work?
The two-gate system runs two mandatory checks before any tool can execute. Gate 0 loads meta-orchestration and validates your context budget stays below 75%. Gate 1 analyzes your query and activates the relevant skills from a library of 30 or more defined patterns. Both must pass before implementation begins.
The system enforces two mandatory checks before any tool can execute. Like buttoning a shirt from the first hole--skip it, and everything else is wrong.
If either gate fails, all implementation tools are blocked. You literally cannot write code until the orchestration layer is ready.
### Gate 0: Meta-Orchestration (Priority 0)
This gate loads immediately and handles three things:
Validates you're under 75% context usage. If you're running hot on tokens, the system warns you before you hit the wall.
Sets up phrase blocking and evidence requirements. The guardrails that make "should work" impossible to say.
Loads the SKILL.md entry point (~200 tokens). Just enough context to route your query.
### Gate 1: Auto-Skill Activation (Priority 1)
This gate analyzes your query and activates relevant skills:
Parses keywords, file patterns, and task type from your query.
Scores against 30+ defined skills using a weighted algorithm.
Applies context boosters and calculates activation thresholds.
Activates top 5 skills: Tier 1 (score ≥50) immediately, Tier 2 (≥30) on first tool use, Tier 3 (≥10) on request.
I love automation. But I spend hours building systems to slow myself down.
## What Is Progressive Disclosure and Why Does It Save 60% of Tokens?
Progressive disclosure loads skill content in three tiers instead of all at once. Tier 1 loads metadata at roughly 200 tokens, Tier 2 loads schemas at 400 tokens, and Tier 3 loads full handler logic at 1,200 tokens only when needed. Sessions that never reach Tier 3 save 60% of their token budget.
Most Claude configurations load everything upfront. Every skill, every rule, every example--thousands of tokens consumed before you've even asked a question. [Anthropic's documentation](https://docs.anthropic.com) covers prompt and context design patterns that help avoid this upfront token burn.
Progressive disclosure flips this. Load metadata first. Load details on demand.
### The 3-Tier System
**Tier 1: Metadata (~200 tokens)**
- Skill name, triggers, dependencies
- Just enough to route the query
**Tier 2: Schema (~400 tokens)**
- Input/output types
- Constraints and quality gates
- Tools available
**Tier 3: Full Content (~1200 tokens)**
- Complete handler logic
- Examples and edge cases
- Only loaded when actively using the skill
The meta-orchestration skill alone: 278 lines at Tier 1, 816 with one reference, 3,302 fully loaded. That's 60% savings on every session that doesn't need the full content.
## What Phrases Does the System Block?
The system blocks phrases that indicate claims without evidence: "should work," "probably fine," "I'm confident," "looks good," and "all set." These phrases are banned not because they are wrong, but because they signal unverified assumptions. Blocking them forces the path of least resistance to be actual build output or test results.
The automated verification system flags specific patterns in code comments and commit messages. Here's the complete breakdown of phrases that indicate insufficient testing or assumptions:
These phrases aren't banned because they're wrong. They're banned because they indicate claims without evidence.
The goal isn't to be pedantic--it's to make evidence the path of least resistance. When "build passed" is easier to say than "should work," you'll naturally verify before claiming.
That hollow confidence of claiming something works without checking--the system makes it impossible.
## How Does AMAO Handle Parallel Execution?
AMAO uses a directed acyclic graph engine to map task dependencies and identify which operations can run concurrently. It runs up to three concurrent tasks with a five-minute timeout and falls back to sequential execution if parallel fails. A context governor tracks token usage and triggers auto-compaction at 70% to prevent overflow.
AMAO (Adaptive Multi-Agent Orchestrator) adds sophisticated orchestration on top of the gate system:
### DAG Engine
- Directed acyclic graph for task dependencies
- Max 50 tasks with cycle detection
- Parallel grouping for independent operations
- Critical path analysis for optimization
### Context Governor
- 75% max budget, 60% warning threshold, 20% reserve
- Predictive usage analysis
- Auto-compact at 70%
- Phase unloading to release memory between stages
### Skill Evolution
- Pattern detection: 5 occurrences triggers skill proposal
- Auto-approval at 85% confidence
- Deprecation at 30% effectiveness
- Weighted feedback: 40% build, 30% test, 20% reverts, 10% user
The parallel execution runs up to 3 concurrent tasks with a 5-minute timeout. If parallel fails, it falls back to sequential--safety over speed.
## What Are the 4 Pillars of Quality?
The four pillars are state and reactivity, security and validation, integration reality, and failure recovery. State enforces Svelte 5 runes only. Security validates all user input. Integration reality requires every component to be used in at least one route. Failure recovery mandates error boundaries and graceful degradation on all async operations.
Every check maps to one of four pillars:
### 1. State & Reactivity
- Svelte 5 runes only (`$state`, `$props`, `$derived`)
- No legacy patterns that cause confusion
- State updates via `$effect` for side effects
### 2. Security & Validation
- All user input sanitized (XSS prevention)
- Form inputs validated with Zod
- API routes validate request schema
- No inline scripts in production
### 3. Integration Reality
- Every component used in at least one route
- No orphaned utility files
- All API routes consumed by UI
- Every feature has verification
### 4. Failure Recovery
- Error boundaries on all route groups
- Graceful degradation for failed API calls
- Loading states for async operations
- User-friendly error messages
## FAQ: Building Quality Systems for AI Code Generation
**What is a two-gate system for AI code generation?**
A two-gate system enforces quality checks before any implementation begins. Gate 0 loads meta-orchestration and validates context budget. Gate 1 activates relevant skills based on your query. Both must pass before tools are unblocked.
**How much do token savings matter with progressive disclosure?**
Progressive disclosure saves 60% of tokens by loading skill metadata first (~200 tokens), then schemas on demand (~400 tokens), then full content only when needed (~1200 tokens). This prevents context overflow on long sessions.
**Why block phrases like 'should work' in AI development?**
Phrases like 'should work' and 'probably fine' indicate unverified claims. Blocking them forces evidence-based completion--actual build output, test results, or screenshots before marking work complete.
**Can I implement this system for my own Claude Code setup?**
Yes. Start with a CLAUDE.md file that enforces gate checks. Add hooks for UserPromptSubmit (skill activation) and Stop (build verification). The meta-orchestration plugin pattern works for any codebase.
**What's the difference between AMAO and Cortex 2.0?**
AMAO handles orchestration--parallel execution, context budgeting, skill evolution. Cortex 2.0 handles skill definitions with 3-tier progressive disclosure. They work together: AMAO decides what to run, Cortex defines how skills work.
---
## What Changed After 6 Months
Six months of running this system daily taught me things I couldn't have predicted from theory alone.
The first surprise: phrase blocking changes how you think, not just what you say. After a few weeks, I stopped forming sentences like "this should work" internally. The habit of reaching for evidence replaced the habit of reaching for confidence. That's not something I expected from a text filter.
The second surprise: Gate 1 skill activation catches mismatches I didn't know were happening. I'd ask about a database query and Claude would activate the wrong skill--something frontend-adjacent because I'd mentioned a component in the same message. The gate surfaces that mismatch immediately. Without it, I'd get a halfway answer and not know why.
The third lesson: the 75% context budget threshold is exactly right, and I didn't trust it at first. I kept thinking "I have 30% left, that's fine." Then I'd hit context compaction 15 minutes later in the middle of implementation. The system wasn't being conservative. I was being optimistic about how much context finishing a feature actually consumes.
After 6 months, my ratio of "it worked first time" to "spent 2 hours debugging" has roughly flipped. Not because Claude got better. Because I stopped accepting "should work" as a completion state.
The hardest part was admitting the problem was my workflow, not the AI's capability. Once I accepted that, building the gates was easy.
I thought I needed better prompts. Well, it's more like... I needed better systems around the prompts. The AI was always capable. I just needed guardrails that made "should work" impossible to say.
Maybe the goal isn't to trust AI more. Maybe it's to trust evidence--and build systems that make evidence the only path forward.
## Adapting the Gates to Different Project Types
The two-gate system was built for a specific project, a SvelteKit blog. The principles generalize, but the implementation details vary.
### Bug Fixes vs. New Features
For bug fixes, Gate 1 skill activation should weight debugging skills higher than implementation skills. The mental model shift: you're investigating, not building. That difference matters because investigation and implementation have different quality criteria.
For a bug fix, "complete" means: the bug no longer reproduces, a test catches the regression, and the root cause is documented in context.md. Not "the code looks right."
For new features, "complete" means: the feature works end-to-end in the happy path, edge cases are documented and handled, and the build passes with types. A feature that "looks right" is in the same category as "should work."
The gate isn't different, evidence over confidence in both cases, but what counts as evidence is.
### Refactoring Sessions
Refactoring is where the gate system earns its keep most visibly. The failure mode for AI-assisted refactoring is: Claude refactors the code, the surface behavior looks the same, but something subtle broke in a path you didn't test.
Before any refactoring session, add one step to Gate 0: document the current behavior you need to preserve. Not informally, write it in context.md. "This function should return null for missing users, not throw. The calling code depends on null, not an exception."
After refactoring, verify that exact behavior explicitly. Don't trust that "the tests pass" if you don't have a test for the specific behavior you documented. If the behavior isn't tested, write the test first, confirm it passes in the original code, then refactor.
### Exploratory Sessions
Some sessions are genuinely exploratory, you're trying to understand an unfamiliar codebase, evaluate a new library, or figure out why something is slow. These sessions don't naturally fit the gate structure because there's no implementation to gate.
For exploratory sessions: skip Gate 1 skill activation and run with a single constraint, everything you learn goes into context.md in real time. Exploration without capture is entertainment.
At the end of an exploratory session, convert your notes into hypotheses for the next session. "Learned: the bottleneck is in the database query at line 45. Hypothesis: adding an index on user_id will fix it. Test this in the next session."
This closes the loop between exploration and implementation, which is the gap where insights evaporate.
### Short Projects vs. Long Projects
Short projects (one to three sessions): use the gate system for quality control, skip the full dev docs workflow. One context file with the current state is enough.
Long projects (weeks to months): invest in the full dev docs structure. The overhead of maintaining three files pays off by session four, when the project would otherwise require full context rebuilds at the start of every session. If task initiation or context-switching is the hard part, I wrote up the [ADHD version of this Claude Code workflow](/blog/claude-code-adhd-workflows) separately.
The gate system is not overhead. It's the minimum viable quality control. Don't scale it down below the gate check, that's where the broken-code problem that started this whole system lives.
---
## Related Reading
This is part of the [Complete Claude Code Guide](/blog/claude-code-complete-guide). Continue with:
- [Context Management](/blog/claude-context-management-dev-docs) - Dev docs workflow that prevents context amnesia
- [Evidence-Based Verification](/blog/ai-code-verification-evidence-based) - Why "should work" is the most dangerous phrase
- [Token Optimization](/blog/reduce-ai-token-usage-progressive-disclosure) - Save 60% with progressive disclosure
- [What is RAG?](/blog/what-is-rag) - Foundational concept behind context-aware AI
--- END POST ---
================================================================================
POST: Reduce Claude Token Usage 60%: Progressive Disclosure
================================================================================
URL: https://chudi.dev/blog/reduce-ai-token-usage-progressive-disclosure
Date: 2025-12-15T00:00:00.000Z
Tags: claude-code, ai, tokens, optimization, workflow
Pillar: ai-building
Reading Time: 11 min
Word Count: 2162
TL;DR: I was burning through $50/month in Claude API costs before I realized the problem. Every session loaded 8,000 tokens whether I needed them or not. Progressive disclosure flips this: metadata first (~200 tokens), schemas on demand (~400 tokens), full content only when needed. Well, it's more like paying for what you actually use.
Key Takeaways:
- A 2,400-line CLAUDE.md loading fully on every session wastes tokens and hits context limits mid-task
- Tier 1 (metadata): skill names and triggers only (~200 tokens), loads immediately
- Tier 2 (schema): input/output types and constraints (~400 tokens), loads on activation
- Tier 3 (full): handler logic and examples (~1200 tokens), loads only for complex tasks
- Counter-intuitively, less context = better output because the AI parses less noise to find what matters
--- CONTENT ---
I was burning through $50/month in Claude API costs before I realized the problem. Every session loaded the same 8,000 tokens of context--rules, skills, examples--whether I needed them or not.
Progressive disclosure in AI context management means loading information in tiers: metadata first for routing, schemas on demand for understanding, and full content only when actively using a feature. The result? 60% fewer tokens consumed with better output quality, because the AI isn't parsing through irrelevant context to find what matters. [Claude's context window documentation](https://docs.anthropic.com/en/docs/about-claude/models/overview) outlines the token limits that make this kind of optimization essential for longer sessions.
## Why Was I Wasting So Many Tokens?
You waste tokens when your entire context file loads on every session regardless of what you are doing. A 2,400-line CLAUDE.md containing every skill definition, constraint, and example costs thousands of tokens upfront, even for a quick one-line fix. Most sessions only need a fraction of that content to produce accurate results.
My original CLAUDE.md file was 2,400 lines. Every skill definition. Every constraint. Every example. It loaded completely on every single session.
That specific dread of seeing "context limit reached" mid-task--when you're 80% done and suddenly the AI forgets everything--became a weekly occurrence.
I thought more context was always better. If the AI knows everything, it can handle anything. Right? [Anthropic's documentation](https://docs.anthropic.com) actually recommends against bloated system prompts for exactly this reason--focused context produces more reliable outputs.
Well, it's more like... drowning in information isn't the same as understanding what matters.
## How Does the 3-Tier System Work?
The 3-tier system loads context in stages based on what you actually need. Tier 1 loads only skill metadata, names and triggers, at around 200 tokens. Tier 2 loads schemas and constraints when a skill activates, around 400 tokens. Tier 3 loads full handler logic and examples only for complex tasks, around 1,200 tokens.
Progressive disclosure splits context into three tiers, each loading only when needed.
### Tier 1: Metadata (~200 tokens)
The routing layer. Just enough to know what skills exist and when to activate them.
```markdown
## Skill: sveltekit_architect
**Triggers:** routes, layouts, prerender
**Dependencies:** None
**Priority:** High for SvelteKit projects
```
This is all that loads initially. Skill names, trigger patterns, quick reference. The AI knows the skill exists without knowing everything about it.
### Tier 2: Schema (~400 tokens)
The contract layer. Input/output types, constraints, quality gates.
```markdown
### Input Schema
- operation: "analyze_routes" | "create_layout" | "optimize_prerender"
- targetPath?: string
### Output Schema
- success: boolean
- summary: string
- nextActions: string[]
### Constraints
- Svelte 5 runes only
- Prerender for static pages
- Bundle under 300KB
```
This loads when the skill activates--when you're actually working with routes. Not before.
### Tier 3: Full Content (~1200 tokens)
The implementation layer. Complete handler logic, examples, edge cases.
```markdown
### Handler: analyze_routes
1. Glob all `src/routes/**/+page.svelte`
2. Check each for data loader presence
3. Verify prerender config matches content type
4. Return recommendations
### Example Output
{
"success": true,
"summary": "Found 12 routes, 3 missing data loaders",
"nextActions": ["Add +page.server.ts to /blog/[slug]"]
}
```
This only loads when you're deep into a complex routing task. Most sessions never need it.
## What Are the Actual Token Savings?
Progressive disclosure saves 60 to 92 percent of tokens per session depending on complexity. A skill that loads 3,302 tokens in full costs only about 200 tokens at Tier 1. Most sessions only activate one or two skills, so the majority of your context never loads at all.
Here's the meta-orchestration skill as a real example:
| Tier | Lines | Tokens | When Loaded |
|------|-------|--------|-------------|
| Tier 1 | 278 | ~200 | Every session |
| Tier 2 | 816 | ~600 | On skill activation |
| Tier 3 | 3,302 | ~2,400 | Complex tasks only |
Without progressive disclosure: 2,400 tokens loaded every session.
With progressive disclosure: 200 tokens for most sessions, 600 for active skill use.
**Savings: 60-92% depending on task complexity.**
I load less to get more. The paradox makes sense once you see it in practice.
## How Does Smart Mode Auto-Detect Verbosity?
Smart mode analyzes your query and selects from four verbosity levels automatically. Simple questions trigger Minimal mode, loading only Tier 1 at around 500 tokens. Standard tasks load activated Tier 2 skills at around 1,500 tokens. Complex implementations load full tiers for active skills at 4,000 tokens. Deep debugging loads everything at 8,000 tokens.
The system doesn't just have three tiers--it has four verbosity levels that auto-adjust:
Smart mode analyzes your query and picks the level. Simple questions get minimal context. Architecture questions get everything.
## What Is the Lazy Module Loader Pattern?
The lazy module loader pattern imports skill files and reference docs only when your task explicitly references them, instead of loading everything upfront. Each module caches for 10 minutes and expires when unused. Deduplication ensures the same module never loads twice in one session, regardless of how many times it is referenced.
Beyond skill tiers, the system uses lazy loading for expensive modules:
```javascript
// Instead of:
import { allSkills } from './skills'; // 8,000 tokens
// Use:
const getSkill = (name) => {
return import('./skills/' + name + '.md');
};
```
### Features
- **Dynamic imports**: Load modules only when referenced
- **10-minute TTL**: Cache loaded modules, expire unused ones
- **Deduplication**: Never load the same module twice per session
This pattern works for skill files, reference docs, and example repositories. Anything that doesn't need to exist in context until explicitly requested.
## How Do Skill Bundles Reduce Redundant Loading?
Skill bundles group related skills into a single deduplicated unit that activates together. Instead of loading three separate frontend skills with overlapping context, one frontend bundle activates and shares content between them. This eliminates redundant token usage when working across related skills and keeps the total context smaller than loading each skill independently.
Related skills often load together. Instead of loading them individually:
```json
{
"frontend-bundle": {
"skills": ["react-patterns", "tailwind-stylist", "component-testing"],
"tokens": 4500,
"triggers": ["*.tsx", "*.css", "component"]
}
}
```
When you're working on frontend code, the bundle activates as a unit. No loading three separate skills with overlapping context--one bundle with deduplicated content.
### Current Bundles
- **frontend-bundle**: React, UI/UX, web standards (4,500 tokens)
- **backend-bundle**: API, database, patterns (4,200 tokens)
- **debugging-bundle**: Error resolution, testing (2,500 tokens)
- **workflow-bundle**: Git, CI/CD, deployment (3,200 tokens)
Each bundle is optimized to eliminate redundancy between related skills.
## What's the Token Budget System?
The token budget system enforces hard context limits with automatic responses at each threshold. At 60 percent usage it starts dropping inactive skills to lower tiers. At 75 percent it stops loading new context. It always reserves 20 percent for AI responses. When limits approach, it summarizes and compresses old context before hitting the ceiling.
The context governor enforces hard limits:
- **75% max budget**: Never exceed this regardless of task
- **60% warning threshold**: Start aggressive tier reduction
- **20% reserve**: Always keep space for AI responses
When approaching limits:
1. **Phase unloading**: Release completed phase context
2. **Tier reduction**: Drop to lower tiers for inactive skills
3. **Auto-compact**: Summarize and compress old context
4. **Graceful degradation**: Warn before hitting hard limits
That anxiety of context overflow--the system makes it manageable by budgeting proactively.
## FAQ: Token Optimization for AI Tools
**What is progressive disclosure in AI context management?**
Progressive disclosure loads AI context in tiers: metadata first (~200 tokens), schemas on demand (~400 tokens), full content only when needed (~1200 tokens). This prevents loading thousands of unused tokens upfront.
**How much can progressive disclosure save on AI costs?**
In practice, progressive disclosure saves 40-60% of tokens per session. A skill that would load 3,302 tokens fully only loads 278 tokens at Tier 1--unless you actually need the deeper content.
**Does loading less context hurt AI performance?**
Counter-intuitively, no. Focused context with relevant information outperforms bloated context with everything. The AI processes fewer tokens to find what matters, leading to more accurate responses.
**What are the three tiers in progressive disclosure?**
Tier 1 is metadata (name, triggers, dependencies). Tier 2 is schema (input/output types, constraints). Tier 3 is full content (handler logic, examples). Each tier loads only when needed based on task complexity.
**How do I implement progressive disclosure for Claude Code?**
Split your CLAUDE.md into a router file (~500 lines) and reference files (loaded on demand). Use skill activation scores to determine which tier to load. Start with metadata, escalate to full content only for complex tasks.
---
## Real Token Savings: What the Numbers Actually Look Like
Theory is one thing. Here's what three weeks of session data showed in practice.
I tracked token usage across 47 sessions--ranging from quick bug fixes to multi-hour feature implementations. Before implementing progressive disclosure, every session loaded my full CLAUDE.md: 2,400 lines, approximately 8,200 tokens consumed before I'd typed a single character.
After the 3-tier system:
| Session type | Old usage | New usage | Savings |
|--------------|-----------|-----------|---------|
| Quick fix (under 20 min) | 8,200 | 320 | 96% |
| Standard feature | 8,200 | 1,840 | 78% |
| Complex architecture | 8,200 | 4,100 | 50% |
| Deep debugging session | 8,200 | 6,800 | 17% |
The quick fixes were the biggest surprise. I was burning 8,200 tokens to answer "what's the right Svelte 5 syntax for this component?"--when the answer needed maybe 200 tokens of context to be accurate.
The 60% average savings held across all session types. Complex architecture sessions saved less because they genuinely need the full skill content. But those sessions are maybe 15% of my total Claude usage. The other 85% are standard features and quick fixes where I was wildly overloading context.
One concrete month: April to May after switching. My Claude API bill dropped from $51 to $19. Same work output. Same code quality. Just the difference between loading everything and loading what's needed.
The counterintuitive part: output quality stayed the same or improved. On the quick fixes especially, removing irrelevant context meant Claude wasn't filtering through 8,000 tokens of routing rules and schema definitions to answer a simple syntax question. The focused context got more focused answers.
The $32 monthly savings was a side effect. The main effect was fewer mid-session context limits hitting right when I needed them least. That quality-of-life improvement mattered more than the cost reduction.
I thought the solution to context limits was bigger context windows. Well, it's more like... the solution was loading less, more intentionally.
Maybe the goal isn't maximum context. Maybe it's minimum necessary context--and systems that know the difference. For retrieval-based approaches to the same problem, the [RAG paper by Lewis et al.](https://arxiv.org/abs/2005.11401) pioneered how on-demand retrieval can substitute for loading all knowledge upfront.
---
## Update (June 2026): The Five Behaviors That Actually Move Cost
A year of running this system surfaced a second layer the 3-tier architecture doesn't cover: token cost is not primarily a prompting problem. It is a session design problem. After instrumenting token audits across my production harness, the expensive behaviors ranked like this.
**Pattern 1: Cache prefix mutation.** The single biggest lever, and the one I could not see without measuring it. The Anthropic prompt cache holds the stable prefix of a conversation for five minutes. When a turn reuses that prefix unchanged, the cached portion bills at roughly 10% of normal input cost. When the prefix changes, even by one line, the cache invalidates and the whole prefix re-bills at full cost. At a 15,000-token prefix (a CLAUDE.md plus a few rules files), editing instructions mid-session costs 15,000 tokens every turn for the rest of the session. In a 40-turn session, that is 600,000 tokens of avoidable cost. The rule is not "edit less." It is "edit at the right moment": batch all CLAUDE.md and rules edits to session end.
**Pattern 2: Reading whole files to find one function.** A 3,000-line module read to find one function costs 3,000 tokens. Grep for the function name, then read the 50 relevant lines with an offset, and it costs roughly 100. Search first, read narrowly second.
**Pattern 3: Exploratory investigation in main context.** An investigation that reads 10 files and hits 3 dead ends permanently loads all of it into the conversation. A subagent runs the same investigation and returns "the bug is in line X because Y" in 500 tokens. The trigger: if a task is exploratory and likely to hit dead ends, it belongs in a subagent. The main agent should receive conclusions, not the investigation trail.
**Pattern 4: Re-reading stable context.** If the same files are read repeatedly across turns, that is a context design failure. Structure the session so stable context loads once at the start. This is the practical argument for well-structured CLAUDE.md files: they pay for themselves by eliminating re-read costs in long sessions.
**Pattern 5: Over-scaffolded output.** Verbose completion messages and repeated summaries accumulate in the conversation and cost tokens on every subsequent turn. Compact output compounds as sessions lengthen.
Two harness changes followed. A post-session audit flags any CLAUDE.md edit that happened before turn 30 in a session over 40 turns; every flag was a cache invalidation that paid for nothing. And a dispatch rule routes investigation language ("why is," "find out," "figure out why") to a subagent by default.
Neither change touched how I prompt. They changed when behaviors happen in a session. The expensive behaviors are the invisible ones: mid-session instruction edits, whole-file reads, exploratory dead ends accumulating in main context. Measure those before optimizing prompts.
---
## Related Reading
This is part of the [Complete Claude Code Guide](/blog/claude-code-complete-guide) and the full [tools and resources collection](/products). Continue with:
- [Quality Control System](/blog/how-i-build-with-claude-code) - Two-gate enforcement that blocks "should work"
- [Context Management](/blog/claude-context-management-dev-docs) - Dev docs workflow that prevents context amnesia
- [Evidence-Based Verification](/blog/ai-code-verification-evidence-based) - Why confidence without proof fails
--- END POST ---
================================================================================
POST: My Trading Bot Adjusts Its Own Bet Size. Here Are the 5 Rules.
================================================================================
URL: https://chudi.dev/blog/self-tuner-adaptive-position-sizing-python
Date: 2025-11-01
Tags: trading, python, position-sizing, automation, ai-building
Pillar: ai-building
Reading Time: 6 min
Word Count: 1047
TL;DR: A self-tuner reads recent trade performance and adjusts position sizing dynamically: scaling up after strong periods, scaling down after weak periods. The critical design question is how much history to use and how fast to react. React too fast and you're chasing variance. React too slow and you miss genuine regime changes. I'll show you the architecture that avoids both failure modes.
Key Takeaways:
- Self-tuning is about adjusting to regime changes, not chasing variance. Tune the lookback window to the signal's mean reversion speed.
- Always clamp output. A self-tuner without hard floor and ceiling limits will occasionally size bets at 0% or 150% during outlier periods.
- Rolling Sharpe is more stable than raw win rate for tuning, it accounts for both return and volatility.
- Backtest the tuner itself, not just the underlying strategy. Tuners can underperform fixed sizing on strategies with high variance.
- Dead-man switch: if the database has no recent trades, default to minimum bet size, not last known size.
--- CONTENT ---
A strategy that works on average might not work in all market conditions. Position sizing that is fixed ignores this. A self-tuner adapts.
This post covers the architecture of a self-tuning position sizing system for a prediction market bot: how it reads its own trade history, computes a performance score, and translates that score into a bet size multiplier, without overfitting to variance or blowing up during quiet periods. If you need the foundational math on expected value and [Kelly criterion](/blog/directional-betting-binary-markets-math) before diving into adaptive sizing, that post covers it from first principles.
## TL;DR
- The tuner reads recent trade outcomes from SQLite and computes a performance score
- Performance score drives a multiplier (0.5x to 1.5x) applied to base bet size
- Lookback window should match your strategy's mean reversion speed, not be as long as possible
- Hard clamps prevent the tuner from ever sizing at 0% or above a safe ceiling
- A circuit breaker (separate from tuning) provides a hard stop on consecutive losses
---
## Why Self-Tune?
A fixed position size of $15 treats a period where your strategy is firing at 75% win rate the same as a period where it is firing at 45% win rate.
The insight behind self-tuning: recent performance is predictive of near-future performance for some strategy types. When the strategy is aligned with current market conditions (momentum strategies during trending markets), it performs above baseline. When conditions shift, performance degrades before you manually notice.
If you can detect that degradation early and reduce size, you protect capital. If you detect outperformance and increase size, you compound gains faster.
The risk: reacting to variance as if it were a regime change. After any 10-trade sample, even a 65% win rate strategy will sometimes have 4-6 consecutive losses purely by chance. A tuner that reduces size aggressively on 6 losses in a row is chasing noise.
## How Is the Self-Tuner Structured?
```
TradeDB (SQLite)
|
v
PerformanceReader (queries recent N trades)
|
v
ScoreComputer (win rate, rolling Sharpe, or custom metric)
|
v
MultiplierMapper (score → bet multiplier, with clamps)
|
v
SizingOutput (base_bet × multiplier → final bet)
```
Each component is stateless except TradeDB. The tuner runs before each trade decision and recomputes fresh.
## How Does TradeDB Persist Trade Outcomes?
The tuner needs trade history. SQLite is sufficient for single-instance bots.
```python
import sqlite3
from dataclasses import dataclass
from typing import List, Optional
import time
@dataclass
class TradeRecord:
trade_id: str
timestamp: float
entry_price: float
size_usdc: float
pnl: float # positive = profit, negative = loss
resolved: bool
class TradeDB:
def __init__(self, db_path: str):
self._conn = sqlite3.connect(db_path, check_same_thread=False)
self._create_table()
def _create_table(self):
self._conn.execute("""
CREATE TABLE IF NOT EXISTS trades (
trade_id TEXT PRIMARY KEY,
timestamp REAL NOT NULL,
entry_price REAL NOT NULL,
size_usdc REAL NOT NULL,
pnl REAL,
resolved INTEGER DEFAULT 0
)
""")
self._conn.commit()
def record_trade(self, record: TradeRecord):
self._conn.execute("""
INSERT OR REPLACE INTO trades
(trade_id, timestamp, entry_price, size_usdc, pnl, resolved)
VALUES (?, ?, ?, ?, ?, ?)
""", (
record.trade_id, record.timestamp,
record.entry_price, record.size_usdc,
record.pnl, int(record.resolved)
))
self._conn.commit()
def get_recent_resolved(self, n: int, max_age_secs: float = None) -> List[TradeRecord]:
query = """
SELECT trade_id, timestamp, entry_price, size_usdc, pnl, resolved
FROM trades
WHERE resolved = 1
"""
params = []
if max_age_secs is not None:
cutoff = time.time() - max_age_secs
query += " AND timestamp >= ?"
params.append(cutoff)
query += " ORDER BY timestamp DESC LIMIT ?"
params.append(n)
rows = self._conn.execute(query, params).fetchall()
return [
TradeRecord(*row[:5], bool(row[5]))
for row in rows
]
```
The critical decision here is `WHERE resolved = 1`. Unresolved trades have no outcome yet and cannot inform performance scoring. Including open positions in win rate calculations produces garbage scores.
## How Does PerformanceReader Compute a Performance Score?
The simplest score: win rate over the last N resolved trades.
```python
from dataclasses import dataclass
from typing import List
@dataclass
class PerformanceScore:
n_trades: int
win_rate: float
avg_pnl: float
confidence: str # ANECDOTAL / LOW / MODERATE
class PerformanceReader:
LOOKBACK_TRADES = 30
ANECDOTAL_THRESHOLD = 10
LOW_CONFIDENCE_THRESHOLD = 30
def __init__(self, db: TradeDB):
self._db = db
def compute(self) -> PerformanceScore:
trades = self._db.get_recent_resolved(self.LOOKBACK_TRADES)
if not trades:
return PerformanceScore(0, 0.5, 0.0, "NO_DATA")
n = len(trades)
wins = sum(1 for t in trades if t.pnl > 0)
win_rate = wins / n
avg_pnl = sum(t.pnl for t in trades) / n
if n < self.ANECDOTAL_THRESHOLD:
confidence = "ANECDOTAL"
elif n < self.LOW_CONFIDENCE_THRESHOLD:
confidence = "LOW"
else:
confidence = "MODERATE"
return PerformanceScore(n, win_rate, avg_pnl, confidence)
```
The `confidence` field matters. A tuner operating on 5 trades is reacting to pure noise. The multiplier mapping should discount heavily for ANECDOTAL confidence.
### Rolling Sharpe: A More Stable Alternative
Win rate ignores the size of wins and losses. Rolling Sharpe accounts for both:
```python
import statistics
def compute_rolling_sharpe(trades: List[TradeRecord], risk_free: float = 0.0) -> float:
if len(trades) < 3:
return 0.0 # not enough data
returns = [t.pnl / t.size_usdc for t in trades] # per-dollar return
mean_r = statistics.mean(returns)
std_r = statistics.stdev(returns)
if std_r < 1e-9:
return 0.0 # all trades identical, no variance info
return (mean_r - risk_free) / std_r
```
Positive Sharpe = strategy is generating return above its variance. Negative Sharpe = return doesn't justify the variance. Sharpe of 1.0+ is a strong signal; Sharpe of 0.3 is borderline.
## How Does MultiplierMapper Convert a Score to Bet Size?
The multiplier maps a performance score to a scaling factor. Linear interpolation between confidence-weighted bounds:
```python
class MultiplierMapper:
# Multiplier bounds
FLOOR = 0.5 # never go below half base bet
CEILING = 1.5 # never go above 1.5x base bet
NEUTRAL = 1.0 # no adjustment when performance is baseline
# Win rate thresholds
BASELINE_WIN_RATE = 0.55 # expected win rate for this strategy
STRONG_WIN_RATE = 0.70 # scale up significantly
WEAK_WIN_RATE = 0.45 # scale down significantly
def compute_multiplier(self, score: PerformanceScore) -> float:
# Insufficient data: default to neutral or slight reduction
if score.confidence == "NO_DATA":
return self.FLOOR # no history = minimum size
if score.confidence == "ANECDOTAL":
return self.NEUTRAL * 0.75 # reduce but don't stop
win_rate = score.win_rate
if win_rate >= self.STRONG_WIN_RATE:
raw = self.CEILING
elif win_rate <= self.WEAK_WIN_RATE:
raw = self.FLOOR
else:
# Linear interpolation between floor and ceiling
t = (win_rate - self.WEAK_WIN_RATE) / (
self.STRONG_WIN_RATE - self.WEAK_WIN_RATE
)
raw = self.FLOOR + t * (self.CEILING - self.FLOOR)
# Hard clamp regardless of edge cases
return max(self.FLOOR, min(self.CEILING, raw))
```
The hard clamp at the end is not redundant. Floating point edge cases, database corruption, or integer overflow in trade records can produce extreme scores. The clamp ensures the tuner never outputs a multiplier that could cause catastrophic position sizing.
## Putting It Together: SelfTuner
```python
class SelfTuner:
def __init__(self, db: TradeDB, base_bet_usdc: float):
self._db = db
self._base_bet = base_bet_usdc
self._reader = PerformanceReader(db)
self._mapper = MultiplierMapper()
def get_bet_size(self) -> float:
score = self._reader.compute()
multiplier = self._mapper.compute_multiplier(score)
sized = self._base_bet * multiplier
# Log for auditing
print(
f"[SelfTuner] n={score.n_trades} wr={score.win_rate:.2f} "
f"conf={score.confidence} mult={multiplier:.2f} "
f"bet=${sized:.2f}"
)
return round(sized, 2)
```
Usage in the signal handler:
```python
async def on_signal(direction: Direction):
bet_size = tuner.get_bet_size()
await executor.execute(direction, size_usdc=bet_size)
```
The tuner runs fresh before each trade. It does not cache its result across the session, market conditions and performance data change throughout the day.
## The Circuit Breaker: Hard Stop vs. Soft Tuning
The self-tuner adjusts gradually. The circuit breaker stops the bot entirely.
They are separate systems and serve different purposes:
- **Self-tuner**: adjusts to gradual regime changes. Soft. Continuous adjustment.
- **Circuit breaker**: prevents catastrophic loss from N consecutive losses. Hard. Binary stop.
```python
import json
import os
class CircuitBreaker:
def __init__(self, state_path: str, max_consecutive: int = 4):
self._path = state_path
self._max = max_consecutive
def _load(self) -> dict:
if not os.path.exists(self._path):
return {"consecutive_losses": 0}
with open(self._path) as f:
return json.load(f)
def _save(self, state: dict):
with open(self._path, "w") as f:
json.dump(state, f)
def record_outcome(self, win: bool):
state = self._load()
if win:
state["consecutive_losses"] = 0
else:
state["consecutive_losses"] += 1
self._save(state)
def is_tripped(self) -> bool:
state = self._load()
return state.get("consecutive_losses", 0) >= self._max
def reset(self):
self._save({"consecutive_losses": 0})
```
The circuit breaker persists to disk. If the bot restarts after N consecutive losses, it remains tripped. You must manually reset it after investigating the losses.
## The Lookback Window Problem
The hardest design decision: how many trades to look back?
**Too short (5-10 trades)**: pure variance. After 10 trades, a 65% win rate strategy will have periods of 4-6 consecutive losses 15-20% of the time by pure chance. A 10-trade tuner reacts to this as a regime change and cuts size, then misses the recovery.
**Too long (50-100 trades)**: too slow. If your strategy genuinely degrades (market conditions shift, new market makers enter, fees change), a 100-trade lookback takes weeks to detect the change and adjust.
**The calibration approach**: measure how quickly your strategy's performance autocorrelates. If today's win rate predicts tomorrow's win rate at r=0.7 over 5-trade windows, use 5-trade windows. If r=0.2 (essentially no autocorrelation at short windows), you need a longer window to find the signal.
For the [Polymarket 5-minute BTC strategy](/blog/how-i-built-polymarket-trading-bot) I use: 30-trade lookback, 72-hour max age. Trades older than 72 hours are excluded because BTC volatility regimes shift faster than that. A quiet Saturday is not predictive of an active Monday.
## Backtesting the Tuner
You need to backtest the self-tuner separately from the underlying strategy, because tuning can underperform fixed sizing.
Scenarios where fixed sizing beats self-tuning:
- High-variance strategies where variance is not predictive (high noise floor)
- Short-lived edge periods where the tuner scales up just as performance reverts
- Strategies with long autocorrelation where the tuner reacts too fast
The backtest loop:
```python
def backtest_tuner(trades: List[TradeRecord], base_bet: float) -> dict:
tuner_equity = 1000.0
fixed_equity = 1000.0
history = []
for i, trade in enumerate(trades):
# Fixed sizing
fixed_pnl = (trade.pnl / trade.size_usdc) * base_bet
fixed_equity += fixed_pnl
# Tuner sizing
if i >= 10: # minimum history
recent = trades[max(0, i-30):i]
wins = sum(1 for t in recent if t.pnl > 0)
wr = wins / len(recent)
mult = max(0.5, min(1.5, 0.5 + wr)) # simple linear
sized = base_bet * mult
else:
sized = base_bet
tuner_pnl = (trade.pnl / trade.size_usdc) * sized
tuner_equity += tuner_pnl
history.append({
"trade": i,
"fixed_equity": fixed_equity,
"tuner_equity": tuner_equity,
})
return {
"fixed_final": fixed_equity,
"tuner_final": tuner_equity,
"outperformance": tuner_equity - fixed_equity,
"history": history,
}
```
If the tuner does not outperform fixed sizing in backtests, use fixed sizing. The tuner adds complexity and latency. It only makes sense if it demonstrably improves risk-adjusted returns.
## What the Production System Looks Like
In production, the self-tuner runs as a lightweight module called once before each trade decision. It queries a local SQLite database, computes the score, and returns a bet size in under 10ms.
The full sequence for each signal:
1. [Signal fires](/blog/binance-polymarket-momentum-signal-pipeline) (Binance momentum detected)
2. Circuit breaker check, is it tripped? If yes, abort.
3. SelfTuner.get_bet_size(), what is the current bet size?
4. Execute at computed size
5. On resolution, record outcome to DB
6. If loss: CircuitBreaker.record_outcome(win=False)
The tuner and circuit breaker are independent layers. You can disable either without affecting the other. This separation makes debugging straightforward: a tripped circuit breaker is visually obvious in the log; a tuner operating at 0.75x is logged with explicit score and multiplier.
--- END POST ---
================================================================================
POST: The Cross-Market Signal Pipeline That Spots Price Moves Early
================================================================================
URL: https://chudi.dev/blog/binance-polymarket-momentum-signal-pipeline
Date: 2025-10-15
Tags: trading, python, websocket, polymarket, binance, ai-building
Pillar: ai-building
Reading Time: 13 min
Word Count: 2532
TL;DR: The pipeline has four stages: Binance aggTrade WebSocket stream, a rolling window momentum detector, a signal guard that prevents duplicate entries, and a CLOB order executor. Calibration matters, 0.3% in 60 seconds is the threshold that generates 3-8 signals per day without trading noise. Below that you drown in false triggers.
Key Takeaways:
- Use aggTrade stream, not kline, you need real-time tick data, not candles.
- A deque-based rolling window with timestamp pruning is the cleanest momentum detector.
- Signal guard prevents re-entering the same direction after a recent fill, critical for P&L.
- Async event loop handles WebSocket receive and CLOB placement on the same thread without blocking.
- Calibrate threshold on historical data before going live, different assets have different noise floors.
--- CONTENT ---
Prediction markets are slow to reprice. Binance spot is fast. The pipeline between them is where the edge lives.
This post covers the full signal pipeline for a Polymarket latency arbitrage bot: from Binance WebSocket stream to CLOB order placement, including the filtering, deduplication, and async architecture that prevents duplicate entries and missed signals. If you want the full system architecture first, start with [how I built the Polymarket trading bot](/blog/how-i-built-polymarket-trading-bot), this post is a deep-dive on Stages 1 through 3 of that architecture.
## TL;DR
- Stream Binance aggTrade via WebSocket, tick-level data, not candles
- Rolling 60-second window detects >0.3% momentum
- Signal guard suppresses re-entry on the same direction
- Async executor places CLOB maker order within 100-200ms of signal detection
- Run on Amsterdam VPS: 5-12ms to Polymarket CLOB in London
---
## Why This Pipeline Exists
Polymarket lists binary markets on 5-minute BTC price movements: "Will BTC be higher in 5 minutes?" Market makers reprice YES/NO probabilities based on Binance spot. When BTC moves sharply, there is a 30-90 second lag before Polymarket odds fully reflect the move.
The pipeline exploits that lag by:
1. Detecting the move on Binance first
2. Placing a maker order on Polymarket before market makers reprice
3. Collecting the spread between entry price and resolved probability, the [expected value math](/blog/directional-betting-binary-markets-math) that makes this profitable is straightforward once you have a calibrated p_true
## Stage 1: Binance WebSocket Stream
The signal source is Binance's aggTrade stream, not klines (candlesticks). Here is why that distinction matters:
- **aggTrade**: fires on every individual trade, sub-second latency
- **klines/1m**: fires once per minute, 0-60 second stale data window
For a strategy where the edge window is 30-90 seconds, klines make the signal source 33-100% as wide as the edge itself. You need tick data.
```python
import asyncio
import websockets
import json
from collections import deque
from dataclasses import dataclass
from typing import Optional
@dataclass
class Tick:
timestamp: float # unix seconds
price: float
class BinanceStream:
SYMBOL = "btcusdt"
URL = f"wss://stream.binance.com:9443/ws/{SYMBOL}@aggTrade"
def __init__(self, on_tick):
self._on_tick = on_tick
self._running = False
async def run(self):
self._running = True
backoff = 1
while self._running:
try:
async with websockets.connect(self.URL) as ws:
backoff = 1 # reset on successful connect
async for raw in ws:
msg = json.loads(raw)
tick = Tick(
timestamp=msg["T"] / 1000,
price=float(msg["p"])
)
await self._on_tick(tick)
except Exception as e:
await asyncio.sleep(backoff)
backoff = min(backoff * 2, 30) # exponential backoff, max 30s
def stop(self):
self._running = False
```
The reconnect loop with exponential backoff is not optional. Binance WebSocket connections drop 2-3 times per 24-hour session in my experience, sometimes silently, no error event, no close frame, just a dead socket. Without reconnect logic, your bot goes silent and you don't notice until you check P&L and find it hasn't traded in 6 hours.
The exponential backoff caps at 30 seconds because Binance rate-limits reconnection attempts. If you hammer the endpoint with immediate retries, you'll get temporarily banned. The backoff resets to 1 second on successful connection, so normal reconnects happen quickly while sustained outages don't burn through your rate limit budget.
One subtlety worth noting: the `on_tick` callback is an async function, which means the entire pipeline from tick receipt through order placement runs on the same event loop. This is intentional. A synchronous callback would block the WebSocket receive loop during order placement, causing missed ticks. The async design lets Python interleave tick processing with CLOB API calls without threading complexity.
## Stage 2: Rolling Window Momentum Detector
The detector maintains a 60-second rolling window of price ticks and fires a signal when the net move exceeds the threshold.
```python
from collections import deque
from enum import Enum
from typing import Optional
class Direction(Enum):
UP = "UP"
DOWN = "DOWN"
class MomentumDetector:
THRESHOLD_PCT = 0.003 # 0.3%
WINDOW_SECS = 60
def __init__(self):
self._window: deque[Tick] = deque()
def update(self, tick: Tick) -> Optional[Direction]:
self._window.append(tick)
self._prune(tick.timestamp)
if len(self._window) < 2:
return None
oldest = self._window[0].price
newest = self._window[-1].price
pct_move = (newest - oldest) / oldest
if pct_move >= self.THRESHOLD_PCT:
return Direction.UP
if pct_move <= -self.THRESHOLD_PCT:
return Direction.DOWN
return None
def _prune(self, now: float):
cutoff = now - self.WINDOW_SECS
while self._window and self._window[0].timestamp < cutoff:
self._window.popleft()
```
The deque pruning keeps memory bounded regardless of how long the bot runs. Without pruning, the window grows unboundedly and the oldest price comparison becomes meaningless.
### How the Momentum Detector Works
The detector solves a core problem in momentum-based trading: distinguishing real directional moves from noise. Traditional momentum indicators use closed candles or fixed time windows, but a prediction market bot can't wait for a candle to close, by then the Polymarket market makers have already repriced.
The rolling window approach is different. Instead of waiting for a time boundary, we track every single tick that arrives and continuously evaluate whether the oldest and newest prices in our 60-second window differ by at least 0.3%. This means the signal can fire at any moment during the window, not just at fixed intervals.
The algorithm's elegance lies in its simplicity: for every new tick, we append it to the deque, prune old ticks that fall outside the window, and check if the price move from oldest to newest exceeds our threshold. The computation is O(1) per tick (minus the O(n) for pruning, which amortizes to O(1) across the session because each tick is pruned exactly once).
Why a deque instead of a circular buffer? A circular buffer would require pre-allocating memory for 60 seconds of ticks (roughly 1000+ ticks per second on active Binance, so 60K+ entries). A deque grows naturally with actual tick volume and shrinks as old ticks are discarded. On slow market days, memory usage is negligible. On volatile days, the deque expands to match real activity.
The deque approach also handles market microstructure better than you might expect. During a flash crash or sudden spike, ticks come in clusters, multiple trades at the same or nearby prices fire within milliseconds. A rolling window sees all of them, whereas a sampled-tick approach would miss intermediate prices and underestimate the move. This is critical for latency arbitrage: you want to detect moves as early as possible, and the deque gives you continuous visibility into the price action rather than discrete snapshots.
Another practical advantage: the deque naturally handles reconnects gracefully. When the WebSocket reconnects, you can reseed the deque from historical REST API data (the last 120 seconds of klines), and the detector will have full context for the next signal. A circular buffer would need explicit reset logic. A rolling window with a deque just works.
### Threshold Calibration
0.3% in 60 seconds is the calibrated threshold for BTC. How I arrived at it:
- **0.15% in 30s**: fires constantly, 15-20 signals per day. Most resolve as noise.
- **0.3% in 60s**: fires 3-8 times per day. 62% of signals land in-range (market resolves in signal direction).
- **0.5% in 60s**: fires 0-2 times per day. Too infrequent to validate or build statistical confidence.
The 62% in-range rate translates directly to your p_true estimate for position sizing, see [the math behind directional betting in binary markets](/blog/directional-betting-binary-markets-math) for how to convert that into expected value and Kelly fraction.
The threshold needs recalibration for other assets. ETH has a different noise floor than BTC. SOL and XRP are noisier at shorter windows.
## Stage 3: Signal Guard
Without a signal guard, a sustained 3-minute BTC rally fires 3 separate signals, and the bot opens 3 long positions on what is effectively one trade. The signal guard prevents this.
```python
import time
class SignalGuard:
COOLDOWN_SECS = 120 # 2 minutes
def __init__(self):
self._last_direction: Optional[Direction] = None
self._last_signal_ts: float = 0
def should_trade(self, direction: Direction) -> bool:
now = time.time()
time_since_last = now - self._last_signal_ts
# Same direction within cooldown: suppress
if (direction == self._last_direction and
time_since_last < self.COOLDOWN_SECS):
return False
self._last_direction = direction
self._last_signal_ts = now
return True
```
The guard resets on direction change. A BTC DOWN signal immediately after a BTC UP position is a new, independent signal, not a duplicate. Only same-direction signals within the cooldown window are suppressed.
The 120-second cooldown is calibrated to the 5-minute market duration. A cooldown shorter than 60 seconds lets the bot stack positions on a single sustained move, you end up with three long entries that are really one trade with 3x the risk. A cooldown longer than 180 seconds suppresses legitimate second signals in volatile markets where BTC can make two independent moves within the same 5-minute window. The 120-second sweet spot prevents stacking while preserving the ability to trade a genuine reversal.
In practice, the signal guard prevents roughly 30-40% of raw signals from reaching the executor. This sounds like a lot of missed trades, but those suppressed signals are almost always duplicates of a sustained move, entering a second position on the same momentum rarely adds edge and always adds risk. The guard is one of the highest-value components per line of code in the entire pipeline.
## Stage 4: CLOB Order Executor
The executor receives a validated signal and places a maker order on the Polymarket CLOB. The `BASE_BET_USDC` here is fixed for simplicity, in production, [a self-tuning system](/blog/self-tuner-adaptive-position-sizing-python) adjusts this value dynamically based on recent win rate.
```python
from py_clob_client.client import ClobClient
from py_clob_client.clob_types import OrderArgs, BUY
class CLOBExecutor:
MIN_SHARES = 5 # Polymarket minimum
BASE_BET_USDC = 15 # dollar amount per trade
def __init__(self, client: ClobClient, market_registry):
self._client = client
self._registry = market_registry
async def execute(self, direction: Direction) -> Optional[str]:
market = self._registry.get_active_5m_btc_market(direction)
if not market:
return None
# Check market has enough time remaining
if market.secs_remaining <= 30:
return None
# Compute shares from dollar amount
mid_price = market.mid_price
shares = self.BASE_BET_USDC / mid_price
if shares < self.MIN_SHARES:
return None
order_args = OrderArgs(
token_id=market.token_id,
price=mid_price,
size=round(shares, 2),
side=BUY,
)
result = self._client.create_and_post_order(order_args)
return result.order_id if result else None
```
The expiry check (`secs_remaining <= 30`) prevents placing orders on markets that are about to close. Without it, you fill on a market that resolves 10 seconds later and cannot place an exit if needed. I learned this the hard way, an early version of the bot placed an order with 8 seconds remaining, got filled, and the market resolved before I could even check the position status. The capital was locked for the full resolution cycle.
The `create_and_post_order` call is critical. The Polymarket SDK has both `create_order` and `create_and_post_order`. The first only builds the order object locally without submitting it, a naming trap that cost me an evening of debugging when orders appeared to succeed but never showed up on the CLOB. Always use `create_and_post_order` for actual submission.
## Wiring It Together: Async Event Loop
The full pipeline runs on a single asyncio event loop. The WebSocket receive loop, momentum detection, and CLOB placement all happen asynchronously without blocking each other.
```python
async def main():
detector = MomentumDetector()
guard = SignalGuard()
executor = CLOBExecutor(client, registry)
async def on_tick(tick: Tick):
direction = detector.update(tick)
if direction is None:
return
if not guard.should_trade(direction):
return
order_id = await executor.execute(direction)
if order_id:
log.info(f"Placed order {order_id} — {direction.value}")
stream = BinanceStream(on_tick)
await stream.run()
asyncio.run(main())
```
The event loop is single-threaded but non-blocking. `on_tick` is called for every Binance trade tick. The CLOB placement is awaited inline, if the CLOB call takes 50ms, the next tick is processed after it completes.
For strategies with heavier logic, you can offload placement to a separate task:
```python
async def on_tick(tick: Tick):
direction = detector.update(tick)
if direction and guard.should_trade(direction):
asyncio.create_task(executor.execute(direction))
```
This lets tick processing continue while placement runs in the background.
## Latency Breakdown
On a well-tuned Amsterdam VPS, the pipeline latency from Binance tick to CLOB order acknowledgement:
| Stage | Latency |
|-------|---------|
| Binance WS → Python receive | 1-5ms |
| Deque update + threshold check | <1ms |
| Signal guard check | <1ms |
| CLOB order construction | <1ms |
| CLOB API round trip (Amsterdam→London) | 5-12ms |
| **Total** | **~10-20ms** |
20ms from Binance tick to CLOB order. The Polymarket repricing lag is 30-90 seconds. You have 30-90,000ms of edge window, and you use 20ms of it.
The math still holds even with US East latency (130-150ms total). The bottleneck is not network, it is whether anyone else detected the signal first.
## What Can Go Wrong
**WebSocket disconnects during a live position.** Your position is still open, but you have no incoming price data to decide when to exit. Solution: implement position health checks on reconnect that query CLOB for open positions before resuming signal processing.
**Market not found for a signal direction.** The 5-minute BTC market rolls over every 5 minutes. If a signal fires at minute 4:58, the current market might have 2 seconds remaining. Your registry needs to handle market lookup gracefully with a fallback to the next available market.
**CLOB rate limits.** If you place too many orders in rapid succession, the CLOB returns rate limit errors. The signal guard helps here, but also implement a per-minute order count limiter with an asyncio.sleep backoff.
**Silent order rejections.** The CLOB occasionally rejects orders with a 200 OK but no `order_id` in the response. Always check the return value, do not assume success from HTTP status alone.
## Testing the Pipeline Before Going Live
Before deploying this pipeline with real money, you need a systematic way to validate that every stage works correctly in isolation and as a connected system. The testing approach I used saved me from at least three bugs that would have cost real capital.
Start with the momentum detector. Feed it historical tick data from the Binance REST API and verify that signals fire at the expected timestamps. I downloaded 24 hours of aggTrade data for three different volatility regimes: a quiet Sunday, a normal weekday, and a day with a major price move. The detector should produce zero signals on the quiet day, three to eight on the normal day, and ten to fifteen on the volatile day. If the numbers are wildly different, your threshold is miscalibrated.
Next, test the signal guard in isolation. Create a synthetic sequence of signals: UP, UP, UP (should suppress the second and third), then DOWN (should pass through immediately), then DOWN again within two minutes (should suppress). This catches the most common guard bug, which is accidentally suppressing direction changes along with duplicates.
The executor is the hardest to test without real money. I built a dry-run mode that constructs the order object and logs it without calling the CLOB API. This validates the share calculation, the minimum share check, and the expiry guard. Run the dry-run executor against live market data for a full trading day and manually verify that every logged order makes sense: correct direction, reasonable share count, valid token ID, and sufficient time remaining on the market.
Finally, run the full pipeline end-to-end in dry-run mode for at least 48 hours. Watch for three specific failure patterns. First, zombie signals where the detector fires but the guard or executor silently drops the signal without logging why. Second, phantom orders where the executor logs an order that should have been suppressed by the guard. Third, timing failures where the executor tries to place an order on an expired market because the registry returned stale data.
The 48-hour window matters because it covers both high-volatility and low-volatility periods. A pipeline that works perfectly during active US trading hours might break during the Asian session when tick frequency drops and the rolling window behaves differently with sparse data.
## Monitoring in Production
Once the pipeline is live, you need real-time visibility into every stage. Logging alone is not sufficient because you cannot watch logs continuously, and by the time you notice a problem in the logs, the damage is already done.
I built a lightweight health monitor that tracks four metrics. Signal rate measures how many raw signals the detector produces per hour. A sudden drop to zero means the WebSocket is probably disconnected. Guard pass rate measures what percentage of signals survive the guard. If this drops below fifty percent, the cooldown might be too aggressive for current market conditions. Executor success rate measures what percentage of passed signals result in a confirmed order on the CLOB. A drop here usually means an API issue or an allowance problem. Average latency measures the time from tick receipt to order acknowledgment. If this creeps above 100 milliseconds, something in the pipeline is blocking.
These four metrics tell you the health of the entire pipeline at a glance. I push them to a Slack channel every fifteen minutes during active trading hours. Any metric that crosses a threshold triggers an immediate alert. The alerting is simple: if signal rate hits zero for fifteen minutes during US market hours, something is wrong and the bot needs attention.
The most valuable metric over time turned out to be the guard pass rate. When market regime shifts from trending to ranging, the pass rate changes significantly. During strong trends, the guard suppresses sixty percent of signals because the same directional move generates multiple triggers. During choppy markets, the pass rate rises to eighty percent because signals alternate direction frequently. Tracking this ratio over weeks gave me insight into when the strategy was in a favorable regime versus when I should expect lower returns.
## Signal Frequency Expectations
On active BTC trading days:
- 3-8 signals with 0.3%/60s threshold
- Each signal potentially captures a 5-minute market (or what remains of it)
- 1-3 of those signals will be suppressed by the guard as duplicates of a sustained move
On quiet days (low volatility): 0-2 signals. This is fine. The strategy is about edge quality, not trade frequency.
**See also:** [I Built a Live Trading Bot in Python. Here's What Actually Works.](/blog/algorithmic-trading-python-ai-complete-guide)
--- END POST ---
================================================================================
POST: I Sized My Polymarket Bets Wrong. Kelly Criterion Fixed It.
================================================================================
URL: https://chudi.dev/blog/directional-betting-binary-markets-math
Date: 2025-10-01
Tags: trading, math, kelly-criterion, polymarket, position-sizing
Pillar: ai-building
Reading Time: 9 min
Word Count: 1611
TL;DR: Binary markets are brutally honest about edge. Expected value = (p_true × payout) - stake. If your p_true exceeds the market's implied probability by a meaningful margin, you have positive EV. Kelly sizing tells you how much to bet. The hard part is honestly estimating p_true, every overestimate directly transfers money to the market.
Key Takeaways:
- EV is the only number that matters. Win rate alone means nothing without payout size.
- Kelly criterion gives the theoretically optimal bet size for a given edge, but half-Kelly is safer in practice.
- Markets in binary prediction platforms are already efficient at popular events. Your edge must come from information or speed.
- Fee drag is larger than it looks. A 1.5% taker fee at p=0.50 requires a 3% edge just to break even.
- Overconfidence in p_true is the primary cause of losses for new prediction market traders.
--- CONTENT ---
Binary markets punish sloppy position sizing immediately. Half-Kelly sizing (f* / 2) is the fix: Polymarket's 2% taker fee at p=0.50 requires roughly a 3% probability edge just to break even, and getting the fraction wrong either leaves money on the table or exposes your bankroll to a single bad estimate. Across 23 live BTC trades using this exact math, the win rate came in at 69.6%.
Binary markets like Polymarket strip away ambiguity. Either the event happens or it doesn't. You either get paid $1.00 per share or you get $0. That clarity makes the math clean and makes it immediately obvious whether you have an edge. The full bot architecture this math sits inside is documented in [How I Built a Polymarket Trading Bot](/blog/how-i-built-polymarket-trading-bot); this post is the directional-math slice.
Most people who lose money in prediction markets are not doing the math wrong. They are skipping it entirely.
## TL;DR
- **Expected value** = (true probability × net gain) - (loss probability × stake)
- **You have edge** when your true probability estimate exceeds the market's implied probability
- **Kelly criterion** sizes the bet to maximize long-run bankroll growth
- **Fees** shrink your EV and require a larger edge to break even
- **The hard part** is honest probability estimation, every overconfidence point is money transferred to the market
---
## The Mechanics of a Binary Market
A binary market works like this: you buy YES shares if you believe an event will occur, or NO shares if you believe it won't. Each share costs between $0.01 and $0.99, and pays $1.00 on resolution if correct.
The price IS the market's implied probability. A YES share at $0.65 means the market believes the event has a 65% chance of occurring.
If you believe the true probability is 75%, you have edge: the market is underpricing the YES outcome. You buy YES at $0.65 and expect to receive $1.00 with 75% probability.
## Expected Value: The Only Metric That Matters
Expected value (EV) is your average outcome per dollar bet, calculated across all possible resolutions.
For a binary bet:
```
EV = (p_true × profit_if_win) + ((1 - p_true) × loss_if_lose)
```
Where:
- `p_true` = your estimate of the true probability
- `profit_if_win` = payout minus entry cost = (1 - entry_price)
- `loss_if_lose` = entry cost = entry_price
Example: Polymarket lists "BTC up in 5 minutes" at $0.62. You estimate the true probability is 72% based on a momentum signal.
```
profit_if_win = 1.00 - 0.62 = $0.38 per share
loss_if_lose = $0.62 per share
EV = (0.72 × $0.38) + (0.28 × -$0.62)
EV = $0.2736 - $0.1736
EV = $0.10 per share
```
You expect to earn $0.10 for every $0.62 wagered, a 16.1% expected return.
If p_true = p_market (0.62), EV = 0. The market is fairly priced for your estimate. No edge.
If p_true < p_market, EV is negative. You are betting against the odds.
### The EV Formula Simplified
For binary markets, EV can be expressed more cleanly:
```
EV per share = p_true - p_market
```
When p_true = 0.72 and p_market = 0.62: EV = 0.72 - 0.62 = $0.10 per share. This holds exactly for $1 payout markets.
This is the core insight: **your EV is the gap between what you believe and what the market believes.** If you can't articulate why your p_true is higher than p_market, you don't have edge.
## Fee Drag
Fees directly reduce EV. They do not appear in the calculation above because they need to be added separately for maker vs. taker orders.
**Taker orders (fill immediately)**: Polymarket charges approximately 2% on winning positions. This means:
```
Effective EV = p_true × (1 - fee_rate) - p_market × (1 - p_true) / p_true
```
Simplified: if you pay 2% on wins, you lose 2% × p_true per share. At p_true = 0.72:
```
Fee drag = 0.02 × 0.72 × $1.00 = $0.0144 per share
Effective EV = $0.10 - $0.0144 = $0.0856 per share
```
The fee consumed 14.4% of your edge. At lower win probabilities, the fee drag is proportionally smaller (fewer wins to charge), but the EV itself is already thinner.
**Maker orders (limit orders, filled later)**: Zero fee on Polymarket. The full $0.10 EV is preserved. This is why maker entry is the only viable structure for thin-edge strategies.
**Break-even edge for taker entry**: At p_market = 0.50 and 2% taker fee, you need p_true ≥ 0.52 just to break even. Every percentage point of edge buys you $0.01 of expected profit per share. Taker fees consume 2 of those percentage points before you start.
## Kelly Criterion: How Much to Bet
Once you have confirmed positive EV, Kelly criterion calculates the optimal bet size. "Optimal" means: the fraction of bankroll that maximizes long-run geometric growth.
The Kelly formula:
```
f* = (p × b - q) / b
```
Where:
- `f*` = fraction of bankroll to bet
- `p` = probability of winning (your p_true)
- `q` = probability of losing (1 - p)
- `b` = net odds (profit per $1 wagered if you win)
For our example: p = 0.72, p_market = 0.62
```
b = (1 - 0.62) / 0.62 = 0.38 / 0.62 = 0.613
q = 1 - 0.72 = 0.28
f* = (0.72 × 0.613 - 0.28) / 0.613
f* = (0.4414 - 0.28) / 0.613
f* = 0.1614 / 0.613
f* = 26.3%
```
Full Kelly says bet 26.3% of your bankroll on this trade. With a $1,000 bankroll, that is $263.
### Why You Should Use Half-Kelly
Full Kelly is theoretically optimal but practically dangerous because of estimation error in p_true. If your true probability estimate is even slightly overconfident, full Kelly over-bets.
Consider: you estimate p = 0.72 but actual win rate turns out to be 0.65. Full Kelly at 0.72 overbets significantly. The result is higher volatility, larger drawdowns, and potential ruin.
Half-Kelly:
```
f_half = f* / 2 = 26.3% / 2 = 13.2%
```
At half-Kelly, you bet $132 from a $1,000 bankroll. You give up roughly 25% of growth rate in exchange for half the variance. For most practitioners, this tradeoff is favorable.
Quarter-Kelly (6.6%) is used by very risk-averse operators or when p_true confidence is low.
## The Direction Question: When Is Your Edge Real?
The math above assumes p_true is accurate. Estimating it honestly is the hardest part.
For price-movement markets (BTC up/down in 5 minutes), there are three legitimate sources of edge:
**1. Information advantage**: You have data the market hasn't priced yet. In a latency arb strategy, the [Binance momentum move is your information](/blog/binance-polymarket-momentum-signal-pipeline), the Polymarket market hasn't repriced it yet. p_true = conditional probability of continued movement given current momentum.
**2. Speed advantage**: You can act on public information faster than market makers can update their quotes. This is pure latency arbitrage. Your edge is time, not information.
**3. Model advantage**: Your probability estimate is more accurate than the crowd's. This requires having a calibrated model that outperforms the market on a sample large enough to be statistically meaningful.
**What is not a source of edge**: gut feeling, recency bias, pattern-matching on a small sample, "I think BTC is going up today."
### Calibration Check
A properly calibrated probability estimate means: when you say 70%, you win 70% of the time over a large sample. When you say 80%, you win 80%.
Most people's estimates are overconfident. They think 70%, they win 58%. The gap is money transferred to the market.
A simple calibration test: track every trade you make with your entry p_true estimate. After 100 trades, compare your estimated win rates to actual win rates by bucket. If your "70%" bucket wins at 58%, your estimates are 12 percentage points overconfident. Adjust.
## A Worked Example: BTC 5-Minute Market
You are running a [Binance momentum detection bot](/blog/binance-polymarket-momentum-signal-pipeline). BTC has moved +0.35% in the past 60 seconds. Based on historical data, when this pattern fires, BTC continues upward and resolves YES 69% of the time, consistent with the [69.6% win rate across 23 live BTC trades](/blog/how-i-built-polymarket-trading-bot).
Current market: "BTC up 5 minutes" YES at $0.58.
```
p_true = 0.69
p_market = 0.58
EV per share = 0.69 - 0.58 = $0.11
b = (1 - 0.58) / 0.58 = 0.72
q = 0.31
f_full = (0.69 × 0.72 - 0.31) / 0.72
f_full = (0.497 - 0.31) / 0.72
f_full = 26%
f_half = 13% of bankroll
```
With $2,000 bankroll: bet $260 (13%). At $0.58 per share: 448 shares.
Expected outcome:
- Win (69% of time): 448 shares × $1.00 = $448. Profit: $448 - $260 = $188.
- Lose (31% of time): 448 shares × $0.00 = $0. Loss: -$260.
EV per trade: (0.69 × $188) + (0.31 × -$260) = $129.7 - $80.6 = **+$49.1**
Over many trades, this compounds. That is why Kelly sizing and honest EV calculation are the foundation, not optional extras.
## Common Mistakes
**Mistaking win rate for edge.** A 65% win rate sounds impressive. At p=0.80 entry, a 65% win rate is catastrophically negative EV. Win rate only matters relative to entry price.
**Ignoring fee drag on marginal edges.** An edge of 3% gross becomes 1% net after taker fees. The difference between "bet aggressively" and "don't bet" can be a single fee calculation.
**Treating correlated bets as independent.** Three bets on "BTC up in the next hour" are not three independent bets, they are one bet expressed three ways. Kelly sizing assumes independence. Correlated positions require reducing bet size proportionally to correlation.
**Updating p_true mid-trade on noise.** Your pre-trade probability estimate should be locked in at entry. Reacting to market moves mid-trade by changing your p_true estimate introduces emotion into what should be a mechanical process.
If your position sizing is still guesswork, that's a fixable gap, not a personality trait. My [prediction market and trading bot lane](/services) runs $1,500 to $5,000: strategy implementation, backtesting harnesses, and live deployment with monitoring, so the sizing math above ships inside a system that logs every decision before it executes.
## The Summary Formula
For any directional bet in a binary market:
1. **Estimate p_true** honestly, with a calibrated model
2. **Calculate EV** = p_true - p_market (simplified for $1 payout markets)
3. **Subtract fee drag** if using taker entry (2% on wins)
4. **If EV > 0**, calculate Kelly fraction: f = (p × b - q) / b
5. **Bet half-Kelly** to account for estimation error
6. **Track and recalibrate** p_true against actual outcomes after every 50+ trades, and consider an [adaptive position sizing system](/blog/self-tuner-adaptive-position-sizing-python) that adjusts bet size automatically as win rate shifts
Binary markets will tell you exactly how accurate your probability estimates are. There is no narrative to hide behind, only resolution and math.
The sizing layer described here runs live inside a production system documented in [How I Built a Claude Code Trading Bot](/blog/claude-code-production-trading-bot): 36,000 lines, real money, Kelly criterion sizing.
--- END POST ---
================================================================================
POST: Polymarket Arbitrage Bot in Python: How I Hit a 69.6% Win Rate
================================================================================
URL: https://chudi.dev/blog/how-i-built-polymarket-trading-bot
Date: 2025-09-15
Tags: trading, python, automation, polymarket, ai-building
Pillar: ai-building
Reading Time: 10 min
Word Count: 1860
TL;DR: I built a latency arbitrage bot for Polymarket by tapping Binance WebSocket price feeds and placing orders on the CLOB before market makers reprice. The core insight: Polymarket's 5-minute BTC up/down markets often lag Binance by 30-90 seconds. That lag is the edge. 69.6% win rate across 23 clean trades. Here's the full architecture.
Key Takeaways:
- Polymarket's CLOB lags Binance spot by 30-90 seconds on large moves, that's your edge.
- Maker entry is free. Taker entry costs 1.56% at p=0.50. Fee structure determines profitability.
- Server location matters more than code quality. Amsterdam is 5-12ms to London. US East is 130-150ms.
- Signature type matters: proxy wallets (type 1) are gasless and required for MagicLink accounts.
- The biggest bugs were silent, wrong allowance calls, wrong balance checks, missing expiry guards.
--- CONTENT ---
Polymarket's CLOB takes 30-90 seconds to reprice after a large Binance spot move. That lag is a tradeable edge: using asyncio WebSocket signals and maker CLOB orders at zero taker fee, this bot turned it into a 69.6% win rate across 23 real-money BTC trades in February 2025.
I spent three months building a trading bot for Polymarket. Not a "signal subscriber" or a wrapper around someone else's strategy: a system that finds mispricing, places orders, and manages exits autonomously with real money on the line. The deployment infrastructure this bot runs on is documented in [Deploy Python Agents to DigitalOcean](/blog/deploy-python-agent-digitalocean) (the infrastructure cluster's pillar).
Here is the full architecture: what works, what broke, and what the numbers look like.
## TL;DR
- **The edge**: Polymarket's 5-minute BTC up/down markets lag Binance price moves by 30-90 seconds
- **The strategy**: Detect a large BTC move on Binance, buy the corresponding YES/NO market on Polymarket before odds reprice
- **The stack**: Python, asyncio, Binance WebSocket, py_clob_client, VPS in Amsterdam
- **The results**: 69.6% win rate, +$11.51 net over 23 clean trades (Feb 13-16, 2025)
- **The hardest part**: The bugs were silent, wrong API calls that succeeded but destroyed your position
---
## Why Prediction Markets Have Exploitable Lags
Polymarket runs binary options on events. The 5-minute BTC up/down markets ask: will BTC be higher or lower in 5 minutes than it is right now?
Market makers on Polymarket reprice these markets in response to Binance spot price moves. But they are not colocated at Binance. There is latency in their observation, decision, and order update cycle.
That window, typically 30 to 90 seconds on large moves, is the edge. When BTC moves 0.3% in 60 seconds, the Polymarket YES probability for "BTC up" often hasn't fully moved from 0.55 to 0.80 yet. You buy YES at 0.60, the market closes at 0.80 (already resolved YES by then), and you capture the spread.
The math on a winning trade with maker entry:
```
Entry: 100 shares at $0.60 = $60
Payout: 100 shares × $1.00 = $100
Profit: $100 - $60 = $40 (66.7% return)
Maker fee: $0 (Polymarket pays makers)
```
The math on a losing trade:
```
Entry: 100 shares at $0.60 = $60
Payout: $0 (market resolves other way)
Loss: -$60
```
This means the strategy needs roughly 60% win rate at p=0.60 entry price to break even, and historically the signal fires at higher-conviction entry points.
## The Architecture
The system has four layers:
1. **Signal layer**, Binance WebSocket detects >0.3% BTC move in 60 seconds
2. **Decision layer**, checks market conditions, filters, existing positions
3. **Execution layer**, places maker order on Polymarket CLOB
4. **Exit layer**, holds to resolution or uses a stop-loss on adverse move
### Signal Layer: Binance WebSocket
For a complete breakdown of this stage, including the full `MomentumDetector`, `SignalGuard`, and reconnect architecture, see [Binance to Polymarket: Building a Real-Time Momentum Signal Pipeline](/blog/binance-polymarket-momentum-signal-pipeline).
```python
import asyncio
import websockets
import json
from collections import deque
THRESHOLD_PCT = 0.003 # 0.3%
WINDOW_SECS = 60
price_window: deque = deque()
async def monitor_binance():
url = "wss://stream.binance.com:9443/ws/btcusdt@aggTrade"
async with websockets.connect(url) as ws:
async for msg in ws:
data = json.loads(msg)
price = float(data["p"])
ts = data["T"] / 1000
price_window.append((ts, price))
# Prune window
cutoff = ts - WINDOW_SECS
while price_window and price_window[0][0] < cutoff:
price_window.popleft()
if len(price_window) < 2:
continue
oldest_price = price_window[0][1]
pct_move = (price - oldest_price) / oldest_price
if abs(pct_move) >= THRESHOLD_PCT:
direction = "UP" if pct_move > 0 else "DOWN"
await on_signal(direction, pct_move, price)
```
The key calibration: 30-second/0.15% threshold produced zero signals in testing. 60-second/0.3% produced 62% of BTC moves landing in-range. You need enough signal frequency to generate trades, but not so much that you're trading noise.
### Execution Layer: py_clob_client
The Polymarket Python SDK has some surprises. Here is the correct order placement pattern:
```python
from py_clob_client.client import ClobClient
from py_clob_client.clob_types import OrderArgs, OrderType
client = ClobClient(
host="https://clob.polymarket.com",
key=PRIVATE_KEY,
chain_id=137,
signature_type=1, # POLY_PROXY for MagicLink/email accounts
)
async def place_maker_order(token_id: str, price: float, size: float):
order_args = OrderArgs(
token_id=token_id,
price=price,
size=size,
side=BUY, # always BUY — YES and NO are separate tokens
)
# This creates AND submits the order in one call
result = client.create_and_post_order(order_args)
return result
```
Two things that will burn you if you don't know them:
**Always use `create_and_post_order`, not `create_order`.** The SDK has both. `create_order` only creates a local order object, it does not submit it. This is not obvious from the name.
**Always use `side=BUY`.** YES and NO are separate tokens on Polymarket. You never "sell NO", you "buy YES." When you want to exit a YES position, you place a SELL order, but entry is always BUY on the token you want to hold.
### Signature Types
This tripped me up for a week. Polymarket has three signature types:
- **Type 0 (EOA)**: Standard private key, EIP-712 signing. Works for wallets you control directly.
- **Type 1 (POLY_PROXY)**: For MagicLink/email accounts. Gasless proxy contract signing. Different from type 0 in ways that matter for allowances.
- **Type 2 (POLY_GNOSIS_SAFE)**: MetaMask browser wallet, Gnosis Safe. Not suitable for automated bots.
If you have a MagicLink account (most consumer Polymarket accounts), you need type 1. Using type 0 on a type 1 account will produce valid-looking signatures that the CLOB silently rejects.
## The Allowance Bug That Cost Real Money
The CLOB tracks internal allowances separately from on-chain state. There are two API calls that sound similar but do opposite things:
```python
# SAFE: reads current CLOB internal allowance
client.get_balance_allowance()
# DANGEROUS: reads on-chain, then OVERWRITES CLOB internal allowance
client.update_balance_allowance()
```
I had code that called `update_balance_allowance()` before placing sell orders to "confirm I had enough allowance." This overwrote the CLOB's internal allowance with a freshly read on-chain value, which, after a recent fill, was zero. Every sell after that failed silently because the CLOB thought I had no allowance.
The fix: never call `update_balance_allowance()` after a fill. Only call `get_balance_allowance()` to read. The CLOB updates its internal state correctly on its own after fills.
## Server Location Is Not Optional
The Polymarket CLOB is hosted in London (eu-west-2). Latency from different locations:
| Location | Latency |
|----------|---------|
| Amsterdam | 5-12ms |
| Frankfurt | 15-25ms |
| US East (NYC) | 130-150ms |
| US West | 180-200ms |
At 130ms from US East, you are behind every European competitor by 120ms. In a strategy where the edge window is 30-90 seconds, 120ms is not catastrophic, but it compounds with every other source of latency.
My bot runs on a [QuantVPS](https://www.quantvps.com?via=chudinnorukam) instance in Amsterdam. The difference in fill quality between the Amsterdam server and testing from my Mac in San Francisco was measurable. I started on a [$6/month DigitalOcean Droplet](https://www.awin1.com/cread.php?awinmid=123996&awinaffid=3002649&ued=https%3A%2F%2Fwww.digitalocean.com%2F) before migrating to a specialized VPS like QuantVPS when my strategy demanded sub-10ms latency. If you want the full Droplet setup guide, it's at [deploy a Python agent on DigitalOcean](/blog/deploy-python-agent-digitalocean), and the 24/7 keep-alive setup (systemd, heartbeats, what a $6 box can actually carry) is in [run your AI agent on a VPS](/blog/run-ai-agent-on-vps).
QuantVPS hosts the live bot: Amsterdam instance, sub-10ms to the venues this strategy needs. DigitalOcean's $6 Droplet runs everything that isn't latency-sensitive (monitors, dashboards, cron jobs). My honest advice: start on the Droplet, and pay for QuantVPS only when your fill data proves latency is costing you money. I used both before either paid me a cent.
## What the Numbers Look Like
Over 23 clean BTC trades (February 13-16, 2025, excluding deployment-disrupted trades):
- **Win rate**: 69.6% (16/23)
- **Net P&L**: +$11.51
- **Average per trade**: +$0.50
- **BTC Down signal**: 75% win rate (strongest direction)
- **BTC Up signal**: 65% win rate
"Clean" means I excluded trades during bot restarts, code deploys, and the 48-hour period after I changed the entry threshold. Deployment-disrupted trades are not strategy losses, they're operational noise.
The win rate of 69.6% at average entry prices around p=0.62-0.68 puts this well above break-even. The position sizing is conservative ($5-15 per trade) because this is early validation, not full deployment, in the full system, an [adaptive self-tuner](/blog/self-tuner-adaptive-position-sizing-python) scales that base bet based on rolling performance. For the expected value and Kelly sizing math behind why that win rate translates to positive P&L, see [The Math Behind Directional Betting in Binary Markets](/blog/directional-betting-binary-markets-math).
## The Exit Problem
Most of the complexity is not in signal generation or order placement, it is in exit management.
Options for exit:
1. **Hold to resolution**: simplest, highest EV if your directional signal is right
2. **Stop-loss on adverse move**: cut the position if odds move against you beyond a threshold
3. **Time-based exit**: exit N minutes before market close
"Hold to resolution" sounds right but requires discipline. A position at 0.65 that moves to 0.40 before resolving YES will feel terrible even if the final outcome is correct. Most people override their bot at exactly the wrong moment.
I use hold-to-resolution as the default with a secondary check: if position moves against me AND the market has more than 3 minutes left AND I am down more than 30% on the position, I exit and redeploy elsewhere. This is not optimal from a pure EV perspective, but it prevents large single-trade losses from ending the session.
## What I Would Do Differently
**1. Build the exit manager before the entry logic.** Entries are easy to tune. Exits determine whether a 69% win rate turns into positive P&L or not.
**2. Validate the expiry check from day one.** Markets close abruptly. If you place an order 1 second after a market closes, the fill happens but you can not exit. Add `secs_remaining <= 0` checks before every order.
**3. Log everything to a DB immediately.** I lost the first two weeks of data because I was logging to a text file that got rotated. SQLite with trades, fills, and exits gives you something to analyze. Every trade should record: entry price, exit price, signal strength, market ID, timestamps, and the reason the exit triggered. Without this data, you cannot distinguish between strategy problems and execution problems. Most early debugging sessions ended with "I think it lost money on that trade but I can not prove why." That is an unacceptable state for a system handling real capital.
**4. Colocation is table stakes, not optimization.** I treated server location as something to optimize later. It is actually part of the core strategy definition.
## Operational Challenges: When Theory Meets Production
Running the bot on a live VPS introduced challenges that don't show up in local testing. The first week was dominated by what I call "invisible failures", operations that succeeded at the API level but broke the underlying strategy logic in ways that took days to diagnose.
The most insidious: after a successful fill, the bot would enter a state where it couldn't exit. It had capital locked in a position but no way to liquidate. The root cause was subtle, the CLOB's internal state tracking after a fill didn't always sync with what the REST API reported for balance. Polling `get_balance()` after a fill would return stale data. The solution was adding a 500ms delay before checking balance post-fill and validating against multiple sources. A single delay. Days of debugging.
Network reliability mattered more than code quality. The Binance WebSocket connection dropped 2-3 times per 24-hour session, sometimes silently (no error event, just dead). The reconnect loop caught most of it, but during the blind period (first 60 seconds after reconnect), the bot would miss signals or, worse, place orders on old signal data. Solution: on reconnect, dump the entire rolling window and reseed from REST API klines for the last 120 seconds before resuming signal processing.
Position management in production revealed another gap: the bot would place an order, get filled, then lose the fill notification if the connection hiccupped between order acknowledge and fill event. The CLOB would have the position, but the bot wouldn't know about it. This left zombie positions open. The fix was polling the CLOB's open positions on every signal, expensive, but necessary for correctness.
How this codebase grew from a 2-hour prototype into a 36,000-line production system, and the five Claude Code principles that kept it correct, is documented in [How I Built a Claude Code Trading Bot](/blog/claude-code-production-trading-bot).
**See also:** [I Built a Live Trading Bot in Python. Here's What Actually Works.](/blog/algorithmic-trading-python-ai-complete-guide) and, on the documentation habit that made this post citable in the first place, [What Actually Predicts Whether AI Cites Your Content](/blog/ai-citability-audit-what-predicts-citations)
--- END POST ---
================================================================================
POST: llms.txt for AI Crawlers: Why robots.txt Is Not Enough
================================================================================
URL: https://chudi.dev/blog/llms-txt-robots-txt-for-ai-crawlers
Date: 2025-01-17
Tags: seo, ai, llms.txt, robots.txt, ai-crawlers
Pillar: ai-building
Reading Time: 17 min
Word Count: 3398
TL;DR: llms.txt is a site-level policy file that tells AI engines how they can use and cite your content. It complements robots.txt by focusing on usage and attribution rather than crawl access.
Key Takeaways:
- llms.txt sets usage and attribution rules for AI systems.
- Publish it at the site root, alongside robots.txt.
- Link to sitemap and RSS so crawlers discover updates.
- Robots.txt controls access; llms.txt controls usage.
--- CONTENT ---
llms.txt is a site-level policy file that tells AI engines how they can use and cite your content. It complements robots.txt by focusing on usage and attribution rather than crawl access. If you want AI systems to cite you correctly, this is the simplest control point.
## What is llms.txt?
llms.txt is a plain-text policy file you publish at your site root that tells AI engines how they may use and cite your content. Where robots.txt controls whether a crawler can access your pages, llms.txt governs what the crawler can do with the content it finds: training, answer generation, attribution format, and which sections to exclude from indexing entirely.
`llms.txt` is a root-level policy file that tells AI engines like Perplexity, Claude, and ChatGPT how they may use and attribute your content. Unlike robots.txt, which controls crawl access, llms.txt governs usage policy and preferred citation format. Publishing it signals explicit consent to AI indexing and helps ensure correct attribution.
`llms.txt` is a new standard file that tells AI engines (Perplexity, Claude, ChatGPT) how to handle your content.
- **robots.txt** = "Can you crawl my site?" (access control)
- **llms.txt** = "How should you use my content?" (usage policy)
Both should exist on your site. This is part of the broader [AI search optimization strategy](/blog/how-to-optimize-for-perplexity-chatgpt-ai-search) that helps your content get discovered and cited.
---
## Why llms.txt Matters
### The Problem: Content Attribution
When OpenAI's ChatGPT answers a user's question, it synthesizes an answer from multiple sources. But where does it cite those sources?
**Without llms.txt:** ChatGPT has to guess your preferred attribution format.
- Maybe it cites the article title
- Maybe it cites your domain
- Maybe it doesn't cite you at all
**With llms.txt:** You explicitly say "Cite me like this: [Title] by [Author] (yoursite.com)"
AI engines follow your preference.
### The Bigger Picture
`llms.txt` emerged in 2024 as a response to AI scraping concerns. Instead of fighting crawlers, creators use `llms.txt` to:
1. **Invite crawlers**, "Please index my content"
2. **Set terms**, "But cite me this way"
3. **Exclude content**, "Don't train on my drafts"
4. **Provide discovery**, "Here's my sitemap and RSS"
It's like putting a "Welcome" sign on your site with conditions attached. This is a foundational piece of what I call [Answer Engine Optimization (AEO)](/blog/aeo-answer-engine-optimization-explained), the practice of making your content discoverable and citable by AI systems. Not sure where your site stands? Run the free [AEO audit tool](/tools/aeo-audit) to check your robots.txt and llms.txt against exactly what AI crawlers look for.
---
## How AI Engines Use llms.txt
When a crawler visits your site:
1. Fetch `/robots.txt` → Check if allowed to crawl
2. Fetch `/llms.txt` → Check usage policy
3. Fetch `/sitemap.xml` → Discover all pages
4. Extract content → Index and train
If `/llms.txt` doesn't exist, the crawler might:
- Crawl your site anyway (risky for them)
- Skip your site entirely (loss for you)
- Use conservative assumptions (minimal indexing)
Having `/llms.txt` shows you've **explicitly consented** to AI indexing.
The distinction between "crawl access" and "usage policy" matters more than it sounds. A crawler that can access your page via robots.txt still faces a question: can I train on this content? Can I quote it in an answer? Should I attribute it, and how? Without llms.txt, those questions have no answers. The crawler either guesses conservatively (you get less visibility) or guesses aggressively (you lose attribution control). Neither outcome is what you want.
---
## How to Create llms.txt
### Step 1: Location
Create a file at: `https://chudi.dev/llms.txt`
It must be at the root, not in `/content/` or `/blog/`. Just like `robots.txt` is at the root. If you're using a static site generator like Next.js, SvelteKit, or Hugo, place the file in your `public/` or `static/` directory so it gets served at the root path during deployment. For SvelteKit specifically, you can also create a server route at `src/routes/llms.txt/+server.ts` that returns the content dynamically, useful if you want to auto-generate sections like your sitemap URL or last-updated date from your build config.
### Step 2: Content
Here's a basic template:
```markdown
# LLM Content Policy for [Your Site]
All articles on this site are available for training and search indexing by large language models.
## How to attribute content
When citing articles from this site, please use the format:
[Article Title] — [Author Name] on [yoursite.com]
Example: "How to Optimize for Perplexity" — Chudi on chudi.dev
## Content discovery endpoints
- Sitemap: https://yoursite.com/sitemap.xml
- RSS feed: https://yoursite.com/rss.xml
- Blog archive: https://yoursite.com/blog
## Content not available for indexing
- Pages marked as draft or private
- Internal documentation
- User-generated content (comments)
- Archived content older than [5] years
## Preferred citation style
Inline: [Article](https://yoursite.com/article-url) by Author Name
Bibliography: Author Name. "Article Title." Your Site, YYYY.
## Questions or Concerns
Email: [your-email@yoursite.com]
Last updated: January 2025
```
### Step 3: Customize for Your Site
Replace:
- `[Your Site]` → your actual site name
- `[Author Name]` → your name
- Email → your contact email
- Dates → today's date
### Step 4: Include Metadata
Optionally, you can include a JSON section:
```json
{
"version": "1.0",
"license": "CC BY-SA 4.0",
"attribution_required": true,
"commercial_use": "allowed",
"modification": "allowed",
"sitemap": "https://yoursite.com/sitemap.xml",
"rss": "https://yoursite.com/rss.xml"
}
```
This helps AI engines parse your policy programmatically.
---
## Where llms.txt vs robots.txt
| Aspect | robots.txt | llms.txt |
|--------|-----------|----------|
| Purpose | Access control | Usage policy |
| Audience | Search crawlers | AI engines |
| Required | Yes (best practice) | No (but recommended) |
| Format | Plain text directives | Markdown + optional JSON |
| Location | `/robots.txt` | `/llms.txt` |
| Blocks access | Yes | No |
| Legally binding | No | No (advisory) |
**robots.txt** is like a gate at your property. **llms.txt** is like a sign on the gate saying "Welcome, but please do X."
The format difference is worth noting. robots.txt uses a strict directive syntax that machines parse rigidly, `User-agent`, `Disallow`, `Allow`. llms.txt uses markdown with optional embedded JSON, which gives you room to express nuance that directive syntax cannot capture. You can explain your attribution preferences in natural language, describe what types of content are available for different use cases, and provide context about your licensing terms. This flexibility is intentional, AI systems are better at parsing natural language than traditional crawlers, so the policy file takes advantage of that capability.
Another key difference: robots.txt is enforced by convention. Well-behaved crawlers respect it. llms.txt is purely advisory, there is no mechanism to force compliance. But the advisory nature is actually a strength for creators who want visibility. You are not blocking anything. You are inviting crawlers in and telling them how to treat your content responsibly. The incentive alignment works because AI engines want to cite correctly, it improves their answer quality, and your llms.txt makes that easy for them.
---
## Common llms.txt Policies
### Policy 1: Fully Open (Creator-Friendly)
```markdown
# LLM Content Policy
All content on this site is available for:
- Training large language models
- Extracting for answer engines
- Commercial and non-commercial use
Just cite us: [Title] — [Author] ([yoursite.com])
```
**Best for:** Indie creators who want maximum visibility
### Policy 2: Attribution Required (Balanced)
```markdown
# LLM Content Policy
Content available for training and use, with required attribution.
Required format: [Article Title] by [Author Name] (yoursite.com)
Prohibited use: Removing or hiding attribution
```
**Best for:** Most creators who want credit
### Policy 3: Non-Commercial Only (Restrictive)
```markdown
# LLM Content Policy
Content available for non-commercial use and training.
Prohibited use:
- Commercial products without permission
- Training proprietary LLMs
- Republishing without modification
```
**Best for:** Creators concerned about exploitation
### Policy 4: Permission Required (Most Restrictive)
```markdown
# LLM Content Policy
All uses require explicit permission. Email [your-email] to request.
```
**Best for:** Creators who want full control
---
## Real-World Examples
### Example 1: Tech Blog
```markdown
# LLM Content Policy
Technical articles on this site are available for:
- AI training (open-source and proprietary)
- Answer generation (Perplexity, ChatGPT, Claude)
- Academic and educational use
Citation format: [Title] by [Author] on [yoursite.com]
Prohibited:
- Removing examples or code without attribution
- Training models specifically to replicate this blog
Updated: January 2025
```
### Example 2: Content Creator
```markdown
# LLM Content Policy
All essays are available for training and synthesis.
Citation: [Essay Title] — [Your Name]
Prefer long-form citations, not snippets.
Excluded:
- Guest posts (ask the author)
- Archived essays older than 3 years
Contact: [email]
```
### Example 3: SaaS Documentation
```markdown
# LLM Content Policy
Documentation is available for indexing and use in AI tools.
Required attribution: Link to the original docs page + software name.
Prohibited:
- Repackaging docs as your own product
- Training models on raw HTML without attribution
Questions? hello@[company].com
```
---
## How to Test if llms.txt Works
### Method 1: Manual Check
```bash
# Verify it exists and is accessible
curl https://chudi.dev/llms.txt
# Should return 200 status code
curl -I https://chudi.dev/llms.txt
```
### Method 2: Check in Perplexity
Search your site name in Perplexity. Are you being cited?
Before llms.txt: Sporadic or no citations
After llms.txt: More consistent citations with proper attribution
### Method 3: Monitor Traffic
Track referral traffic from:
- `perplexity.com`
- `openai.com`
- `anthropic.com`
A week after publishing llms.txt, you should see an uptick.
---
## Does llms.txt Actually Matter?
Yes, llms.txt matters in practice even though it is advisory and not legally required. Major AI engines including Perplexity, Claude, and ChatGPT recognize the file and use it to determine attribution preferences. Sites with llms.txt configured tend to receive more consistent citations and better crawl coverage from AI systems within four to eight weeks.
**Short answer:** Yes, but not as much as robots.txt.
**Longer answer:**
- **Required by law:** No, it's advisory
- **Followed by all AI engines:** Not yet (but major ones do)
- **Necessary for indexing:** No, but it helps
- **Better than nothing:** Absolutely
Think of it like the difference between:
- A locked door (robots.txt: blocks crawling)
- A welcome mat with terms (llms.txt: invites crawling with rules)
You still need the robots.txt. But llms.txt gets you better attribution and signaling.
---
## The Future of llms.txt
Standards bodies like IETF and W3C are discussing llms.txt as a formal standard. As it becomes more official:
1. AI engines will prioritize crawling sites with llms.txt
2. LLMs will automatically cite in your preferred format
3. Licensing and commercial rights will be more enforceable
For now, it's early adoption. But early adopters get:
- Better attribution from AI engines
- Clearer signal to crawlers
- Documented content policy (good for SEO too)
The timing advantage is real. When ChatGPT or Perplexity improves their citation systems, and they will, because user trust depends on source attribution, the sites that already have clear llms.txt policies will be prioritized over sites where the AI engine has to guess. You are building citation infrastructure now that will compound as AI search grows. The same logic applies to [structured data and schema markup](/blog/aeo-answer-engine-optimization-explained), the earlier you adopt, the more crawl history and citation data accumulates in your favor.
---
## llms.txt and the Evolving AI Content Ecosystem
The emergence of llms.txt reflects a broader shift in how content creators relate to AI systems. For the past two decades, the relationship between websites and search engines was mediated by a single file: robots.txt. That file was designed for a world where crawlers fetched pages, indexed keywords, and ranked results. The crawler's job was discovery and ranking. The content stayed on your site, and users clicked through to read it.
AI engines fundamentally change this dynamic. When a user asks ChatGPT a question, the AI synthesizes an answer from multiple sources and presents it directly. The user may never visit your site at all. This means the old model of "let crawlers in, they send traffic back" no longer applies cleanly. AI engines consume your content and may deliver the value to the user without a click. llms.txt is the first attempt to create a new social contract for this relationship: you let AI engines use your content, and in return, they attribute it properly so that users know where the information came from and can choose to visit your site for more depth.
This is not a solved problem. The current llms.txt format is informal, advisory, and unevenly adopted. But the direction is clear. As AI search grows, and it is growing rapidly, with Perplexity alone processing millions of queries daily, creators who have clear, machine-readable content policies will be better positioned than those who remain silent. Silence is ambiguous, and ambiguity favors the platform, not the creator.
The practical takeaway is straightforward: spend five minutes creating an llms.txt file today, and you are ahead of ninety-five percent of websites. The standard will evolve, but having any explicit policy is better than having none. You can always update the file as best practices solidify. What you cannot do is retroactively claim attribution for the months when your content was being used without any policy in place.
## Checklist: Set Up llms.txt
- [ ] Create file at `/llms.txt`
- [ ] Include attribution format
- [ ] Link to sitemap.xml
- [ ] Link to RSS feed
- [ ] Specify excluded content
- [ ] Add contact email for questions
- [ ] Test with `curl https://chudi.dev/llms.txt`
- [ ] Announce on Twitter/LinkedIn
- [ ] Monitor Perplexity citations week 1-4
---
## Measuring llms.txt Impact on Your Content
Setting up llms.txt is one thing. Measuring whether it's actually working is another. The infrastructure is new enough that most analytics tools don't have native llms.txt tracking yet. But you can observe the impact indirectly.
### Tracking Citation Changes
Start tracking your current citation baseline BEFORE setting up llms.txt:
1. Search your site name in Perplexity, note how many results cite you and in what format
2. Search common queries you'd expect your content to answer (e.g., "ADHD and productivity"), capture screenshots of current attribution
3. Repeat the same searches weekly for 4 weeks after deploying llms.txt
What you'll likely see: citations move from sporadic to consistent, and the format increasingly matches your preferred citation style. One creator reported 3x increase in Perplexity citations within 6 weeks of adding llms.txt with explicit citation format instructions.
### Server Logs & Referral Traffic
Enable detailed logging for referrals from AI engines. Check your analytics dashboard for:
- Traffic from `perplexity.com` referrer (often appears as direct, but you can trace it)
- Traffic from `openai.com` (ChatGPT search and plugin references)
- Traffic from `anthropic.com` (Claude search, coming in 2026)
Most sites with llms.txt see a small but measurable uptick in "direct" traffic that you can attribute to AI search referrals. The traffic isn't massive (not like Google), but it's concentrated and high-intent, these are people asking AI systems questions and following the generated citations back to your site.
### SEO Secondary Effects
llms.txt itself doesn't affect Google ranking. But the behavior it enables does:
- **More inbound links from AI answers** → Higher domain authority (slow effect, 3-6 months)
- **Lower bounce rate from AI referrals** → Better engagement signal
- **Branded search visibility** → People search your name after seeing it cited, improving brand recall
The SEO value isn't direct. It's that llms.txt positions your content better upstream, which flows downstream to traditional SEO.
---
## Common Implementation Gotchas
### Gotcha 1: Forgetting the Sitemap Link
Many creators add llms.txt but forget to include the sitemap URL. AI crawlers find pages two ways:
1. Following links from your homepage
2. Checking the sitemap (if you link to it)
Without the sitemap link, crawlers discover only top-level pages and pages linked from your nav. Blog archives, portfolio details, and deeply nested pages might be skipped.
**Fix:** Always include both `sitemap.xml` and RSS feed URLs in your llms.txt.
### Gotcha 2: Making Attribution Too Restrictive
Some creators specify citation format so narrowly that crawlers treat it as legally risky. Example: "Must cite as [Title] by [Author] (site.com) or don't use at all."
This is advisory, not legally binding. But crawlers are conservative. If the policy feels overly restrictive, they might skip your site rather than risk a citation violation.
**Better approach:** Specify preferred format but allow flexibility. Example: "Preferred: [Title] by [Author] on yoursite.com. Acceptable: Article title with link."
### Gotcha 3: Conflicting robots.txt and llms.txt
If `robots.txt` blocks all crawlers with `User-agent: *` and `Disallow: /`, then llms.txt becomes useless, you've already said "no crawling allowed."
llms.txt is a refinement on top of robots.txt, not a replacement. The order is:
1. Check robots.txt → if blocked, stop
2. Check llms.txt → if explicit policy, follow it
3. Use default assumptions → treat site as opt-out
**Fix:** Make sure robots.txt permits crawlers to access the paths you want indexed.
### Gotcha 4: Not Versioning or Dating the Policy
llms.txt is new. Standards might change. Crawlers might update. If you never update your llms.txt, you'll fall behind.
At minimum:
- Add a "Last updated" date (forces you to think about freshness)
- Include a version number
- Monitor crawler behavior quarterly
---
## Real-World Adoption: Who's Using llms.txt?
Early adopters include:
- **Tech blogs and documentation sites**, Want deep indexing by Claude and ChatGPT for technical Q&A
- **News publishers and media sites**, Want proper attribution in AI-generated news summaries
- **Educational platforms**, llms.txt gives them granular control over how content is used
- **Creator platforms** (Substack, Medium), Want to compete with Google for AI-driven discovery
Most consumer blogs? They're still ignoring it. Most of the internet doesn't have llms.txt yet. This means adopting early gives you a small competitive advantage, you're signaling to crawlers that you welcome them, while competitors are silent.
---
## How I Implemented llms.txt on My SvelteKit Blog
When I added llms.txt to chudi.dev, I chose the dynamic route approach over a static file. The reason was simple: I wanted the sitemap URL, post count, and last-updated date to stay accurate without manual updates. A static file in the public directory would go stale the moment I published a new post and forgot to update it.
The implementation took about twenty minutes. I created a server route that reads the blog post metadata at build time, counts the total published posts, finds the most recent publication date, and renders the llms.txt content with those values interpolated. The route returns plain text with the correct content type header so crawlers parse it as a text file rather than HTML.
The content itself follows the balanced attribution model from the policies section above. I explicitly welcome AI training and answer generation, specify my preferred citation format, link to both the sitemap and RSS feed, and exclude draft posts and archived content older than two years. I also include a machine-readable JSON block with licensing information because some crawlers parse structured metadata more reliably than natural language instructions.
One decision I made early was to include a section listing my content pillars and the types of questions each pillar answers. This gives AI engines a semantic map of my site that goes beyond what a sitemap provides. A sitemap tells a crawler which URLs exist. The pillar section tells it what topics those URLs cover and what kinds of user queries they can answer. This distinction matters because AI engines are not just indexing pages, they are building knowledge graphs, and explicit topic mapping helps them place your content in the right nodes.
After deploying, I tested the implementation by fetching the URL with curl and verifying the content rendered correctly. I also searched my site name in Perplexity and noted the baseline citation format before the llms.txt could take effect. Four weeks later, I repeated the search and found that citations had shifted from inconsistent domain-only references to the preferred format I specified in the file. The sample size was small, about twelve citations across different queries, but the pattern was clear.
The maintenance burden is effectively zero because the route generates dynamically. When I publish a new post, the post count and last-updated date update automatically on the next build. The only manual update I foresee is if the llms.txt standard evolves to include new fields or if I change my licensing terms.
## Why Most Sites Get llms.txt Wrong
The most common mistake I see when reviewing other sites' llms.txt implementations is treating it as a legal document rather than a communication tool. Long paragraphs of legalese about intellectual property rights, DMCA provisions, and liability disclaimers miss the point entirely. AI crawlers are not lawyers. They are parsers looking for structured signals about how to handle your content.
The second most common mistake is being too vague. Saying "please attribute properly" without specifying a format gives the AI engine nothing actionable to work with. You need to spell out exactly what a correct citation looks like: the article title, your name, your domain, and ideally a link back to the original URL. The more specific you are, the more likely the AI engine will match your preference.
A third mistake is forgetting to update the file after initial creation. I have seen sites where the llms.txt references a sitemap URL that no longer exists, or lists content categories that were reorganized months ago. Stale metadata is worse than no metadata because it actively misleads crawlers. If you include a last-updated date (and you should), treat it as a commitment to actually review the file when that date approaches.
The fourth and most subtle mistake is not aligning llms.txt with your actual content strategy. If your business model depends on gated content behind a paywall, but your llms.txt says "all content available for training," you are giving away the content you charge for. Conversely, if you are a creator who wants maximum visibility, an overly restrictive llms.txt that requires permission for every use case will reduce your AI search presence. The policy should reflect your actual goals, not a generic template you copied from a tutorial.
## What's Next?
After setting up robots.txt and llms.txt, focus on structured data using schema.org markup, content structure with semantic headers and lists, and content freshness by updating articles regularly. These three areas improve how AI engines parse and extract your content, which directly affects how often you are cited in AI-generated answers.
Once you have robots.txt and llms.txt set up, focus on:
1. **Structured data** (schema.org), Helps AI parse your content
2. **Content structure** (headers, lists), Makes extraction easier
3. **Freshness** (update articles), Recent content ranks higher
4. **Specificity** (answer common questions directly), Better for AI synthesis
The combination of these creates what we call **AEO (Answer Engine Optimization).** For a complete walkthrough of these techniques, see my [full AEO optimization guide](/blog/how-to-optimize-for-perplexity-chatgpt-ai-search).
**Start here:** Add llms.txt to your site today. It takes 5 minutes and can improve your visibility in AI search engines.
Then, run a full AI-citability audit at [citability.dev](https://citability.dev/?utm_source=chudidev&utm_medium=referral&utm_campaign=llms-txt-robots-txt-for-ai-crawlers) to measure exactly which AI engines are citing your URLs and where the gaps are.
Sites with both `robots.txt` and `llms.txt` properly configured consistently see better AI crawl coverage and citation rates within 4-8 weeks of implementation, the infrastructure compounds over time as AI systems return to re-index updated content.
--- END POST ---
================================================================================
POST: Get Cited by ChatGPT and Perplexity: 5 Concrete Steps
================================================================================
URL: https://chudi.dev/blog/how-to-optimize-for-perplexity-chatgpt-ai-search
Date: 2025-01-16
Tags: seo, ai, perplexity, chatgpt, content-optimization
Pillar: ai-building
Reading Time: 11 min
Word Count: 2040
TL;DR: To optimize for Perplexity, ChatGPT, and Claude, you need crawl access, explicit AI crawler permissions, and answer-first formatting with schema. This guide lays out the exact steps: robots.txt, llms.txt, structured data, and content structure.
Key Takeaways:
- Allow AI crawlers in robots.txt and publish llms.txt for attribution.
- Use sitemap and RSS to help answer engines discover updates.
- Add BlogPosting and FAQ schema to priority posts first.
- Lead with answers and structured headings to improve extraction.
--- CONTENT ---
If ChatGPT and Perplexity cannot crawl or parse your site, they cite a competitor instead, and that buyer never sees you. Fixing it takes five concrete changes: crawler access in robots.txt, an llms.txt file, schema markup, and answer-first content structure. This guide walks through each step and how to verify it actually worked.
## How Do You Optimize a Website for ChatGPT and Perplexity?
To optimize your website for ChatGPT and Perplexity, do five things in order: allow their crawlers (GPTBot, ClaudeBot, PerplexityBot) in `robots.txt`, publish an `llms.txt` attribution file, add BlogPosting and FAQPage schema, structure every section answer-first under question-based headings, and keep your highest-value pages fresh. Everything below is the exact implementation of those five steps.
## Quick Wins (Do These Today)
1. Update `robots.txt` to allow AI crawlers
2. Create `/llms.txt` at site root
3. Add BlogPosting schema to 10 top articles
4. Structure one article with 15+ headers instead of 3
That's a 30-minute investment that unlocks visibility in Perplexity, ChatGPT, and Claude.
## What Software Checks Whether ChatGPT and Perplexity Cite Your Posts?
[citability.dev](https://citability.dev/?utm_source=chudidev&utm_medium=referral&utm_campaign=how-to-optimize-for-perplexity-chatgpt-ai-search) is the tool I built to answer that: it asks ChatGPT and Perplexity real questions in your topic and records which URLs from your blog they actually cite, keeping the raw AI responses as receipts, so a citation gained or lost is traceable to what the engine said. The free [30-second scan](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=how-to-optimize-for-perplexity-chatgpt-ai-search) checks the infrastructure half of the five steps below before you spend time on the content half.
---
## Why Do AI Engines Need Different Optimization Than Google?
Google ranks pages based on backlinks and authority signals. AI engines, Perplexity, ChatGPT, Claude, retrieve content by semantic relevance and structure. They need explicit crawler permission, clean schema markup, and answer-first formatting. Without these signals, your content is invisible to AI search regardless of how well it ranks on Google.
## Step 1: Update robots.txt for AI Crawlers
Your `robots.txt` is the bouncers list for search crawlers. Most sites have something like:
```
User-agent: *
Disallow: /admin
Disallow: /api
Sitemap: https://yoursite.com/sitemap.xml
```
This is Google-focused. AI engines need explicit permission. Update it:
```
# Allow all standard crawlers
User-agent: *
Disallow: /admin
Disallow: /api
Disallow: /private
# Explicitly allow AI crawlers
User-agent: GPTBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Googlebot-Extended
Allow: /
User-agent: CCBot
Allow: /
Sitemap: https://yoursite.com/sitemap.xml
```
**Why each one matters:**
| Bot | Owner | Used By |
|-----|-------|---------|
| GPTBot | OpenAI | ChatGPT, GPT-5 |
| ClaudeBot | Anthropic | Claude.ai, API users |
| PerplexityBot | Perplexity | Perplexity search |
| Googlebot-Extended | Google | Google's AI Overview, SGE |
| CCBot | CommonCrawl | Hugging Face, open-source models |
**Test it:** Use this to verify:
```bash
curl -I https://yoursite.com/robots.txt
```
---
## Step 2: Create `/llms.txt` (New File)
While robots.txt controls *access*, `/llms.txt` controls *attribution*. Create a new file at the root. For a comprehensive deep-dive on this file, see my guide on [llms.txt and robots.txt for AI crawlers](/blog/llms-txt-robots-txt-for-ai-crawlers):
`https://chudi.dev/llms.txt`
```markdown
# LLM Content Policy
All articles on this site are available for training, search indexing, and answer generation by LLMs.
## How to attribute our content:
For articles: [Article Title] by [Author Name] (sitename.com)
For data: Link to the specific section
For code: Preserve license headers
## How we'd like to be credited:
If citing multiple articles, link to: https://yoursite.com
## Content discovery:
- Sitemap: https://yoursite.com/sitemap.xml
- RSS: https://yoursite.com/rss.xml
- Blog archive: https://yoursite.com/blog
## Content we don't want indexed:
- Drafts (marked `draft: true`)
- Private tools or dashboards
- Archived content older than 5 years
## Preferred citation format:
[Article Title] — Author Name on yoursite.com
---
Last updated: January 2025
```
**Why it matters:** AI engines scan for `llms.txt` to understand your content policy. Without it, some engines might skip you (too risky). With it, you're explicitly inviting them in.
---
## What Schema Markup Should You Add for AI Search?
Start with BlogPosting schema on every article, it tells AI engines the content type, author, date, and topic. Add FAQPage schema wherever you have Q&A sections, and HowTo schema on step-by-step guides. These three schema types cover most blog content and have the highest citation impact in AI search.
## Step 3: Add Schema.org Structured Data
AI engines parse JSON-LD to understand content structure. This structured approach to content is what enables the extraction and synthesis that powers answer engines. Add this to your blog post template:
### For Articles (BlogPosting)
Add this in your page's ``:
```html
```
### For How-To Content (HowToSchema)
If you're teaching a process:
```html
```
### For FAQ Content (FAQPage)
```html
```
**Verify schema:** Use [Google's Rich Results Tester](https://search.google.com/test/rich-results) to validate.
---
## Step 4: Structure Content for Extraction
AI engines need to *find* the answer within your content. This means:
### Use Headers to Break Up Content
**Bad structure:**
```
# Article Title
Long paragraph explaining the concept...
More context...
Finally, the key insight...
```
**Good structure:**
```
# Article Title
## What is AEO?
AEO (Answer Engine Optimization) is the practice of structuring content so AI engines can extract, cite, and surface it in generated answers. It focuses on crawl access, structured data, and answer-first formatting.
## Why does AEO matter for visibility?
AI engines like Perplexity and ChatGPT now answer queries directly without sending users to search results. If your content is not optimized for extraction, you are invisible to this growing traffic source.
## How to implement AEO
Steps...
## Common mistakes
What to avoid...
## FAQ
- Q1: Answer
- Q2: Answer
```
Every 2-3 paragraphs, add a header. This makes it easier for AI to:
1. Find the specific section answering a user's question
2. Extract just that section (not the whole article)
3. Cite the correct part of your content
### Use Lists for Dense Information
Instead of:
> "To optimize your site, you need to update your robots.txt file, create an llms.txt file, add schema to your articles, and structure your content with headers."
Write:
> "To optimize your site:
> 1. Update robots.txt for AI crawlers
> 2. Create llms.txt at site root
> 3. Add schema to articles
> 4. Structure with semantic headers"
AI engines can extract lists more reliably than paragraph prose.
### Put the Answer at the Top
Don't make readers scroll for the punchline. If your headline is "Why AEO Matters," answer it in the first paragraph:
> "AEO matters because 30% of searches now go through AI engines instead of Google. If your content isn't optimized for Perplexity, ChatGPT, and Claude, you're invisible to an entire audience."
Then expand with context, examples, and proof.
### Use Definition Boxes
For key concepts, use a highlighted box:
```markdown
> **Definition:** AEO (Answer Engine Optimization) is optimizing your content to be found, extracted, and cited by AI search engines.
```
This signals to AI engines: "This is important context."
### Include Tables and Structured Data
Tabular data is easier for AI to extract:
| Factor | SEO | AEO |
|--------|-----|-----|
| Ranking | Backlinks | Content structure |
| Speed | Important | Less important |
Don't just describe in paragraphs. Use tables.
---
## How Do You Know If AI Engines Are Citing Your Content?
Search your target keywords directly in Perplexity, ChatGPT search, and Claude. If your site doesn't appear in sources within 4-6 weeks of publishing, you likely have a crawl access problem, a freshness issue, or your content isn't structured as direct answers. Manual citation checks take five minutes weekly.
## Step 5: Monitor Visibility in AI Engines
### Search Your Topics in Perplexity
Go to [perplexity.ai](https://perplexity.ai) and search your main keyword. Do you see your content cited?
If yes ✅ → Your content is discoverable
If no ❌ → You need to audit (usually a robots.txt or freshness issue)
### Check ChatGPT Search
OpenAI's ChatGPT now searches the web. Search your site name + keyword. Does your content appear?
### Use Perplexity Citation Tracking
When your content is cited, you'll see traffic from `perplexity.com` in your analytics. Track this growth.
---
## 2026 Update: Platform-Specific Citation Strategies
Since this article was first published, research has revealed that each AI platform has distinct source preferences. A one-size-fits-all approach to AI search optimization is no longer sufficient.
### Perplexity: The Reddit Connection
Perplexity cites approximately 6.6 sources per answer, more than any other platform. Its source pool leans heavily on Reddit, with 46.7% of top-cited sources coming from Reddit discussions. This means genuine participation in subreddits relevant to your expertise (r/SEO, r/webdev, r/artificial) directly increases your chances of being cited.
The mechanism: Perplexity indexes Reddit threads and cites them as sources. When your expertise appears in those threads (with or without links to your site), Perplexity associates your knowledge with your brand.
### ChatGPT: Recency and Authority
ChatGPT cites only about 2.6 sources per answer, making it the most selective platform. It shows strong preference for Wikipedia (7.8% of all citations) and recently updated content. 95% of ChatGPT citations come from content published or updated within the last 10 months.
To improve ChatGPT citations: update your highest-value articles quarterly with new data, add `dateModified` schema, and ensure your content contains specific statistics AI can attribute.
### Google AI Overviews: Still Tied to Rankings
Google AI Overviews show 76% overlap with traditional top 10 results, making it the most SEO-correlated AI feature. If you rank well on Google, you're likely to appear in AI Overviews. Focus traditional SEO efforts here.
### Measuring Your AI Visibility
You can now audit your site's AI infrastructure readiness with automated tools. [citability.dev](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=how-to-optimize-for-perplexity-chatgpt-ai-search) runs 10 checks against your site covering robots.txt, sitemap, structured data, answer-first content, and freshness signals. It measures three tiers: AI visibility (can AI find you?), AI recommendability (does AI suggest you?), and AI citability (does AI link to you?).
To measure your citation rate directly, query ChatGPT, Perplexity, and Claude with 20 questions your site should answer. Track two metrics per query: whether the AI mentions your brand (visibility) and whether it links to a URL on your domain (citation). In our benchmark of 7 sites, the gap between these two numbers ranged from 25 to 95 points. Ahrefs (DR 92) was 100% visible but only 5% cited. A site with DR under 10 achieved 15% citation rate by focusing on original data and answer-first structure. Authority did not predict citations. Structure did. The full step-by-step methodology, including the 20-query template and per-engine scoring sheet, is published on freeCodeCamp: [How to Measure Your AI Citation Rate Across ChatGPT, Perplexity, and Claude](https://www.freecodecamp.org/news/how-to-measure-your-ai-citation-rate-across-chatgpt-perplexity-and-claude/). The companion case study is [I Audited 7 Websites for AI Citability](/blog/ai-citability-audit-what-predicts-citations).
---
## Complete Optimization Checklist
- [ ] robots.txt allows GPTBot, ClaudeBot, PerplexityBot
- [ ] `/llms.txt` exists at site root
- [ ] Top 10 articles have BlogPosting schema
- [ ] How-to content has HowToSchema
- [ ] FAQ content has FAQPageSchema
- [ ] Content has 10+ headers (not 3)
- [ ] Key answers in first paragraph
- [ ] Important data in tables, not paragraphs
- [ ] Meta descriptions under 160 chars (Google habit)
- [ ] Images have descriptive alt text
- [ ] No critical content in images only
- [ ] Content updated in last 12 months (fresh signal)
- [ ] Run infrastructure scan at [citability.dev](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=how-to-optimize-for-perplexity-chatgpt-ai-search) to identify gaps
- [ ] Measure AI citation rate with 20 manual queries across 3 platforms
- [ ] Add original data (benchmarks, tables, unique statistics) to top articles
---
## Why Isn't My Content Showing Up in AI Search Results?
The four most common causes: AI bots blocked in robots.txt, conflicting canonical URLs from cross-posting, duplicate content diluting citation chances, and schema errors preventing structured data from registering. Run through each diagnostic in order, most optimization failures trace back to one of these four fixable issues.
## When Your Optimization Isn't Working
If you've done the five steps and still don't see citations in Perplexity or ChatGPT after 4-6 weeks, run this diagnostic.
**Check if AI bots are reaching your pages.** Look in your server logs or analytics for user-agent strings containing GPTBot, ClaudeBot, or PerplexityBot. If you see no traffic from these bots, there's a crawl access issue, either robots.txt is still blocking them, or your hosting provider's firewall doesn't recognize these newer bot names.
**Verify your canonical is consistent.** AI systems avoid citing content with conflicting canonical signals. If you're [cross-posting to Dev.to](/blog/devto-cross-posting-automation), Medium, or LinkedIn, make sure each cross-post has your original URL in the canonical field, not the platform's default. One conflicting canonical can suppress citations across all AI engines.
**Check for duplicate content.** AI engines, like Google, downweight duplicate content. If you have two posts targeting the same query, consolidate them. Two 800-word posts on the same topic compete with each other and dilute citation chances compared to one 1,600-word authoritative post.
**Test your schema.** Run your top 3 posts through Google's Rich Results Test. If schema errors appear, fix them, AI engines use the same structured data signals to understand content type.
Most optimization failures trace back to one of these four issues. The good news: all four are diagnosable and fixable in an afternoon.
## The Advantage
Here's the thing: **most creators still treat AEO as optional.**
You just did these 5 steps. Your competitors haven't. This gives you a 6-12 month window where your content will be cited more often in AI answers, driving visibility and traffic.
This is the opposite of SEO, where the first-mover advantage is gone. AEO is still day one. Want a quick score first? Run the free [AEO audit tool](/tools/aeo-audit) to check crawler access, schema, and structure before you dig into the full strategy in my [detailed AEO guide](/blog/aeo-answer-engine-optimization-explained).
Five changes is the DIY path and it works.
**Next:** Run a free AI visibility scan at [citability.dev](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=how-to-optimize-for-perplexity-chatgpt-ai-search) to check your infrastructure readiness, or dive deeper with the [full AEO guide](/blog/aeo-answer-engine-optimization-explained).
When that scan comes back and you cannot tell which two of the five actually moved anything, [that is the read you are buying](/services).
**Related on citability.dev:** if your citation rate is stuck at zero despite good rankings, the diagnosis path is [The 0% ChatGPT Citation Trap](https://citability.dev/blog/the-0-percent-chatgpt-citation-trap?utm_source=chudidev&utm_medium=referral&utm_campaign=how-to-optimize-for-perplexity-chatgpt-ai-search); the full signal framework behind every check above is [What Is AI Citability? The Complete Framework](https://citability.dev/blog/what-is-ai-citability?utm_source=chudidev&utm_medium=referral&utm_campaign=how-to-optimize-for-perplexity-chatgpt-ai-search).
--- END POST ---
================================================================================
POST: GTD for ADHD Doesn't Work: The Energy-Based System I Built Instead
================================================================================
URL: https://chudi.dev/blog/adhd-engineer-productivity-system
Date: 2025-01-15T00:00:00.000Z
Tags: adhd, productivity, notion, ai-tools
Pillar: neurodivergent
Reading Time: 12 min
Word Count: 2310
TL;DR: Traditional productivity systems assume neurotypical brains. After years of failed experiments, I built Notion templates that work WITH ADHD: single-surface design, energy-aware scheduling, context capture, and AI as an external processor.
Key Takeaways:
- ADHD doesn't run on importance and deadlines, it runs on interest, urgency, novelty, and challenge
- Single-surface systems work because if you can't see it, it doesn't exist
- Energy-aware scheduling matches tasks to your brain's current capacity
- Context capture before switching saves the 23-minute refocus cost
- AI can serve as an external prefrontal cortex for task organization
--- CONTENT ---
I've tried every productivity system.
GTD. Bullet journaling. Pomodoro. Time blocking. The "second brain" thing. Each one worked for about two weeks before joining the graveyard of abandoned systems.
The pattern was always the same: find new system, get excited, customize obsessively, use it perfectly for 10 days, miss one day, feel guilty, avoid it, never open it again.
Sound familiar?
An ADHD-compatible productivity system works with your executive function rather than assuming neurotypical energy, attention, and motivation. That means energy-aware scheduling, external structure for task initiation, and evidence-based completion rather than willpower-based follow-through. Here's the system that actually sticks.
## Why Do Traditional Productivity Systems Fail for ADHD?
Traditional productivity systems assume consistent energy, linear task progression, and motivation driven by importance and deadlines. ADHD brains run on interest, urgency, novelty, and challenge instead. When the motivation system doesn't match the system's assumptions, the result isn't failure through laziness. It's a structural mismatch that discipline alone cannot bridge.
Here's what took me years to understand: these systems aren't built for ADHD brains.
They assume you have consistent energy levels. You don't.
They assume you can maintain complex routines. You can't, not without enormous friction that eventually wins.
They assume linear task progression. But your brain wants to hyperfocus on thing C while thing A sits there judging you.
Traditional productivity is built for neurotypical consistency. ADHD runs on interest, urgency, novelty, and challenge. Not importance and deadlines.
When you try to force a system designed for one type of brain onto another, you don't get productivity. You get guilt. You get shame spirals. You get the creeping suspicion that you're fundamentally broken.
You're not broken. Your productivity system is.
## What Is the Two-Week Death Spiral?
The two-week death spiral is the predictable collapse of any new productivity system after novelty wears off. Setup phase generates dopamine. Active use follows. One missed day creates guilt. Guilt creates avoidance. The app becomes a reminder of failure rather than a tool for work. Systems that punish inconsistency don't survive contact with ADHD. They need to expect gaps.
Every ADHD person knows this cycle intimately. You discover a new system, maybe it's Notion, maybe it's a fancy app, maybe it's the latest productivity guru's framework. The novelty hits your dopamine receptors like a shot of espresso.
You spend hours setting it up. Days, even. The customization is part of the appeal. You're not procrastinating, you're *preparing*. This time will be different.
And for a while, it is. The system is new. It's interesting. Your brain rewards you for engaging with it.
Then life happens. You miss a day. The system doesn't account for missed days. It assumes you'll be back tomorrow, picking up where you left off. But you won't. The guilt has already started building.
By day three of avoidance, opening the app feels like confronting a disappointed parent. By week two, you've mentally filed it under "things that don't work for me" and started looking for the next solution.
The problem isn't your discipline. The problem is that these systems punish inconsistency instead of expecting it.
## What Actually Works
After enough failed experiments, I started noticing what *did* work:
### Single-Surface Systems
If I can't see everything in one place, it doesn't exist. Nested folders and pages are where my tasks go to die.
The best day I ever had with a task manager was when I kept everything on a single Notion page. No navigation. No "where did I put that?" No clicking through three levels of hierarchy to find my actual work.
One screen. Everything visible. That's it.
### Energy-Aware Scheduling
My 10am brain can do deep coding. Solving complex architectural problems. Writing intricate logic.
My 3pm brain can answer emails. Maybe do some admin work. Definitely not debug a race condition.
Traditional systems say "do the important things first." But importance doesn't correlate with cognitive demand. Sometimes the important thing is a draining two-hour meeting. Sometimes it's a quick email.
What works: matching task difficulty to current energy level. A simple dropdown, high energy, medium, or low, that filters what I should be working on right now.
### Context Capture Before Switching
The [research says it takes 23 minutes to fully refocus after a context switch](https://www.apa.org/topics/research/multitasking). That number haunted me. With ADHD, context switches happen constantly: sometimes because I'm distracted, sometimes because a new thought pulled me away, sometimes because someone needed something.
But I discovered something: the 23 minutes isn't mandatory. The reason it takes so long is because you're rebuilding context from scratch. Your brain has to remember what you were doing, where you were in the task, what your next step was, what files you had open.
If you write that down before switching, you can recover in 2-3 minutes instead. Just a quick note: "Working on the authentication flow. Just finished the login form. Next: add password validation. Open files: auth.ts, login.svelte."
When you come back, whether it's 10 minutes later or three days later, you have a breadcrumb trail back to exactly where you were.
### AI as an External Processor
This one changed everything for me.
My brain generates ideas faster than it can organize them. Thoughts arrive like popcorn: random, unpredictable, piling up. Traditional advice says to capture them in an inbox. But the inbox becomes another pile to process, another guilt source.
Claude and ChatGPT became my external prefrontal cortex.
When I'm overwhelmed by a project, I dump everything at an AI and ask it to organize it into steps. When I can't figure out what's important, I describe the situation and ask for priority sorting. When I'm stuck, I ask "what am I probably forgetting?"
The AI doesn't judge. It doesn't get tired. It doesn't sigh when I come back to the same problem for the fifth time. It just processes.
I've written extensively about [how I build with Claude Code](/blog/how-i-build-with-claude-code), [ADHD-specific Claude Code workflows](/blog/claude-code-adhd-workflows), and [preventing context loss](/blog/claude-context-management-dev-docs). These systems emerged directly from working with my ADHD brain, not against it.
### Forgiveness Built In
Systems that punish inconsistency don't survive. I needed templates that expect gaps and make returning easy.
No streak counters. No "you missed 3 days" notifications. No graphs showing my declining engagement. Just a clean slate every time I open it, ready for whatever I can give it today.
The best productivity tool for ADHD is one that never makes you feel bad for being human.
## The Templates I Built
I packaged what actually works into Notion templates, starting with the core system:
### Daily Driver Dashboard (Available Now)
Everything on one screen. Today's tasks, energy level selector, brain dump capture, and current focus indicator. No navigation required.
The energy level dropdown is the magic piece. Set it when you sit down, and the dashboard shows you only tasks that match your current capacity. High energy: deep work items. Low energy: quick wins and admin tasks.
This is the foundation. If you only use one template, this is the one.
### Coming Soon (Free for Early Buyers)
When you buy now, you'll automatically receive these additional templates as I complete them:
**Project Hyperfocus Tracker** - Captures your context when you go deep on something. When you come back after an interruption (or three days later), you can pick up in minutes instead of rebuilding from scratch.
**10-Minute Weekly Review** - Three questions, ten minutes, done. Designed for inconsistency, because most weekly reviews get abandoned after you skip one.
**AI Prompt Library** - 40+ prompts I actually use with Claude and ChatGPT for ADHD-specific challenges. Breaking down tasks, unsticking paralysis, reality-checking timelines, and more.
I'll email you when each one is ready. No extra charge. Early buyers get everything.
## Who This Is For
You might find these useful if:
- You've abandoned more productivity systems than you can count
- Your Notion is either completely empty or an overwhelming maze
- You know you're capable but can't seem to consistently execute
- Traditional advice like "just break it into smaller tasks" makes you want to scream
- You've started suspecting the problem isn't laziness
You probably don't need these if:
- GTD or time-blocking already works great for you
- You prefer building systems from scratch
- You don't use Notion
- You're looking for a magic solution that requires zero effort
I'm not selling discipline. I'm selling friction reduction.
## Get the Templates
I'm selling these as a bundle for $19.
**[Get the ADHD Engineer's Productivity System](https://chudi.dev/products)**
One purchase, lifetime access, all future updates included.
I'm genuinely proud of these. They're the system I wish existed five years ago, and the one I actually use every day now.
The templates won't fix your ADHD. Nothing will "fix" ADHD, because it's not broken. It's different. But they can reduce the friction between your brain and your work. They can stop punishing you for being inconsistent. They can meet you where you are instead of where productivity culture thinks you should be.
That's worth something.
## What Single Metric Changed Everything?
Tracking context switches per day, not tasks completed or hours worked, revealed where productivity was actually being lost. Each unrecorded switch costs 23 minutes of rebuild time. Adding 30-second context notes before every switch dropped recovery time to 2-3 minutes and reclaimed roughly four productive hours daily, without requiring more focus or fewer interruptions.
I track one number now: context switches per day.
Not tasks completed. Not hours worked. Not streaks. Context switches--the number of times I interrupted one thing to start another without capturing where I was.
Before I built the context capture system, I averaged 14-18 context switches per day. Each one cost me the 23-minute rebuild. Do the math: on bad days I was spending 6+ hours just recovering from interruptions.
After making context capture automatic--30 seconds before every switch, write down where you are and what comes next--my switches didn't decrease much. I still get distracted. Still jump between things. But the recovery time dropped from 23 minutes to 2-3.
That one change added roughly 4 productive hours to every workday. Not by forcing me to focus more. By making interruptions cheaper.
I hated journaling. But I love 30-second context notes.
If you take nothing else from this system: before you switch tasks, write one sentence about where you are. That's it. That's the minimum viable version of everything in this post.
## Failure Modes of ADHD Productivity Systems
ADHD productivity systems fail in predictable ways: setup becomes procrastination, review guilt accumulates until the system is abandoned, energy-level filters drift to static lists, and added features compound into cognitive load that collapses the system. Knowing these failure modes in advance doesn't grant immunity, but it makes them recognizable while they're happening, which is early enough to correct.
After building these systems and watching others try to adopt them, I've noticed the same failure patterns repeat. Knowing them in advance doesn't make you immune, but it does make them recognizable when they're happening.
**Failure Mode 1: System Setup as Procrastination**
The most common trap. You spend four hours building the "perfect" Notion template instead of doing the actual work. The setup feels productive. It looks like progress. It registers as engagement in your dopamine system.
It's avoidance.
The tell: if you're spending more time designing the system than using it, the system has become the object of novelty, not the work it's supposed to support. The two-week death spiral often starts with an over-engineered setup phase.
Fix: impose a time budget. One hour for initial setup, then you must use it with actual work. Imperfect and in use beats perfect and abandoned.
**Failure Mode 2: The Review Guilt Loop**
Weekly reviews are supposed to be a reflection tool. For ADHD systems, they become another thing to fail at.
You miss one week. The review feels pointless because it's not current. You avoid it. Another week passes. Now two weeks of guilt have accumulated around a task that was supposed to take ten minutes.
Fix: make the minimum viable review three questions and ten minutes, maximum. Not "did I hit all my goals" - just "what are the three things on my plate right now?" That's a review you can do from your phone in a waiting room. If that's all you do, you haven't failed.
**Failure Mode 3: Energy-Aware Scheduling Drift**
You set up the energy level filter on day one. You use it for a week. Then you stop updating it and just use the system as a flat task list, which is exactly what you had before.
Energy-aware scheduling only works if the energy level is accurate. If you set it to "high" in the morning and forget to update it, the filter surfaces wrong tasks all afternoon.
Fix: tie the energy level update to a specific transition, not a reminder. Mine is: every time I make coffee or a meal, I update the energy level. Habits attached to existing physical transitions survive longer than habits attached to notifications.
**Failure Mode 4: The Sophistication Ceiling**
Every feature you add to the system is another thing to maintain. At some point, the system becomes so sophisticated that keeping it current is itself a cognitive load.
The sign you've hit the ceiling: you open the system and feel overwhelmed by all the things in it. At that point, the system is working against you.
Fix: delete aggressively. If a feature hasn't been used in two weeks, remove it. The system should shrink, not grow, as you find what actually helps. The minimum viable version is almost always the sustainable version.
---
## ADHD and Architecture
The cognitive traits behind this productivity system translate directly to technical skills. See [My ADHD Brain Thinks in Distributed Systems](/blog/adhd-systems-architecture-engineering) for how pattern recognition, parallel processing, and chaos tolerance make ADHD brains natural architects.
---
*Reply in the comments or [email me directly](mailto:hello@chudi.dev). I read and respond to everything.*
--- END POST ---
================================================================================
POST: How ChatGPT and Perplexity Decide Which Sources to Cite
================================================================================
URL: https://chudi.dev/blog/aeo-answer-engine-optimization-explained
Date: 2025-01-15
Tags: seo, ai, content-optimization, search-engines
Pillar: ai-building
Reading Time: 24 min
Word Count: 4643
TL;DR: AEO (Answer Engine Optimization) is the practice of structuring content so AI answer engines can extract, trust, and cite it. It emphasizes crawl access, clear definitions, and machine-readable structure over classic link-based ranking.
Key Takeaways:
- AEO prioritizes extractable answers over traditional ranking signals.
- Crawl access and clear definitions are the fastest wins.
- Schema like BlogPosting and FAQPage clarifies structure for AI.
- Start with your top pages, then scale the pattern.
--- CONTENT ---
If ChatGPT and Perplexity never cite your site, the buyers asking those questions go to whoever they do cite instead. My site ranked on Google, had schema, fast load times, and decent backlinks. Then I searched my own main topic in Perplexity. My site wasn't cited once. My competitors, including one with a domain rating under 10, were showing up in every answer. That gap between ranking and being cited is what changed how I think about content structure entirely, and it's what the rest of this post explains how to close.
Answer Engine Optimization (AEO) is the practice of structuring content so AI answer engines can extract, trust, and cite it. It prioritizes crawl access, clear definitions, and machine-readable structure over classic link-based ranking signals. If SEO gets you into search results, AEO gets you into the answer itself.
## TL;DR
**AEO (Answer Engine Optimization)** is optimizing your content to be found and cited by AI search engines like Perplexity, Claude, ChatGPT, and Google's AI Overview. Not traditional Google Search. The core insight: Google ranks pages by popularity; AI engines cite pages by extractability. Those are different problems requiring different solutions.
- 60% of reputable news sites block at least one AI crawler, and most indie sites have never checked their own crawler access
- Blocking is usually accidental: a blanket robots.txt disallow or a bot-protection layer can shut out AI crawlers without you noticing
- Content that ranks on Google doesn't automatically appear in AI search results
- AEO is simpler than SEO: fewer competitors, clearer rules, higher ROI
---
## What Platform Shows Which URLs ChatGPT Cites From Your Site?
[citability.dev](https://citability.dev/?utm_source=chudidev&utm_medium=referral&utm_campaign=aeo-answer-engine-optimization-explained) is the platform I built for exactly this question: it asks ChatGPT and Perplexity real questions in your topic and records which URLs from your site they actually cite, with the raw AI responses kept as receipts. When a page loses citations, the recorded responses show what got cited in its place, so you can see what changed and why. Start with the free [30-second infrastructure scan](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=aeo-answer-engine-optimization-explained) to clear crawl-access and structure blockers, then run the citation test for URL-level results. The rest of this guide is the manual version of what it measures.
## What Is AEO and Why Does It Matter?
Answer Engine Optimization is the practice of structuring content so AI systems like Perplexity, ChatGPT, and Claude can extract, trust, and cite it. It matters because ranking on Google no longer guarantees visibility in AI-generated answers. A separate optimization layer is now required for that.
## The Problem: You're Invisible to AI
Ranking on Google does not mean you appear in AI-generated answers. A 2025 study found 60% of reputable news sites block at least one major AI crawler, and most indie sites have never audited their robots.txt for AI crawler access at all. Meanwhile, AI engines like Perplexity are already answering your audience's questions with your competitors' content.
You probably optimized your site for Google Search in 2024. Good job. But Perplexity, Claude, ChatGPT, and Microsoft Copilot are **answering questions from your competitors' content instead of yours**.
Here's why:
### Google vs AI Search
**Google Search:** "Show me the 10 best pages matching my query"
- Your meta description and title matter
- Backlinks prove authority
- Domain age signals trust
**AI Search:** "Synthesize an answer from multiple sources, cite them, move on"
- Your content is extracted, not ranked
- Meta descriptions are ignored (not shown to users)
- Titles matter less than content quality
- Backlinks don't matter at all
AI engines ask: *"Is this content accurate, specific, and extractable?"*
Google asks: *"Is this content popular and authoritative?"*
These are not the same thing.
---
## 2026 Platform Data: How AI Engines Actually Cite
Perplexity cites roughly 21.9 sources per answer; ChatGPT cites only 7.9. Only 12% of URLs cited by ChatGPT, Gemini, and Copilot appear in Google's top 10 for the same prompt, and Perplexity is a notable outlier at closer to 29%. These numbers mean your Google ranking strategy and your AI citation strategy are nearly independent problems that require separate solutions.
Research published in early 2026 reveals specific citation behaviors across platforms:
### Citation Volume by Platform
| Platform | Citations Per Answer | Primary Source Preference |
|----------|---------------------|-------------------------|
| Perplexity | ~21.9 | Reddit (46.7% of top sources) |
| Google Gemini | ~17.1 | No single dominant source (~8% overlap with Google top 10) |
| ChatGPT | ~7.9 | Wikipedia (4.8% of all citations) |
### Key Findings
- **Only 12% of URLs cited by ChatGPT, Gemini, and Copilot appear in Google's top 10.** AI citation and Google ranking are largely separate systems, except for Google AI Overviews (76% overlap with traditional results).
- **Pages with dateModified schema receive 1.8x more citations.** Freshness signals matter, but only when backed by substantive content updates. Bumping dates without changing content triggers penalty signals.
- **Pages with specific numbers, data tables, and original research are significantly more likely to be cited as sources.** Extractable, quotable data gives an answer engine something concrete to attribute.
- **AI-cited pages skew markedly fresher than Google's results.** Ahrefs measured AI-cited URLs at roughly 26% fresher than the pages ranking in traditional search for the same queries.
- **Reddit dominates Perplexity's source pool.** If you want Perplexity citations, genuine Reddit engagement with your expertise topics matters more than on-site optimization alone.
### Original Benchmark Data: The Visibility-Citation Gap
We ran the [AI Visibility Readiness](/framework) framework on 7 websites and measured both visibility (brand mentioned) and citability (URL linked) across AI platforms. The results quantify the gap between being known and being cited:
| Site | DA | AI Visible | AI Cited | Gap |
|------|-----|-----------|----------|-----|
| ahrefs.com | 92 | 100% | 5% | 95 pts |
| citability.dev | Under 10 | 44% | 15% | 29 pts |
| chudi.dev | 28 | 25% | 0% | 25 pts |
The finding: domain authority failed to predict AI citation rates in the 7-site sample. citability.dev (DA under 10) achieved 3x the citation rate of Ahrefs (DA 92). The differentiator was original benchmark data and answer-first content structure, not backlinks. Full audit methodology and results in [I Audited 7 Websites for AI Citability](/blog/ai-citability-audit-what-predicts-citations).
These numbers reframe the AEO strategy: platform-specific optimization outperforms universal approaches. You can now audit your own site's AI readiness with tools like [citability.dev](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=aeo-answer-engine-optimization-explained), or run the free [AEO audit tool](/tools/aeo-audit) to check crawler access, schema, and extractability in 60 seconds.
---
## How Do AI Engines Decide What Content to Cite?
AI engines prioritize content that is crawlable, clearly structured, and directly answers a specific question. Unlike Google, which weighs backlinks and engagement, AI systems evaluate accuracy, extractability, and metadata completeness. Technical access and content format are the primary ranking levers.
*An AI answer engine only cites a page after it clears all four gates in order: crawl access, extractable structure, selection signals, then a scarce citation slot. Most sites lose out at the first two gates, crawl and retrieve, long before schema, freshness, or citation volume ever come into play.*
*Sources: [Steinacker-Olsztyn et al., 2025](https://arxiv.org/abs/2510.10315) (news-site crawler blocking); [Semrush, citing AirOps](https://www.semrush.com/blog/answer-engine-optimization/) (1.8x freshness lift); [Qwairy Q3 2025](https://www.qwairy.co/blog/provider-citation-behavior-q3-2025) (citations per answer, 7.92 vs 21.87).*
## The 6 AEO Factors
The six factors that determine whether AI engines cite your content are: crawler access, llms.txt, structured data schema, content extractability, metadata completeness, and answer-ready format. Unlike SEO's 200+ signals, these six cover the full decision stack AI engines use to select and extract sources.
If SEO has 200+ ranking factors, AEO has 6 critical ones:
### 1. AI Crawler Access
First, your site needs to be crawlable by AI bots. [Google's robots.txt documentation](https://developers.google.com/search/docs/crawling-indexing/robots/intro) covers the standard. AI crawlers follow the same protocol using their own user-agent strings. Check your `robots.txt`:
```
User-agent: *
Disallow: /admin
Disallow: /private
# AI Crawlers
User-agent: GPTBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Googlebot-Extended
Allow: /
```
**The stat:** A 2025 study found 60% of reputable news sites block at least one major AI crawler. Most indie sites fall into one of three buckets:
- Block all crawlers with `Disallow: /`
- Use `X-Robots-Tag: noai` headers
- Never heard of this and rely on old robots.txt defaults
If you block AI crawlers, you're invisible.
### 2. llms.txt (Robots.txt for AI)
You know about `robots.txt`. Now there's [`llms.txt`](https://llmstxt.org). I wrote a full implementation guide in [llms.txt: Robots.txt for AI Crawlers](/blog/llms-txt-robots-txt-for-ai-crawlers).
`llms.txt` is a human-readable file that tells AI crawlers *what* to index and *how* to cite you. It should live at `chudi.dev/llms.txt`:
```
# Our content policy for LLMs
All content on this site is available for training and search.
Please credit sources as: [Article Title] by [Author Name] (yoursite.com)
Sitemap: https://yoursite.com/sitemap.xml
RSS: https://yoursite.com/rss.xml
```
**Why it matters:** Without `llms.txt`, AI engines might skip your site or misattribute content. With it, you're explicitly inviting them in and setting citation rules.
### 3. Structured Data
AI engines parse [JSON-LD schemas](https://schema.org/). If your content has:
- [`BlogPosting` schema](https://schema.org/BlogPosting) → AI knows it's an article
- [`FAQPage` schema](https://schema.org/FAQPage) → AI knows it's answerable questions
- [`HowTo` schema](https://schema.org/HowTo) → AI knows it's a tutorial
Content without schema is harder for AI to structure. `Learn how to build a SaaS
` is vague. Schema says `{"@type": "HowTo", "step": [...]}`. That's machine-readable.
### 4. Content Extractability
AI engines don't need your layout. They need your *text*. This means:
- **Semantic HTML:** Use ``, ``, proper `` → `` → `` hierarchy
- **No text in images:** AI can't read screenshots. Use actual text + `![...]()
`
- **Lists > paragraphs:** Bullet points are easier to extract than walls of text
- **Scannable structure:** Headers every 2-3 paragraphs
A 2,000-word article with 3 headers is harder to cite than a 2,000-word article with 15 headers. AI needs to find the specific section that answers the user's question.
### 5. Metadata Completeness
Even though AI ignores ``, it checks:
- Canonical URL (to avoid duplicate content)
- Open Graph image (for preview context)
- Article `datePublished` (to know how old your content is)
Stale content (3+ years old) gets deprioritized. Fresh content gets cited more often.
### 6. Answer-Ready Format
The best AEO content directly answers common questions:
- "What is X?" → Definition box at the top
- "How do I X?" → Step-by-step with numbered lists
- "Why does X matter?" → Clear benefits, quantified where possible
Content written as "walls of paragraphs" is less likely to be extracted. Content structured as "question → answer → proof" is gold. I built my [autonomous blog agent](/blog/openclaw-autonomous-blog-agent) around this principle. Every draft follows the question-answer-proof structure automatically.
---
## AEO vs SEO vs GEO: Three Optimization Disciplines
The search landscape now has three distinct optimization disciplines. SEO targets traditional search rankings, AEO targets AI answer citations, and GEO (Generative Engine Optimization) targets how generative AI models represent your brand and content in synthesized responses. They overlap but reward different signals.
| Factor | SEO (Google) | AEO (AI Engines) | GEO (Generative AI) |
|--------|-------------|-----------------|---------------------|
| Goal | Rank in search results | Get cited in AI answers | Shape AI's representation of your brand |
| Crawling | robots.txt | robots.txt + llms.txt | robots.txt + llms.txt + ai.txt |
| Ranking | Backlinks + engagement | Content accuracy + structure | Entity clarity + topical authority |
| Signal count | 200+ factors | ~6 critical factors | Emerging, entity-centric |
| Meta descriptions | Shown to users | Ignored | Ignored |
| Title tags | Shown to users | Used for context | Used for entity recognition |
| Content format | Long-form preferred | Any length, needs structure | Definition-dense, claim-backed |
| Freshness | Can rank for years | Deprioritized after ~10 months | Training data snapshots, slower refresh |
| Citations | Implied (link juice) | Explicit (source linked in answer) | Implicit (model "knows" you) |
| Key differentiator | Popularity | Extractability | Entity recognition |
### When to prioritize which
- **SEO first** if your revenue depends on organic click-through traffic today.
- **AEO first** if you produce reference content (definitions, tutorials, comparisons) that AI engines synthesize into answers.
- **GEO first** if brand perception matters more than individual page traffic. You want AI to describe your company correctly when users ask about your category.
Most sites benefit from all three, and the technical foundations overlap heavily. The good news: structured data, semantic HTML, clear headings, and answer-first formatting improve all three simultaneously. The differences are in emphasis, not in contradiction.
**The opportunity:** A well-structured blog post with schema and semantic HTML will rank on Google, appear in AI answers, *and* inform how generative models represent your expertise. The cost of doing all three is marginally higher than doing one.
---
## How Do You Start Optimizing for Answer Engines?
Start by confirming AI crawlers can access your site via robots.txt, then create an llms.txt file at your site root. Audit your top five pages to add schema markup, restructure them with more headers, and rewrite openings to directly answer the question in the title. Those four actions deliver the most AEO impact.
## How to Start with AEO
Four actions deliver 80% of your AEO impact: unblock AI crawlers in robots.txt, create a two-paragraph llms.txt at your site root, add FAQPage schema to your three best posts, and rewrite each post's opening paragraph to directly answer the title question. You can finish all four in under an hour.
### Step 1: Check if AI Can Find You
```bash
# Can Perplexity, Claude, etc. access your site?
curl -I yoursite.com/robots.txt
# Look for GPTBot, ClaudeBot, PerplexityBot allow rules
```
### Step 2: Create `/llms.txt`
Add this file to your site root:
```
# Content policy for LLMs
All content available for training and search.
Please attribute as: [Article] by [Author] (yoursite.com)
Sitemap: https://yoursite.com/sitemap.xml
RSS: https://yoursite.com/rss.xml
```
### Step 3: Audit Your Best Content
Pick your 5 best-performing pages and:
- Add schema (BlogPosting, HowTo, FAQ)
- Restructure with more headers
- Move key info to the top
- Add a definition box for the main question
### Step 4: Monitor in Perplexity
Search your main topics in Perplexity. Are you being cited? If not, your content isn't being discovered.
---
## Platform-by-Platform AEO Strategy
Each AI engine has distinct citation behavior and source preferences. Perplexity favors Reddit and reference-style content; ChatGPT is highly selective and freshness-sensitive; Google AI Overview mirrors traditional search rankings; Claude rewards structured technical writing. A platform-specific approach compounds your overall citation rate significantly beyond what generic AEO delivers.
### Perplexity
Perplexity cites the most sources per answer (~21.9) and favors Reddit heavily (46.7% of top sources, per Profound's 2025 citation-pattern study). To optimize for Perplexity:
- **Participate in Reddit discussions** about your expertise topics. Perplexity's retrieval pipeline weights Reddit as a high-trust source for niche questions. Genuine contributions with links to your detailed writeups outperform any on-site optimization alone.
- **Structure content as reference material.** Perplexity favors pages that read like authoritative references: tables, definitions, numbered lists, and comparative data. Pure opinion pieces get cited less.
- **Keep content updated.** Perplexity's real-time retrieval means recently updated pages get priority. Pages with `dateModified` schema that reflect genuine content updates outperform stale pages.
- **Include original data.** Benchmark results, survey findings, and original research get cited at higher rates because they provide information Perplexity can't synthesize from other sources.
### ChatGPT (SearchGPT / Browse)
ChatGPT is the most selective citer (~7.9 sources per answer) and leans heavily on Wikipedia (4.8% of all citations, per Qwairy's Q3 2025 provider study). To optimize:
- **Answer the exact question in paragraph one.** ChatGPT extracts more aggressively than other platforms. If your answer is in paragraph four, it may not get extracted at all.
- **Target Wikipedia-adjacent queries.** ChatGPT cites Wikipedia for broad topics but turns to specialized sources for niche questions. The sweet spot is questions too specific for Wikipedia but too authoritative for forums.
- **Prioritize freshness.** AI-cited pages skew markedly fresher than traditional search results (Ahrefs measured AI-cited URLs at roughly 26% fresher than same-query SERP pages). A page that hasn't been updated in a year is competing at a measurable disadvantage.
- **Use canonical URLs consistently.** ChatGPT deduplicates aggressively. If your content appears at multiple URLs (www vs non-www, trailing slashes, paginated versions), citations may be split or lost.
### Google AI Overview
Google AI Overview draws 76% of its citations from the traditional top 10 results. This makes it the most SEO-correlated AI citation source.
- **Rank in Google first.** Unlike other AI engines, Google AI Overview primarily cites pages that already rank well in traditional search. AEO-only optimization without SEO foundations won't work here.
- **Add FAQPage schema.** Google AI Overview preferentially extracts from pages with structured FAQ markup. The question-answer format maps directly to how Overview constructs its responses.
- **Target featured snippet queries.** Queries that currently trigger featured snippets are the most likely to trigger AI Overview. If you can win the snippet, you're positioned for the Overview citation.
### Claude
Claude doesn't have a live search product with citations in the same way, but its training data preferences and retrieval behaviors are worth understanding:
- **Maintain an llms.txt file.** Claude's parent company (Anthropic) respects llms.txt as a content policy signal. Having one is a positive signal for ClaudeBot crawling.
- **Provide structured, well-organized technical content.** Claude's training pipeline favors content with clear hierarchical structure, code examples with context, and explicit reasoning chains.
- **Avoid content that reads like AI-generated filler.** Claude's quality filters are sensitive to low-information-density content. Dense, opinionated, experience-backed writing gets weighted higher than generic explainers.
---
## Common AEO Mistakes
The six mistakes that most often keep well-written content invisible to AI engines are: blocked crawlers, buried answers, image-only key information, assuming Google rank equals AI visibility, bumping date metadata without changing content, and skipping FAQ schema. Each is fixable in under an hour once you know to look for it.
### Mistake 1: Blocking AI crawlers unintentionally
The most common AEO failure is a robots.txt that blocks crawlers the site owner doesn't know about. A blanket `Disallow: /` for unlisted user agents, or a hosting platform that adds `X-Robots-Tag: noai` by default, silently makes your entire site invisible. Run `curl -I yoursite.com/robots.txt` and check what it actually says. Don't assume.
### Mistake 2: Burying the answer
SEO rewards content that keeps users scrolling: long intros, context-setting, narrative buildup. AEO rewards the opposite. If your page title is "What is Answer Engine Optimization?" and the definition doesn't appear until paragraph three, AI engines may extract from a competitor who puts it in paragraph one. Move the answer up. Context can follow.
### Mistake 3: Using images instead of text for key information
Infographics, diagrams, and screenshots are invisible to AI extraction. If your comparison table is an image, AI engines can't read it. If your step-by-step tutorial is a screenshot of a terminal, AI can't extract the commands. Use real HTML tables, real code blocks, and real text. Add `alt` attributes to images, but don't rely on `alt` for primary content.
### Mistake 4: Assuming Google ranking equals AI visibility
This is the most expensive mistake. A page ranking #1 on Google may not be cited by any AI engine. Google and AI engines evaluate different signals: Google weighs backlinks and click-through rate; AI engines weigh extractability and structural clarity. The only AI engine with strong Google correlation is Google AI Overview (76% overlap). For Perplexity, ChatGPT, and Claude, your Google rank is largely irrelevant.
### Mistake 5: Updating `dateModified` without changing content
Pages with `dateModified` schema get 1.8x more AI citations, but only if the content actually changed. Bumping the date on unchanged content is detectable (AI engines can diff cached versions) and triggers a trust penalty. When you update `dateModified`, make substantive changes: new data, expanded sections, corrected claims, or added examples.
### Mistake 6: Ignoring FAQ schema
FAQPage schema is the single highest-ROI structured data for AEO. It maps directly to how AI engines construct answers: question in, answer out. A page with five well-structured FAQ entries gives AI engines five separate extraction opportunities. A page without it gives AI engines one: the entire article body, which they then have to parse themselves with lower accuracy.
---
## How Do You Measure AEO Performance Without a Dashboard?
Measure AEO manually each month by querying ChatGPT, Perplexity, Claude, and Gemini with the exact questions your content answers, then checking whether you are cited. Track AI referral traffic in analytics using domain segments, and monitor Google's AI Overview for your target queries to identify citation gaps.
## Measuring AEO Progress
Track AEO performance with four monthly checks: a manual citation audit across ChatGPT, Perplexity, Claude, and Gemini; AI referral traffic segments in analytics; an automated infrastructure scan via citability.dev; and a Google AI Overview review for your target queries. No single dashboard does this yet.
The hardest part of AEO is knowing if it's working. Traditional SEO has rankings and impressions in Search Console. AEO doesn't have a dashboard yet.
### Manual citation audit (monthly)
Open ChatGPT, Perplexity, Claude, and Gemini. Ask the exact questions your content answers:
- "What is AEO?"
- "How do I optimize for Perplexity?"
- "What is llms.txt?"
Are you cited? If not, who is? Read the content that does get cited and compare it to yours. The differences are usually structural: their definition is in the first paragraph, yours is in paragraph four. Their page has 12 question-format headers, yours has three. These gaps are fixable.
### Want to See Which URLs ChatGPT Cites From Your Site?
The manual audit above is free but slow, and it only samples a few questions. That exact problem is why I built [citability.dev](https://citability.dev/?utm_source=chudidev&utm_medium=referral&utm_campaign=aeo-answer-engine-optimization-explained): the paid citation audit asks ChatGPT and Perplexity real questions in your topic and records which URLs from your site they actually cite, with the raw AI responses kept as receipts. Nothing is scored that wasn't tested. Start with the free [30-second infrastructure scan](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=aeo-answer-engine-optimization-explained) to clear crawl-access and structure blockers first, then run the citation test for your URL-level results.
### AI referral traffic in analytics
Create a segment for sessions from AI engine domains: `chatgpt.com`, `perplexity.ai`, `claude.ai`, `gemini.google.com`, `copilot.microsoft.com`. Track this monthly. Growth here is a leading indicator of citation growth. Direct AI traffic often comes before organic traffic from AI-influenced searches.
### Automated infrastructure auditing
Manual citation checks tell you the outcome but not the cause. Infrastructure audits identify the specific technical gaps preventing citations. Tools like [citability.dev](https://citability.dev/assess?utm_source=chudidev&utm_medium=referral&utm_campaign=aeo-answer-engine-optimization-explained) run 10 automated checks against your site: robots.txt permissions, sitemap completeness, answer-first content structure, content freshness via dateModified schema, JSON-LD coverage, meta descriptions, canonical URLs, HTTPS, heading hierarchy, and social sharing tags. The scan takes under 30 seconds and shows exactly which signals pass and which need fixing.
### Google AI Overview tracking
Search your target queries in Chrome incognito. Does Google's AI Overview cite you? If it does, you're in the 2–7% of pages that get sourced for that query. If it doesn't but competitors are cited, run their pages through the AEO checklist. FAQPage schema and answer-first formatting are usually the gap.
### The minimum viable AEO setup
If you want to start today with 30 minutes of work:
1. Add `User-agent: GPTBot / Allow: /` and similar for ClaudeBot, PerplexityBot to your robots.txt
2. Create a 200-word `llms.txt` at your root with your sitemap URL and preferred attribution format
3. Add FAQPage schema to your three best posts
4. Rewrite the opening paragraph of each post to directly answer the question in the title
That's it. Nothing else in the AEO checklist will have as much impact as those four actions. Do them before optimizing for any specific engine.
## Advanced AEO: Beyond the 6 Factors
After the six core factors are in place, three techniques compound your AI visibility further: publishing an ai.txt policy file, implementing WebMCP so AI agents can query your site programmatically, and stacking entity mentions across Wikidata, GitHub, LinkedIn, and Crunchbase. These are differentiation plays, not baseline requirements.
### ai.txt: Declaring your AI interaction policy
While `llms.txt` tells AI crawlers what content to index and how to cite it, `ai.txt` goes further. It declares your site's full AI interaction policy including tool use, embedding permissions, and content licensing terms. Place it at `yoursite.com/ai.txt` alongside your `robots.txt` and `llms.txt`.
An `ai.txt` file signals to AI systems that your site is intentionally designed for AI interaction, not just passively crawlable. This is a differentiator: most sites are accidentally AI-visible (or accidentally invisible). A site with `ai.txt` is declaring intent, which AI systems can use as a trust signal.
### WebMCP: Making your site callable by AI agents
AEO gets your content cited in AI answers. WebMCP gets your content *used* by AI agents. WebMCP is a browser-side protocol that registers tools AI agents can call (search your site, query your data, interact with your product) directly from the browser context.
This is the frontier of AI-visible web architecture: a site that isn't just readable by AI, but usable by it. When an AI agent needs to find information in your domain, WebMCP lets it query your site programmatically rather than scraping HTML.
### Entity stacking: Building your knowledge graph footprint
AI engines identify entities (people, companies, products) by cross-referencing structured data across the web. The more high-authority platforms that contain consistent information about your entity, the more confidently AI engines will cite you.
Key platforms for entity stacking:
- **Wikidata:** Add a structured entry for yourself or your company. Wikidata is a primary knowledge source for most AI systems.
- **Crunchbase:** For companies and products. Crunchbase data feeds into multiple AI training pipelines.
- **GitHub:** Active repositories with clear README files and consistent author attribution.
- **LinkedIn:** Detailed profile matching your site's author schema exactly (same name, same title, same description).
- **Product Hunt:** For SaaS products. Product Hunt pages are high-authority and frequently cited by AI when discussing tools.
The key is consistency: the same name, same description, same URL across all platforms. Inconsistent entity data confuses AI systems and splits your citation authority across multiple "versions" of you.
### Topic hubs: Building topical authority for AI
AI engines evaluate topical authority differently than Google. Google uses backlinks as authority proxies. AI engines use content depth and internal linking density. A site with one article about AEO is a mention. A site with ten interconnected articles about AEO (definitions, tutorials, case studies, comparisons, tooling) is a topical authority.
Build topic hubs by:
1. **Writing a pillar article** (like this one) that defines the topic comprehensively.
2. **Creating supporting articles** that go deep on subtopics: each of the 6 factors, platform-specific guides, case studies, tool comparisons.
3. **Internal linking densely.** Every supporting article links to the pillar. The pillar links to every supporting article. AI engines follow these internal link graphs to gauge content depth.
4. **Using consistent terminology.** If your pillar calls it "Answer Engine Optimization," every supporting article should use the same phrase, not alternate between "AEO," "AI SEO," and "answer engine marketing."
---
## The Future is Plural Search
Over 30% of searchers now use AI answer engines for complex queries, and that share is growing monthly. AEO is not a replacement for SEO. It is a second optimization layer for a second type of search engine, one that extracts passages directly rather than ranking pages for users to click.
Google won't be the only search engine anymore. By 2026, over 30% of searchers use answer engines for complex queries. You need to be visible in *all* of them.
AEO isn't replacing SEO. It's extending your reach to a new search engine that's growing fast and underserved.
The technical foundation is the same as good SEO: well-structured, authoritative content with clear headings and direct answers. What changes is the mental model. SEO rewards findability: rank high enough and users click through. AEO rewards extractability: your H2 sections get lifted verbatim into AI responses. A page that ranks #3 on Google but buries its main answer in paragraph five won't get cited by AI even if Google loves it.
Write for systems extracting specific passages, not just readers scanning for reasons to click. Each H2 should be a complete, self-contained answer to the question it poses. Enough context to stand alone if extracted.
The easiest time to optimize for AEO was 2024. The second easiest time is today.
The six factors above are the whole method, and they are yours to run. If you would rather have the diagnosis handed to you, page by page, with the fixes already ordered, that is [the AEO audit I run at a fixed price](/services).
**Next:** Check out the [optimization checklist for AI search](/blog/how-to-optimize-for-perplexity-chatgpt-ai-search).
**See also:** [Why Domain Authority Is Irrelevant for AI Search (And What to Build Instead)](/blog/domain-authority-irrelevant-ai-search) and [How to Structure Content So AI Actually Cites Your URL](/blog/how-to-optimize-for-perplexity-chatgpt-ai-search)
--- END POST ---
================================================================================
POST: RAG Explained: How to Stop LLMs From Making Things Up
================================================================================
URL: https://chudi.dev/blog/what-is-rag
Date: 2025-01-15T00:00:00.000Z
Tags: ai, rag, llm, tutorial
Pillar: ai-building
Reading Time: 7 min
Word Count: 1371
TL;DR: Without me realizing it, I had been using RAG every time I asked Claude to help me understand a codebase. RAG (Retrieval-Augmented Generation) gives LLMs access to external knowledge at inference time. Well, it's more like grounding AI in actual documents instead of hoping training data is enough.
Key Takeaways:
- RAG retrieves relevant documents at inference time, grounding responses in current, factual sources
- The pattern follows query, retrieve, augment, then generate. Context gets added before the LLM responds
- Use cases include documentation chatbots, customer support, research assistants, and code Q&A
- Every time you feed context to Claude before asking questions, you're using the RAG pattern
--- CONTENT ---
## TL;DR
RAG (Retrieval-Augmented Generation) combines language models with real-time data retrieval to provide accurate, up-to-date responses. **Key benefit**: Reduces hallucination by grounding responses in actual documents.
## What is RAG?
RAG (Retrieval-Augmented Generation) is a technique that gives large language models access to external knowledge at inference time. Rather than relying solely on training data that may be months old, RAG retrieves relevant documents and adds them to the prompt before the model generates a response, grounding answers in current, factual sources.
RAG is a technique that gives LLMs access to external knowledge at inference time. Instead of relying solely on what the model learned during training--which could be months or years old--RAG pulls in relevant documents before generating a response.
Without me realizing it, I had been using a form of RAG every time I asked Claude to help me understand a codebase. Feeding it context before asking questions? That's the RAG pattern in action.
### How RAG Works
1. **Query Processing**: User question is received
2. **Retrieval**: Relevant documents are fetched from a knowledge base
3. **Augmentation**: Retrieved context is added to the prompt
4. **Generation**: LLM generates a response using both its training and the retrieved context
I thought RAG was only for enterprise systems. Well, it's more like... the pattern exists everywhere we add context to AI conversations.
### The Three Core Components
Every RAG system has three parts that need to work together:
**Embedding model**: Converts text into vectors, lists of numbers representing semantic meaning. "How do I authenticate users?" becomes a numerical vector where similar questions produce similar vectors. Common choices include OpenAI's `text-embedding-ada-002`, Cohere Embed, or open-source models from `sentence-transformers`. The embedding model determines how well your system understands semantic similarity. Two sentences that mean the same thing should produce vectors that are close together in vector space, even if they use different words.
**Vector database**: Stores those embeddings and enables fast similarity search. When a query arrives, the database finds documents whose embeddings are most similar to the query embedding. Options range from Pinecone (managed cloud) to Chroma (local dev) to pgvector (PostgreSQL extension). For most small-to-medium projects, Chroma locally or pgvector in Supabase is more than sufficient, no need for a dedicated vector database service until you're storing millions of documents.
**Retrieval strategy**: Decides which documents to include and how many. Naive retrieval takes the top-k most similar chunks. Hybrid search combines vector similarity with keyword matching (BM25). Reranking uses a second model to re-score results for relevance. Getting retrieval right often matters more than which database or embedding model you choose.
## Why Does RAG Matter for Builders?
I hated the feeling of asking an AI a question and getting confidently wrong information. But I love being able to trust responses when they're grounded in actual sources.
That specific relief of knowing where information comes from--it changes how you build with AI entirely.
### Common RAG Use Cases
## When Should You Use RAG vs. Fine-Tuning?
Both approaches customize LLM behavior, but they solve different problems.
| | RAG | Fine-Tuning |
|---|---|---|
| Best for | Factual queries, current info | Style, tone, task format |
| Knowledge updates | Instant (update the DB) | Requires retraining |
| Hallucination risk | Lower (grounded in docs) | Higher (model memorizes) |
| Source transparency | Can cite sources | Black box |
| Setup cost | Medium (need vector DB) | High (need training data) |
I tried fine-tuning a model on internal documentation once. Updates required retraining. Months later, the model confidently cited outdated API versions. RAG would have been simpler and more reliable for that use case, the knowledge was too dynamic for fine-tuning.
**Use RAG when:**
- Your knowledge changes frequently (documentation, product updates, news)
- You need to cite sources and show your work
- You're querying factual or domain-specific information
- You want explicit control over what the model can reference
**Use fine-tuning when:**
- You need a specific response format or output structure
- You're adjusting tone or communication style
- Task-specific patterns need to be baked in at training time
- You have thousands of high-quality labeled examples
Most production systems end up using both: fine-tuned models for consistent format and tone, RAG for factual grounding.
## How Do You Get Started with RAG?
Here's a working minimal implementation with ChromaDB and Claude:
```python
import chromadb
import anthropic
# 1. Set up vector store and add documents
client = chromadb.Client()
collection = client.create_collection("docs")
collection.add(
documents=[
"Users authenticate via JWT tokens stored in httpOnly cookies.",
"Database connection uses PostgreSQL on port 5432.",
"API rate limit is 100 requests per minute per IP."
],
ids=["auth", "db", "rate-limit"]
)
# 2. Retrieve relevant context
def retrieve(query: str, n: int = 3) -> list[str]:
results = collection.query(query_texts=[query], n_results=n)
return results["documents"][0]
# 3. Generate with context
anthropic_client = anthropic.Anthropic()
def rag_query(question: str) -> str:
context = retrieve(question)
context_str = "\n".join(f"- {doc}" for doc in context)
response = anthropic_client.messages.create(
model="claude-opus-4-8",
max_tokens=1024,
messages=[{
"role": "user",
"content": f"Use this context to answer the question.\n\nContext:\n{context_str}\n\nQuestion: {question}"
}]
)
return response.content[0].text
print(rag_query("How do users log in?"))
# → "Users authenticate via JWT tokens stored in httpOnly cookies."
```
ChromaDB handles embedding automatically using its default model. For production, swap in a dedicated embedding service and a persistent vector database. The key is step 3, context gets prepended to the prompt before the LLM generates. That's it. That's RAG.
## Common RAG Mistakes
I've made most of these. Some took months to diagnose.
### Chunk size too large
Retrieving entire documents when only one paragraph is relevant. LLMs have context windows, stuffing in irrelevant text wastes tokens and dilutes useful content. Worse, the LLM might confabulate based on unrelated content that got retrieved alongside the correct answer.
**Fix:** Chunk documents into 256–512 token segments with 50-token overlap to preserve context at boundaries.
### Chunk size too small
Splitting so aggressively that context is lost. A 50-word chunk might contain the answer but not the surrounding context that makes it meaningful. "Use bcrypt" without "when storing passwords" misses the point.
**Fix:** Experiment per content type. For prose, 512 tokens with overlap. For code, split by function or class boundary.
### Skipping reranking
Taking top-k vector search results at face value. Vector similarity finds semantically related text but not always the most *relevant* text. A passage about "fast running" can score well for "quick deployment" when you wanted DevOps documentation.
**Fix:** Add a cross-encoder reranker as a second pass. It re-scores results by relevance to the specific query, not just semantic proximity. Slower but significantly more accurate for precision-sensitive use cases.
### Missing metadata filtering
Treating all documents equally. If you have docs for v1 and v2 of an API, you want v2 docs for v2 questions. Vector similarity doesn't understand versioning or recency.
**Fix:** Store metadata (version, date, author, section) alongside embeddings. Filter before or after retrieval based on query context or user-supplied parameters.
### No evaluation
Building RAG and measuring success by feel. "It seems better" is not a metric.
**Fix:** Create a test set of 20–50 question-answer pairs. Measure retrieval precision (did the right document get retrieved?) and answer faithfulness (did the LLM actually use the retrieved content?). Tools like [RAGAS](https://docs.ragas.io/) automate this evaluation loop.
## Production RAG Considerations
The working example above gets you a prototype. Production systems need a few more pieces.
**Chunking strategy matters more than the database.** The most common reason RAG systems underperform isn't the LLM or the vector store, it's that chunks are either too large (stuffing in irrelevant context) or too small (losing surrounding meaning). Start with 512-token chunks with 50-token overlap and test against your actual question set before committing to a strategy.
**Evaluate retrieval separately from generation.** Two distinct failure modes exist: the right document wasn't retrieved, or the right document was retrieved but the LLM generated a bad answer anyway. Evaluate each independently. If retrieval precision is low (wrong documents retrieved), improve embedding quality or add metadata filtering. If answer faithfulness is low (documents retrieved but ignored in the response), improve the system prompt to be more directive about using context.
**Monitor costs in staging.** Embedding thousands of documents at 10,000 tokens each adds up. Audit your embedding cost before scaling to production. For most developer knowledge bases, the one-time embedding cost stays under $5, but knowing that number before you scale matters. OpenAI's `text-embedding-3-small` costs $0.02 per million tokens; for a 500-document knowledge base, that's effectively free at initialization.
---
Since I no longer need to second-guess every AI response, I can focus on what I actually want to build. I like to see it as a comparative advantage--understanding RAG means building more reliable AI applications.
---
## Related Reading
This is part of the [Complete Claude Code Guide](/blog/claude-code-complete-guide) and the [tools and resources collection](/products). Continue with:
- [Quality Control System](/blog/how-i-build-with-claude-code) - Two-gate enforcement for AI code generation
- [Context Management](/blog/claude-context-management-dev-docs) - The dev docs workflow is essentially manual RAG
- [Self-Improving RAG for Claude Code](/blog/self-improving-rag-claude-code) - Building a system that learns from your sessions
--- END POST ---