A Developer’s Guide to Building RAG Systems: Lessons from the Trenches

I have spent a good while now building RAG pipelines across a number of engagements, mostly for documentation sites, and the gap between the tutorials and the reality of production is substantial.

This post is not a getting started guide. There are plenty of those.

This is a collection of things I got wrong, things I got right eventually, and things I wish someone had told me before I started, drawn from the work rather than from any single project.

It spans the whole pipeline rather than one corner of it: not just ingestion and chunking, but the retrieval and synthesis side where slow queries and silently duplicated context live, and the operational side where ingestion jobs die quietly and indices drift out of date.

Some of the hardest lessons were the ones furthest from the embedding model.

I will also talk about how my build process changed over time, specifically how I went from writing everything by hand to leaning heavily on Claude Code to accelerate development without sacrificing quality or understanding.

Let’s dig in.

~

How It Started

The arc tends to be the same on these engagements. The first pipeline I ever built was entirely manual. I wrote the crawler, the chunker, the embedding step, and the query layer from scratch in .NET 9.

No AI assistance, no scaffolding. I wanted to understand every moving part before I let anything generate code for me.

That was the right call.

The problem was that the iteration cycle was slow. Every new documentation site meant going back into the code to adjust selectors, tweak chunking parameters, or add a new filter.

What I had built was a single-purpose pipeline masquerading as something more general, and that is a trap worth naming because it is the default outcome if you are not deliberate about it.

So at some point I started using Claude Code more deliberately, not to write code for me, but to accelerate the parts I already understood. I would describe what I wanted, review the output, adjust it, and move on. The key shift was treating it as a collaborator that needed direction rather than a tool that would figure it out on its own.

I want to be direct about this because there is a version of this post where I undersell it. Claude Code materially accelerated the work, not by replacing my understanding of what I was building, but by removing the friction between understanding something and having it implemented.

When I knew what I wanted, describing it clearly and getting back something close to correct was significantly faster than writing it from scratch, and that time went back into thinking about the design, testing the behaviour, and figuring out what to build next.

The parts that went wrong were the parts where I gave it a vague brief and accepted the output without reviewing it carefully. The field naming mismatch I describe later in these lessons is a good example. I described what I needed but did not think through the schema detection logic myself, and the first version made assumptions that did not hold.

The pattern that works is to understand the domain well enough to know what correct looks like, use Claude Code to get there faster, and review the output with the same care you would apply to your own code.

It is a collaboration, not a delegation. That combination of me knowing the domain and Claude Code handling the boilerplate is what has let me stand up the moving parts of a pipeline quickly on each new engagement without losing the plot.

~

The Tools You End Up Building

Across these engagements, the same two tools tend to emerge. I doubt this is specific to me. If you do this work for any length of time you will likely build your own versions of both, so it is worth describing them as patterns rather than products.

The ingestion pipeline is the obvious one. You give it a URL and a config file and it handles the rest: crawling the documentation site, extracting the meaningful content, chunking it, generating embeddings, and indexing everything into a vector store. You can then send it a natural language question and get back an answer synthesised by an LLM with numbered citations.

Each project gets its own config, its own index, and its own system prompt, which is what lets the same pipeline serve more than one site.

The admin tool is the one people underestimate. It is a localhost panel for managing the health of a RAG pipeline. It sits on top of a content index and a vector index and gives you a live view of what is in good shape and what is not. It lets you un-delete documents, re-queue them for embedding, delete orphaned vector chunks, and re-vectorise individual documents when something goes wrong.

Both patterns come out of the same realisation: building a RAG pipeline is not hard, but keeping it healthy over time is.

~

Part 1: Architecture & Ingestion

Getting the pipeline stood up is one thing. Getting it stood up in a way that does not fight you for the rest of the engagement is another. The decisions you make here — about storage abstractions, content scoping, chunking, IDs, and configuration — either compound in your favour or accumulate as debt. Most of the mistakes in this section are things I had to retrofit rather than build in from the start.

~

Lesson 1: Start Without Infrastructure

The first thing I get wrong on every new project is reaching for a proper vector database too early. Spinning up a managed vector store or a dedicated search cluster for an experiment that might not go anywhere is overkill.

The ingestion pipeline has a dual storage strategy for this reason. The vector store sits behind an interface with two implementations: a file-backed store that runs cosine similarity in memory, and a heavier production store that runs proper vector searches at scale. Switching from one to the other is a single field change in the config file. The point is the abstraction, not the particular backends. Pick whichever lightweight option and whichever production option suit your stack.

The workflow this enables is simple. Start with the in-memory store, validate that the crawl and chunking are producing sensible results, then switch to the production store when you are ready to scale. No code changes, no migration scripts, just a config update.

If I had insisted on the production store from day one, I would have spent the first week setting up infrastructure instead of figuring out whether the chunking strategy was actually working.

~

Lesson 2: Content Scoping Is Not Optional

When you crawl an HTML page and dump the raw text into a chunk, you get the navigation bar, the cookie banner, the footer links, and the social share buttons alongside the actual content. All of that ends up in your embeddings.

The result is that queries about real content compete with chunks full of “Home About Blog Contact” and “Subscribe to our newsletter.” Your retrieval quality degrades and it is not obvious why.

The ingestion pipeline uses XPath selectors from the config to scope extraction to the meaningful part of the page:

"parser": {
  "contentSelector": "//main | //article",
  "excludeSelectors": ["//nav", "//footer", "//aside", "//script", "//style"]
}

This is the single most impactful thing you can do for retrieval quality before you touch chunking or embedding models. Get the right content in, and everything downstream gets easier.

~

Lesson 3: Chunking Strategy Actually Matters

The naive approach is to split on a fixed character count. It works well enough to demo but poorly enough in production to be worth fixing.

The problems are twofold. First, fixed-character splits break sentences in the middle, which produces chunks that are confusing to the embedding model and useless to users who see them in search results. Second, if a key sentence falls right at the boundary between two chunks, without overlap it only appears in one of them. Depending on where the query lands semantically, you might miss it entirely.

One approach that works well is a paragraph-aware chunker: split on paragraph boundaries first, then enforce a size cap, with overlap between adjacent chunks. Fenced code blocks are preserved in full regardless of length, because splitting a code example mid-block produces something that is not useful to anyone.

A sliding window approach works too, and the pipeline defaults to 800 tokens with 100-token overlap. The overlap is the critical part. It means a sentence near the end of one chunk also appears at the start of the next, which makes retrieval at boundaries much more reliable.

One practical detail: I use a 4-chars-per-token approximation rather than running a tokeniser on every chunk. It is good enough for chunking purposes and avoids a dependency.

~

Lesson 4: Give Each Chunk a Deterministic ID

This one sounds obvious in retrospect but I did not do it correctly the first time.

If your chunk IDs are random UUIDs, re-running the ingestion pipeline creates duplicate entries. You end up with two copies of every chunk, which doubles your vector store size and skews retrieval scores. The fix is to make chunk IDs deterministic.

The approach that works is to derive each chunk ID from a hash of the source URL combined with the chunk index. That way the same chunk always lands on the same ID, so running the pipeline twice overwrites the first run rather than duplicating it. If you are running multiple projects or tenants out of one store, fold a project identifier into the ID as well so they cannot collide. This makes re-ingestion safe and removes a whole category of data quality problems.

~

Lesson 5: Separate Your Content and Vector Lifecycles

Most RAG implementations use a single index where each document has both its metadata and its vector representation. I took a different approach and split them across two indices, a content index and a vector index.

The reason is that content and vectors have different lifecycles. Content gets updated by the crawler on whatever schedule you run it. Vectors depend on the embedding model you are using, the chunking algorithm, and whether the source content has changed. When you update the chunking logic, you need to re-embed everything, but you do not need to re-crawl. When a document is deleted from the source site, you want to soft-delete it in the content index and eventually clean up its vectors, but those are separate operations.

The tradeoff is that the two indices can get out of sync, which is precisely why a split like this needs tooling to keep them reconciled. But the flexibility it gives you is worth the operational overhead, especially if you are likely to change your embedding model or chunking strategy over time.

~

Lesson 6: Zombies Are a Real Problem

A zombie is a soft-deleted document that still has vector chunks in the vector index. It is not serving any users because the content is marked as deleted, but it is polluting the vector space and skewing search results. These accumulate silently over time as content gets updated or removed from the source site.

I did not have visibility into this problem until I built the admin tool. A stats panel shows a live zombie count and you can filter for them directly. Cleaning them up is a one-click operation rather than a sequence of ad hoc queries against the index.

The broader point is that RAG pipelines are not fire-and-forget. You need visibility into the state of your indices and the ability to intervene at the document level. Building the tooling to do that is not glamorous work, but it is what keeps retrieval quality from degrading over time.

~

Lesson 7: Configuration Belongs in Config, Not Code

My first pipeline had site-specific logic scattered through the code: hardcoded content selectors, a specific chunking size, a particular system prompt for the LLM. Anything that varied from one documentation site to the next was baked into the source, so adapting it to a second site meant going back into the code.

The better approach is to make every site-specific detail a property of a config file and keep the pipeline logic generic. A config describes the seed URLs, the content selectors, the chunking parameters, the embedding model, the storage backend, and the system prompt. The pipeline code never changes; only the config does. Whether you are serving one corpus or fifty, the same principle holds, and it is what makes multi-tenancy almost free when you do need it, because a new tenant is just another config rather than another code path.

The practical benefit is that standing up a new site is a matter of minutes, not hours. You drop a config file in place and point the pipeline at it. The pipeline handles the rest.

~

Part 2: Embeddings & Retrieval

Getting content into the index correctly is only half the problem. The retrieval side has its own failure modes, and most of them are quieter. Embeddings silently mismatched. Chunker fast paths that only reveal themselves when a 40KB code block hits the API. Token limits that look fine by approximation and aren’t. Duplicate pages flooding the synthesis context. And logging gaps that let all of it hide in plain sight. These are the lessons that cost me the most time to diagnose.

~

Lesson 8: Query Embedding Must Match Ingestion Embedding

This is a simple one but easy to get wrong if you are iterating quickly. The model you use to embed chunks at ingestion time must be the same model you use to embed queries at retrieval time. If they diverge, cosine similarity scores become meaningless.

In the ingestion pipeline the embedding model is set in the config file, and the same config drives both ingestion and query, so there is no way to accidentally mismatch them. However you manage it, make sure the model is a property of the index, not a runtime parameter.

~

Lesson 9: Watch Out for the Fast Path in Your Chunker

This one took me a while to track down because the symptom and the cause were in different places. A common optimisation in chunkers is to pull code blocks out and replace them with short placeholders before measuring the content length. The idea is to reason about the prose without the code getting in the way. The problem is what happens next. If the total length with the placeholders in place looks small enough to skip splitting, the whole thing gets returned as a single chunk once the placeholders are swapped back out for the real code.

The failure mode is a page with a few hundred characters of prose wrapped around one large code block. With the placeholder in place it looks tiny, so it sails straight through the fast path. Then the real code goes back in and you have a chunk of tens of thousands of characters. The embedding API rejects it and the whole document fails to vectorise. The fix is to measure the restored length, not the placeholder length, before deciding whether to split. Whenever you substitute a stand-in for the real content to make a decision easier, make sure the decision is made against what you are actually going to send.

~

Lesson 10: Atomic Code Block Protection Needs a Size Cap

Treating code blocks as atomic units is the right instinct. I said as much back in the chunking lesson, and I stand by it. Splitting a code example mid-block produces something useless to a reader and confusing to the embedding model. But “never split” cannot quietly become “no size limit”. A large markup layout or a long fenced code block will be emitted whole regardless of how far it exceeds the embedding model’s token limit, and at that point your good intention has become the thing that breaks ingestion.

Code blocks still need a maximum size check. The right behaviour is that blocks over the limit get split at a safe boundary, a closing brace or a blank line, rather than emitted whole or dropped entirely. You lose the guarantee that a block is never broken, but you only lose it for the blocks that were too big to keep anyway, and a block split at a sensible boundary is far better than an embedding call that fails outright.

~

Lesson 11: Token Count Approximations Break Down for Code-Heavy Content

Using a character-based approximation for token counting is the right call, and I said as much back in the chunking lesson. Running a full tokeniser on every chunk adds latency and a dependency you do not need during chunking. The standard approximation of around four characters per token works well for natural language prose. The trouble is that it does not work as well for code, markup, or XML.

Code tends to run closer to two or three characters per token because of the density of special characters, short identifiers, and whitespace. A chunk that looks comfortably within your size budget by the character approximation can sail past the embedding model’s actual token limit once it is measured properly. This is the same failure as the oversized code blocks from earlier, just arriving through a different door. If your content is documentation-heavy with a lot of code examples, the answer is to be more conservative where it counts. Apply a tighter size cap to chunks you have identified as code, or treat the approximation as a soft limit and give anything near the boundary more careful handling rather than waving it straight through.

~

Lesson 12: Apply the Same Guards to Ingestion That You Apply to Queries

Both the query path and the ingestion path call the same embedding model, but they are usually written at different times and by someone focused on different problems. Query-side guards get added when you are thinking about search quality and edge-case user input. Ingestion-side guards get added when something breaks during a crawl. The result is that the two paths drift apart even though they are talking to the same model with the same limits.

The way this bit me was a character limit. I added one before calling the embedding API on the query side, while I was thinking about what a user might paste into a search box, and forgot entirely that the exact same limit applies when generating embeddings for chunks. The chunk side then failed silently on anything long enough to hit the model’s context limit, and I only found out because a document never appeared in search results. The fix is a habit rather than a line of code. Any defensive logic that exists on one path should exist on the other. Review them together, not independently, because the model does not care which path the text arrived on.

~

Lesson 13: Deduplicate Retrieved Chunks by Document URI

Lesson 4 was about preventing duplicate chunks at ingestion time. This is the other duplication problem, and it lives on the retrieval side where it is much easier to miss.

A vector search will happily return several chunks from the same source page, often with identical relevance scores, because they are paragraph-level slices of one document that all match the query equally well. I had a query return five chunks from the same page, every one scoring 8.665282, different paragraph offsets, same URL. Nothing in the pipeline flagged it. The synthesis step was being handed the equivalent of an entire documentation page, around 19,700 tokens, when it should have had a few focused chunks.

The fix is to group by document URI and take one chunk per document before applying your top-N limit:

return chunks
    .Where(c => c.Score >= MinRelevanceScore)
    .OrderByDescending(c => c.Score)
    .GroupBy(c => c.DocumentUri)
    .SelectMany(g => g.Take(1))   // Take(2) is reasonable for multi-section pages
    .Take(MaxChunksForSynthesis);

That one change dropped the same query from roughly 19,700 tokens to around 12,000 and took a meaningful chunk off the latency. Take one per document if you want maximum diversity, two if a single chunk feels too thin for long pages. The point is that without this step you are silently sending whole pages to the model and paying for it in both latency and answer quality.

~

Lesson 14: Before You Blame the Model, Log What You Are Sending It

When a RAG query is slow, the instinct is to suspect the model. I spent time convinced the problem was chunk size, then convinced it was the synthesis model being too slow. Both were wrong, and I only found that out by adding diagnostic logging to the synthesis step: number of chunks retrieved, total token count going to the model, and the document URIs of those chunks.

The logs made the real cause obvious in seconds, and it was the duplicate-page problem from the previous lesson. Token volume was the variable that mattered, and I had been guessing at everything except the one number that would have told me.

The habit worth forming is that token count is the first thing you log in any slow LLM pipeline, before you touch the model choice or the chunking parameters. The model is rarely the bottleneck. What you are feeding it usually is, and you cannot reason about that without measuring it.

~

Lesson 15: Count Your LLM Calls Per Request

While I was logging token counts I found something I was not looking for. A single query was making three separate synthesis calls, not one, and one of those calls was receiving roughly 62,860 tokens because it ran through a code path that bypassed the chunk-cleaning logic entirely.

I had been reasoning about the pipeline as though one query meant one model call. The mental model was wrong, and the cost of that was multiplied latency that no amount of tuning the single call I knew about would have fixed.

Instrument the number of model calls per request as a first-class metric, the same way you instrument token count. A pipeline that quietly fans out into multiple synthesis calls, or has an older path that skips your guards, will not show up in any single-call profiling. You have to count.

~

Lesson 16: A Cache Is Only Worth Having If You Can Prove It Hits

I added a semantic cache to the query path: embed the incoming query, check it against recently logged queries with a vector search, and if something scores above a similarity threshold, return the cached answer instead of hitting the model. Sensible idea, and it appeared to work.

It was dead code. Two problems compounded. The cache hits were logged at debug level, invisible at the default log level, so I had no way to see whether it was ever hitting without reconfiguring logging. And the similarity search used a different score normalisation from the one used elsewhere in the pipeline, which meant the threshold I had set was almost certainly never being met. The cache looked alive and did nothing.

The lesson is that a cache you cannot observe is a liability, not an optimisation. Log hits and misses at a visible level from the first day, and verify that the scores your threshold compares against are on the scale you think they are on. An optimisation you cannot confirm is firing is just untested code sitting in the hot path.

~

Lesson 17: More Context Is Not a Free Optimisation

I had an MCP tool returning paraphrased API documentation through a RAG pipeline, and the results were modest. My theory was that paraphrasing was losing exact signatures, so I built a second tool that skipped synthesis and returned raw type definitions straight from the package: precise, machine-readable, no translation loss. It felt obviously better.

The benchmarks said otherwise. Measured against compile rate, render rate, and hallucination count, the raw type definitions made code generation worse. A single request pulled in tens of kilobytes of type information, and that volume displaced the model’s attention and introduced noise rather than grounding it. Models adapt better from a close working example than they interpret from a specification, and interpretation is exactly where hallucinations creep in. More context is not a free optimisation; it has a cost in attention, and on smaller models and simpler tasks that cost can outweigh the benefit.

There are two lessons here. The narrow one is to prefer retrieving a working example the model can adapt over injecting raw specifications it has to interpret. The broader one is that without benchmarks I would have shipped a tool that actively hurt results and told people it was an improvement. Feeling useful is not the same as being useful, and the only thing that tells them apart is measurement.

~

Part 3: Operations & Production

Getting a RAG pipeline working is not the same as keeping it working. The operational problems are slower to surface and harder to diagnose. Status flags that lie. Indices that drift without anyone noticing. Jobs that die silently at 3am. Health checks that can fail without telling you. These lessons are the ones you only learn once the pipeline has been running long enough to accumulate history, and they are the ones I would build for from day one if I were starting over.

~

Lesson 18: Audit Your Index Mappings Before You Write Any Query Logic

This one cost me time I did not need to spend. The content and vector indices in my setup were written by different services at different times. They ended up with different field naming conventions: camelCase in one, a legacy capitalisation artifact in the other (uRL instead of url).

The admin tool detects the field structure at runtime by inspecting the index schema on startup rather than assuming it. This is extra work, but it is much better than having queries silently return nothing because a field name does not match what you hardcoded.

The lesson is to audit your index schema before you write any query logic, and to treat field names as something that can vary rather than something you can assume.

~

Lesson 19: Batch Your Index Queries

The first version of the admin tool’s results table made an individual vector count query per row. A 25-row results page fired 25 separate requests to the index. Page load time was painful.

Switching to a single batched request that carried all 25 sub-queries cut the load time by around 80%. If you are building any kind of admin or inspection interface on top of a vector store, batch your queries wherever the store supports it rather than making per-row individual calls. Most engines offer some form of multi-query or bulk request for exactly this reason, and the round-trip savings add up fast.

~

Lesson 20: Synchronous Re-Vectorisation Is Fine for Admin Work

I considered adding streaming progress updates for the re-vectorisation operation in the admin tool. A document with 40 chunks means 40 embedding API calls, plus bulk indexing, which takes 20 to 40 seconds.

I did not add streaming. The reason is that for a one-at-a-time admin operation, a 30-second wait is acceptable and the simplicity of keeping the HTTP request open is worth it. I added exponential backoff for when the embedding provider’s rate limits kick in (2, 4, 8, 16, 32 seconds) and left it synchronous.

If I were processing hundreds of documents at once I would revisit this. For the admin workflow I actually have, synchronous is fine.

~

Lesson 21: Do Not Trust a Single Status Flag as Ground Truth

Most RAG pipelines end up with some boolean or status field that records whether a document has been vectorised. It is a convenient thing to query against, and that convenience is exactly what gets you into trouble. The flag describes what you believe happened, not what is actually in the vector index, and those two things can diverge.

The way they diverge is through partial failure. If a bulk write to the vector index succeeds for some chunks and then fails on a later one, the error path tends to mark the whole document as not vectorised. Now the vector index holds some chunks for that document while your status field insists it holds none. The two have quietly disagreed and nothing tells you. Worse, the document shows up in your non-vectorised report, so you retry, hit the same failing chunk, the flag flips back to not vectorised, and you are exactly where you started. It never clears on its own.

The lesson I took from this is that a status flag is a cache of an answer, not the answer itself. Do not treat it as ground truth. Periodically cross-reference it against the actual contents of the vector index and reconcile any mismatch, and when something looks stuck, go and count the real chunks rather than trusting the field. And of course, fix the underlying failure too, because a flag that keeps flipping is usually pointing at a real problem upstream, not just a bookkeeping error.

~

Lesson 22: Build an Audit for Orphaned Flags

The inverse problem also exists, and it is sneakier because nothing errors. A document can be marked as vectorised while the vector index holds no chunks for it at all. This happens when content is soft-deleted and the cleanup job does not run, or when an index migration goes partially wrong and leaves the flag set without the data behind it. It is the same divergence as the previous lesson, just pointing the other way.

These documents are invisible to users, but they skew your health metrics and cause confusing retrieval results when you expect chunks to be there and they are not. A dedicated audit that cross-references your document records against the actual vector index and surfaces the mismatches in both directions is worth building as a first-class part of the pipeline. It is the kind of thing you do not know you need until the numbers stop adding up, and once they do, having the audit already in place is the difference between a five-minute answer and an afternoon of poking at the index by hand.

~

Lesson 23: Automate a Daily Health Email

An admin dashboard is genuinely useful, but only when you remember to open it, and the whole point of these lessons is that drift happens silently. So I added a daily email that summarises the key stats: total documents, zombie count, non-vectorised count, and the orphaned flags from the previous two lessons. Now I notice drift without having to go looking for it, which is exactly when you want to notice it.

I kept it simple on purpose. A few headline numbers, a table of anything currently in a bad state, and a red flag at the top if the API was unreachable when the report ran. The one rule I gave myself is that the email should always arrive. If it does not turn up one morning, that absence is itself a signal that something is wrong, either with the pipeline or with the reporting around it. A health check that can fail silently is not really a health check.

~

Lesson 24: Treat Ingestion as a Batch Workload, Not a Web Workload

My ingestion crawler died overnight with no error. No exception, no stack trace, no shutdown message, the log just stopped mid-run on a routine line. It turned out the process had been killed by the operating system under memory pressure: a long single-process run, full of large transient allocations, sharing a small instance with several web apps. When the instance sat near its memory ceiling, a normal allocation spike from the crawler was enough to get it terminated, and a non-essential background job is exactly what the OS reaches for first.

The deeper issue is a mismatch. A RAG ingestion pipeline holds full HTML documents, parsed DOM trees several times the source size, chunked text with overlap, embedding vectors, and buffered index batches. Most of those allocations are large enough to land on the large object heap, which is not compacted by default, so a long-running ingestion process grows its working set even when logical usage looks stable. That is a batch workload’s memory profile running on infrastructure shaped for request-response web apps, and the two do not fit.

Two takeaways. First, “no error” is itself a diagnostic signal: a clean kill with no exception and no crash dump is the platform telling you it terminated the process for you, and the next question is always why. Second, give ingestion infrastructure that matches its shape. Stream responses instead of buffering them, process and discard documents one at a time rather than accumulating them, flush index batches small, and do not co-tenant a multi-hour batch job with customer-facing web apps. Better still, chunk the run into smaller triggered invocations so a failure has a small blast radius and you get restart safety for free.

~

Lesson 25: Monitor Index Freshness, Not Just Job Status

The silent death in the previous lesson had a second sting. A job-success alert would not have caught it cleanly, and even if it had, job status is the wrong thing to watch. What actually matters is whether the data in the index is current, and that is a different question from whether the last run exited zero.

This is the classic operational gap in RAG. The job can succeed while the index quietly goes stale, or fail in a way that leaves the index partially updated, and a model retrieving stale or partial data will confidently produce wrong answers. Unlike a 500 error, nobody notices until enough users complain, by which point the trust damage is done.

Surface a “last successful ingestion” timestamp as a metric and alert when it goes stale, rather than alerting only on job failure. A staleness alert catches both outright job failures and the subtler data-quality drift that a success-or-failure signal misses entirely. It sits alongside the health email from Lesson 23 but measures a different axis: that one tells you the index is internally consistent, this one tells you it is current.

~

Lesson 26: Design for the Long Run from the Start

A few of these lessons are things I only learned because I did not do them early enough, and if I were starting over they are the ones I would build in from day one rather than retrofit under pressure.

I would build the admin tooling earlier. It felt like a nice-to-have when I started and became obviously essential once the indices started drifting. The visibility it provides is not optional, it is part of the pipeline, and every engagement has eventually reached the point where I wished it had been there sooner.

I would design for re-indexing from the start. Chunking strategies change and embedding models improve, so the ability to re-index everything without re-crawling is worth designing for early. Deterministic chunk IDs and the separation of content and vector storage are both things I retrofitted rather than planned for, and retrofitting them is more painful than building them in would have been.

I would not skip authentication even for internal tools. An ingestion pipeline running on a trusted network with no API authentication is fine right up until it is not, and it is a maintenance debt either way. Adding it later is more work than adding it at the start. The common thread across all three is that the cost of these decisions is invisible at the beginning and obvious by the time you are keeping the thing healthy in production, which is exactly the theme running through everything above.

~

Closing Thoughts

RAG is not hard to get started with. It is hard to keep working well over time.

The tutorials give you the happy path. The reality involves content drift, index degradation, chunking regressions, and embedding model updates. The tools and patterns I have described here are all responses to real problems I encountered, not theoretical concerns.

If you are building RAG systems for production use, invest in the observability and tooling early. The code that keeps the pipeline healthy is as important as the code that runs it.

The ingestion pipeline and admin tool patterns I keep rebuilding across these engagements are not currently in a public repository, but if there is enough interest I am happy to write them up in more detail and share a reference implementation.

~