The hard problem is evidence

The hard problem is deciding what counts as evidence, and refusing to answer from anything else.

Three unrelated projects I've been following are quietly built on that sentence, even though none of them state it out loud. A blast-radius aware code review tool built on Gemini File Search. An AI-assisted genealogy workflow that has mapped 1,200 people and 13 generations on a single branch. A perspective paper on retrieval-augmented generation in healthcare.

All three treat RAG as a verification problem first. Retrieval is the easy part. And all three skip the usual stack: no vector database, no embedding pipeline, no reranker, no weekend of infrastructure. What they use instead is hosted retrieval, two model calls, and a quality ladder for sources.

RAG without owning the R

The Go project is the sharpest example. The code review tool's author wanted a system where you paste a design proposal, an ADR, or a postmortem draft, and it answers with what is actually being proposed, the pattern underneath it, and the three places on the shelf where the team already ran into that shape. Grounded. With citations. Not "an LLM read your doc and had feelings about it."

The retrieval side is almost insultingly small. Gemini File Search is a hosted store: you create a store, upload documents, and Google chunks, embeds, and indexes them. Then you attach the store to a normal generateContent call as a tool, and the model searches it by itself, mid-answer, and hands back the chunks it used as groundingMetadata.

No pgvector. No Pinecone bill. No embedding model to pick and then regret.

The pricing is what makes this copyable. Storage is free. Search-time embeddings are free. You pay once at indexing time, at embedding prices, and the chunks the model pulls in are billed as ordinary context tokens on the call you were already making. On the free tier the store caps at 1 GB. The entire shelf, nine books plus essays plus postmortems converted to markdown, is 5.3 MB.

Key Numbers5.3 MB: the full corpus as markdown. Nine books, every long essay, sixty-odd internal posts. 71 files: what the corpus becomes after PDF conversion. 2 model calls: per question, total. That is the whole architecture. 1 GB: the free-tier store cap. The shelf uses about half a percent of it. 1.4 million tokens: what "we already learned this once" adds up to.

What you trade for that cheapness is control, and the trade is easy to evaluate.

Dimension	Hosted File Search	The classic stack
Chunking	Google chunks on whitespace with a token budget; you set maxTokensPerChunk and overlap	You build the chunker and debug it forever
Embeddings	Paid once at indexing; query-time embeddings are free	Paid per chunk, plus storage and compute
Query cost	Retrieved chunks bill as context tokens on a call you already make	Embedding queries plus a reranker cost per ask
Operations	None. One store per API key, free up to 1 GB	You run the pipeline, the database, and the reranker
The catch	The store pins to the key that created it; JSON mode plus a tool is not guaranteed	You own every failure mode, including the ones you haven't met yet

Search with the pattern, not the post

The idea I would keep even if I threw out the whole stack: embedding search finds things that sound alike, and that is usually the wrong target.

Paste a proposal that says "we should standardize on one MCP transport across all our agents" straight into retrieval, and you get back every chunk containing "MCP", "agents", and "standardize". Those are your own recent blog posts about MCP and agents. The retrieval returned a mirror where you needed precedent.

What a good reviewer brings instead is the shape of the situation with the nouns removed: "fragmented, incompatible implementations, then one open standard, then mass adoption." Search the shelf with that and the browser wars come back. Rickover on standardizing the nuclear navy comes back. A 2,300-year-old Legalist essay on uniform law comes back. Same pattern, different clothes.

The pipeline enforces this with two calls.

Call 1 gets the proposal plus a corpus index: a one-line summary of every file, generated once by gemini-3.5-flash-lite and keyed by file hash. That index is what lets the model aim at sources that exist instead of guessing. Call 2 gets the analysis, the search tool, and the instruction to search with the generalized queries, then draft. The raw document is in call 2's context, so the draft can quote it. But the search terms came from call 1, and they are about the pattern, not the subject.

Quick Take: The model should search for the shape of the problem, never the raw text, because raw-text retrieval returns a mirror of your own vocabulary instead of precedent.

Every claim carries its source

The two-call pipeline would be theater without the receipts. This is where the Go project gets serious.

The response carries candidates[0].groundingMetadata.groundingChunks, each with the file's display name and the retrieved text. Those get shown to the user in full, next to the draft. If the authority the review leans on is not in that list, it did not come from the shelf.

Prompts are not free text. They come from an AgentLaws lawbook: a folder of markdown where every instruction is a numbered law, grouped into chapters, versioned in git, and compiled once at startup. The model must return applied_laws, the numbers it relied on, and the pipeline resolves each one back to file:line in the lawbook. A number that does not resolve fails the run. That sounds pedantic until the first time the model confidently cites law 7.2.3 in a lawbook with six chapters.

The practical win: "why did it do that" has an answer that is a file and a line number, and "make it stop doing that" is a pull request against a markdown file, not archaeology in a Go string. The output contract sits at the end of the prompt, where the model weights it most. Tone rules go first, the JSON schema last.

The model also fills a checks block, a self-audit that the author does not trust. It still works as a second signal. Before that gets read, deterministic checks in plain Go run on every draft: length bounds, no markdown, at most one question, no "see point N" references, every cited law resolved. A failure gets appended to the user message as a plain list, the draft regenerates, and after two retries it ships anyway with the failures listed. A draft with a warning is more useful than a spinner that gave up. A third call acts as a reviewer pass with a "ship" or "revise" verdict, and a "revise" with rounds left sends the draft back through the loop.

Trust, but verify

The genealogy project reaches the same conclusion from a different direction, with a quality ladder that predates LLMs.

The GEDCOM format has a QUAY directive: the submitter's quantitative evaluation of the credibility of a piece of information, based on its supporting evidence. The official table defines four levels, and the genealogy author's interpretation is a small, clean policy.

QUAY	GEDCOM meaning	What the project maps it to
0	Unreliable evidence or estimated data	Not used
1	Questionable reliability: interviews, census, oral genealogies, potential bias	Not used
2	Secondary evidence, data officially recorded sometime after the event	Genealogy sites (Geni, Geneanet, similar)
3	Direct and primary evidence, or dominance of the evidence	Official civil or parish acts

The loop is simple: mine the data on genealogy sites, set it to QUAY 2, check whether the data also appears in an official record, and if yes, promote it to QUAY 3. If no, discard it. That is the same verification stance as the lawbook resolution, with a source rating replacing a line number.

When I shared this workflow, the pushback was predictable: AI can hallucinate. In theory, yes. In practice the objection loses most of its force once every answer carries a source pointer, and the answer gets dumped when the pointer is missing. The comment that landed hardest on that thread was about authority: an assistant that can search is not the same as an assistant you can trust with research. People liked the conservative match policy best, because one confident false merge can contaminate an entire branch of the tree far beyond the original record.

From PDF to searchable shelf

Before any of the clever retrieval, the corpus has to exist as text, and that step costs an evening even in a small project.

The first PDF tool the Go author tried, markitdown, produced output where every word was glued to its neighbor: "Theubiquityoffrustrating,unhelpfulsoftwareinterfaceshasmotivateddecadesofresearch". The embedding model does not know what that is, and neither does retrieval. pdftotext fixed the spacing and threw away every heading, so a 300-page book became one undifferentiated scroll. The winner was pymupdf4llm, which reads font sizes to decide what is a heading and keeps bold and italics. Forty-four real headings out of one essay, versus zero.

Headings matter more than they look like they should, because File Search chunks on whitespace with a token budget. A chunk that starts at a heading means something on its own. A chunk that starts mid-sentence in a wall of text is noise with an embedding attached.

The chunking config landed at 400 max tokens per chunk with 60 tokens of overlap. The documentation example is 200 and 20, which for a book feels like reading through a letterbox.

Ingest is a diff, not an upload. Every file gets a sha256. A manifest in SQLite records path, hash, and document id per store. Unchanged files are skipped. Changed files have their old document deleted first, then get re-uploaded. Vanished files get deleted from the store. Each upload is a multipart POST that returns a long-running operation, and you poll it until done before trusting the document id. Four uploads run in parallel. Serial, a 71-file corpus takes about ten minutes. Parallel, a few.

Archives, transcription, and the long tail

The genealogy project makes the point that the messy 10% of the corpus eats 90% of the effort. Archive sites differ along two axes: open versus account-required access, and bot protection. The lowest protection level answers to curl. The middle needs a full browser via a small Playwright project. The highest is Cloudflare-protected, where even the "I'm a human" checkbox sometimes fails. The only workaround the author found was Chrome in debug mode, which is fragile: anything that closes the browser ends the run.

Logins follow the same principle as the lawbook: never hand the assistant your credentials. For Cloudflare sites there is no sane alternative to Chrome debug mode. For everything else, log in once in a persistent Playwright profile, close the browser, and let the assistant resume that session later. Sessions expire, so refresh them before unattended runs.

Transcription of handwritten acts is where the author burned the most time. The assistant is careful, setting a placeholder where a human might infer a word. Transkribus, a paid service with 50 free transcriptions a month, output pure gibberish without per-document training, and the author will not use it again. kraken, a local OCR tool with era- and language-specific models, beat Transkribus but stayed below the assistant, so that skill got scrapped. The bright spot was Filae, a small French genealogy site whose transcription of French handwriting was second to none, until the service disappeared.

The workflow also loops. After confirming new ancestors in official records, the author returns to the genealogy sites, where other people's trees may already contain those people and skip a couple of generations of searching. Smart matches are dangerous: accept a bad one and unrelated people join the tree, corrupting a whole branch. The author stays conservative, and the side benefit is discovering spelling variants that uncover acts the assistant missed by searching only the obvious spelling.

When the stakes are medical

The healthcare perspective paper makes the same argument where the stakes are highest. RAG lets models generate more reliable content by retrieving external knowledge, and the authors analyze what that buys in equity, reliability, and personalization, plus what it costs to implement in medical scenarios.

Reliability maps to the QUAY ladder: a clinical claim should trace to a retrievable primary source, ideally with the resolve-or-fail discipline of the lawbook. Personalization maps to the two-call pattern: pull patient-specific context, guidelines, and formularies into the answer the way the code review tool pulls the proposal and corpus index into call 2. The uncomfortable one is equity. Retrieval returns whatever the corpus contains, and a corpus that underrepresents certain populations does not get fixed by a smarter model. The retrieval layer amplifies the bias. The corpus is a policy decision, not an implementation detail.

Then there is the privacy squeeze. The Go project is cheap because the store is hosted. Patient data in a hosted store is a compliance conversation your security team will finish for you, and the conclusion is usually on-prem or VPC-scoped retrieval. That is where the hosted approach stops working, and a healthcare RAG plan should assume it owns retrieval from day one.

Common Pitfalls

Five concrete traps from these projects, each with the fix.

1. Rotating API keys across a tool-attached call. A File Search store belongs to the key that created it. Create a store with key A, search it with key B from a different project, and it does not error helpfully. It is just not there. The free tier's per-minute caps push people toward key rotation, but a tool-attached call cannot rotate, because the store pins you. Give keys roles instead: a few store-owning keys, each with a complete copy of the corpus, and a longer list of rotating keys for calls with no tool attached. Indexing runs twice, and at embedding prices for a 5 MB corpus that is coffee money.

2. Treating 503 as a quota problem. Gemini returns 503 "high demand" when Google is busy. Switching keys does nothing except burn another key's quota. Wait and retry the same key a couple of times before falling over.

3. Putting the key in the URL. The key goes in the x-goog-api-key header, never the query string. In the URL, the first transport error prints your key into your own logs. Ask me how I know.

4. Trusting JSON mode plus a tool. responseMimeType application/json together with File Search worked most of the time, and then occasionally returned prose with a JSON object somewhere in it. The fix is boring: if the reply does not parse, resend once without JSON mode and pull the first JSON object out of the text. Log both attempts.

5. Storing the ingest manifest in the per-user session database. A new teammate's empty manifest makes every file on disk look new, so all 71 get uploaded again, and the old document ids are unknown, so nothing gets deleted. The store ends up with two of everything. Do it a few times and search quality quietly rots, because every query returns the same chunk repeatedly and crowds out the second-best hit. The manifest is a property of the store, not of whoever is asking questions today. Split the corpus database from the session database.

One thing to remember

All three projects, different domains, same architecture of trust: decide what counts as evidence, force the model to prove every claim against that evidence, and run deterministic checks before anything ships. Grounding is not a retrieval feature. It is a verification policy.

The Bottom Line

1. If you're building an internal knowledge tool and want to skip the vector database entirely, use a hosted search store with tool-attached retrieval. The corpus lives in the free tier, and retrieved chunks bill as context tokens on a call you were already making.

2. If your domain stacks crowd-sourced data on top of official records, genealogy, legal, audit, enforce a two-tier evidence ladder. Treat the crowd-sourced layer as secondary until it checks out against a primary record, and discard it when it does not.

3. If the wrong answer carries real harm, healthcare, finance, safety, don't ship RAG until every citation resolves to a file and line and deterministic checks gate the output. Plan for retrieval inside your compliance boundary from day one, because hosted stores and patient data do