No reranker can find what ingest threw away.

Back to stages 1-2. How you cut and encode text decides the ceiling on everything downstream.

Spaced recall (mix of L1-L3): (1) which stage does plain search skip? (2) which search nails an exact order id? (3) which encoder runs query and doc together? Answers: synthesize · BM25 · cross-encoder. Now we go upstream.

The one idea: Retrieval can only return chunks that exist. If ingest splits a document badly, the answer gets cut in half and no amount of hybrid search or reranking can reassemble it. Chunking is the most decisive, least glamorous lever in the whole pipeline.

Why you chunk at all

You do not embed whole documents. You embed chunks: paragraph-sized slices. Two reasons. First, an embedding is a fixed-size vector, so a 50-page doc squashed into one vector loses all detail. Second, you want retrieval to return the relevant paragraph, not the whole file, so the LLM's limited context fills with signal, not noise.

The size and overlap rules of thumb

The defaults that work for roughly 80% of cases, before you tune anything:

Knob	Default	Why
Chunk size	~400-512 tokens	Big enough to hold one idea, small enough to stay specific
Overlap	10-20% of chunk size	A sentence spanning a boundary survives in both chunks
Method	recursive character split	Splits on paragraphs, then sentences, then words. Cheap, robust

Smaller chunks sharpen fine-grained detail retrieval; larger chunks capture broad themes. Overlap exists so a fact sitting on the seam between two chunks is not lost from both.

When to climb the ladder

Fancier chunking has a cost. Climb only when your retrieval metrics say you need it:

Recursivedefault. Split on structure. Start here.

Heading-awarerespect doc structure (markdown, sections)

Semanticgroup sentences by embedding similarity

A January 2026 analysis found plain sentence chunking matched expensive semantic chunking up to ~5,000 tokens, at a fraction of the cost. Translation: do not pay for semantic chunking until you have proven recursive is failing you. (Lesson 1's ladder, applied to ingest.)

The embedding choice

An embedding model turns each chunk into its vector. The model decides what "similar meaning" means, so it sets the quality ceiling for all vector search. Three things matter: its dimensions (vector length), its context window (max chunk it can read, which should match your chunk size), and whether it is multilingual.

Connect it back, and a Rejig flag: DuckyCutz runs Thai, English, and Russian. A monolingual English embedder would mangle Thai retrieval. For any Thai-Rejig KB, a multilingual embedding model is not optional. gbrain lets you pick from 12+ providers (OpenAI, Voyage, ZeroEntropy, Ollama) precisely so you can match the model to the language and budget.

Retrieval practice

Why give chunks a 10-20% overlap?

A perfect reranker still returns a half-answer. Likeliest cause?

A Thai-language client KB returns nothing useful. First thing to check?

Read this next (primary source)

★ Best Chunking Strategies for RAG (Firecrawl). Current, practical, with code. ~12 min. Pair with Weaviate's chunking guide for the size/overlap numbers.

Ask your teacher. Try: "what chunk size would you use for DuckyCutz call transcripts?", "which multilingual embedder is cheapest for Thai?", or "how does Graphiti chunk, given it stores a graph not documents?" (good question — graphs change the chunking story, which is Lesson 5).

← Lesson 3: Reranking Next → Lesson 5: Knowledge graphs (the fourth signal)