Few-Shot Prompting Patterns for Getting Consistent Format From Any LLM
Few-shot prompting — giving the model 2-4 input/output examples before the actual task — is consistently one of the most reliable ways to get consistent format and style, especially for tasks the model doesn't naturally know how to structure. The technique is well-established in the research literature but the practical implementation details matter enormously. I've spent a lot of time optimizing few-shot example quality across different output types — JSON, classification labels, writing style, and structured analysis — and the patterns that work reliably are quite specific.
Writing Few-Shot Examples That Actually Transfer to New Inputs
The most common mistake with few-shot prompting: examples that are too similar to each other. If all your examples are about software companies and the real task involves healthcare, transfer quality drops sharply. Examples should demonstrate the format and reasoning pattern in diverse enough domains that the model learns the underlying structure, not a surface-level pattern tied to one subject matter. A rule I follow: use examples from 2-3 different domains if the actual task will be run on diverse inputs. For a sentiment classification task that will run on product reviews from multiple industries, my few-shot examples would include a software review, a food review, and a travel review — not three software reviews. The other critical factor: every example must be a 'clean' demonstration of the exact pattern you want. No hedging, no exceptions, no best-of-two-worlds examples. If your format shows bullet lists, every example uses bullet lists. Inconsistent examples teach inconsistent behavior.
For generative tasks (writing, summarization, analysis), the example outputs should be roughly the length you want from real outputs. If you show 500-word examples but want 100-word real outputs, specify the target length explicitly — examples anchor the model's length expectations strongly.
Use examples from 2-3 different domains to teach format, not subject-specific pattern
Every example must perfectly demonstrate the exact target format — no exceptions
Example output length anchors real output length — match examples to your target
2 high-quality examples beat 5 mediocre ones — quality dominates quantity
For classification: always show at least one example for each possible output class
Test format consistency by running 20 varied inputs before deploying few-shot prompts
Few-Shot for JSON and Structured Data Output
For reliably generating structured data (JSON, CSV, YAML), few-shot examples are more reliable than instruction-only prompting for most models. The prompt pattern: provide 2 complete input-output pairs showing exactly the JSON structure, then provide the real input. Key details that make this work: (1) show the JSON as valid, parseable output — not pseudo-code with ellipses, (2) include examples with edge cases like null values, empty arrays, and nested objects, because these are where models break format most often, (3) if the output has optional fields, show one example with and one without the optional field so the model knows both are valid. For GPT-4o specifically, combine few-shot JSON examples with the response_format=json_object parameter for near-perfect format compliance. Claude with XML output templates is slightly more reliable than few-shot JSON, but both approaches reach >95% compliance when implemented correctly.
The most common few-shot JSON failure: the model adds explanatory text before or after the JSON object ('Here is the JSON you requested:'). Add an explicit instruction: 'Output only valid JSON. No text before or after the JSON.' and this disappears. Without that instruction, ~30% of GPT-4o responses add preamble text that breaks JSON parsers.
Show full valid JSON examples, not pseudo-code with '...' placeholders
Include examples with null values, empty arrays, and nested objects
Show optional fields in one example present and one absent
GPT-4o: combine few-shot with response_format=json_object for near-100% compliance
Claude: XML output templates work better than few-shot for strict JSON compliance
Add: 'Output only valid JSON. No text before or after.' to every JSON generation prompt
Dynamic Few-Shot: Selecting Examples Based on the Input Query
Static few-shot examples (the same examples in every prompt) work well for consistent task types. For tasks with high input diversity, dynamic few-shot — selecting examples based on similarity to the incoming query — produces better transfer. The implementation with embeddings: pre-compute embeddings for a library of input-output example pairs; at runtime, embed the incoming query; find the top-K most similar examples by cosine similarity; include those K examples in the prompt. Libraries like LlamaIndex and LangChain have built-in retrievers for this. My implementation uses OpenAI's text-embedding-3-large for embeddings and a FAISS index for fast similarity search. With a library of 50 examples and dynamic retrieval, format and style consistency improved 40% over static 3-shot examples on a customer support classification task I worked on. The library size sweet spot is 20-100 examples — small enough to curate quality, large enough to cover diverse inputs.
Building the example library is the hidden cost: each example needs to be manually curated to demonstrate the exact desired output. A poorly curated example in a dynamic retrieval system is worse than a static bad example because it's retrieved for inputs similar to it, concentrating its negative effect. Quality control on the example library is non-negotiable.
Dynamic few-shot: embed incoming query, retrieve top-K similar examples at runtime
Stack: text-embedding-3-large + FAISS or Pinecone for similarity search
Example library size: 20-100 examples is the practical sweet spot
40% format improvement over static few-shot on diverse-input tasks
LangChain SemanticSimilarityExampleSelector or LlamaIndex retrievers for easy implementation
Every example in the library must be manually curated — quality control is mandatory