Skip to main content

Prompt Engineering for LLMs

Master prompt engineering for large language models: from basic composition to Chain-of-Thought, few-shot, and advanced techniques.

Specifying a task through language

Prompt engineering changes a model's input without changing its trained weights. Instructions, examples, and context specify what to do and what a useful answer should contain. Their effectiveness depends on the model, task, and evaluation setup.

This lesson uses authored examples to explain the mechanisms. None of the figures calls a language model or reports an accuracy gain. Treat a proposed prompt as a hypothesis to test on representative inputs.

The recipe analogy

A recipe can specify ingredients, quantities, and a sequence of operations. Similarly, a prompt can specify the audience, task, reference material, and output format. More detail can reduce ambiguity, but irrelevant or conflicting requirements can make the task harder. Specific instructions do not guarantee factual accuracy.

Compare three authored instructions, then step through their recipe analogy.

Instructions specify a task

Compare authored recipes and prompts. Detail can reduce ambiguity; adding words does not guarantee a better result.

Step 1 / 4

Make pasta

    Authored equivalent prompt
    Write about AI.

    With minimal instructions, the chef (model) has to guess your preferences. The intended audience, scope, and format are unspecified.

    3 recipe steps · 3 prompt words

    This analogy does not run a model or measure output quality. Word counts are whitespace-separated words, not tokenizer counts.

    Prompt anatomy

    Different sections serve different purposes. A system instruction can request behavior or tone; reference context supplies material to use; demonstrations show an input-output pattern; constraints specify requirements; and the query identifies the current task. A role such as “expert” does not create expertise or certify the answer.

    There is no fixed attention percentage assigned to these sections. All their serialized tokens pass through the model's layers. The prompt information flow lesson separates possible causal connections from measured influence.

    Toggle and reorder the sections below to inspect the composed text. The counts measure included sections and words, not quality. Real chat templates encode roles separately, so the text builder is not a role-permission simulator.

    Assemble a prompt

    Edit, toggle, and reorder authored sections to inspect the resulting text. The counts describe structure, not prompt quality or model attention.

    Inspect and edit system instruction

    The system instruction sets the persona and behavioral guidelines for the model. It requests a tone and approach; compliance must be evaluated.

    Inspect and edit context

    Context provides the reference material the model should use. Without it, the model relies solely on its training data, which may be outdated or too general.

    Inspect and edit few-shot examples

    Examples demonstrate the exact format, style, and depth you expect. They demonstrate a desired input-output mapping.

    Inspect and edit constraints

    Constraints set boundaries on format, length, style, and content. They specify requirements that can be checked after generation.

    Inspect and edit query

    The query is the actual question or task. It should be clear and specific. Everything above exists to ensure this query gets the best possible answer.

    5 of 5 sections included · 91 whitespace-separated words

    Composed authored prompt
    System Instruction:
    You are a senior data scientist with 10 years of experience. You explain complex topics clearly using analogies and examples.
    
    Context:
    The following is a research paper abstract about transformer architectures and their application to computer vision tasks.
    
    Few-Shot Examples:
    Example:
    Input: "Explain backpropagation"
    Output: "Think of backpropagation like a teacher grading papers backwards..."
    
    Constraints:
    Keep your response under 200 words. Use bullet points. Do not include code snippets. Write at a college freshman reading level.
    
    Query:
    Explain how Vision Transformers (ViT) differ from traditional CNNs for image classification.
    This is a text composition exercise. Real chat APIs may encode system and user roles separately; moving text here does not change message privileges.

    Intermediate reasoning text

    In a causal language model, generated tokens become part of the prefix used for later predictions. Writing intermediate calculations can therefore make partial results available to subsequent steps. A direct answer can also contain several generated tokens; it is not generally a single forward pass for the whole response.

    Wei et al. (2022) evaluated chain-of-thought demonstrations on arithmetic, commonsense, and symbolic tasks, including PaLM 540B on GSM8K. Those experiments support results for their evaluated setups, not a universal improvement from requesting steps. Model behavior varies, and a fluent explanation can contain errors or fail to faithfully describe internal computation.

    The next figure compares an intentionally incorrect direct attempt with authored worked solutions. Step to the final answer and check the arithmetic; these examples are not sampled model responses.

    Inspect a worked solution

    All responses below are authored teaching examples. The direct attempts intentionally contain errors; this is not a model evaluation or an accuracy comparison.

    A train travels at 60 mph for 2.5 hours, then at 80 mph for 1.5 hours. What is the total distance traveled?

    Step 1 / 2

    Ready. Play or Next reveals the authored response.

      Intermediate text becomes available to later generated tokens, but a written explanation can still contain errors or fail to reflect the model's internal computation. Verify the calculation and final answer.

      Choosing an intervention

      Few-shot examples add demonstrations. A format request specifies a checkable output structure. Intermediate steps expose a proposed calculation. Self-consistency samples multiple candidate answers and aggregates them, adding generation cost. A role request changes wording and context but does not guarantee domain knowledge.

      Select a task and technique to see the exact instruction change and a task-specific validation method.

      What changes in the prompt?

      Choose a task and technique to inspect its concrete instruction. These authored transformations are not measured gains.

      Authored prompt
      A train travels at 60 mph for 2.5 hours and 80 mph for 1.5 hours. Find the total distance.

      19 whitespace-separated words · 0 example

      How to check the result: Check both products and their sum against the problem.

      Self-consistency requires a sampling and answer-aggregation procedure beyond prompt wording. Majority agreement can repeat a shared error; its cost and accuracy depend on the model, task, sample count, and decoding setup.

      No technique is best for every task. Compare a simple baseline with one change at a time, using the same checkpoint, test inputs, scoring rubric, and decoding settings. Record token use and latency alongside task performance. For tool workflows, evaluate actual tool results and error handling.

      Prompting mechanisms and tradeoffs

      Zero-Shot

      Mechanism
      Direct question with no examples or extra context
      Possible use
      Simple, well-defined tasks
      Limitations
      Struggles with multi-step reasoning, ambiguous tasks, or uncommon formats

      Few-Shot

      Mechanism
      Include representative input/output examples before the query
      Possible use
      Format-sensitive tasks, classification, structured output
      Limitations
      Token-expensive, examples may bias output, useful example count depends on the task and model

      Chain-of-Thought

      Mechanism
      Add "Let's think step by step" or provide step-by-step examples
      Possible use
      Math, logic, multi-step reasoning, analysis
      Limitations
      Increases latency and token usage, may produce plausible but wrong steps

      Self-Consistency

      Mechanism
      Sample multiple CoT paths and pick the most frequent answer via majority vote
      Possible use
      Math, code debugging, tasks with checkable final answers
      Limitations
      Requires multiple API calls, significantly higher cost, latency tradeoff

      Tree-of-Thoughts

      Mechanism
      Explore multiple reasoning branches, evaluate each, prune and continue the best
      Possible use
      Planning, puzzles, creative problem-solving
      Limitations
      Very token-expensive, complex orchestration, overkill for simple tasks

      ReAct

      Mechanism
      Interleave Reasoning and Acting steps, using external tools between thoughts
      Possible use
      Tool use, fact-checking, knowledge-grounded QA
      Limitations
      Requires tool integration, non-trivial error handling, can loop on failures

      These are mechanism descriptions, not a ranking or universal accuracy guarantee. Compare alternatives on the same held-out tasks with a fixed model and token budget.

      Common pitfalls

      Conflicting or unnecessary requirements

      Keep the objective clear and include the reference material needed to answer it. Test whether each extra example or instruction helps. Prompt length alone is not a quality measure.

      Ambiguous success criteria

      “Make it better” leaves the target unclear. “Rewrite this paragraph for a first-year student in under 100 words” supplies an audience and a checkable length requirement. Factual correctness still needs separate verification.

      Treating demonstrations as evidence

      A few examples can illustrate a pattern without representing the deployment distribution. Use held-out inputs to test generalization, and verify requested citations against their sources. A model repeating a claim or several samples agreeing does not establish truth.

      If you found this explanation helpful, consider sharing it with others.

      Mastodon