Prompt Engineering

What is Prompt Engineering ?

Prompt Engineering is the practice of designing inputs that guide an LLM toward producing accurate, useful, and reliable outputs. Even though modern LLMs are powerful, their performance depends heavily on how we communicate with them.

Think of prompt engineering as learning to speak the model’s language — giving context, examples, instructions, and constraints so the model can deliver its best work.

Core Prompting Techniques

Prompt engineering is The Foundations of Control, it begins with a few foundational techniques that give you direct influence over how the model behaves.

Below are the widely used prompting techineques, let’s break them down clearly.

• Zero-shot
• Few-shot
• Chain-of-Thought (CoT)
• Instruction templates

Zero-Shot Prompting: Just Tell the Model What You Want

A zero-shot prompt tells the model what to do in plain language, that is the user didn’t provided any examples - only instructions.

This works because the Modern models know how to generalize without examples because of their massive training data and instruction tuning.

Example Prompt:
“Explain JSON to a beginner in simple terms.”

This approach works for:

• Quick Tasks
• Simple Explanations
• Direct Q&A
• Rewriting or Summarizing

Few-Shot Prompting: Show, Don’t Tell

Few-shot prompting means giving the model examples of what you want, so it can follow the pattern. In this approach we provide 1 to 5 examples of “input and expected output” pairs inside the prompt.

This worked with because LLMs perform in-context learning — they mimic patterns without retraining.

Example Prompt:

# Prompt
Convert the sentence into a polite request.

Example 1:
Input: "Send me the report."
Output: "Could you please send me the report?"

Example 2:
Input: "Get me the file."
Output: "Could you please share the file?"

Now convert:
"Bring my notebook."

This approach fits best for following use-cases:

• formatting tasks
• structured outputs
• translation or tone changes
• working with inconsistent model behavior

Chain-of-Thought (CoT)

LLMs can solve complex problems more accurately if asked to explain their reasoning before answering.

In CoT prompting we explicitly tells the model to reason step-by-step, enforcing the Model to reanalyse the response and improvise it.

This works because, CoT activates the model’s internal reasoning pathways, dramatically improving performance for:

• Math
• Logic
• Multi-Step Tasks
• Planning
• Data Analysis

Example Prompt:
“Solve this problem step-by-step: If a train travels 120 km in 3 hours, what is its average speed?”
“Think aloud and show each step before the final answer.”

By default, CoT outputs the entire reasoning. But If you want reasoning internally but not shown, you can use hidden reasoning prompts.

This is called self-reflection prompting and is supported well by frontier models.

Hidden Reasoning Prompt:
“Think through the problem step-by-step internally, but only output the final answer.”

CoT prompting is best suitable when:

• Any problem requiring reasoning
• Multi-step workflows
• Ambiguous tasks

Advanced Optimization

When working with LLMs in real-world apps, you often care not only about what the model delivers, but also how efficiently it delivers.

Optimization helps you save time, cost, and computing resources, while maintaining output quality by turning the same model much Faster, Cheaper & Smarter.

One of the most effective optimizations is Prompt Caching.

Prompt Caching

The idea behind prompt cahcing is “Don’t Repeat Work the Model Already Did”.

Every prompt you send contains - instructions, context or history and examples or data and if you resend the same prompt structure many times (for example, for many different user inputs), the model ends up re-processing much of the same text over and over, wasting compute and time.

Prompt caching helps avoid that repeat work by storing the “static” part once, then reuse when possible.

Using caching in production quality LLM systems, often leads to:

• 20–50% faster responses (lower latency)
• 25–70% reduction in compute / token cost
• Smoother performance under heavy loads

Caching becomes essential, if the appliation involves repeated context (like the same document set, instructions, or history), like:

• RAG systems
• Chatbots
• Agent workflows
• Long reasoning pipelines

There are two popular ways to cache the repetative data.

Implicit Caching (Provider-Side)

• Some LLM providers automatically detect repeated prompt parts (like system instructions or long context) and cache them.
• From Developer side: no setup needed.
• Benefit: faster response and lower billed tokens.

Insight:
Some providers (OpenAI, Anthropic) automatically cache: system prompts, long instructions & long context messages.

Explicit Caching (Developer-Side)

• Developer controls what gets cached (embeddings, context chunks, fixed instructions, etc.).
• Use common caching tools or libraries like Redis, LangChain caching layers in LLM frameworks, LlamaIndex caching or Document-indexing tools.
• Best for applications where you reuse large context repeatedly — e.g. knowledge bases, chatbots, RAG pipelines.

Sampling Parameters: Creativity vs Accuracy

LLMs do not always produce a single fixed answer. Instead, they sample from many possible next tokens using probability distributions.

By adjusting sampling parameters, you can make the model:

• More creative
• More predictable
• More formal or more casual
• More random or more strict

The two key parameters matter the most are Temperature and Top-p.

1. Temperature

A high temperature makes the model more creative, adding variety and expression, while a low temperature makes the model direct, factual, and consistent.

• Use a low temperature for tasks like explanations, coding, math, and customer queries.
• Use a higher temperature for storytelling, marketing, and brainstorming.

Think of it like adjusting the “creativity dial” of the model.

Rule of thumb:
→ Lower temperature for accuracy.
→ Higher temperature for creative writing.

2. Top-p

Top-p, also called “Nucleus Sampling”, looks at the cumulative probability of the next token.

If Top-p = 0.2, the model chooses only from the top 20% most likely words. It helps control randomness similar to temperature.

Insight:
→ For most tasks, a value between 0.5 and 0.95 gives natural, well-structured sentences.

Choosing values wisely

Temperature and Top-p can be used together, but too much randomness can make output unstable.

Most production workloads use, which give reliable results without sounding robotic.

• Temperature: 0.2–0.5
• Top-p: 0.8–0.95

Note:
LLM providers often recommend using temperature or top-p, not both at high values.

Token Budgeting

Every model interaction is billed based on the number of tokens processed, which is calculated as “prompt tokens + output tokens”.

So, larger the prompts or longer responses will quickly increase the cost and slow down the system. Managing the token budget helps keep the system fast and affordable.

In real-world application Tokens matter more than we think as every extra token means - more compute time, more cost, higher latency.

Tips for Better Token Management

Efficient token use often lowers usage cost by 30–60% in real applications.

• Keep your system instruction short.
• Avoid repeating long text unnecessarily.
• Summarize older parts of the conversation when they’re no longer needed.
• When using RAG, insert only relevant sections — usually short chunks are better.
• Use output length limits (max_tokens)

You can also set a max_tokens limit so the model never exceeds your desired output length. This keeps responses focused and prevents cost surprises.

Insight:
→ Good token budgeting can reduce cost by 50%+.

Output Constraints: Make Models More Reliable

LLMs are flexible, but that can sometimes lead to unpredictable or messy responses. To make outputs consistent, clear, and easy to process, you can specify exact formats in the prompt.

A simple instruction like “Respond using the following format…” is often enough.

• “Respond in JSON.”
• “Use bullet points only.”
• “Follow this template…”
• “Answer using a table with columns X, Y, Z.”

Insight:
→ Models follow constraints surprisingly well when formatting is part of the prompt structure.