An airline’s customer assistant looked great in demos—until flight schedules started changing daily. The model spoke confidently, but it mixed old and new refund rules, forgot to cite sources, and sometimes granted refunds that didn’t exist. The fix wasn’t a bigger model. It was context engineering: getting the right evidence to the model in the right shape with the right rules.

The Real-Life Use Case

A traveler messages: “My flight was moved from 8:00am to 11:45am—am I eligible for a refund?” The correct answer depends on:

Everything is knowable, but only if the assistant pulls and uses the right context—today’s policy, this traveler’s booking, this route’s rules.

What Went Wrong (Before)

The Context Engineering Fix

We didn’t retrain the model. We changed how it receives and uses evidence:

  1. Contract (the operating rules): Define freshness windows, source tiers, tie-breaks, abstention rules, and output schema.

  2. Retrieval (policy-aware, not just relevant): Pull only eligible policies (correct airline, region, and effective dates), the ticket record, and the schedule change event.

  3. Shaping (atomic claims): Turn documents into timestamped facts (threshold=90, effective_date, exceptions list), each with a source ID.

  4. Reasoning (apply the contract): Rank by score → break ties by recency → prefer primary sources → check required fields → decide answer vs. ask-for-more.

  5. Validation (before user sees it): Enforce JSON schema, minimal-span citations, and an uncertainty check; refuse if key fields are missing.

A One-Page Contract (Drop-In)

System role: Airline refund assistant. Use only provided context.

Rules:
- Rank evidence by retrieval_score; break ties by newest effective_date.
- Prefer primary sources: "policy.md" > email > blog.
- Quote minimal spans; include source_ids for each quoted claim.
- If key fields missing (fare_class OR schedule_change_minutes), ask for them.
- If sources conflict, do not harmonize. Flag discrepancy and present both claims with dates.
- Refuse if evidence is outside freshness window (>= 60 days) unless explicitly marked "current".
Output JSON:
{
  "answer": "...",
  "citations": ["policy:2025-09-29#threshold", "pnr:AB1234#fare"],
  "missing": [],
  "uncertainty": 0.0-1.0,
  "rationale": "one concise sentence"
}

The Context Pack We Feed the Model

{
  "task": "Refund eligibility for PNR AB1234",
  "claims": [
    {
      "id": "policy:2025-10-02#threshold",
      "text": "Refund eligible if schedule change >= 90 minutes.",
      "effective_date": "2025-10-02",
      "retrieval_score": 0.94,
      "source_type": "primary"
    },
    {
      "id": "pnr:AB1234#delta",
      "text": "Original dep 08:00 → new dep 11:45 (change=225 minutes).",
      "effective_date": "2025-10-11",
      "retrieval_score": 0.91,
      "source_type": "system"
    },
    {
      "id": "pnr:AB1234#fare",
      "text": "Fare class: Basic Economy (B).",
      "effective_date": "2025-10-11",
      "retrieval_score": 0.88,
      "source_type": "system"
    }
  ],
  "policies": {
    "freshness_days": 60,
    "tie_break": "newest",
    "max_tokens_ctx": 3500,
    "required_fields": ["fare_class", "schedule_change_minutes"]
  }
}

Shaping: From Documents to Atomic Claims

This makes the model’s job simple: apply rules to a clean set of facts.

Validation: Guardrails That Run Before Output

A thin post-processor checks:

What We Measured (and Improved)

Within two weeks, grounded accuracy rose from 78% → 95%, citation precision from 62% → 93%, and unnecessary escalations fell by a third. Token spend dropped because the model stopped thrashing between irrelevant sources.

Rollout Playbook You Can Reuse

  1. Policy inventory: Centralize refund/return/eligibility rules with versioned dates.

  2. Contract first: Write the one-page rules + JSON schema and treat it like code.

  3. Shaper: Build a small service that emits atomic, timestamped claims with source IDs.

  4. Retriever: Add filters for tenant, region, and freshness before relevance.

  5. Validator: Enforce schema, citations, and uncertainty before showing output.

  6. CI evals: Record real tickets; replay them as “context packs” on every change.

  7. Dashboards: Track grounded accuracy, policy adherence, abstention quality, cost, and latency.

The Result

The assistant now answers, cites, and—when it must—asks for exactly the missing field (e.g., fare family or country of departure) instead of guessing. Agents trust it. Auditors can trace it. Customers get consistent outcomes. That’s context engineering: not bigger models, but better evidence, rules, and proof.

Takeaway

If you want a quick start: write the one-page contract, shape policy into atomic, dated claims, and enforce citations and uncertainty on output. Do this for any high-stakes workflow—refunds, warranties, fee disputes, benefit eligibility—and you’ll convert fluent text into reliable decisions that stand up in the real world.