Context Engineering in AI: The Secret Sauce for Better Models

By gerald, 9 July, 2025
AI photo

Photo by Google DeepMind: https://www.pexels.com/photo/an-artist-s-illustration-of-artificial-int…

As AI models like GPT-4, Claude, and Llama become more powerful, simply crafting good prompts isn’t always enough. To truly harness their potential, you need context engineering—a structured way to provide models with the right background knowledge, constraints, and framing.

AI photo
Photo by Tara Winstead: https://www.pexels.com/photo/robot-pointing-on-a-wall-8386440/

In this post, we’ll break down:
✔ The difference between prompt engineering and context engineering
✔ Why context matters for software engineers & DevOps
✔ How to build a kick-ass context for AI systems
✔ External resources to level up your AI workflows

Prompt Engineering vs. Context Engineering

Prompt Engineering

  • Focuses on crafting the immediate input to the AI.

  • Example:

    • "Write a Python function to sort a list in descending order."

Context Engineering

  • Focuses on shaping the AI’s understanding before it processes the prompt.

  • Example:

    • "You are an expert Python developer with 10 years of experience. Your task is to write clean, efficient, and production-ready code. Here’s the problem: [insert problem]."

Key Difference:

  • Prompt engineering = What you ask

  • Context engineering = How you frame what you ask

Why Context Engineering Matters for Engineers & DevOps

1. Better Code Generation

Without context, AI might generate naive or insecure code. With context:

  • "You are a senior backend engineer optimizing for performance. Write a Go microservice that handles 10K RPS with low latency."

2. Smarter Debugging & Log Analysis

Instead of:
"Why is my Kubernetes pod crashing?"
Try:
"You are a DevOps engineer troubleshooting a Kubernetes cluster. The pod logs show ‘OOMKilled’. Suggest optimizations for memory limits and requests."

3. Infrastructure as Code (IaC) Improvements

A generic prompt might give you a basic Terraform config. But with context:
"You are an AWS solutions architect. Design a fault-tolerant, multi-AZ Terraform module for an EKS cluster with autoscaling."

How to Create a Kick-Ass Context

1. Define the AI’s Role

  • "You are a principal engineer reviewing a pull request."

  • "You are a Site Reliability Engineer (SRE) analyzing system metrics."

2. Provide Background Knowledge

  • For coding: Include relevant libraries, frameworks, or constraints.

    • "Use Python 3.10, type hints, and prefer pathlib over os for file operations."

  • For DevOps: Specify cloud providers, tools, and best practices.

    • "Optimize this Ansible playbook for idempotency and minimal downtime."

3. Set Constraints & Guardrails

  • "Do not suggest eval() or shell injection-prone code."

  • "Follow the AWS Well-Architected Framework for this design."

4. Use Few-Shot Learning (Examples in Context)

Instead of just asking for a query, provide a schema and sample:

# Database schema: users(id, name, email), orders(id, user_id, amount)
# Example query: "Get users who spent more than $200"
SELECT u.name, SUM(o.amount) AS total_spent FROM users u JOIN orders o ON u.id = o.user_id GROUP BY u.name HAVING SUM(o.amount) > 200;
# Now write a query: "Find users who haven’t ordered in the last 30 days"

5. Iterate & Refine

Test different contexts and measure the quality of the output. Adjust based on:

  • Accuracy

  • Relevance

  • Compliance with best practices

First Example: Context Engineering for a React + Node.js App with GitHub Copilot

To maximize GitHub Copilot’s effectiveness when building a React frontend and Node.js backend, you need to provide clear, structured context. Below is a practical example of how to do this in Visual Studio Code (or Visual Studio with Node.js tools).

Scenario: Building a Task Management App

1. Define the Full-Stack Architecture (Context for Copilot)

Start by outlining the tech stack, dependencies, and requirements in comments:

Backend (Node.js/Express) – server.js

// BACKEND CONTEXT:
// You are building a REST API for a Task Manager app using:
// - Node.js 20 + Express
// - MongoDB (Mongoose for ODM)
// - JWT authentication
// - API Best Practices: RESTful routes, error handling, input validation

// Database Model:
// Task { title: string, description: string, completed: boolean, userId: ObjectId }
// User { username: string, password: string (hashed) }

// Required npm packages:
// express, mongoose, jsonwebtoken, bcryptjs, cors, dotenv
// API Endpoints Needed:
// - POST /api/auth/register (User registration)
// - POST /api/auth/login (JWT token generation)
// - GET /api/tasks (Protected route, fetch user's tasks)
// - POST /api/tasks (Create a new task)

Now, Copilot will generate more accurate CRUD logic:

// Example: Copilot suggests a secure login route
app.post('/api/auth/login', async (req, res) => {
    const { username, password } = req.body;
    const user = await User.findOne({ username });
    if (!user) return res.status(401).json({ error: 'Invalid credentials' });
 
    const isMatch = await bcrypt.compare(password, user.password);
    if (!isMatch) return res.status(401).json({ error: 'Invalid credentials' });

    const token = jwt.sign({ userId: user._id }, process.env.JWT_SECRET, { expiresIn: '1h' });
    res.json({ token });
});

Frontend (React) – App.jsx

// FRONTEND CONTEXT:
// You are building a React 18 frontend for a Task Manager app:
// - Libraries: React Query (data fetching), Axios (HTTP), React Router 6
// - UI: Tailwind CSS for styling
// - Authentication: JWT stored in localStorage
// - Required Pages:
// - Login / Register
// - Task List (fetch/add/delete)
// - Protected routes (redirect if unauthenticated)
 
// API Base URL: http://localhost:3000/api

Copilot generates context-aware React components:

function TaskList() {
    const { data: tasks, error } = useQuery('tasks', async () => {
        const token = localStorage.getItem('token');
        const res = await axios.get('/api/tasks', {
            headers: { Authorization: `Bearer ${token}` }
        });
        return res.data;
    });

    if (error) return <div>Failed to load tasks</div>;
    return (
        <div className="p-4"> {
            tasks?.map(task => (
                <div key={task._id} className="border p-2 my-2">
                    <h3>{task.title}</h3>
                    <p>{task.description}</p>
                </div>
            ))}
        </div>
    );
  }

Second Example: Context Engineering for Terraform + AWS with GitHub Copilot

Scenario: Deploying an EKS Cluster with Autoscaling

1. Define Infrastructure Requirements (Context for Copilot)

# TERRAFORM CONTEXT:
# You are an SRE deploying a production-grade AWS EKS cluster with:
# - Managed node group (auto-scaling from 2 to 5 nodes)
# - IAM roles for least privilege (EKS + EC2 permissions)
# - Network: Public/private subnets, NAT gateway
# - Security: SSH access only via bastion host
# - Tools: Helm for add-ons (metrics-server, aws-load-balancer-controller)

# AWS Provider Requirements:
# - region = "us-east-1"
# - Terraform version >= 1.5.0
# - Required providers:
#   hashicorp/aws = "~> 5.0"
#   hashicorp/kubernetes = "~> 2.0"

# Reference Architecture:
# https://docs.aws.amazon.com/eks/latest/userguide/eks-reference-architecture.html

2. Copilot Generates Terraform Based on Context

module "eks" {
    source = "terraform-aws-modules/eks/aws"
    version = "~> 19.0"

    cluster_name = "prod-eks-cluster"
    cluster_version = "1.28"

    vpc_id = module.vpc.vpc_id
    subnet_ids = module.vpc.private_subnets

    # Auto-scaling node group
    eks_managed_node_groups = {
        default = {
            min_size = 2
            max_size = 5
            desired_size = 2
            instance_types = ["t3.medium"]
            iam_role_additional_policies = {
                AmazonEC2ContainerRegistryReadOnly = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
            }
        }
    }
}

Third Example: Context Engineering for FastAPI with Copilot

Scenario: Building a Secure Inventory API

1. Define API Specifications (Context for Copilot)

# FASTAPI CONTEXT:
# You are building a production-ready inventory API with:
# - Python 3.11 + FastAPI
# - JWT authentication (OAuth2PasswordBearer)
# - Database: SQLAlchemy + PostgreSQL
# - Pydantic v2 for validation
# - Security: Rate limiting (100 reqs/min), CORS

# Required packages:
# fastapi, uvicorn, python-jose[cryptography], passlib, sqlalchemy

# Data Models:
# User (id, username, hashed_password)
# Item (id, name, quantity, owner_id)

# Endpoints Needed:
# POST /token (JWT login)
# GET /items/ (protected, paginated)
# POST /items/ (protected, admin-only)

2. Copilot Generates Secure Endpoints

from fastapi import Depends, FastAPI, HTTPException, status
from fastapi.security import OAuth2PasswordBearer

app = FastAPI()

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

# Copilot suggests this JWT dependency check
async def get_current_user(token: str = Depends(oauth2_scheme)):
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Invalid credentials"
    )
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        user = get_user(payload.get("sub"))       
        if user is None:
            raise credentials_exception
        return user
    except JWTError:
        raise credentials_exception

@app.get("/items/")
async def read_items(
    current_user: User = Depends(get_current_user),
    skip: int = 0,
    limit: int = 10
):

# Copilot auto-completes SQLAlchemy query
items = db.query(Item).filter(
    Item.owner_id == current_user.id
).offset(skip).limit(limit).all()
return items

External Resources

  1. OpenAI Cookbook - Techniques to Improve Reliability

  2. Microsoft’s Guidance on Contextual Prompts

  3. Anthropic’s Prompt Engineering Guide

  4. Google’s PaLM 2 Best Practices

Final Thoughts

While prompt engineering gets most of the attention, context engineering is what separates good AI outputs from great ones. By carefully structuring the AI’s understanding upfront, you can get more precise, reliable, and production-ready results, whether you’re coding, debugging, or designing infrastructure.

What’s your experience with context engineering? Have you found tricks that work particularly well? Share in the comments!

Have you enjoyed this post? Follow me for more on AI, DevOps, and software engineering best practices! 🚀

Tags

Comments