Context Engineering: How to Structure a Codebase So AI Agents Actually Help

Why agents struggle in some codebases#

An agent starts each session knowing nothing about your project. It has to work out how to install dependencies, run tests, where things live, and which of three similar patterns is the "right" one. Every guess is a chance to be wrong. Common causes of bad agent output:

• No clear build, test or lint commands.
• Two or more ways of doing the same thing, with no note on which is current.
• Unwritten rules ("never import from that folder") that live only in people's heads.
• No tests, so the agent cannot check its own work.
• Vague names like utils2.js or helpers/misc.

The fix is the same thing that helps a new human teammate: make the project explain itself.

Instruction files: AGENTS.md and CLAUDE.md#

An instruction file is a Markdown file the agent reads at the start of every session. Think of it as a README for agents, which is exactly how the AGENTS.md site describes it.

You do not need to maintain both. According to the Claude Code docs, Claude Code reads AGENTS.md when your repository has no CLAUDE.md. If you want Claude-only extras, create a CLAUDE.md that imports the shared file:

@AGENTS.md

## Claude Code only
- Use the project's /review command before opening a PR.

The @path syntax pulls the other file into context, so every tool reads the same core instructions.

A good AGENTS.md, section by section#

Here is a complete example for a Next.js portfolio site, similar to the one this blog runs on:

# Portfolio site

Next.js 16 (App Router) portfolio and blog. React 19, Tailwind CSS v4, JavaScript (no TypeScript).

## Commands
- Install: `npm install`
- Dev server: `npm run dev` (http://localhost:3000)
- Build: `npm run build` (must pass before every commit)
- Lint: `npm run lint` (0 errors required; warnings are OK)

## Project structure
- `src/app/` routes. Pages are server components unless they need state.
- `src/content/blogs/` one .jsx file per post; metadata in `metadata.js`.
- `src/components/blog/` shared post building blocks (Prose, CodeBlock, Faq).
- `src/data/` static data (projects, experience). Edit data here, not in components.

## Conventions
- New blog posts use the components in `src/components/blog/`; do not copy layout code into posts.
- Use the `@/` import alias, never long relative paths.
- Colours come from CSS variables in `globals.css`; do not hard-code new colours.

## Testing changes
- Run `npm run build` and `npm run lint` after changes.
- For UI changes, describe what to check in the browser.

## Do not
- Do not add new npm dependencies without asking.
- Do not edit `.env.local` or commit secrets.
- Do not rename blog slugs: they are public URLs.

Why each section is there

• One-line overview: the stack and its versions stop the agent from writing code for the wrong framework version.
• Commands: the most valuable lines in the file. With them the agent can build, lint and test without guessing.
• Structure: where things live, in one line each, so the agent goes straight to the right folder.
• Conventions: the rules a new teammate would otherwise learn in code review.
• Do not: boundaries with the reason implied. "Slugs are public URLs" explains why renaming is dangerous.

Keep it short and current#

• Aim for under 200 lines. The Claude Code docs recommend this for each CLAUDE.md, because longer files use more context and reduce how well instructions are followed.
• Remove contradictions. If two instructions disagree, the agent may pick either one.
• Don't repeat the code. Skip anything the agent can find by reading files. Explain the why and the non-obvious instead.
• Update it in the same pull request that changes a command, folder or convention.
• Start fast: in Claude Code, /init generates a first draft by analysing your codebase. Treat it as a starting point and edit it down.

Scope instructions to where they apply#

Nested files in larger repos

In a monorepo, put a short instruction file in each package. AGENTS.md uses the file closest to the code being edited. Claude Code combines files from the repository root down to the folder it is working in, and loads a subdirectory's file when it starts working there.

repo/
├── AGENTS.md            # shared: commands, PR rules
├── apps/web/AGENTS.md   # Next.js specifics
└── apps/api/AGENTS.md   # Express specifics, database rules

Path-scoped rules in Claude Code

Claude Code also supports rule files in .claude/rules/ that load only when Claude works on matching files (docs):

---
paths:
  - "apps/api/**/*.js"
---

# API rules
- Validate every request body with the schemas in apps/api/schemas.
- Return errors as { error: string } with the right status code.
- Never send err.message to the client.

Your main file stays short, and the API rules only use context when they are relevant.

Context beyond instruction files#

An instruction file can only point the way. The codebase itself is most of the context.

Tests: the agent's feedback loop

A fast, reliable test command is the single biggest boost to agent quality. With it, the agent can make a change, run the tests, see the failure and fix it, all before you look. Without it, every mistake reaches you.

• Make one command run the relevant tests quickly, and document it.
• Keep tests deterministic. Flaky tests teach the agent to ignore failures.
• Ask for a failing test first when fixing a bug, then the fix.

Names and structure

• Descriptive file names (formatDate.js, not utils2.js) help the agent find code by searching.
• One clear pattern per job. If you are migrating from an old pattern, say so in the instruction file.
• Group by feature when it helps: everything for "billing" in one place is easy to find.

Decision records

Short architecture decision records (ADRs) explain why things are the way they are: "We use server components for data fetching because…". Link the folder from your instruction file so the agent reads them when it needs the background.

Enforce what must never break

Instruction files are context, not configuration. The agent will usually follow them, but nothing guarantees it. For rules that must always hold, use tools that enforce them: linters, formatters, type checks, CI, and in Claude Code, hooks that run before or after a tool call.

Prompting with good context#

The instruction file sets the baseline. Each task still benefits from a clear request:

Strong prompts name the files, describe how to verify the result, and set limits.

Context engineering checklist#

• An AGENTS.md (or CLAUDE.md) under ~200 lines, with exact build, lint and test commands.
• A short map of the folder structure.
• Conventions and "do not" rules written as specific, checkable instructions.
• Nested files or path-scoped rules for different parts of a large repo.
• A fast, reliable test suite the agent can run.
• Descriptive names and one clear pattern per job.
• Decision records for the important "why" questions.
• Linters, CI and hooks for the rules that must never break.
• Instruction files updated in the same PR as the changes they describe.

Official references#

• AGENTS.md: the open format and list of supported agents
• How Claude remembers your project: CLAUDE.md, AGENTS.md, imports and rules
• Hooks guide, Claude Code documentation
• Architecture decision records