Coding agent in the repository: gates before the first PR

Giving a coding agent permission to open pull requests looks like a natural extension of autocomplete. It is not. The agent can now explore the repository, edit multiple files, run commands, and propose a change that someone may approve in a hurry because CI turned green.

The desired result is not to "trust the model more." It is to build a path where a small task can become a draft PR, but an unsuitable change cannot reach the protected branch just because it was written with great confidence.

The division of responsibilities is straightforward:

• AGENTS.md records context and boundaries for the agent;
• CODEOWNERS, required reviews, and CI provide the actual enforcement;
• a short audit reveals when failures are recurring;
• high-impact tasks remain with people from the start.

The minimum context before autonomy

A coding agent still works with limited context, may choose an API that does not exist, and may produce code that looks valid but does not meet the requirement. A huge context does not eliminate the first problem: the Lost in the Middle study found degradation when relevant information appeared in the middle of long inputs. So keep instructions short, specific, and close to the code they apply to instead of dumping the company's entire manual into the prompt. The linter already knows the style guide; it does not need a dramatic reading by the agent.

Invented dependencies are not folklore either. Spracklen et al., in We Have a Package for You!, analyzed 16 LLMs and 576,000 samples. The abstract reports an average of at least 5.2% hallucinated packages in commercial models, 21.7% in open-source models, and 205,474 unique invented names. The practical fix is to require confirmation in the lockfile, registry, and official documentation before adding a dependency. The production-safe observation is to record every manifest or lockfile change in the PR and require owner review and a dependency scanner.

There are also limits in the selected product. On GitHub, the Copilot cloud agent works in one repository and one branch per task, opens a draft PR, and has a hard timeout of 59 minutes. The branch uses the copilot/ prefix. Workflows submitted by the agent require a person to select Approve and run workflows; there is an additional approval when the PR is not assigned to a person. Those controls exist because an unreviewed workflow can still reach permissions and secrets. The observable signal is direct: an incomplete draft PR, a session terminated at the limit, or a workflow awaiting approval should not be "fixed" by relaxing the ruleset; reduce the task's scope.

AGENTS.md: versioned policy, not a turnstile

AGENTS.md is an open Markdown convention with no required schema. Compatible tools read the file at the root and can apply instructions closer to the edited directory. That allows specific rules in a monorepo without coupling policy to a single vendor.

But it is worth stating plainly once: AGENTS.md is a convention, not enforcement. It guides the agent; it does not prevent a commit, require a review, or replace authorization on GitHub.

A root file can start like this:

# AGENTS.md

## Escopo
- Trabalhe somente neste repositório, em um branch e um draft PR por tarefa.
- Não faça commit direto em `main`.
- Mantenha a mudança restrita aos arquivos e critérios de aceite da issue.

## Regras por tipo de arquivo
- Código em `src/`: toda mudança de comportamento exige teste novo ou atualizado.
- Arquivos em `tests/`: podem ser alterados sem o gate inverso.
- Não altere `migrations/`, arquivos de autenticação, `.env*`, secrets ou
  `.github/workflows/`. Pare e solicite intervenção humana.
- Documentação pode mudar sem teste de código.

## Verificação
- Execute a suíte, o linter e o type checker definidos pelo repositório.
- Não adicione dependências sem confirmação humana.
- Confirme pacotes, APIs e flags no lockfile e na documentação oficial.

## Pull request
- Use o título `[agent] <área> — <objetivo>`.
- Informe os testes executados e o que ficou fora do escopo.

The remediation for ignored instructions is to reduce conflicts, create a nested AGENTS.md when the rule is local, and turn critical boundaries into external controls. Watch for this in PRs: if the same deviation appears twice, adjust the instruction; if the deviation has security impact, do not wait for the third attempt to create a gate.

CODEOWNERS and rulesets: where policy grows teeth

The file below requests the appropriate reviewers for sensitive paths:

# .github/CODEOWNERS
*                      @org/dev-leads
/src/                  @org/dev-leads
**/migrations/**       @org/db-owners
**/*auth*              @org/security
**/.env*               @org/security
.github/               @org/dev-leads
.github/CODEOWNERS     @org/dev-leads

Replace @org/... with the real teams, with write permission, and protect CODEOWNERS itself. Then, in the main branch ruleset, enable Require review from Code Owners, require at least one approval, and make the CI checks mandatory. GitHub documents CODEOWNERS behavior and syntax rules.

If an agent touches authentication or a migration, the remediation is to block the merge until an owner reviews it, not ask the agent to "review more carefully." The safe observation is the review history by path: every sensitive change must record who approved it and which checks ran.

CI: production changed, so the test must change too

The rule "behavior changed, test changed" must leave Markdown and enter CI. The script below fails when it finds a production file in the diff but no test file:

#!/usr/bin/env python3
"""Falha se o diff tocar produção sem tocar testes. Heurística, não cobertura."""
import os
import subprocess
import sys

base = os.environ.get("GITHUB_BASE_REF") or os.environ.get("BASE_SHA") or "origin/main"
try:
    output = subprocess.check_output(
        ["git", "diff", "--name-only", f"{base}...HEAD"],
        text=True,
    )
except subprocess.CalledProcessError:
    output = subprocess.check_output(
        ["git", "diff", "--name-only", "HEAD~1"],
        text=True,
    )

files = [line.strip() for line in output.splitlines() if line.strip()]


def is_test(path):
    name = path.replace("\\", "/").lower()
    return (
        "/test/" in f"/{name}/"
        or "/tests/" in f"/{name}/"
        or name.endswith(
            (
                "_test.py", "_test.go", "_test.ts", "_test.tsx",
                ".test.js", ".test.ts", ".spec.ts", ".spec.js", "test.java",
            )
        )
        or name.startswith("tests/")
    )


def is_production(path):
    name = path.replace("\\", "/").lower()
    if is_test(name) or name.endswith((".md", ".txt", ".rst")):
        return False
    if name.startswith((".github/", "docs/", "automation/")):
        return False
    return name.endswith(
        (".py", ".go", ".ts", ".tsx", ".js", ".jsx", ".java", ".kt",
         ".rs", ".rb", ".c", ".cc", ".cpp")
    )


production = [path for path in files if is_production(path)]
tests = [path for path in files if is_test(path)]
if production and not tests:
    print("Produção alterada sem arquivo de teste no mesmo diff:")
    print("\n".join(production))
    print("Isto é heurística de presença, não prova de cobertura.")
    sys.exit(1)

print(f"ok: {len(production)} prod, {len(tests)} test files")

Save it as scripts/require_tests_with_prod.py. The complete workflow uses actions/checkout@v7, the release documented on 2026-08-22, and fetches the history required to compare the branch:

name: agent-gates

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]

jobs:
  require-tests-with-prod:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - name: Verificar mudança de produção sem teste
        env:
          GITHUB_BASE_REF: origin/${{ github.base_ref }}
        run: python3 scripts/require_tests_with_prod.py

Mark agent-gates and the real test suite as required checks. This heuristic checks for the presence of a test file, not coverage or quality. A test can exist and exercise nothing—an old tradition, now automated. The remediation is to combine the gate with test-suite execution and behavioral review. In production, observe the rate of PRs blocked by the check and, after merge, failures associated with the changed areas; recurrence calls for a more specific test or a smaller scope accepted for the agent.

A weekly audit without inventing the bot's username

Once a week, list recent PRs and filter those created by the agent. First confirm the author field in a real PR; the identifier can vary and should not be hardcoded on a guess.

gh pr list \
  --search "is:pr created:>=$(date -u -d '7 days ago' +%F)" \
  --state all \
  --limit 50 \
  --json number,title,author,createdAt,mergeable,reviews,statusCheckRollup,files

Look for four patterns in the same routine:

1. a package or import that does not exist in the registry;
2. production code without a meaningful test;
3. an attempt to change auth, secrets, migrations, or workflows;
4. an incomplete PR, failing CI, or a long sequence of change requests.

For each pattern, record the occurrence, the applied correction, and the gate that will prevent a repeat. The observation method is GitHub's own trail: author, files, checks, reviews, and time until the PR leaves draft. In Enterprise organizations, usage metrics can complement that reading, but they do not replace opening the diff. No dashboard notices that the test only calls the happy path and leaves satisfied.

Also enable secret scanning. If there is an alert, revoke or rotate the secret outside the agent, remove it from history according to the team's procedure, and review how it entered the context.

When the agent should stay out

The GitHub best-practices guide recommends avoiding broad, ambiguous, production-critical tasks and work involving security, PII, authentication, or incident response. In practice, keep these outside autonomous operation:

• migrations, schemas, and database write operations;
• authentication, authorization, IAM, and access rules;
• secrets, credentials, .env, and their rotation;
• workflows with permissions or access to secrets;
• production-critical fixes and incident response.

The 2026 OWASP GenAI LLM Top 10 ranks Sensitive Information Disclosure second and Excessive Agency third. Improper Output Handling now includes insecure code generated by assistants. The answer is not to write a sterner prompt: reduce tools and permissions, do not provide credentials in context, and require authorization in the system that executes the action. Observe denied attempts, sensitive files touched, and secret-scanning alerts without exposing the secret's contents in logs.

DevDojo would allow an agent to open PRs when the task is small, reversible, has objective acceptance criteria, and does not involve auth, secrets, migrations, or production criticality; the PR would be a draft, with mandatory CI and CODEOWNERS. The team would fall back to assisted suggestions—with no PR autonomy—after invented dependencies, repeated deviations, decorative tests, out-of-scope changes, secret alerts, or persistent review ping-pong.

Next step

Choose one small documentation or test issue today, add AGENTS.md, CODEOWNERS, and the agent-gates check, protect the branch, and run a single pilot draft PR. Expand the scope only after reviewing that PR's diff, checks, and approval trail.