Code review with AI agents: when a pull request is ready

A pull request can solve the assigned problem and introduce another one in the same place. When the tests written to accompany it also pass, the person deciding whether to accept the change gets a reassuring answer to the wrong question.

This deserves attention with AI agents: an agent can prepare both the implementation and the evidence it presents for review. The reviewer's job is to reconstruct the required behavior and look for a case that challenges the solution. A second agent can help find it; the acceptance decision still needs a reason grounded in the code and its result.

To make that reasoning visible, I use a small teaching pull request: checking whether a meeting room is available. The code, defect and tests are constructed for this example. You can reproduce the demonstration in Docker without calling a model or an external service.

Start with the requirement

The room is booked from 10:00 to 11:00. The system must allow a booking from 11:00 to 12:00 and reject any interval that occupies even part of an existing booking. The current version mistakenly rejects consecutive bookings too.

We represent times as minutes after midnight: 600 means 10:00, and 660 means 11:00. An interval includes its start and excludes its end, written [start, end). Two bookings can therefore meet at 11:00 without overlapping.

Before reading the change, I would write down these cases:

The third case makes the review interesting. It follows from the overlap rule; we need no implementation code to establish the expected result.

The change passes three tests

The canBook function receives the requested booking and the existing bookings. It returns true if the interval is available. The initial check applies the same input validation to every version. The following diff changes only the expression that detects a conflict:

 export function canBook(candidate, existing) {
   validate(candidate, existing);
   return !existing.some((slot) =>
-    candidate.start <= slot.end && candidate.end >= slot.start);
+    candidate.start === slot.start && candidate.end === slot.end);
 }

The first expression treats adjacent intervals as conflicting. The new expression compares only identical bookings. The 11:00 case now works, but a booking from 10:30 to 11:30 is accepted too.

The proposed suite checks an exact duplicate, a consecutive booking and a free interval. All three tests really pass: they enter the changed function and assert results. Partial overlaps are the behavior left untested.

This changes the question worth asking. “Did you run the tests?” gets a truthful yes here. The useful question is: “Which test would stop this function from accepting two overlapping bookings?” Google's code review practices specifically recommend checking whether tests would fail when the code is broken.

Discovering that a file was imported does not settle this either. Read the assertion: code can execute without the test distinguishing the intended result from the wrong one. In discussions about agents and the tests they run, that distinction also appears as an objection to checks based solely on file reachability.

The test that changes the decision

For the current proposal, I would request changes. The finding can be short:

With an existing booking from 10:00 to 11:00, canBook({ start: 630, end: 690 }, existing) returns true. The requirement calls for false: both intervals occupy the room for the same half hour. The equality check introduced in the diff catches only exact duplicates.

A reproducible test makes that finding more useful than a general judgment about quality:

test('rejects an overlapping booking', () => {
  assert.equal(canBook(
    { start: 630, end: 690 },
    [{ start: 600, end: 660 }],
  ), false);
});

This test fails on the proposal: the observed result is true. The correction must recognize an intersection with positive duration. The requested booking starts before the existing one ends, and ends after the existing one starts:

-    candidate.start === slot.start && candidate.end === slot.end);
+    candidate.start < slot.end && candidate.end > slot.start);

The strict inequalities allow adjacent intervals. To check the rest of the rule, I add an overlap from the other side, containment, an empty calendar and a conflict with the second booking in the list. With the correction, both the original three tests and the eleven-test extended suite pass.

The check has led to a decision: reject the first change and accept the corrected predicate with its tests. That acceptance concerns this in-memory function. If two users can book at the same instant, checking and saving must also be coordinated in the database; a separate read followed by a write can still assign the room twice.

Reproduce it in Docker

You need a running Docker installation, a terminal with a sh-compatible shell and an editor. The first invocation downloads the official Node image if it is missing; the tests then run without network access. You do not need Node installed on the host, npm dependencies, an AI account or API keys.

Create an empty directory:

mkdir pr-review-demo
cd pr-review-demo

Create the following six files in that directory. You write all of them; the test runner generates none of them.

pr-review-demo/
  validation.mjs
  baseline.mjs
  proposed.mjs
  fixed.mjs
  proposed.test.mjs
  contract.test.mjs

validation.mjs enforces the input contract: integer minutes, a start before the end, and intervals within the day.

export function validate(candidate, existing) {
  const valid = (slot) => slot !== null && typeof slot === 'object'
    && Number.isInteger(slot.start) && Number.isInteger(slot.end)
    && slot.start >= 0 && slot.start < slot.end && slot.end <= 1440;
  if (!valid(candidate) || !Array.isArray(existing) || !existing.every(valid)) {
    throw new TypeError('Expected slots with integer minutes: 0 <= start < end <= 1440');
  }
}

baseline.mjs preserves the starting behavior, including the adjacency defect.

import { validate } from './validation.mjs';

export function canBook(candidate, existing) {
  validate(candidate, existing);
  return !existing.some((slot) =>
    candidate.start <= slot.end && candidate.end >= slot.start);
}

proposed.mjs contains the change under review.

import { validate } from './validation.mjs';

export function canBook(candidate, existing) {
  validate(candidate, existing);
  return !existing.some((slot) =>
    candidate.start === slot.start && candidate.end === slot.end);
}

fixed.mjs contains the correction.

import { validate } from './validation.mjs';

export function canBook(candidate, existing) {
  validate(candidate, existing);
  return !existing.some((slot) =>
    candidate.start < slot.end && candidate.end > slot.start);
}

proposed.test.mjs is the initial suite. The VERSION variable selects the implementation while keeping the tests identical.

import test from 'node:test';
import assert from 'node:assert/strict';

const { canBook } = await import(`./${process.env.VERSION ?? 'proposed'}.mjs`);
const existing = [{ start: 600, end: 660 }];

test('rejects an identical booking', () => {
  assert.equal(canBook({ start: 600, end: 660 }, existing), false);
});
test('allows a booking immediately after the existing one', () => {
  assert.equal(canBook({ start: 660, end: 720 }, existing), true);
});
test('allows a booking in a free interval', () => {
  assert.equal(canBook({ start: 780, end: 840 }, existing), true);
});

contract.test.mjs checks the cases derived from the requirement and interval validation.

import test from 'node:test';
import assert from 'node:assert/strict';

const { canBook } = await import(`./${process.env.VERSION ?? 'proposed'}.mjs`);
const existing = [{ start: 600, end: 660 }];

for (const [name, candidate, expected] of [
  ['identical interval', { start: 600, end: 660 }, false],
  ['overlap on the right', { start: 630, end: 690 }, false],
  ['overlap on the left', { start: 570, end: 630 }, false],
  ['contained interval', { start: 615, end: 645 }, false],
  ['containing interval', { start: 570, end: 690 }, false],
  ['adjacent on the right', { start: 660, end: 720 }, true],
  ['adjacent on the left', { start: 540, end: 600 }, true],
  ['separate interval', { start: 780, end: 840 }, true],
]) {
  test(name, () => assert.equal(canBook(candidate, existing), expected));
}

test('allows a booking in an empty calendar', () => {
  assert.equal(canBook({ start: 600, end: 660 }, []), true);
});
test('checks all existing bookings', () => {
  assert.equal(canBook({ start: 810, end: 870 }, [
    ...existing, { start: 780, end: 840 },
  ]), false);
});
test('rejects invalid intervals', () => {
  for (const candidate of [null, { start: 600, end: 600 },
    { start: -1, end: 20 }, { start: 1430, end: 1441 },
    { start: 600.5, end: 660 }]) {
    assert.throws(() => canBook(candidate, existing), TypeError);
  }
});

In the terminal, staying in the directory with the six files, set the image and define the function that runs a suite. The digest identifies a multi-platform OCI index. With the default configuration and no --platform flag, Docker automatically selects the native variant; you do not need to force arm64 or set up emulation. The demonstration reported here ran on 24 October 2026 with Node 22.23.4 on Linux arm64. The command mounts only this directory, read-only; it exposes no ports and removes the container when it exits.

export REVIEW_IMAGE='node@sha256:de170b39439eb0e5a3444ad7d60b015614a381e10fecae2e30ec9e735122f38d'

review_test() {
  docker run --rm --network none --read-only --cap-drop ALL \
    --security-opt no-new-privileges --pids-limit 64 --memory 128m --cpus 1 \
    --user node --mount "type=bind,source=$PWD,target=/work,readonly" \
    --workdir /work -e VERSION="$1" "$REVIEW_IMAGE" \
    node --test "$2"
}

First run the tests accompanying the proposal:

review_test baseline proposed.test.mjs
review_test proposed proposed.test.mjs

The baseline reports 2 passing tests and 1 failure: the adjacency case that needs fixing. The proposal reports 3 passes and no failures. Now run the extended suite, followed by both suites against the correction:

review_test proposed contract.test.mjs
review_test fixed proposed.test.mjs
review_test fixed contract.test.mjs

Exit code 1 is expected in the two cases that expose a defect. The Node test runner uses it to report failing tests. A failing command is therefore part of the demonstration: it shows which implementation the test can reject.

When finished, you can delete the example directory. The image remains in Docker's cache; remove it only if you downloaded it for this demonstration and no other project uses it.

Where an agent reviewer helps

A second reading by an agent is useful when it has a task whose output can be checked. For this pull request, I would ask it to derive examples from the requirement, look for incompatible behavior and prepare a test. To separate the choice of cases from the proposed solution, I split the request into two steps:

1. I use a separate session with only the requirement and no access to the diff, and ask for cases with expected results, including cases that should reject a solution.
2. I then provide the base version, proposed change and tests, identifying the commits, and ask it to check those cases and explain its findings.

For the second step, I use this request:

Review this pull request against the acceptance requirement and the base version.
Do not edit files or approve the pull request.
Use the requirement-derived cases supplied separately; add missing cases with a requirement-based rationale.
For each material finding, report:
- input and relevant preconditions;
- expected behavior and its requirement;
- observed behavior, or "not executed";
- the relevant file and changed expression;
- a test that fails on the defect and passes on the correction.
Separate a confirmed defect from a hypothesis and from a style preference.
If execution is unavailable, provide the test and leave the result unverified.

The report should lead to a line in the diff and a result another person can inspect. When the agent can run the test, it needs to provide the command, code revision and output; when it can only read the code, the observation remains a hypothesis to test. GitHub's AI code review documentation also requires validating its findings and supplementing them with human review.

A sound finding can be accepted even when the rest of the review contains noise. Conversely, agreement between three models does not explain which requirement they checked. Indiscriminately requesting more reviews increases the material someone must read; a counterexample lets them settle a specific question.

How much code should reach review?

Very large pull requests make it difficult to reconstruct intent and separate changes. In discussions among people receiving them from agents, some want to read everything, while others prefer to focus attention on consequences. The disagreement concerns available time and responsibility before it concerns the choice of reviewer.

Here the change is one expression long and is still wrong. A line limit would have let it through. It is more useful to ask for a pull request with a recognizable objective: correct interval conflicts and include the related tests, leaving a calendar refactor for another change. Google's guidance on small changes emphasizes the same conceptual completeness, including relevant tests.

Risk also determines how much attention a change needs. Editing an internal explanation and changing a rule that allocates resources to customers deserve different paths. Gergely Orosz's survey of code review approaches describes practices ranging from manual reading to impact-based triage: a more useful comparison than an absolute choice between reading everything and trusting the agent.

For an adoption decision across a team, the next step is to observe the work required to accept changes. I discuss that in the article on AI agents in a team: quality, costs and responsibility. For this pull request, the endpoint is narrower.

The decision to leave in the pull request

An acceptance comment can preserve the reasoning without becoming a meeting transcript:

Accept the corrected interval check: it allows consecutive bookings and rejects overlaps and containment. Both suites pass on the corrected version: 3/3 and 11/11. The previous proposal passes the initial suite but fails five cases in the extended suite. Concurrent persistence is outside this change.

In a real pull request, I would add the verified commit and a link to the run. When the pull request changes, rerun the tests on the new commit and update the acceptance comment with the verified commit and run. Keep the earlier results in the history. Branch protections can require fresh checks and approvals when the diff changes.