Meta repositories and AI agents: a project map or pinned revisions?

The product I am working on has four repositories: backend, web application, public website and infrastructure. I wanted to keep them separate, with their own dependencies, instructions and release cycles. I also needed one place to explain how the pieces fit together, including when an AI agent takes on a change.

A meta repository, or metarepo, coordinates other repositories through shared documentation, workspace configuration, instructions and project links. Application source stays in the components, with independent histories and remotes. The pattern predates AI agents: Burke Libbey described it in 2017, proposing a project hierarchy and coordination of its revisions.

Do you need a project map or a selected set of revisions?

There is a decision to make before the setup, because it changes the maintenance involved.

If the aim is finding your way around the projects, coordination can start with a README listing repositories, responsibilities and instructions to read. For example, the API defines the orders contract, the web application consumes it, and infrastructure describes where they run. Checkout locations belong in configuration; each project continues to select its own versions. This gives people and agents a shared entry point when service or package releases already manage the product's composition.

If the aim is recovering the same pair of source revisions, those revisions need to be recorded too. With submodules, Git places the components in subdirectories of the parent repository and records a reference to a commit for each one, called a gitlink. A SHA identifies that commit. A recursive clone can retrieve the chosen API and client rather than whichever revisions their branches happen to name when someone downloads them.

I chose this second arrangement. Its cost became clear as I worked with it: a commit in the component, then another in the metarepo to update its reference. The code had changed, yet cloning the parent still returned the previous version. That leads to the question I would ask today: which combinations are worth retaining together, and who maintains them?

A fixed reference is useful for investigating a regression or recovering a checked pair. If the parent follows every commit merely to keep the projects in one workspace, we have added version-selection work to the original need for orientation. The setup below is for readers who want those recorded revisions; a project map can stop short of that responsibility.

In a monorepo, the sources belong to one Git history, so one commit can change the API and client together. With submodules, histories remain separate and the parent commit records the composition. That is the distinction to consider before copying the layout.

A runnable Git example makes the mechanics visible. Two small JavaScript modules demonstrate an incompatible pair, a rebase that changes a SHA, rejection of an unpublished reference and a fresh clone of the recorded pair. Download the source and read it before running it. It requires Node.js 22 and Git 2.28 or later on macOS or Linux:

node metarepo-pin-flow.mjs

The fixture creates temporary repositories and local remotes, then removes them. It checks the two modules. The setup below reconstructs the source; dependencies, services and runtime requirements stay with the components.

The alternatives I considered

The original decision compared these options:

I chose submodules because they preserved repository independence using Git, which was already available. At that point, another tool was not worth the additional setup.

Submodule setup: connect two existing repositories

The setup requires Git 2.28 or later with an author name and email already configured, and Bash or Zsh. Start with two remote repositories, API and web, each containing at least one published commit on its default branch. Create a third, empty remote repository for coordination and select main as its default branch in your hosting service. This is the branch we will create in the parent. Your account needs permission to clone the components and push to the parent.

Replace the three placeholders with those repository URLs. Start in a directory that does not already contain coordination:

API_URL='<API_REPOSITORY_URL>'
WEB_URL='<WEB_REPOSITORY_URL>'
META_URL='<META_REPOSITORY_URL>'

mkdir coordination
cd coordination
git init --initial-branch=main
git remote add origin "$META_URL"
git submodule add --name api "$API_URL" components/api
git submodule add --name web "$WEB_URL" components/web

Git has cloned the components, created .gitmodules with their names, URLs and paths, and staged the two gitlinks for the parent's first commit.

Now create the coordination files. The completed setup will have this layout:

coordination/
├── .gitmodules
├── workspace.manifest.json
├── AGENTS.md
├── CLAUDE.md
├── tasks/
│   └── orders-pagination.md
└── components/
    ├── api/                    # independent Git repository
    └── web/                    # independent Git repository

Create workspace.manifest.json at the root of coordination with this content:

{
  "schemaVersion": 1,
  "projects": [
    {
      "id": "api",
      "submodule": "api",
      "role": "HTTP API and request contracts",
      "dependsOn": []
    },
    {
      "id": "web",
      "submodule": "web",
      "role": "Web application and API consumer",
      "dependsOn": [
        "api"
      ]
    }
  ]
}

This JSON is the example's own format, read by the person or agent receiving the task. It describes roles and dependencies; it does not install or start projects. The submodule field refers to the name assigned with --name. The path remains in .gitmodules and can be read with:

git config -f .gitmodules --get submodule.api.path

The last command reads the path associated with api. Anyone reading the manifest needs to make that same lookup. In this setup, moving the submodule with git mv updates its path in .gitmodules; the JSON keeps its name.

dependsOn records that the application consumes the API. Whoever reads that map still needs to select the relevant checks: the relationship alone does not describe the contract.

Pinned SHAs stay in the gitlinks. Repeating them manually in the JSON would mean updating them in two places. A test result can instead record the SHAs that were actually checked.

The shared instructions had started to drift

Different copies of shared instructions had appeared in my workspace. Some supposedly reusable procedures still contained the team identifier or assumed a project path already declared in the manifest. One component had local rules mixed into the section intended to be shared.

Changing a rule means working out which copies need updating and which differences are intentional. The workspace tooling has a shared source and a mode that reports differences without writing files. I want to see those differences before distributing a change.

For this setup, create AGENTS.md in the parent root with this shared entry point:

# Cross-repository work

Read workspace.manifest.json and the selected task before editing.
For submodules, resolve each manifest submodule name to its path in .gitmodules.
Use project IDs in instructions.
Before changing a component, read its applicable agent instructions and build configuration.
Keep component commands and runtime requirements with that component.
Report conflicting project rules before choosing how to proceed.

Record the exact revisions and the checks actually run in the task.
Leave unexecuted checks pending. Do not reset or clean another task's checkout.

Also create CLAUDE.md in that root, containing the single line @AGENTS.md. This shares the entry point with Claude Code, as described in Anthropic's documentation. Current versions also support reading AGENTS.md directly under the conditions listed there.

Codex builds an instruction chain from the project root to the working directory. A session started in an independent checkout may therefore miss the coordination entry point. I would name the coordination repository and relevant components in the task assignment, then ask the agent to identify the applicable instructions before editing.

When distributing rule copies, compare the shared portion separately from local adaptations. Check removals too: a rule deleted from the source may survive in an older copy. Review those differences, particularly when they affect permissions or commands.

A task someone can resume

Consider adding pagination to an order search. The current API returns an array; the new response contains items and nextCursor. Changing the response shape on the existing endpoint would break the previous client.

Create the tasks directory and tasks/orders-pagination.md. This example task records the decision that avoids that failure and the information needed to resume work:

# Orders pagination

Projects: api, web
Status: planned

Contract:
- GET /orders keeps returning the existing array.
- GET /v2/orders returns {items, nextCursor}.
- The web application moves to /v2/orders after that API is available.

Checks:
- Old client against updated API: pending.
- Updated client against the new endpoint: pending.
- Last page and invalid cursor: pending.

For each executed check, record:
- API and web commit SHAs, with source files matching those commits.
- Exact command, result and retained output location.

Next action: implement the compatible API endpoint, then update the client.
Release order: API first, web second.
Integration owner: assign before starting.

For a small task, one file can do this job. I would split the plan, state and handoff when they have different readers or update cycles. A decision that outlives the task can become an ADR, an Architecture Decision Record. The task links to it instead of copying its rationale every time work resumes.

Anthropic's work on long-running agents uses progress artifacts and Git history to orient the next session. A meta repository adds a useful question: which component revisions do these results describe? When a SHA changes, the old result still describes the previous combination.

This connects to the article on verifying agent work: a “completed” status needs supporting evidence. Here, that evidence must also cover the repository boundary.

Save the first workspace and clone it elsewhere

The parent root should now contain the manifest, both instruction files and the task you created. From the root of coordination, record those files alongside Git's configuration and component references:

git add -- .gitmodules components/api components/web \
  workspace.manifest.json AGENTS.md CLAUDE.md tasks/orders-pagination.md
git diff --cached --submodule=log
git commit -m "Set up coordinated API and web workspace"
git push --recurse-submodules=check -u origin main

Try reconstructing it from a directory outside the checkout, setting META_URL to the same parent remote:

META_URL='<META_REPOSITORY_URL>'
git clone --recurse-submodules "$META_URL" coordination-check
git -C coordination-check submodule status
git -C coordination-check status --short

submodule status should show one line for API and one for web. Neither freshly reconstructed component should have a - prefix (uninitialized), + (different from the parent's index) or U (conflict). The parent's status --short should be empty.

If you cloned without --recurse-submodules, complete that clone with:

git -C coordination-check submodule update --init --recursive

This is the flow described in Pro Git. You have reconstructed the source and coordination documents. Installing dependencies and starting services remain component-specific steps described in their READMEs; the parent does not provide a shared installation for different stacks.

For the first agent task, open the coordination checkout and give it an explicit assignment:

Read AGENTS.md in this coordination repository, then workspace.manifest.json
and tasks/orders-pagination.md. Resolve api and web through .gitmodules.
Read each component's applicable instructions and build configuration.
Before editing, explain the contract, current revisions and planned files.
Work in the relevant component, then record checks and the next action in the task.

The manifest identifies the projects; local instructions explain how to work in them. The agent can then propose changes in the relevant repository rather than guessing commands or locations.

Why the parent reference stays behind

Even setting branch = main in .gitmodules leaves the reference pinned until it is updated. That configuration selects the branch for operations such as update --remote, as described in the Git submodule manual.

From the meta repository root, compare the saved reference with the checked-out component. This example places the API at components/api:

git ls-tree HEAD -- components/api
git -C components/api rev-parse HEAD

The first command reads the reference in the parent's latest commit. The second reads the component's current commit. They can differ immediately after you commit in the API. git diff --submodule=log shows the commits between the component checkout and the reference in the parent's index.

That difference may be intentional during development. When reproducing a failure, it matters which revision was tested. A branch name cannot answer that reliably: it may point somewhere else tomorrow.

When I would update the pins

For a change involving both API and application, I would group the pin updates after the component merges. Intermediate commits would stay in their own histories. The parent would record the final combination we had checked.

There is a good reason to choose a different cadence. If the parent's CI must check every component merge, frequent pin updates trigger useful work: detecting incompatibilities early. Waiting for a larger batch would delay that feedback.

Recording the composition after the merges

Prepare the parent commit with the revisions produced by the component merges and available from their remotes. A rebase merge can change the SHAs from the original working branch.

I would use a separate integration clone. Start in a directory outside the checkouts, with no existing integration directory. In the block, replace META_REPOSITORY_URL with the parent repository URL and the two PUBLISHED_FINAL_*_COMMIT_SHA placeholders with the full final commit SHAs, already published to the component remotes after the merges. The paths follow .gitmodules in the example layout:

META_URL='<META_REPOSITORY_URL>'
API_SHA='<PUBLISHED_FINAL_API_COMMIT_SHA>'
WEB_SHA='<PUBLISHED_FINAL_WEB_COMMIT_SHA>'

git clone --recurse-submodules "$META_URL" integration
git -C integration switch -c integration/orders-pagination

git -C integration/components/api fetch origin
git -C integration/components/api switch --detach "$API_SHA"
git -C integration/components/web fetch origin
git -C integration/components/web switch --detach "$WEB_SHA"

git -C integration/components/api status --short
git -C integration/components/web status --short

Check the source state before and after bootstrap and the task's checks. The sources must match the selected SHAs. If a test needs modified or new source files that have not been committed, stop recording the composition: integrate those changes in the components, then rerun the checks on the new final revisions. Do not reset local work to obtain a clean checkout.

Update tasks/orders-pagination.md with the revisions, results and links to retained output. Include it in the same PR as the gitlinks and review the full diff:

git -C integration add -- components/api components/web tasks/orders-pagination.md
git -C integration diff --cached --submodule=log
git -C integration commit -m "Record verified orders pagination composition"
git -C integration push --recurse-submodules=check -u origin integration/orders-pagination

The check mode described in Pro Git rejects the push if the new submodule commits are not found on known remotes. The CI identity still needs access to the component repositories.

A detached HEAD is intentional in the integration clone: we are checking specific commits. For parallel development, I would use branches and worktrees of the individual components. The git-worktree manual still describes incomplete submodule support and advises against multiple checkouts of a superproject. I would also avoid recursive updates in a checkout another session is using.

When I would change the arrangement

Today I would also include west in the comparison. west manifest --freeze can produce a manifest with revisions resolved to SHAs. It uses west's manifest-rev references, rather than capturing a developer's local work. This option emerged from research for the article; it was not one of the original discarded candidates.

If a client consumes a released library, package versions and consumer lockfiles may be the right coordination boundary. If almost every feature requires simultaneous changes in the same components, I would revisit a monorepo. Recreating one through scripts and conventions can leave a team bearing the coordination overhead without being able to commit related changes together.

To decide whether to keep the pins, I would start with their next user: who needs to recover this pair of revisions, and for which check? A concrete answer gives the parent commit a purpose and an owner. If all we need is an entry point to the workspace, the map and instructions can serve that purpose without updating a composition for every change.