Vision: proposed architecture
From a proven open-source core to an enterprise platform
DevBench runs today as an operator-run system: started by hand on a workstation or VM, against a workspace that every client reaches through file-level access, with progress watched in a terminal (and, if switched on, in Slack). Concurrency is already there: named sessions on one host, kept apart by a scope-overlap guard. This page is a proposal: what the same core would look like if it were run as an internal, multi-tenant service inside a large enterprise. It is a design direction, not a shipped product and not a roadmap commitment. Nothing described in purple below exists.
Exists today: open source
The core this vision is built on
A working spec → merge orchestration, published under Apache-2.0. Ten agents ship in the orchestrate plugin: an executor, a review supervisor, four review judges, a security reviewer, a manifest amender, a blocker resolver and a task factory. The done-gate refuses to mark a work unit done without passes from five judges (the four review judges plus the security reviewer). Blocked work is classified in code into seven distinct block-resolution paths, each with its own operator surface and notification event. The configuration surface is 24 top-level YAML keys (112 leaf keys), 56 CLI commands and 75 environment variables read by the Python source. State lives on disk, not in memory, so a run resumes after a restart. See the full architecture diagram →
- Green: exists today. Verified against the open-source repository. Solid borders.
- Purple: proposed. Not built, not scheduled. Dashed borders, and always written as “would” or “designed to”.
What changes at enterprise scale
Vision: proposed architecture
The pipeline itself would not change. What would change is where it runs, what it talks to, and how many people can use it at once. The proposal is a service decomposition: take the loop that works today on one machine, put it behind an authenticated boundary, and give the backlog a durable home that is not a directory of files.
Requests flow downward; execution state, verdicts and telemetry flow back up. Layers 6 and 7 are the system as it exists today; layers 1–5 are the proposal.
| Concern | Today: open source, source-verified | Proposed at enterprise scale |
|---|---|---|
| Where the loop runs | devbench start is a single non-interactive Claude Agent SDK
session that runs the orchestrate skill to completion on a workstation or VM. --daemon
detaches it and writes a PID file; instances lists every live orchestrator on the host. |
The same loop would be packaged as a container image and run as an authenticated service on ECS or EKS. A scheduler would provision it on request, one container per session, 1 to N per person, instead of a process launched by hand. |
| Backlog source of truth | Plain Markdown work-unit files on a filesystem, plus a
BACKLOG.md index. No database server, no message broker, no issue tracker anywhere in
the data path. (SQLite appears only as a rebuildable report cache.) |
beads (an open-source issue tracker, absent from the devbench repository today) would be forked and extended into a durable backlog service with its own web front end and back end, becoming the authoritative store. |
| How clients read and write work | The CLI reads and writes files directly under
DEVBENCH_WORKSPACE_ROOT. Concurrent claims are serialized by an exclusive lock on
BACKLOG.lock, with the status re-read under the lock before any write. |
The CLI would talk to the platform API. File-level access would stop being the integration surface, so a client no longer needs a copy of the workspace to participate. |
| Run configuration | One YAML file per workspace (backlog/config/devbench.yaml),
resolved environment > YAML > default, with scope selectors restricting
which work units a run may touch and named sessions kept apart by a scope-overlap guard. |
Per-developer sessions defined in the portal: which repos, which branches, which merge and PR strategy, how much concurrency, stored per tenant rather than per workstation. |
| Identity and credentials | The operator's own GitHub token and Claude Code credentials, read from their environment. | Enterprise SSO for people, scoped service identities for benches, and credentials issued and rotated by the platform rather than held on a laptop. |
| Observability | report, watch and hook-tail in a
terminal: 19 metric rows per window including estimated cost so far and estimated total cost at
completion, plus 16 Slack notification event types (all off by default). |
The same telemetry rendered in a shared portal, per developer, per team and per tenant, so cost and progress are visible to people who never open a terminal. |
Proposed
One control plane, two clients
The row above about client access has a consequence worth stating on its own. The CLI would keep its full command surface and change only what it talks to: the platform API instead of a directory of files. That is the relationship a cloud CLI has with the service behind it, and it would mean anything the portal can do, a script can do headlessly, because both would be clients of the same API. Continuous-integration jobs, scheduled automation and people who would rather stay in a terminal would drive exactly the same control plane as the web interface, and no capability would be reachable only by clicking.
Exists today: open source
The headless surface is already the whole surface
That is not a new way of working; it is the current one, re-pointed. The command line is already how the
system is driven: 56 commands cover the lifecycle end to end, from start, drain,
stop, sessions and instances through status transitions, dependency
wiring, scope selection, proposal promotion, git-ops and reporting, with machine-readable
--json output on devbench instances. There is no graphical surface today, so there is
nothing that only a UI can do. The one thing the CLI deliberately does not do is author a work unit's body prose:
scaffolding and drafts come from templates, and the substance stays a file edit. What the platform would change is where those commands send their reads and
writes.
Coexistence with the tools an enterprise already runs
Vision: proposed architecture
No enterprise is going to abandon its planning system, and the proposal does not ask it to. The design would keep the backlog service authoritative for execution state while the enterprise's planning tool stays authoritative for intent, with the direction of truth stated per field rather than left implicit.
| What is being synced | Proposed owner | Proposed direction of truth |
|---|---|---|
| Business intent: epic and story descriptions, priority, assignment, target dates | Enterprise planning tool (Jira, Azure DevOps) | Planning tool → backlog service |
| Execution state: status, dependencies, blocked classification, judge verdicts, PR links | Backlog service (the beads fork) | Backlog service → planning tool |
| The work-unit contract: acceptance criteria, changes manifest, definition of done | Backlog service | Authored in the platform, mirrored outward for visibility |
Proposed
Bidirectional, but not ambiguous
Sync would run in both directions, with every field owned by exactly one side. A planner re-prioritizing in their own tool would be reflected in the backlog; a bench blocking a task would be reflected on the planning ticket. Where both sides can write, the owning system wins and the loser is recorded: no silent merge, no last-writer-wins.
Proposed
One backlog, many producers
Because intake would be an API rather than a directory, other tools could later feed the same backlog (a spec-authoring assistant, a defect triage system, a modernization scanner) without each one inventing its own queue. The execution contract stays identical no matter who filed the work.
Exists today: open source
The audit trail sync would carry already exists
Every agent action and every judge verdict is already appended as a timestamped comment to the work unit itself, and the repository's own description of the system is “auditable by default … state lives on disk, not in memory.” The proposal is to transport that trail into the tools an enterprise already reads, not to invent a second record of what happened.
The portal: one pane of glass
Vision: proposed architecture
Today the operator surface is a terminal. The proposal is a web portal that shows every bench in a team or tenant, and lets a person act on what they see. Each portal capability below is paired with the mechanism that already exists in the core: the portal would be a front end for behavior that is already implemented, not a new engine.
Proposed
Status and cost at a glance
A live board of what is in queue, in progress, in review, blocked and done, filtered by repo, team or tenant, with spend visible alongside progress rather than reconstructed after the fact.
Today
- The lifecycle already has 9 canonical statuses.
- Blocked work is already split into the 7 classified buckets by
devbench statusanddevbench report. - The report already renders estimated cost so far and estimated total cost at completion, from
per-model rates that
devbench cost-calibratetunes against a real invoice.
Proposed
A chat agent that resolves blocks
A conversational agent with backlog-management skills, so unblocking work does not require knowing the command surface: ask why a unit is blocked, get the classification and the audit trail, and choose a resolution in the same conversation.
Today
The resolutions it would call are the operator paths that already exist: set-status,
hold/unhold, decline, add-dep,
sync-blocked, reconcile-cascade, promote-proposal and
reject-proposal, mapped onto the
seven block-resolution paths.
Proposed
Start, stop and drain controls
Role-scoped buttons to start a bench, stop it, or drain it gracefully, so a team lead can quiet a fleet before a release freeze without asking each developer to run a command.
Today
- Graceful stop is already a first-class operation:
devbench drain(with--reason,--cancel,--status,--sessionand--all) lets the current work unit finish and then exits cleanly. - Alongside it:
stop,restart,sessionsandinstances.
Proposed
Session configuration, provisioned on request
A form instead of a file: which repositories a bench may touch, which branches it works from, the merge and PR strategy it must follow, and how much concurrency it is allowed, versioned and attributable to the person who changed it. Submitting it would be the request. The scheduler would provision the session and it would appear in the same pane of glass as every other session that person or team is running, so someone who needs three benches against one backlog would ask for three rather than opening three terminals.
Today
- The same knobs exist in YAML: a required
repos:map with per-repo default branch and merge strategy. - A top-level merge strategy of
merge,squashorrebase, plus the git-ops modes. - Scope selectors that bound which work units a run may claim.
- A session is already a first-class named thing on disk: with
DEVBENCH_SESSION_NAMEset it gets its own state directory and its own scope file, and the session registry records its process and liveness.
What does not exist is the part that would place it for you.
Multi-user and multi-tenant
Vision: proposed architecture
A bench today binds to one workspace on one host. The interesting engineering problem at enterprise scale is several developers, each running several benches against the same backlog at the same time. The proposal answers it with a mechanism the core already has.
Proposed
Sessions as scheduled workloads, not processes started by hand
A person would request a session rather than launch one: name the repositories, the branches, the merge and PR strategy, the scope and the concurrency, and the platform's scheduler would provision it. The same person could hold several sessions at once against the same backlog, and the scheduler would run 1 to N containerized orchestrator loops to match what has been asked for, placed on a shared pool rather than on anyone's machine. That is what would make the system multi-user and multi-session-per-user at once: capacity, isolation and limits would be scheduling decisions the platform makes, records and enforces, rather than a consequence of who opened a terminal and when.
Exists today: open source
Concurrent named sessions are already real; the placement is what is manual
Several named sessions already run against one backlog. Each has a name, a registry entry recording its process and liveness, its own state directory and its own scope file.
devbench sessionslists them and clears the stale ones.devbench instanceslists every live orchestrator on the host.devbench stop-instance,devbench stop --sessionanddevbench drain --allend them.- Overlap is arbitrated rather than assumed: a session whose scope overlaps an active one is refused
unless
--allow-overlapis passed.
All of it runs on one host and is started by hand, which is exactly the part a scheduler would take over.
Proposed
First merge wins; the loser becomes a block
Two benches would be free to work the same repository concurrently. When the first one merges, the second one's branch is now behind. Rather than attempting an automatic rebase of unreviewed work, the design would turn that conflict into a queued block on that work unit. The unit would carry the failure surface in its audit trail, be resolved (by the bench, by the chat agent, or by a person), and be resubmitted through the same gates as any other work. Losing a merge race would be an ordinary, recorded outcome rather than an exception.
Exists today: open source
Why that is a small change rather than a new subsystem
The core already treats “this could not proceed” as a first-class, classified state rather than a crash.
A git-ops hard failure blocks the unit and writes a [BLOCKED] audit comment naming the exact
failure surface; a pull request closed without merging blocks the unit with the PR number and URL in the
reason. Concurrency is already guarded where it matters: a claim takes an exclusive lock, re-reads the
on-disk status under that lock, and refuses the claim if another party got there first. Two named sessions
with overlapping scopes are refused unless the operator explicitly allows the overlap.
Isolation and controls the design would have to satisfy
These are stated as requirements on the proposed design. None of them is implemented in the open-source core today, and none should be read as a current capability.
Tenant isolation
Separate backlog stores, queues, credentials and telemetry per tenant, with no path by which one team's bench can read another team's work, secrets or logs.
SSO and RBAC
Authentication through the enterprise identity provider (no local accounts), and distinct roles for who may start or stop a bench, who may approve a merge, and who may change a team's trust settings.
Audit
Every action attributable to a person or a named service identity, retained on the enterprise's schedule, and exportable to the systems that compliance teams already review. Proving the record →
Proving the record: tamper-evident history and replay
Of those three requirements, audit is the one that does not stop at access control. An enterprise handing implementation work to agents has to answer a harder question than “what is the status of this work unit?”: it has to be able to show an auditor, a regulator or an incident review that the recorded history of what the machine did is complete and unaltered. That is a property of the record itself, not a report someone runs afterwards.
Proposed
Hash-chain the orchestration event history
Every event the platform records (a claim, an agent handoff, a judge verdict, a commit, a human approval) would be appended to one ordered history, and each entry would carry the cryptographic hash of the entry before it. Verification would then cover the sequence, not only the individual entries.
How it would work
- Each entry carries its predecessor's hash
- The history has exactly one valid order, and that order can be rechecked by recomputing the chain.
- Append-only by construction
- Altering or deleting anything would break verification for everything recorded after it, so tampering would surface as a broken chain rather than as a silent edit.
- Content addressing for what an entry refers to
- Entries would name the diff, the verdict and the work-unit revision they describe by hash, so the history could not drift away from the artifacts it points at.
- Optional signing and external anchoring
- The head of the chain could be signed, timestamped and anchored outside the platform, so the record would be attestable independently of the system that produced it.
Deliberately generic: hash chaining, append-only writes, content addressing and optional signing are the whole of the design. No specific algorithm, key-management scheme, vendor or ledger product is proposed here.
Proposed
Replay: stepping back through a verified history
Once a chain verifies, the history could be replayed rather than merely read. The platform would reconstruct the run in order and let a reviewer step through it:
- Which agent acted, on which work unit, at which position in the sequence.
- Which verdict each judge returned, against which diff, and what the next stage did with it.
- Where the loop went backwards: a failed review, a block, a retry, a cascade.
- What a human approved, rejected, held or merged, and when.
The limit, stated plainly: replay reconstructs and re-examines the record; it does not re-execute the agents. Language models are not deterministic, so running the same work unit again would not be expected to produce the same diff, and nothing here claims bit-for-bit reproducibility of agent output.
Proposed
What a verifiable history would be for
- Audit evidence: a history whose completeness can be demonstrated rather than asserted.
- Incident forensics: when a change caused harm, the exact sequence that produced it, including which gates passed and where a person intervened.
- Disputes about approval: what was approved, by whom, against which diff, without a second version of events.
- Segregation of duties over time: showing that the party which implemented a change was not the party which approved it, across a period rather than in one screenshot.
No compliance claim attaches to any of that. Whether such evidence satisfies a particular standard, audit or regulator is a determination for the enterprise and its own auditors, and this page does not make it.
Exists today: open source
The record is already real; the chain is what is missing
The proposal above is an extension of something the core already does, not a new discipline bolted onto it. Verified in the open-source repository:
- State lives on disk rather than in memory, which is why a run resumes after a restart instead of losing what it knew.
- Each work unit's
## Commentssection is append-only: one line per entry, stamped in UTC and attributed to the agent that wrote it. Judge verdicts land there as[REVIEW_PASS]or[REVIEW_FAIL], which are the only two tokensdevbench log-verdictwrites for any judge, the security reviewer included (cli.py:3607), alongside the status transitions and the TDD cycle log. - One honest gap in that record, of exactly the kind a verifiable chain would surface.
The done-gate reads
[REVIEW_REJECTED]as the boundary of the most recent review round (manager.py:1083,:1092), but nothing in the source writes that token: it exists only as a read pattern, so in practice the boundary falls back to the whole comment history. The repository's documentation describes a security failure writing[SECURITY_FAIL]and[REVIEW_REJECTED]; the code writes neither. A record that verifies as a sequence is how a gap like that stops being something a reader has to notice by hand. - The hook event log (
hook-logs.jsonl) takes one JSON line per Claude Code event, so every tool call by every agent is recorded, plusSubagentStartandSubagentStopat each agent boundary (executor → review-supervisor → each judge → security-reviewer), each line stamped with the orchestrator session. - The Stop hook writes its own JSON file on every loop-continuation block under
.devbench/stop-hook-diag/, recording the exact payload it emitted plus the circuit-breaker counters, so a hang can be post-mortemed from evidence rather than speculation. - In the default git-ops mode each work unit lands as its own branch and its own commit, with the work-unit id in both the branch name and the commit message. Git's object model is content-addressed, so that part of the history is already a hash-linked record.
Those are four separate records today, each trustworthy only as far as the storage holding it. What the proposal would add is one verifiable chain across the whole orchestration event stream, so the history could be checked as a single sequence rather than corroborated by hand.
What would come next
Vision: proposed architecture
Fifteen further capabilities, grouped by what each one would actually take to build rather than by how appealing it sounds. The first group extends a seam the core already has. The second gates at the authenticated boundary the platform introduces, so it cannot come before it. The third would be built from nothing. Every item is a proposal, and the order is the argument for its feasibility.
Proposed
Group 1: extends a seam that already exists
Each of these attaches to a mechanism that is already in the code, which is why it comes first.
Proposed capability / how it would work
- More chat transports, Microsoft Teams as an example
- A Teams endpoint, and other chat platforms, would register alongside today's Slack endpoint and publish the same lifecycle events, so the per-event toggle surface would not change and every transport would stay off until a team switches it on. Naming Teams here is an example of the category, not a built integration.
- Plug-and-play specialty judges
- An infrastructure-as-code judge, a DevOps judge, or whatever role a particular organization cares about, would be added as review roles registered with the done-gate rather than as extra opinions running beside it. The change would be making the required judge set configuration instead of a constant.
- Waiting on model-provider capacity, and resuming without a person
- When a provider has no capacity to give, a session would park and wait rather than fail, then resume on its own when capacity returns, reusing the restart-and-resume machinery the orchestrator already has instead of a new mechanism. Where a plan's capacity replenishes on a schedule the provider sets, that automatic resumption is what would let a session carry on across the whole of it without someone watching for the moment to relaunch.
- Sessions provisioned to fit the environment the work needs
- A session request would name the machine shape the build actually needs, including memory, disk and chipset architecture, and the platform would provision to it. Instance family and disk size are already the inputs to the workstation stack; what would be new is a service choosing them per session instead of a person choosing them once per laptop replacement.
Exists today: open source
The seams those four would attach to
- Notifications. Slack is the only endpoint in the code, carrying 16 event types behind three independent switches that all default to off. The configuration schema keeps endpoints in their own sub-block on purpose, and states the reason itself: so new transports can be added “without touching the event-toggle surface.” Its own parenthetical is “Slack today; Discord/Teams/etc. later”, which is intent recorded in a schema description, not an integration.
- Judges. The review supervisor already discovers its judges dynamically, by listing the
agent files in
agents/review_team/, and each agent's model is separately overridable indevbench.yaml. The set the done-gate counts is not dynamic:ALL_REQUIRED_JUDGE_NAMESpins five canonical names inconstants.py, andmark-donerefuses any unit missing a pass from each. A judge file added today would be dispatched and would still not be required, which is exactly why plug-and-play is a code change and not a drop-in. - Restart and resume. Automatic recovery already exists for one classification: a run
whose only blocks are
runtime-degradationexits with code 42, themake startwrapper relaunches it up toDEVBENCH_MAX_AUTO_RESTARTS(default 3), then fails fast for a human. State lives on disk rather than in memory, which is why a relaunched run picks up what it knew. - Machine provisioning. The repository ships infrastructure code for per-developer remote
workstations: three Terraform modules, Terragrunt wiring and nine Ansible roles, with instance type
(
DEVBENCH_INSTANCE_TYPE, defaultc8g.2xlarge) and volume size (DEVBENCH_EBS_SIZE_GB, default 256) already operator inputs. The instance family carries the chipset architecture with it, so selecting one is already how the architecture gets chosen. Today a person runs those targets by hand, one stack per developer.
Proposed
Group 2: needs the authenticated control plane first
Each of these is a decision made at the boundary where a request is admitted. That boundary is itself proposed, so these follow the API rather than precede it.
Proposed capability / how it would work
- Quotas and budgets the organization sets, and a way to ask for more
- A tenant, a team or a person would carry ceilings on concurrent sessions and on spend, enforced where session requests are admitted, because that is already where the scheduler would decide what to provision. Raising a ceiling, permanently or for a defined window, would be a request that a named role approves and that is recorded like any other audited action, not an edit to a file on a workstation.
- Access control lists on every point a human can act
- The API, the portal, the chat surfaces, each approval role, and the start, stop and drain controls would each carry their own list of who may use them, extending the SSO and RBAC requirement already stated above rather than introducing a second authorization model beside it.
- Custom network restrictions around the bench
- Ingress and egress rules per tenant, and where wanted per repository, would be declared with the environment rather than adjusted after it, so a bench would reach the registries, package indexes and internal services an organization has decided it may reach, and nothing else.
- Consumption scheduling across the day
- Where a provider offers differential pricing by time of day, the scheduler could hold work that is not urgent and place it in the cheaper window, since queueing and placement would already be its job. Whether that is worth doing depends entirely on what a given provider charges, which this page does not assume.
- Remote block resolution, including from a phone
- An authenticated person would clear a blocked work unit from a chat app or a web chat, and from a mobile device where the organization chooses to permit it, calling the same operations the CLI exposes for the seven classified blocked states over the API instead of a shell. It depends on the access control lists above: a remote surface is only as safe as the list of who can reach it.
- An optional development-manager process outside the loop
- A wrapper process would own every human-in-the-loop interaction, block handling included: it would watch for the units the loop parks, route each to the right person on the surface that person actually uses, and hand the resolution back. Sitting outside the orchestrator means the loop itself would never wait on a human. It would be optional, because a team that prefers the loop to stop and wait should be able to keep that behavior.
Exists today: open source
What those six would build on, and the one thing that is missing
- The numbers a budget would check are already computed.
devbench reportrenders estimated cost so far and estimated total cost at completion, from per-model rates thatdevbench cost-calibratetunes against a real invoice. - The operations a remote surface would call already exist. Against the seven classified
blocked states:
set-status,hold/unhold,decline,add-dep,sync-blocked,reconcile-cascade,promote-proposalandreject-proposal. Three of the seven classifications always require a person, which is the work a development-manager process would take on. - Network posture is already declared in code, not clicked. The workstation stack creates one VPC and security group per AWS account, and that group is declared with zero ingress and egress only: the operator reaches the instance over an SSM tunnel rather than public SSH. The instance's own IAM role reads the operator's GitHub credentials from Secrets Manager, so no credential transits on a refresh.
- What is genuinely absent is the boundary itself. There is no account model, no role model and no quota anywhere in the core: access to a bench is access to the host, and the credentials in play are the operator's own GitHub token and Claude Code credentials read from their environment.
Not a compliance claim: quotas, access control lists and network rules are engineering controls. Nothing above asserts that any of them satisfies a standard, certification or regulation. That determination belongs to the enterprise and its own auditors.
Proposed
Group 3: new subsystems
These would be built rather than extended. They are last for that reason, not because they matter least: two of them change the quality of what enters the queue, which is where rework is decided.
Proposed capability / how it would work
- A retrieval layer over the organization's business context
- Specs and backlogs would be authored against the organization's own domain knowledge, its glossary, its policies, its prior specs and its recorded decisions, rather than from the prompt and one operator's answers alone. The authoring skills already accept supplied context, so the seam exists; a retrieval service that keeps that context current is what would be new.
- A second retrieval layer over reusable resources
- An index of what the organization already runs, its services, libraries, patterns and infrastructure modules, would be consulted while the backlog is written, so a work unit would be pointed at an existing component instead of quietly paying to recreate it. It would be the standing version of a discovery pass the repository already documents doing by hand, per project.
- Multi-person review of the spec before work starts
- Several people would review the specification together, with independent human security review and enterprise architecture review as distinct approval roles, each of which has to sign before the backlog is released into the queue. The approval roles are the new part: the review target, a written spec, already exists.
- Always-on maintenance loops
- Bot-style orchestrator loops would stay up and watch a queue continuously for ongoing maintenance, with dependency-update review and merge as the obvious first case and the work filed by whatever more specialized agentic service produces it. This one depends on API intake, and on a run becoming long-lived rather than finite.
- The platform itself provisioned as code
- Network, data stores, queues, the orchestrator fleet and the portal would all be declared in infrastructure code, released as immutable versioned artifacts with zero-downtime replacement rather than in-place edits, and autoscaled on queue depth. Feasible mainly because nothing about this design wants to be imperative: the run configuration is already declarative, and the repository already treats its own infrastructure this way.
Exists today: open source
Where those five would start from
- Authoring context is supplied, not retrieved.
create-specgets its material from one structured interview (seven question blocks, 17 numbered questions), an optional exemplar spec named byskills.exemplar_spec_path, and an optional directory of discovery artifacts. There is no retrieval layer and no index of anything. - The discovery pass is documented and manual. The repository describes a pre-authoring phase whose stated purpose is to inspect current state before the spec is written, so a project does not collide with what already exists. It runs per project, by hand.
- Spec review today is one person. The skill presents each section for operator spot-check as it is written and shows a final review at the end, to the same operator. The security review that exists today is an agent gate on the implemented diff, run only after the four review judges pass and hard-failing on any critical or high finding, which is a different thing from a human security review of the spec beforehand.
- A run is finite.
devbench startis not a daemon watching a queue: it runs the loop to completion and exits when the backlog is done or nothing is actionable.--daemondetaches that single run; it does not make it long-lived. - Infrastructure code exists, for the wrong target. What the repository provisions is per-developer workstations, and its own documentation states that the workstation stack is independent of the orchestrate pipeline. Nothing in the repository deploys devbench as a service, so a multi-tenant platform delivered as code starts from nothing.
What this list is: fifteen capabilities that illustrate the direction, ordered so the dependencies are visible. It is not a committed roadmap, nothing here is built or scheduled, and no delivery order, effort or benefit is being promised.
Progressive trust: how a team would adopt it
Mechanism exists today
No enterprise turns on autonomous merging in week one, and it does not have to. The adoption arc runs from “a human approves every task and every merge” to “merge on green”, and every rung of that ladder is an existing configuration setting, not a feature that would need building. What the platform would add is making the line a managed, visible policy instead of a YAML file on someone's laptop.
The ladder at a glance
| Rung | Who releases work into the queue | Who merges |
|---|---|---|
| 1. A human approves the work going in and the merge coming out | A person: new units start as draft and are promoted by hand |
A person |
| 2. The queue runs unattended; a human still merges every pull request | Normal intake: units land in-queue by default |
A person, one pull request per work unit |
| 3. The bench batches the work; a human merges the batch | Normal intake, as rung 2 | A person, one pull request per repository |
| 4. Merge on green | Normal intake, as rung 2 | devbench, once CI reports green, and still gated by review decisions and bot reviewers |
Each rung below names the configuration that produces it.
-
A human approves the work going in and the merge coming out
The most conservative operational configuration: nothing runs until a person releases it, and nothing lands until a person merges it.
Mechanism today
backlog.default_status_for_new_work_units: draft- New units are invisible to the orchestrator until promoted with
devbench promoteordevbench set-status … in-queue. task_factory.enabled: false- The default: the task factory is off unless it is enabled.
task_factory.auto_accept_proposals: false- Where the factory is enabled, each generated task is left at
proposedforpromote-proposalorreject-proposalrather than auto-promoted. git_ops.pause_before_merge: true- Pushes the pull request, watches CI to green, then moves the unit to
in-reviewand stops.devbench check-mergereconciles once a person merges or closes it.
-
The queue runs unattended; a human still merges every pull request
The first thing to relax is intake: work is queued the normal way, and the human decision moves to the merge button alone.
Mechanism today
backlog.default_status_for_new_work_units- Left at its default (
in-queue), so intake needs no promotion step. git_ops.pause_before_merge: true- The same setting as rung 1. The bench claims, implements, passes the five-judge done-gate and opens the pull request per work unit, then stops.
Without
pause_before_merge, that per-unit pull request is merged by devbench once CI is green, so this setting is the difference between a person merging and the bench merging. -
The bench batches the work; a human merges the batch
For teams that would rather review one coherent change than a stream of small ones.
Mechanism today
git_ops.single_branchwithdefer_pr- Holds commits locally until the batch is finalized into one pull request per repository.
auto_finalize- Opens that pull request automatically once every unit is terminal, leaving the merge to a person.
-
Merge on green
The end of the arc, and still gated: by CI, by review decisions, and by whatever bot reviewers the team requires.
Mechanism today
git_ops.auto_merge: true- Merges once CI reports green. Requires
auto_finalizeanddefer_pr. - A human “changes requested” review decision
- Hard-blocks the merge when the pull-request review phase is enabled.
- Named review bots
- Can block the merge too.
- Sixteen notification event types
- Can page the team on any of it, all off until switched on.
Exists today: open source
Per tool-call approval is deliberately not a rung
It is not an operational mode.
- What
devbench startruns: the orchestrate skill through the Claude Agent SDK withpermission_mode="bypassPermissions". - What the repository says about interactive mode: the one mode that does prompt for
each tool call (a live interactive Claude Code session) is documented as intended for observation only and
“almost never operational”, and a user-scope plugin install, the alternative to a per-session
--plugin-dir, registers hooks that then block every other Claude session on that machine from writing tobacklog/**. - What observation actually is:
devbench hook-tail,devbench reportanddevbench status: the same firehose, without the console.
Proposed
What the platform would add
The rungs above are per-workspace YAML today, which means the trust line is wherever each operator last edited a file. The platform would make it a policy:
- Set per repository and per team.
- Enforced at the API rather than by convention.
- Visible in the portal.
- Changed only through an audited action by someone with the role to do it.
Different teams, and different repositories inside one team, would sit on different rungs at the same time, deliberately.
Ways of working: the operator role
Vision: proposed architecture
The most consequential change is not architectural. If benches do the implementation, the scarce human activity stops being writing code and starts being deciding what should be built, and judging what came back.
Shaping the work
The operator's first job would be the specification and the backlog it becomes: what is in scope, what the acceptance criteria are, what “done” means. Ambiguity is what produces rework, and it is resolved here or not at all.
Resolving operator blocks
The second job would be the exception queue: the work the system deliberately refuses to decide on its own. That queue is the operator's inbox, and keeping it short is the skill.
Managing the backlog
The third job would be curation: sequencing, dependencies, holding what should wait, declining what should not be built. Closer to a development manager's craft than to an individual contributor's.
Exists today: open source
This is already how the core is shaped
The repository describes its own operating model in one line: “One operator writes the spec; the orchestrator drives every task from claim to merged PR.” And the exception queue is not rhetorical. Of the seven block-resolution paths, three always require a person (a unit an operator deliberately held, a unit blocked behind a held unit, and the catch-all where no automation path exists), while the other four are designed to clear themselves. The role described above is the role the system already implies; the platform would give it better tools.
No productivity, throughput or headcount claim is made on this page. Any such number would need measurement that has not been done.
Start with what is real
Everything above with a purple marker is a proposal. Everything with a green marker is running code you can read today: the orchestration, the judges, the guards, the block-resolution paths and the whole configuration surface, published under Apache-2.0.
To be unambiguous: the multi-tenant platform, the backlog service, the portal, the chat agent and the enterprise-tool sync described on this page are not built. They are a design direction for a proven open-source core, described here so the engineering conversation can start from a shared picture.