Alper Semerci

Case Study 03 · Agentic Engineering

Building an AI agent team that ships issues end to end.

How I designed a six-role team of AI workers that plans, designs, implements, reviews, and tests real product issues, and what a week of its own analytics showed about where delivery time goes.

Brix, a cartoon beaver in a navy hoodie holding a blue block, the mascot of the agent team dashboard
Role
CTO; designed and built the system
Context
Product engineering at Proteams
Focus
Multi-agent delivery, guardrails, and observability
Evidence
Live dashboard captures, 13–19 November 2026

An agent can write a change. Delivery is everything around it.

Getting a model to produce a plausible diff is no longer the hard part. A change only counts once it has been planned against the real requirement, reviewed by someone who did not write it, merged at the commit that was reviewed, deployed to a test environment, and checked in a browser by someone acting as the user. That chain is where an engineering team spends most of its discipline, and it is the part a single agent in a terminal does not cover.

The agent team is my answer to that for our own product work. One person, the owner, adds an issue link to a queue. Six AI workers take the issue from a plan to a production-ready handover. The owner is asked only for the decisions that are really theirs: product trade-offs, accepted risks, and access the team cannot get on its own.

Dashboard card reading: Brix, Builder. 6 of 12 blocks stacked this week, one for each issue completed in 7 days. Status: Building. Complete one more issue to stack the next block.
The dashboard's overview card. Brix stacks one block for each issue the team completed in the last seven days.

Six workers, each with a narrow job and narrower permissions.

The team splits responsibility the way a careful human team does. The role names matter less than what each role is not allowed to do.

01 · Lead

Plans, and never writes code.

Owns the plan, the contract between backend and frontend, the merge order, and the test duties. Every decision is recorded with its reason.

02 · UX/UI

Settles the interface before code exists.

Writes an implementation-ready specification before the plan is approved, then checks the finished interface after QA.

03 · Backend

Implements in one repository.

One branch and one merge request per change. It publishes its work and stops; it never starts a build and never merges.

04 · Frontend

Builds against the agreed contract.

Works under the same rules as the backend worker and treats the other repository as read-only reference.

05 · Review

Is never the author.

Gets a read-only checkout of the exact commit and writes only structured findings. It cannot change the code it reviews.

06 · QA

Tests like a user.

Works in prepared, signed-in test browser sessions and never sees a credential. A missing sign-in is recorded as incomplete, never as a pass.

Team workers panel listing Lead, UX/UI, Backend, Frontend, Review and QA. Frontend and Review are marked Working; the others are Idle.
The worker roster while two issues are in progress at once.

Workers do not write to the issue tracker or the code host themselves. They return results in a fixed format, and a controller checks and publishes them. The parts that only decide, such as routing work or stating that a merge is intended, run without network access. The part that acts is ordinary, tested code.

Seven stages from an issue link to production-ready.

Every issue follows the same path, and the dashboard shows that path as a row of stage cards. A stage that does not apply, such as backend work on an interface-only change, is marked not required instead of being skipped silently.

  1. LeadDrafts the plan. The owner approves it, and the approved plan is locked for every later step.
  2. UX designSized to the change, with upper bounds. Not needed for backend-only work.
  3. BackendImplements and publishes a merge request.
  4. FrontendImplements against the backend contract in the order the plan sets.
  5. ReviewAn independent review after the build passes. Findings go back to the author, and a changed commit gets a fresh review.
  6. QAAcceptance testing in a browser once the merged change is confirmed on the test environment. A failed retest reopens the finding.
  7. UX acceptanceThe finished interface is checked against the approved design.
Seven stage cards for one issue. Lead finished with a QA fix plan, UX design finished, Backend merged, Frontend working for 12 minutes, Review waiting after fixes, QA fix in progress with a retest after deploy, UX acceptance not started.
One issue in the middle of delivery. QA found a problem after deployment, the lead wrote a fix plan, the frontend worker is fixing it, and review and QA are queued to run again.
Seven stage cards for a completed issue. Lead finished, UX design finished, Backend not required, Frontend merged, Review merged, QA approved, UX acceptance accepted by the owner.
An interface-only issue after delivery: backend not required, QA approved, and the interface accepted by the owner.

Between the stages sit gates that cannot be talked past. The build must pass on the exact commit that will merge. The merge is pinned to the reviewed commit, so if the branch has moved, the merge is refused. A green build is never treated as QA, and QA starts only after the deployment of that commit has been confirmed.

The model is the smallest part of the system.

Most of the engineering went into the controller around the workers. These are the rules that made the team safe to leave running.

01 · Evidence

Saved words are not facts.

A status line a worker wrote earlier cannot prove that anything is still running. Liveness comes from the process, state comes from a fresh read, and every write is read back.

02 · Decisions

One function picks the next step.

The controller reads the saved evidence and the current facts, then decides. Steps go to an append-only journal, so a crashed run resumes at the first unmet step instead of starting over.

03 · Waiting

A wait is not a failure.

A busy build queue, another writer on the same merge request, or a slow API are normal states. The team looks again by itself and involves the owner only after a set time.

04 · Limits

Everything is bounded.

Review rounds, QA rounds, repairs, and retries all have ceilings. Token use per issue is capped at twice the rolling weekly average; past that, the team stops and asks.

05 · Isolation

No ambient authority.

Each worker starts in a fresh, sandboxed session with only the folders and hosts its role needs. Issue text and web content are treated as data, never as instructions.

06 · Language

Plain words for the human.

Owner-facing text carries no identifiers, hashes, or file paths. A pause between two workers reads “Preparing the next step”, so silence is never mistaken for a stop.

The technology is deliberately plain: a Python runner and controller built on the standard library, files on disk as the record, a local dashboard, and more test code than product code.

The owner is a decision-maker, not a debugger.

The team never asks an open question. When it needs a person, it offers two or three options with their consequences: approve or reject a plan, accept a named risk, release with follow-ups or send the work back, keep an issue ready for production or hand it to an independent tester. If a worker drafts a question full of internal detail, the controller refuses it, asks for one correction, and otherwise falls back to a safe default.

Everything that needs a person sits in one “Waiting on you” list. The mascot carries the same signal. Brix has five moods, in order of urgency: troubled when a run has failed, waiting when a decision is due, building while implementation runs, thinking during planning and review, and resting when the team is idle. His rank follows the last seven days of completed issues, from Rookie through Stacker, Builder, and Architect to Master builder.

The same Brix card with an exclamation badge and the status Waiting for your decision.
The same card when a decision is due. The mood is always written in words as well, never signalled by the picture alone.

A mascot on an engineering dashboard is a small thing, but it has a job. A status page that people glance at many times a day needs one signal that can be read in a second.

The agents were not the bottleneck.

The dashboard keeps three measures apart on purpose: the tokens the workers used, the time they actually ran, and the time issues spent waiting. Adding them together would hide the most useful finding so far.

6Issues delivered through 333 successful worker tasks
51 hTotal time the workers were running
365 hTotal time the same issues spent waiting
Eight overview tiles: 6 workers, 2 working now, 0 queued issues, 7 active issues, 1 waiting on you, 1 completed today, 6 completed in 7 days, 333 worker tasks in 7 days.
The team overview at the end of the week.

In the week of 13 to 19 November the team completed six issues through 333 successful worker tasks. The workers ran for about 51 hours in total. The same issues spent about 365 hours waiting, roughly seven times longer. Several issues wait in parallel, so the hours add up past the clock; the ratio is the point.

Waiting time chart for 13 to 19 November: 365 hours in total. CI build 117 hours, your decision 107 hours, standing still about 68 hours, test deployment about 63 hours, QA sign-in about 2.5 hours, other about 7 hours.
What the issues waited for. Builds come first, the owner second.

The two largest queues were the build system, at 117 hours, and me, at 107 hours of waiting for a decision. Test deployments added another 63. Once agents do the typing, a faster model changes little. Faster builds, faster deployments, and faster decisions change a lot. The team now also limits how many of its changes may wait for a build at once, so it stops feeding a queue that is already full.

Token usage chart for 13 to 19 November: 978 million tokens. Frontend 363 million, QA 316 million, Backend 96.2 million, Review 91.3 million, Lead 58.5 million, UX/UI 53.5 million. On 19 November, 192 million of 198 million input tokens were cached.
Token use by worker. Frontend work and browser-based QA account for about 70 percent.
Worker time chart for 13 to 19 November: 51 hours 20 minutes. Frontend 17 hours 41 minutes, QA 14 hours 6 minutes, Lead 5 hours 42 minutes, Backend 5 hours 42 minutes, UX/UI 4 hours 14 minutes, Review 3 hours 54 minutes. On 19 November the average run took 10 minutes.
Running time by worker. A typical run takes about ten minutes.

Two more patterns stood out. Testing costs about as much as building: QA used nearly as many tokens and hours as frontend implementation, because checking real behaviour in a browser is slow, careful work for an agent as well. And coordination is frequent but cheap: the lead handled 136 of the 333 tasks, about 40 percent, on roughly 6 percent of the tokens. On the last day of the week, about 97 percent of input tokens were served from cache, which is what makes many short, focused runs affordable.

Daily completions chart for 13 to 19 November: 6 completed issues. Successful tasks by worker: Lead 136, Frontend 69, Review 43, QA 32, UX/UI 28, Backend 25.
Issues completed per day, with successful tasks per worker. An issue counts once, when the production-ready handover is recorded.

Delivery that runs without being watched.

The result is not a model that writes code faster. It is a delivery process that keeps moving while nobody is watching it, stops when it should, and can explain where it is at any moment.

Issues delivered end to end From a queued link to a production-ready handover, with review, QA, and interface acceptance on record.
Decisions instead of supervision The owner approves plans and settles real trade-offs instead of watching terminals.
A pipeline you can measure Tokens, running time, and waiting are tracked apart, so the next bottleneck is visible.

Manage agents as a team, not as a tool.

What made this work is what makes a human team work: clear roles, limited authority, independent review, evidence before claims, and a calm way to say “I am waiting”. The difference is that with agents all of it has to be written down as code, because nothing is understood implicitly.

The operating model matters more than the model. Narrow permissions, bounded retries, and honest status language are what make autonomous work trustworthy enough to leave running. And once it is running, the numbers point away from the agents and toward the pipeline and the people around them.

Where the visuals and numbers come from.

Every image on this page is a cropped capture of the team's own dashboard, taken on 19 November 2026. The figures cover the seven days from 13 to 19 November and are counted by the system from its recorded runs. Issue titles and other project details were kept out of frame.