jguillaumesio
teamprocessagile

A practical playbook for running a remote dev team

One QA, eight people, a multi-tenant codebase, and too many bugs in prod. Here are the five ceremonies a new lead used to fix it, with the exact way we run each one.

We are eight people, fully remote: one lead, four developers, two product (one of them a designer), and one QA. We build a multi-tenant product: one shared codebase, plus features specific to individual tenants.

For a while it did not work. Bugs reached production regularly. We picked up work ad-hoc. Nobody had a clear picture of what was shipping next. And one QA cannot cover four developers on a codebase where a change for one client can quietly break another.

Then a new lead arrived. He did not rewrite our process from a book. He introduced five ceremonies, one at a time, each one aimed at a specific thing that was hurting us. This is the playbook: what each ceremony is, exactly how we run it, and what changed.

One honest caveat up front: we do not run fixed-length sprints. When I say “sprint review” below, read it as “release review.” We run regular ceremonies over a continuous flow of work, not textbook Scrum. I think that is fine, and I will come back to it at the end.

Ship fewer bugs: put quality in the ticket, not just in QA

The worst bugs we shipped were per-tenant: a change for one client broke another. You cannot hire your way out of that fast enough. The only thing that scales is moving quality earlier, before a line of code is written. The industry name is “shift left,” and the numbers are blunt: ambiguous or incomplete requirements are blamed for an estimated 50 to 60% of production defects, and a bug caught at the requirements stage costs a fraction of the same bug caught in production.

Requirements-phase defects (ambiguous acceptance criteria, missing edge cases, contradictory business rules) account for an estimated 50 to 60% of production defects, and teams that shift QA left reduce defect costs dramatically compared to teams that catch bugs in production.

Source: synthesised from IBM’s Systems Sciences Institute data and Capgemini’s World Quality Report

So we changed what a ticket must contain before anyone picks it up.

Every ticket gets acceptance criteria before it is estimated. Not a vague title. A concrete list of what must be true for the ticket to be done.

If a ticket is unclear, it gets an investigation first. We do not estimate something nobody understands. A short spike answers the open questions, and only then do we size it. This alone killed a class of “we did not realise it also touched X” bugs.

The multi-tenant rule: every ticket names the blast radius. Because our worst failures were cross-tenant, each ticket answers one question explicitly: does this touch shared code or a single tenant, and if shared, which tenants need checking? That one line turns an invisible risk into a checklist item.

Here is a template we paste into every non-trivial ticket:

## Acceptance criteria
- [ ] Given <state>, when <action>, then <result>
- [ ] Negative case: given <bad input>, then <handled how>

## Blast radius
- Touches: [ ] shared core   [ ] single tenant
- If shared, tenants to verify: <list>

## How to test
- <steps QA can follow without asking us>

Here is the part we have not fully solved, and I would rather be honest about it. Today the ticket creator writes the acceptance criteria, and we developers write the “how to test” notes. That works until it does not: sometimes those notes are thin, the ticket gets marked ready for release, and it should not have been. The root cause is not who typed the notes. It is that the author is effectively self-certifying. The person who wrote the code is the last person who should be the gate that says “this is releasable.”

Where I think this goes next is a lighter version of what the industry calls the “Three Amigos”: product, QA, and a developer shaping the criteria together, product owning the “what,” QA owning “what could break,” dev owning “what it touches.” We do not run that ceremony yet. But even a lightweight Definition of Ready would help, and the release gate should be someone other than the author, QA when possible, a peer when QA is swamped.

Here is the checklist we are converging on. Tick it against your own next ticket:

Is this ticket actually ready?

Not ready. Do not estimate it yet.

Tick what is true for your next ticket.

The regression net: the same cases, tested before every release

Acceptance criteria stop new bugs. They do nothing for the old feature that silently breaks when you touch shared code. For a multi-tenant product that is the most dangerous failure of all, and it needs its own guard: a fixed set of checks that runs before every release, not per ticket.

There are two acceptable versions of this, and you should be climbing from the second toward the first:

  • Automated end-to-end tests covering the critical paths: login, the core workflow, billing, and the per-tenant features that diverge from the shared core. Green suite or no release.
  • At minimum, a written list of must-pass scenarios, one set per important client, that QA runs before every release. Nothing ships until that list is green.

The multi-tenant rule applies here too: the list must include at least one scenario for every tenant whose behaviour differs from the shared core, because that is exactly where a “harmless” change detonates. And the list is a ratchet: the moment a regression slips through, its scenario gets added, so the same bug can never reach production twice.

This is the difference between “we tested the thing we changed” and “we proved we did not break the things we did not touch.” Only the second one keeps a multi-tenant product stable as it grows.

What changed anyway: more tickets are genuinely ready for release, because more eyes hit them before they get there. The remaining gap is the self-certification one above, and naming it is the first step to closing it.

The retro: a Miro board, and votes that force focus

At the end of each release we open a Miro board with four columns:

  • What is good
  • What we learned
  • What is not going well
  • What we would want

Everyone adds notes, privately, at the same time. Writing before talking is the whole trick: it avoids the loudest voice setting the agenda and it avoids groupthink. Then everyone gets a small number of votes (we use three) to spend on the topics they most want to discuss. We only discuss the top-voted items. The rest are logged, not lost.

Try the flow. You have three votes:

3 Votes left

What is good

What we learned

What is not going well

What we would want

Discussed this round
  • Nothing yet. Vote on what matters most.

You get 3 votes. Spend them, then we discuss the top items only.

The retro is where every other ceremony on this page actually got tuned. It is also the ceremony most likely to become theatre. The single most common way retros fail is not the format, it is the follow-through:

The reason most retrospectives fail is not the meeting itself, it is what happens (or does not happen) afterwards. Action items get discussed, then forgotten by the next sprint.

Source: Easy Agile, Why Retrospectives Fail

This is exactly where we are weakest, so I will not pretend otherwise. We discuss the top items and we track them loosely, but we do not assign a named owner or turn each one into a Jira ticket. So some drift, and the same complaint resurfaces a few retros later. The fix we know we need is boring: two or three action items maximum, each with an owner and a ticket, revisited at the start of the next retro. An action with no owner did not really happen. We are not there yet, and writing it down here is partly to shame ourselves into it.

Planning poker: the size is disposable

We size tickets with planITpoker, but with one deliberate change: we do not vote in numbers. We vote in t-shirt sizes, XS, S, M, L, XL. Numbers invite false precision (is it really a 5 and not an 8?) and, worse, they invite someone to add them up into a velocity number and wave it around. Sizes resist both. “This is a Large” starts a conversation. “This is 8 points” starts a spreadsheet.

Everyone votes at the same time, hidden, then we reveal together. The blind reveal is the point: it kills anchoring, so a junior does not just echo the senior’s card.

Have a go. Pick a card, then reveal the team:

You are estimating "Add per-tenant export to CSV". Pick your card.

Notice what the widget shows: a wide spread, S next to XL. That is not a failure of the estimate, that is the estimate doing its job. When the sizes disagree, someone knows something the others do not, a hidden dependency, an edge case, a tenant that behaves differently. The conversation that follows is the actual deliverable.

I want to be fair to the critics here, because they are the reason we dropped numbers in the first place. There is a loud, credible backlash against story points:

I don’t like story points because they give a false sense of precision, and the moment management starts treating them as a productivity metric, they are weaponised against the very developers who produce them.

Source: paraphrasing Tomasz Lakomy, Why I don’t like story-point-driven estimates, and Scott Logic, Story points are wasting time

They are right about the failure mode. Points become poison the instant a manager turns them into a burndown scoreboard. T-shirt sizes are our defence: there is nothing to sum, nothing to chart, nothing to weaponise. The size is disposable, the discussion is the deliverable. We use the vote to trigger a conversation, and then we mostly forget the letter.

The daily: coordination, not a status report

Fifteen minutes, same time every day, everyone on camera. Here is what it is for, and what it is not.

It is not a status report to the lead. The dailies people rightly hate are the ones where each person recites yesterday’s tasks to a manager who writes them down. If that is all your daily is, it should have been an async written update, a Slack message. The critics are right about that. Keep the synchronous time for the blockers. So that is what each person actually covers: what is blocking me, and what I am about to touch, especially if it is shared code. Not a recital of yesterday’s tickets.

Being fully remote, we felt the pull toward async written standups, and for pure status they would be better. But we kept the live daily for one reason it does well: coordination and surfacing blockers in real time. It is where “I am about to touch the shared billing code” meets “wait, so am I.” That collision does not happen in a threaded update. So the rule is: keep it short, no solving problems in the meeting, take the deep dives to a breakout right after.

Less ad-hoc drift is one of the payoffs, not the biggest but a real one: because we sync every morning, fewer people quietly start random work, and everyone has a rough idea of what is being picked up. The bigger win is still the collisions we avoid by saying out loud what we are about to touch.

The sprint review: shared visibility, not a demo

Call it a release review in our case, since we do not run fixed sprints. The trap everyone falls into is treating this as a demo: a parade of every task, performed for an audience. Ours has no external audience at all, no clients, just the developers, product, and QA, which is exactly why we can optimise it for shared understanding instead of performance. Scrum’s own guidance pushes back on the demo trap too:

The Sprint Review is not a demo. It is a working session to inspect the outcome and figure out what to do next. If it is a one-way presentation, you are doing it wrong.

Source: Scrum.org, Sprint Review: Much More Than Just A Demo

For us the real value is not the demo at all, it is shared visibility. We walk the whole internal team through what is going into the next release and what is queued behind it. Three things fall out of that:

  • Everyone learns the whole product. Because we discuss many tickets, including ones we will not personally pick up, more people understand more of the system. New joiners ramp faster.
  • Fewer PRs stuck in the review queue. When everyone already has context on a change, reviews are quick instead of archaeological.
  • We catch duplicates and collisions. Discussing tickets together is when someone says “that is the same as the thing I did last month” or “those two will conflict.”

We do keep a separate slot for real demos: a Friday session you sign up for to show something you shipped. Honestly, it is our weakest ceremony. It tends to run long, not many people sign up, and half the time the interesting work never gets shown. The visibility review above is what earns its keep. The demo is still looking for its format.

The unglamorous truth about why this works

None of this is clever. The reason shared visibility beats a polished demo is that the demo optimises for looking productive to outsiders, while the review optimises for the team actually knowing what is happening. The first is theatre. The second is the point.

The actual lesson

None of these five ceremonies is novel. You have heard of all of them. The thing worth copying is not the ceremonies, it is that each one was introduced to fix a specific, named pain:

  • Bugs in prod → acceptance criteria and blast radius in the ticket.
  • Ad-hoc chaos → the daily as coordination.
  • Estimates nobody trusted → blind planning poker in t-shirt sizes, with the size thrown away.
  • No shared picture → the review as visibility, not demo.
  • Everything drifting → the retro, with owned action items.

We do not do textbook Scrum. No fixed sprints, no formal Three Amigos, and one honest gap where authors still self-certify their own releases. That is fine. The goal was never to run the ceremonies correctly. It was to stop shipping bugs and start knowing what the team is doing. Pick the pain you actually have, and add the one ceremony that targets it. Then run the retro, and let it tell you what to add next.