The tests were already written. Nothing ever ran them.

A regression reached production. Not an exotic one: a behaviour that used to work, quietly stopped working, and a customer noticed before I did.

The humiliating part came during the post-mortem. The test that would have caught it was already sitting in the repository. It had been written months earlier, it was correct, and it had never run a single time in the life of the project.

This is part 3 of a series on hardening a solo-built SaaS in production. The setup and the full list of weak spots are in the pillar article. This one is about the cheapest quality win available to anyone: running the tests you already have.

Three layers of nothing

I assumed I had “some tests.” What I actually had was five test files and no mechanism whatsoever to execute them. The failure had three layers, and each one alone would have been enough.

There was no test script. Not in the root package.json, not in the API’s. The tests are written with Bun’s built-in runner:

import { describe, expect, test } from 'bun:test';

So the only way to run them was to know, from memory, to type bun test in the right directory. Nothing in the project told you that. A new contributor cloning the repo would have no way to discover the test suite existed.

Nothing triggered them. The repo does have a .github/workflows/ directory, which is why I had a comfortable feeling that “CI exists.” But the only workflow in it starts like this:

on:
  repository_dispatch:
    types: [jira-ticket-moved-to-progress]

It fires when a Jira ticket moves to “in progress.” It is a useful automation, and it has absolutely nothing to do with my code being correct. There was no push trigger and no pull_request trigger anywhere. Code could go from my editor to production without a single automated check.

The linter never ran either. There is a perfectly good lint script in the root package.json. Same story: it only ran when I remembered to run it, which was rarely, and never before a deploy.

The lesson I would put on a poster: a test that does not run automatically does not exist. It is a file. Files do not catch regressions.

Step 1: make the suite runnable

Before automating anything, the tests need one obvious entry point. This is a two-line change and it is the highest-value change in the whole article:

{
  "scripts": {
    "test": "bun test",
    "lint": "eslint 'apps/**/*.{ts,js}' 'packages/**/*.{ts,js}'"
  }
}

Now bun run test works from the root, for me and for anyone else, and there is a single command CI can call. If you take one thing from this article, take this: if your test command is tribal knowledge, it is already broken.

Step 2: make them run on every push and pull request

The workflow itself is short. Bun makes the setup trivial:

# .github/workflows/test.yml
name: test

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
      - run: bun install --frozen-lockfile
      - run: bun run lint
      - run: bun run test

That is the whole thing. Every push to main and every pull request now runs the linter and the full unit suite. The five tests that had never executed started executing, on every change, forever.

Note --frozen-lockfile. It makes CI fail if the lockfile does not match package.json, instead of silently resolving different dependency versions than the ones you develop and deploy with. On a project where the lockfile is committed, you want this.

Step 3: the test I did not have

Unit tests are good at pure logic. They are useless at telling you whether the application actually boots, whether the routes are wired, or whether auth still guards what it should. That whole category of failure was untested.

The funny part is that I already had a perfect smoke test written. It just only ran in production. Look at the healthcheck in the compose file:

healthcheck:
  test: ['CMD', 'bun', '-e', "fetch('http://localhost:8000/health')..."]

That is an end-to-end assertion: boot the app, hit a real HTTP route, expect a good answer. Docker runs it faithfully against production, every 30 seconds, after I have already deployed. Which is exactly the wrong time to discover the app does not start.

So I moved that assertion earlier, into CI, and added the one check unit tests kept missing:

// apps/api/src/__tests__/smoke.test.ts
import { describe, expect, test } from 'bun:test';

const BASE = process.env.API_URL ?? 'http://localhost:8000';

describe('api smoke', () => {
  test('the app boots and answers on /health', async () => {
    const res = await fetch(`${BASE}/health`);
    expect(res.ok).toBe(true);
  });

  test('a protected route still rejects anonymous callers', async () => {
    const res = await fetch(`${BASE}/api/crud/deal`);
    expect(res.status).toBe(401);
  });
});

The second test is the valuable one. It does not check business logic, it checks that the auth middleware is still attached to the route. That is precisely the kind of wiring that unit tests never notice and that breaks during a refactor.

Running it in CI needs the real dependencies, which GitHub Actions gives you as service containers:

  smoke:
    runs-on: ubuntu-latest
    services:
      db:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB: test
        options: >-
          --health-cmd pg_isready --health-interval 10s
          --health-timeout 5s --health-retries 5
      redis:
        image: redis:7
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
      - run: bun install --frozen-lockfile
      - run: bun run --cwd apps/api server.ts &
      - run: bunx wait-on http://localhost:8000/health
      - run: bun test smoke

Start small here. A smoke suite that proves the app boots and that auth is still enforced catches a shocking share of real regressions, and it costs about thirty lines. Full end-to-end coverage of every flow can come later, or never.

What this still does not fix

Being honest about the limit: CI now runs and it tells me when something is broken. It does not stop me from merging anyway. Nothing prevents a direct push to main, and a red pipeline is currently just a red icon I am free to ignore.

Turning “CI reports” into “CI blocks” needs branch protection and a required status check, and that comes bundled with an actual deploy pipeline. That is the next article in this series.

The takeaway

I did not need to write a single new unit test to make this project meaningfully safer. The tests existed. What was missing was a test script so they could be run at all, and a workflow trigger so they were run without me remembering.

If you have a repo with tests in it, check two things right now: does npm test or bun run test actually do something, and does any workflow trigger on pull_request? If either answer is no, your test suite is decoration. The series overview has the rest of the hardening list.