-
The alert channel nobody read, and the catalogue that fixed it
Every monitor paged, so none of them meant anything. Staging woke people at night, one quiet tenant could report a 100 percent error rate off a single request, and the team had learned to swipe the notification away. Here is the P1 and P2 split we settled on for an ECS stack, and the two mechanics that stopped the flapping.
- prod-opsdevops
Users emailed me about 500s before I ever saw them
Every error in production was logged with console.error and then died in a log file nobody read. Customers became my error tracker. Here is how I wired Sentry-compatible tracking into Express and React with sourcemaps, grouping, and one alert rule that actually fires.
-
I was grepping JSON over SSH while production was down
During an incident, my observability stack was an SSH session, docker logs, and grep. Here is the zero-budget setup that replaced it on the same VPS: Uptime Kuma on the healthchecks I already had, Dozzle for live logs, and the alert that now reaches my phone.
- prod-opsdevops
The Friday push that taught me to ban commits on main
A git push at 6pm on a Friday deployed straight to production, untested, and took the API down. Nothing in the pipeline could have stopped it, because there was no pipeline. Here is the one I built: protected main, CI that runs the tests, images built off the box.
-
Database backups that actually restore: the pgdata I lost
I deleted a Postgres data directory that lived in a bind mount with nothing behind it. My backup plan was hope. Here is the cheap, single-VPS backup setup I should have had, including the step everyone skips.
-
CPU utilization lies: autoscaling a single-threaded service
A service was slow under load, but autoscaling never fired because CPU sat at 30 percent. A single-threaded app on a multi-core task saturates one core and reads as barely busy. Why, and how to fix it.
-
Docker never restarted my database: the price of restart: no
A container died at night and nothing brought it back, because every service was set to restart: no. The healthchecks I did have could not help, and the reason why surprises most people.
-
The tests were already written. Nothing ever ran them.
A regression reached production, and the test that would have caught it was already in the repo. There was no test script, and the only CI workflow never triggered on a push. How I wired it up.
-
Docker bypassed my firewall: the database port was open to the internet
My host firewall blocked port 5432. It was reachable from the internet anyway, because Docker writes its own iptables rules that your INPUT and ufw rules never see. Why it happens, and how to actually close it.
-
The docker build that filled the disk and took down production
Production went down with no code change. The cause was a docker compose build orphaning a new <none> image on every deploy until the disk filled. What actually cleared it, and the deeper fix.
-
Shipping a SaaS on one VPS with Docker Compose: an honest audit
The whole product runs on a single VPS behind one docker-compose.yml. Here is why that was the right call at launch, and an honest map of every single point of failure it hides.
-
Cloudflare under the hood: how it works and how attackers try to get around it
What actually happens when a request hits a Cloudflare-protected site, how Turnstile distinguishes bots from humans, and the techniques attackers use to find the origin server behind the proxy.
- localhostnginxhttpsdevopsfullstackdx
Sharing cookies across subdomains locally
A prod-identical local setup where cookies are shared across subdomains over real HTTPS, with zero app code changes. The trick lives in /etc/hosts + nginx.
- architecturedevops
Architecture matters more than clean code
I spent 2 weeks making my serverless functions clean. The architecture was wrong. I rewrote everything as a single Express server and it was simpler.
- devopsarchitecture
Debugging is more important than features
My blog returned 404 on every page at 2 AM. I had zero visibility into why. Here is the debugging infrastructure I wish I had set up from day one.
- devopsaws
Testing Lambda + EventBridge locally
How to test AWS Lambda functions and EventBridge rules locally using SAM, LocalStack, and docker-compose. No cloud deployment needed.
-
AI infrastructure: a maintenance nightmare
I asked AI to generate Terraform for a standard AWS setup. It produced 2,400 lines that worked perfectly. Here's why I'd never deploy it to production.