Scale-up Engineering Team

The client had grown from 5 to 40 engineers in 18 months. Their infrastructure had scaled with them but their operational practice hadn't. There were no runbooks, no on-call rotation, and no alerting beyond "the CTO sees it when it breaks."

We came in under a retained ops agreement. The first month was observability — deploying Prometheus and Grafana across the cluster, building dashboards that reflected what the system was actually doing, and wiring Loki for centralised log aggregation. We wrote the first 12 runbooks alongside the engineers who understood the services, not in isolation.

The second month we built a proper on-call rotation using PagerDuty, starting with ourselves as primary and the client's lead engineers as secondary. Every page generated a post-mortem, however brief. After three months the client's team was primary on most rotations; we were the backup escalation path.

At the 14-month mark, MTTD had dropped from 40 minutes to 4. The CTO had not been woken by an incident in over six months. The client eventually hired a dedicated SRE and we handed the runbooks and dashboards to them cleanly.