Site Reliability Engineering (SRE)

Overview

Site Reliability Engineering (SRE) is an engineering discipline founded by Ben Treynor Sloss at Google in 2003. Treynor famously defined SRE as:

“What happens when you ask a software engineer to design an operations team.”

Rather than managing systems manually through sysadmin playbooks, SRE treats operations as a software problem, designing automated, self-healing platforms.

Core Tenets of SRE

1. Embracing Risk & Error Budgets in SRE:
   • 100% reliability is almost always the wrong target (the user cannot tell the difference between 99.99% and 100% due to local ISP failure). The remaining percentage is the team’s budget to ship features quickly and take calculated risks.
2. Service Level Framework (Service Level Terminology: SLI, SLO, & SLA):
   • Quantify system health via measurable SLIs, agree on internal target thresholds (SLOs), and formalize customer commitments (SLAs).
3. Eliminating Toil (Toil Identification & Reduction):
   • Operational work that is repetitive, manual, tactical, and devoid of enduring value must be capped at 50%. The remaining $\ge$50% must be spent on engineering project work (automation, architecture, scaling).
4. Monitoring & Observability:
   • Focus on the Four Golden Signals:
     • Latency: Time taken to service a request.
     • Traffic: Demand placed on system (HTTP req/sec, bandwidth).
     • Errors: Rate of failed requests (explicit HTTP 500s or implicit wrong content).
     • Saturation: How full the subsystem is (CPU, RAM, I/O, disk).
5. Blameless Postmortems:
   • When outages occur, assume good intentions and competent engineers. The system allowed the failure; investigate systemic, procedural, and tooling causes rather than punishing human error.

SRE Tooling & Ecosystem

The SRE toolchain is predominantly written in The Go Programming Language (Golang), powering projects like Prometheus, Kubernetes, Grafana, OpenTelemetry, Alertmanager, and Terraform.

Recommended Reading & Bookmarks

• Google SRE Books: Full online text of Google’s flagship SRE books.