Interview Questions for Site Reliability Engineer

Use the STAR method. Describe a specific project where you implemented or managed Kubernetes. Detail the problem (e.g., slow deployments, resource waste, lack of resilience). Explain your actions (e.g., deployed a new service, optimized resource requests/limits, set up HPA/VPA, implemented a custom operator, configured network policies). Quantify the results (e.g., 'reduced deployment time by 50%', 'improved resource utilization by 20%', 'achieved zero downtime during upgrades'). Mention specific components like Helm, Kustomize, or specific cloud provider's managed Kubernetes service (EKS, GKE, AKS).

Start by defining the problem and its scope (e.g., 'First, I'd confirm the latency increase using monitoring dashboards like Grafana/Datadog, checking SLOs/SLIs.'). Then, outline your diagnostic steps: check recent deployments/changes, examine logs (ELK stack, Splunk), trace requests (Jaeger, Zipkin), inspect resource utilization (CPU, memory, network I/O) on relevant services/databases, look for external dependencies or network issues. Explain how you'd narrow down the root cause and potential mitigation strategies (e.g., 'If it's a database bottleneck, I'd check query performance or connection pools; if it's a specific service, I'd look at its internal metrics and logs.'). Conclude with post-mortem considerations.

Begin by clarifying requirements (e.g., message volume, latency tolerance, data durability, consistency model). Propose a high-level architecture: message ingestion (e.g., Kafka, Kinesis, MQTT broker), processing (e.g., serverless functions, microservices on Kubernetes), storage (e.g., NoSQL database like Cassandra, DynamoDB), and analytics. Emphasize reliability aspects: redundancy (multi-AZ/region deployments), load balancing, auto-scaling, circuit breakers, dead-letter queues, idempotent processing, and robust error handling. Discuss observability (SLOs/SLIs, tracing) and disaster recovery strategies.

Define an Error Budget as the acceptable amount of unreliability (downtime, latency, errors) for a service over a period, derived from its Service Level Objective (SLO). Explain that it's a shared metric between development and SRE teams. For implementation, describe: 1. Defining clear SLIs (e.g., request success rate, latency percentiles) and SLOs (e.g., '99.95% availability'). 2. Monitoring these SLIs in real-time. 3. Calculating the remaining error budget. 4. Using the budget to drive decisions: if the budget is healthy, teams can take more risks (e.g., feature launches); if it's depleted, focus shifts to reliability work (e.g., bug fixes, tech debt, performance improvements). Provide an example of how a depleted budget would trigger specific actions.

Clearly define 'toil' (manual, repetitive, automatable, tactical, no lasting value). Describe a specific instance (e.g., manual certificate rotation, repetitive server patching, manual deployment steps, log aggregation). Explain the pain points of the manual process. Detail your automation solution (e.g., Python script, Ansible playbook, Terraform module, CI/CD pipeline integration). Mention specific tools (e.g., Python, Go, Bash, Terraform, Ansible, Jenkins, GitLab CI). Quantify the impact: 'reduced weekly operational hours from 5 to 0', 'eliminated human error', 'sped up deployments by 70%', 'freed up engineers for strategic work'.

Start by defining IaC and its benefits (version control, consistency, speed, auditability, disaster recovery). Describe a project where you used Terraform or Ansible. For Terraform: explain how you defined cloud resources (e.g., AWS VPC, EC2 instances, RDS databases, Kubernetes clusters) in HCL, managed state, used modules, and implemented CI/CD for deployments. For Ansible: describe automating server configuration, application deployments, or patching across a fleet of machines. Highlight how IaC improved reliability, reduced manual errors, or accelerated delivery. Mention specific practices like modularity, testing, and peer review.

Use the STAR method. Describe a situation where a development team pushed for a feature that could impact reliability (e.g., a new service with untested scaling, a change to a critical database). Explain your role in identifying potential risks. Detail your actions: 'I collaborated with the development team to establish clear SLOs for the new feature, proposed a phased rollout with canary deployments, implemented additional monitoring, and suggested performance testing before full launch.' Emphasize communication, data-driven decision-making, and finding a solution that balanced both goals. The result should show a successful feature launch without compromising reliability.

Use the STAR method. Clearly describe the incident (e.g., 'a critical service outage due to a misconfigured load balancer'). Explain your specific role during the incident (e.g., 'I was the incident commander,' 'I diagnosed the network issue,' 'I implemented the rollback'). Detail the steps taken to mitigate and resolve the issue. Crucially, discuss what you learned from the incident (e.g., 'the importance of better pre-deployment checks,' 'the need for more robust monitoring'). Conclude with your contributions to the post-mortem process and the preventative actions taken (e.g., 'implemented automated configuration validation,' 'updated runbooks,' 'developed new alerting thresholds,' 'conducted a chaos engineering experiment').