Principal SRE

Remote, USA

Join us as we pursue our disruptive vision to make machine data accessible, usable and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun and most significantly to each other’s success. Learn more about Splunk careers and how you can become a part of our journey!


Splunk's Cloud Services group is looking for a Principal Site Reliability Engineer to help lead, design and build the next generation of our large scale cloud offering. You will be working on core services and applications that form the primitives for our current and future cloud service offerings.  Principal Site Reliability Engineers in this role will be engaging with multiple service owners across the platform to teach and implement modern interpretations of SRE, observability, Chaos Engineering and DevOps. This role is highly visible and impactful to the organization and will help shape Splunk's Engineering culture for years to come. Your job, in a nutshell, is to make every team around you better... including your own!

This is a remote role available in all US states except AK, ND, and WY. You also have the option of an office desk in some locations if that's convenient and desirable for you!

You will:

  • ¬†OwnSplunk Cloud in FedRAMP environments.
  • ¬†Work across the organization to deliver quality products that delight Splunk's passionate users.
  • ¬†Lead teams of tight-knit engineers who are building a state-of-the-art, cloud-based environment for massive-scale data processing.
  • ¬†Mentor and help new engineers to achieve more than they thought possible. You enjoy making other teams successful and are fulfilled through the success of others.


  • ¬†You have experience or an interest in working with regulated computing environments such as FISMA and/or FedRAMP and are enthusiastic about doing it better.
  • ¬†This is a fully remote, US-based/work-from-home position. You must be a US Citizen working on US soil to be considered.
  • ¬†You have owned and operated Kubernetes clusters and the associated ecosystems. ¬†Kubernetes certifications or an interest in obtaining these certifications are a plus, such as those from the Cloud Native Computing Foundation; Certified Kubernetes Administrator (CKA), Certified Kubernetes Application Developer (CKAD), or Certified Kubernetes Security Specialist (CKS).
  • ¬†You enjoy building and running distributed systems at scale in production. You understand the challenges and trade-offs to be made when building and deploying systems to production.
  • ¬†Deep understanding of linux systems (network stack, file system, OS services) and networking (L2 vs. L3, network architecture, VLANs, etc)
  • ¬†Experience with at least one programming language, preferably Golang (go) or python. ¬†Knowledge of working with and automating linux systems tasks using this language is required, including working with configuration files and system services. ¬†Knowledge of common data structures and algorithms, as well as their performance characteristics is required.
  • ¬†Knowledge of standard methodologies related to security, performance, and disaster recovery.
  • ¬†Highly skilled in identifying performance bottlenecks, identifying anomalous system behavior, and resolving root cause of service issues.
  • ¬†You have assembledOpen Source components into cohesive services.
  • ¬†You've demonstrated the skills to effectively work across teams and functions to influence design, operations and deployment of highly available software.
  • ¬†You are interested in working hard to make the users of Splunk's products happier every day.

Preferred skills:

  • ¬†Experience monitoring cloud environments with Splunk.
  • ¬†Experience with development and deployment in a hosted cloud environment, preferablyAWS& GCP. ¬†Cloud certifications are a plus or an interest in obtaining these certifications, such as AWS Certified Solutions Architect, AWS Certified DevOps Engineer, or Google Associate Cloud Engineer (ACE).
  • ¬†Experience with large scale distributed cloud service development, infrastructure, traffic management and architecture.
  • ¬†Experience with distributed architectures/systems with optimized and scalable software that operates on a large number of nodes.

We value diversity, equity, and inclusion at Splunk and are an equal employment opportunity employer. Qualified applicants receive consideration for employment without regard to race, religion, color, national origin, ancestry, sex, gender, gender identity, gender expression, sexual orientation, marital status, age, physical or mental disability or medical condition, genetic information, veteran status, or any other consideration made unlawful by federal, state, or local laws. We consider qualified applicants with criminal histories, consistent with legal requirements.  



Apply for this job
Share this job opening

DevOps and Dev jobs in your inbox every week.

Thank you! You'll receive a confirmation shortly
Oops! Something went wrong while submitting the form.
Made with loveÔłŹ by Mohamed Labouardy.