The posting
At Lilly, the work is demanding because patients are waiting. We unite caring with discovery to help make life better for people around the world, knowing that every decision, every detail, and every day matters. Headquartered in Indianapolis, Indiana, our over 50,000 employees around the globe take on complex challenges to discover and deliver life-changing medicines, strengthen how health is understood and managed, and support the communities we serve. This is hard, urgent, selfless work—but it’s work worth doing. If you’re driven by purpose and ready to bring your best to work that truly matters for patients, we invite you to join us.
About Lilly
At Lilly, everything we do starts with patients. We unite caring with discovery to make life better for people around the world. Headquartered in Indianapolis, Indiana, our global team of over 50,000 employees work with urgency and purpose to discover and deliver life-changing medicines, strengthen how health is understood and managed, and support the communities we serve. We bring our best to this work because people depend on it. If you're driven by purpose and determined to make a meaningful difference for patients, we invite you to bring your skill and your commitment to Lilly.
About the Technology Organization
At Lilly, technology is not a support function. It is how a global medicine company operates, innovates, and delivers. Lilly in Bengaluru builds the capabilities that make this possible, cloud platforms, AI systems, and automation at enterprise scale, all in service of a purpose that makes this technology work genuinely distinctive, from advancing drug discovery to enabling connected clinical trials to keeping a global medicine company running at the standard patients deserve.
About the Business Function
The Software Product Engineering (SPE) team is a specialised engineering group that delivers strategic solutions and differentiated capabilities. We take a forward-thinking approach, focusing on an enterprise platform and product mindset, ensuring that the solutions we build can be leveraged across Technology teams for broader impact and efficiency.
Role Summary
As a Senior Principal Engineer, you will design, build, and operate Lilly's Kubernetes-based hosting platform across multiple clouds. You will own platform capabilities end to end: writing Kubernetes controllers in Go and platform automation in Go or Python, defining infrastructure as code, evolving CI/CD and GitOps delivery, and raising the reliability and security of the platform. You will also lead a small team of engineers, guiding them technically and driving delivery.
What You'll Do
- Build, deploy and operate Kubernetes clusters end to end, from provisioning and configuration through upgrades, networking, scaling and decommissioning, on a managed offering such as EKS or ROSA, or an equivalent platform.
- Design, operate, and troubleshoot Kubernetes networking, including CNI plugins (Cilium, AWS VPC CNI), Services and kube-proxy replacement, CoreDNS, Ingress and Gateway API, network policies, pod-to-pod and cross-cluster connectivity, and egress control.
- Develop and maintain custom Kubernetes controllers and operators in Go (controller-runtime, Kubebuilder, or Operator SDK), including CRD design, reconciliation logic, finalizers, and admission webhooks.
- Operate and extend controller-based platform tooling such as AWS Controllers for Kubernetes (ACK), Crossplane, kro, Argo CD, and cert-manager.
- Deploy and operate the Istio service mesh (sidecar and ambient modes), including mTLS, traffic management (Gateway, VirtualService, DestinationRule, ServiceEntry), authorization policies, mesh observability, and version upgrades.
- Author and maintain automation, tooling, and services in Go or Python that streamline platform operations, onboarding, and review processes.
- Build and manage infrastructure as code using Terraform, Helm, and Kustomize, with strong module design, state management, and policy-as-code practices.
- Own and evolve CI/CD and GitOps pipelines using GitHub, GitHub Actions, Argo CD, Artifactory, and Checkmarx, with a security-first approach to software supply chain.
- Implement node and workload autoscaling with Karpenter, Cluster Autoscaler, HPA/VPA, and KEDA, including node image and upgrade strategies.
- Review and approve pull requests for platform hosting, deployment configurations, and infrastructure changes, ensuring they are secure, reliable, and aligned with platform standards.
- Lead a small team of platform engineers, providing technical direction, architecture and design guidance, and mentoring to grow engineers’ skills and independence.
- Own team delivery: break down roadmap items into sprint-sized work, plan and track sprints, manage dependencies and risks, and communicate status to stakeholders and leadership.
- Partner with application teams during onboarding, migration, and go-live, providing hands-on technical guidance and clear documentation.
- Participate in on-call and incident response for platform issues, including root cause analysis (RCA).
How You Will Succeed
- Demonstrate deep, hands-on Kubernetes expertise: scheduling, networking, storage, RBAC, API machinery, and troubleshooting production issues.
- Write production-quality Go (and Python where needed), including tests, and apply good software engineering practices to platform code.
- Apply strong knowledge of AWS/Azure, including compute, networking (VPC/VNet, routing, private endpoints), IAM (IRSA, Pod Identity, workload identity), and managed Kubernetes services.
- Lead and mentor the team, setting engineering standards and technical direction while keeping delivery predictable, with clear priorities, estimates, and outcomes.
- Use Istio to secure and manage service-to-service traffic, and set clear guidance on where mesh, CNI, and network policy responsibilities begin and end.
- Operate CI/CD and GitOps toolchains effectively, with sound judgment on release strategies such as progressive delivery, canary, and rollback.
- Use observability tooling to diagnose platform and application issues quickly, and drive improvements from the findings.
Required Skills
- Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent practical experience).
- 9+ years of experience in Platform or DevOps engineering, including running Kubernetes in production.
- Experience technically leading or mentoring a small team of engineers, with a track record of guiding design decisions and delivering platform work on schedule.
- Strong, hands-on experience with at least one managed Kubernetes offering (EKS, AKS, ROSA, or ARO); exposure to multiple is strongly preferred.
- Strong programming experience in Go, including hands-on experience building Kubernetes controllers and operators; Python for tooling and automation is a plus.
- Hands-on experience with Kubernetes manifests, rollouts and Helm charts (deployments, rollbacks, upgrades), along with a solid understanding of core Kubernetes concepts such as storage (PV/PVC, StorageClasses, CSI), networking (Services, Ingress, DNS, network policies) and scheduling.
- Strong Kubernetes networking knowledge: CNI plugins, Service and kube-proxy modes, CoreDNS, Ingress and Gateway API, network policies, and troubleshooting pod, node, and VPC connectivity.
- Hands-on experience operating the Istio service mesh in production, including mTLS, traffic management, authorization policies, ingress and egress gateways, and upgrades.
- Hands-on experience with CI/CD and GitOps tooling: GitHub, GitHub Actions, Argo CD or Flux, and artifact management (Artifactory or equivalent).
- Infrastructure-as-Code experience with Terraform.
- Working knowledge of AWS and/or Azure core services, networking, and IAM.
- Practical experience with observability tooling (Prometheus, Grafana, Loki, Tempo, Mimir, or equivalents) and log analysis in Splunk or similar.
- Experience with Linux, containers, and container runtimes, and strong troubleshooting skills across the network and OS layers.
Preferred
- Experience with ACK, Crossplane, kro, or similar infrastructure-from-Kubernetes approaches.
- Experience with security scanning and policy tools such as Checkmarx, Trivy, Kyverno, or OPA.
- Experience supporting platforms in regulated industries (Life Sciences, Healthcare).
- Kubernetes certifications (CKA, CKAD, CKS) or cloud certifications (AWS or Azure professional level).
- Experience with multi-cluster management, disaster recovery, and cost optimization (FinOps).
- Exposure to Agile methodologies and SDLC.
Additional Information
This is a full-time employee position. Availability for flexible hours may be needed to support platform operations, and some on-call or non-standard-hours support may be required depending on platform incident volume.
Lilly is dedicated to helping individuals with disabilities to actively engage in the workforce, ensuring equal opportunities when vying for positions. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form (https://careers.lilly.com/us/en/workplace-accommodation) for further assistance. Please note this is for individuals to request an accommodation as part of the application process and any other correspondence will not receive a response.
Lilly does not discriminate on the basis of age, race, color, religion, gender, sexual orientation, gender identity, gender expression, national origin, protected veteran status, disability or any other legally protected status.
#WeAreLilly
