Skip to content

Open nowPosted 40 days ago

Engineering Manager, Inference Router

Cerebras132 open roles

Where
Sunnyvale, CA
Work mode
Hybrid
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowEngineering Manager, Inference RouterCerebras · Sunnyvale, CA
  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Cerebras's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

14.5% of postings close within 7 days. Measured by our own scanner across the market. Cerebras postings stay open a median of 77 days.

Share of postings closed within
  1. 2.2%1 day
  2. 4.8%3 days
  3. 14.5%7 days
  4. 23.2%14 days
  5. 39.9%30 days
  6. 59.3%60 days
This job: posted 40 days ago

Cerebras median: 77 days open

The posting

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services.

This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.

Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership https://openai.com/index/cerebras-partnership/ with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

ABOUT THE ROLE

The Inference Router team at Cerebras builds the systems that decide where every inference request runs. We route traffic across all of Cerebras’ data centers and third-party integrations, balancing heterogeneous compute, maximizing cache utilization, and delivering the industry’s lowest time-to-first-token (TTFT) at massive scale.

As an Engineering Manager, Inference Router, you will lead the team responsible for the intelligent control plane of the fastest generative AI inference platform in the world. Your team owns global request routing, prefill/decode (P/D) disaggregation, prefix-aware routing for high cache hit rates, autoscaling, and the high-availability infrastructure that keeps the platform fast and reliable under real-time, bursty load.

This role operates at the intersection of distributed systems, Kubernetes-based infrastructure, routing-algorithm design, and machine learning. It combines technical leadership, people management, and execution ownership, with direct impact on the latency, throughput, and reliability experienced by every Cerebras inference customer.

WHAT YOU’LL DO

TECHNICAL LEADERSHIP

- Own the architecture and evolution of the inference routing layer across all Cerebras data centers and third-party inference integrations.

- Guide the design of:

- global, multi-region request routing and load balancing;

- routing for heterogeneous compute, including prefill/decode (P/D) disaggregated serving;

- prefix-aware / cache-aware routing to maximize KV-cache reuse and utilization;

- low-TTFT, high-throughput request scheduling under concurrent, bursty workloads.

- Drive the routing algorithms that trade off cache locality, queueing, load, and latency in real time.

- Invest in simulation: build high-fidelity models of the fleet and its workloads so the team can validate routing policies, failover behavior, and scaling decisions under a wide range of conditions without tying up precious accelerator capacity.

- Partner with inference runtime, cloud/platform, hardware, and ML teams to optimize end-to-end performance.

TEAM LEADERSHIP

- Build, manage, and grow a team of distributed systems, infrastructure, and ML systems engineers.

- Provide technical direction, mentorship, and career development.

- Foster a culture of ownership, velocity, and operational excellence.

- Recruit top talent in distributed systems, Kubernetes/infrastructure, and routing/ML systems.

EXECUTION & DELIVERY

- Drive execution of complex, cross-functional initiatives spanning inference runtime, cloud/infrastructure, and ML teams.

- Own delivery of capabilities such as:

- prefix- and cache-aware routing policies;

- P/D-disaggregation-aware scheduling and placement;

- autoscaling and capacity management across heterogeneous fleets;

- high availability, failover, and graceful degradation;

- simulation and load-generation frameworks that exercise the router under diverse traffic patterns, failure modes, and fleet topologies — without consuming scarce production compute;

- observability, SLOs, and reliability across the routing stack.

- Ensure high-quality releases through strong testing, load/chaos validation, and operational rigor.

PLATFORM & PERFORMANCE OWNERSHIP

- Scale the routing layer to handle very large volumes of concurrent requests at the lowest possible latency.

- Drive improvements in TTFT, throughput, cache-hit rate, and compute efficiency across the fleet.

- Design for reliability and high availability: failover, redundancy, scale-out, and fail-closed behavior under partial outages.

- Identify and prioritize technical debt and system bottlenecks.

- Maintain Cerebras’ industry-leading inference speed advantage.

CROSS-FUNCTIONAL COLLABORATION

- Partner with:

- inference runtime teams (execution and serving);

- cloud/platform and SRE teams (deployment, scaling, reliability);

- ML researchers and engineers (model enablement, routing signals);

- third-party integration partners.

- Act as a bridge between infrastructure, routing, and production systems.

WHAT YOU BRING

- 8+ years of experience in large-scale software engineering, distributed systems, or infrastructure.

- 2+ years of engineering management experience.

- Strong programming skills in C++ and/or Go for performance-critical, concurrent production systems.

- Deep experience operating production services on Kubernetes: deployments, autoscaling, rollout strategies, and multi-cluster/multi-region topologies.

- Proven track record building and operating highly available, scalable systems: load balancing, failover, scale-out, and reliability engineering.

- Experience designing or operating request routing, load balancing, scheduling, or traffic-management systems at scale.

- Experience with cloud infrastructure and best practices for building scalable microservices.

PREFERRED SKILLS

- Experience with large-scale LLM/inference serving and the systems around it (e.g. vLLM, TensorRT-LLM, SGLang, or comparable).

- Familiarity with prefill/decode (P/D) disaggregation, KV-cache management, and prefix/cache-aware routing.

- ML background: understanding of inference workloads, batching, and the performance characteristics of serving LLMs and multimodal models.

- Experience with autoscaling and capacity management across heterogeneous compute.

- Experience building simulation, modeling, or load-testing frameworks for distributed systems to validate behavior cost-effectively at scale.

- Strong grounding in observability (metrics, tracing, SLOs) and incident response for high-traffic production systems.

WHY THIS ROLE MATTERS

The Inference Router is the control plane behind Cerebras’ core promise: the fastest AI inference in the world. The routing decisions your team makes — across data centers, across heterogeneous compute, with cache-awareness and high availability — directly determine the latency, throughput, and reliability every customer experiences. Your work keeps real-time AI applications fast and dependable, and unlocks new capabilities across enterprise and frontier AI use cases.

Why Join Cerebras

People who are serious about software make their own hardware. At Cerebras, we have built a breakthrough architecture that is unlocking new opportunities for the AI industry. With dozens of model releases and rapid growth, we’ve reached an inflection point in our business. Members of our team tell us there are five main reasons they joined Cerebras:

1. Build a breakthrough AI platform beyond the constraints of the GPU.

2. Publish and open source their cutting-edge AI research.

3. Work on one of the fastest AI supercomputers in the world.

4. Enjoy job stability with startup vitality.

5. Our simple, non-corporate work culture that respects individual beliefs.

Find out more about what it's like to work at Cerebras here https://www.cerebras.ai/join-us!

Apply today and become part of the forefront of groundbreaking advancements in AI!

Cerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them.

This website or its third-party tools process personal data. For more details, click here https://www.cerebras.net/privacy/ to review our CCPA disclosure notice.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Cerebras's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.
  3. We apply on Cerebras's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.
  4. Every reply, sorted

    Cerebras's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.
  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.