Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 431 days ago

Senior Backend Engineer, Inference Platform

Together AI85 open roles

Pay
$200,000 – $290,000 a year
Where
San Francisco
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowSenior Backend Engineer, Inference PlatformTogether AI · San Francisco

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Together AI's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

12.3% of postings close within 7 days. Measured by our own scanner across the market. Together AI postings stay open a median of 32 days.

Share of postings closed within
  1. 2.1%1 day
  2. 4.5%3 days
  3. 12.3%7 days
  4. 19.7%14 days
  5. 35.6%30 days
  6. 58.0%60 days
This job: posted 431 days ago

Together AI median: 32 days open

The posting

About the Role

Together AI is building the Inference Platform that brings the most advanced generative AI models to the world. Our platform powers multi-tenant serverless workloads and dedicated endpoints, enabling developers, enterprises, and researchers to harness the latest LLMs, multimodal models, image, audio, video, and speech models at scale.

If you get a thrill from optimizing latency down to the last millisecond, this is your playground. You’ll work hands-on with tens of thousands of GPUs (H100s, H200s, GB200s, and beyond), figuring out how to fully utilize every FLOP and every gigabyte of memory.

You’ll collaborate directly with research teams to bring frontier models into production, making breakthroughs usable in the real world. Our team also works closely with the open source community, contributing to and leveraging projects like SGLang, vLLM, and NVIDIA Dynamo to push the boundaries of inference performance and efficiency.

  • Shape the core inference backbone that powers Together AI’s frontier models.
  • Solve performance-critical challenges in global request routing, load balancing, and large-scale resource allocation.
  • Work with state-of-the-art accelerators (H100s, H200s, GB200s) at global scale.
  • Partner with world-class researchers to bring new model architectures into production.
  • Collaborate with and contribute to the open source community, shaping the tools that advance the industry.
  • A culture of deep technical ownership and high impact — where your work makes models faster, cheaper, and more accessible.
  • Competitive compensation, equity, and benefits.

Responsibilities

  • Build and optimize global and local request routing, ensuring low-latency load balancing across data centers and model engine pods.
  • Develop auto-scaling systems to dynamically allocate resources and meet strict SLOs across dozens of data centers.
  • Design systems for multi-tenant traffic shaping, tuning both resource allocation and request handling — including smart rate limiting and regulation — to ensure fairness and consistent experience across all users.
  • Engineer trade-offs between latency and throughput to serve diverse workloads efficiently.
  • Optimize prefix caching to reduce model compute and speed up responses.
  • Collaborate with ML researchers to bring new model architectures into production at scale.
  • Continuously profile and analyze system-level performance to identify bottlenecks and implement optimizations.

Requirements

  • 5+ years of demonstrated experience building large-scale, fault-tolerant, distributed systems and API microservices.
  • Strong background in designing, analyzing, and improving efficiency, scalability, and stability of complex systems.
  • Excellent understanding of low-level OS concepts: multi-threading, memory management, networking, and storage performance.
  • Expert-level programming in one or more of: Rust, Go, Python, or TypeScript.
  • Knowledge of modern LLMs and generative models and how they are served in production is a plus.
  • Experience working with the open source ecosystem around inference is highly valuable; familiarity with SGLang, vLLM, or NVIDIA Dynamo will be especially handy.
  • Experience with Kubernetes or container orchestration is a strong plus.
  • Familiarity with GPU software stacks (CUDA, Triton, NCCL) and HPC technologies (InfiniBand, NVLink, MPI) is a plus.
  • Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or related field, or equivalent practical experience.

About Together AI

Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.

Compensation

We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $200,000 - $290,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at https://www.together.ai/privacy

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Together AI's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Together AI's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Together AI's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.