Skip to content

Open nowPosted 457 days ago

Member of Technical Staff - Distributed Training Engineer

Liquid AI20 open roles

Where
San Francisco
Work mode
Hybrid
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowMember of Technical Staff - Distributed Training EngineerLiquid AI · San Francisco
  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Liquid AI's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

12.7% of postings close within 7 days. Measured by our own scanner across the market. Liquid AI postings stay open a median of 41 days.

Share of postings closed within
  1. 2.2%1 day
  2. 4.5%3 days
  3. 12.7%7 days
  4. 20.4%14 days
  5. 35.6%30 days
  6. 58.7%60 days
This job: posted 457 days ago

Liquid AI median: 41 days open

The posting

ABOUT LIQUID AI

Spun out of MIT CSAIL, we build general-purpose AI systems that run efficiently across deployment targets, from data center accelerators to on-device hardware, ensuring low latency, minimal memory usage, privacy, and reliability. We partner with enterprises across consumer electronics, automotive, life sciences, and financial services. We are scaling rapidly and need exceptional people to help us get there.

THE OPPORTUNITY

Our Training Infrastructure team is building the distributed systems that power our next-generation Liquid Foundation Models. As we scale, we need to design, implement, and optimize the infrastructure that enables large-scale training.

This is a high-ownership training systems role focused on runtime/performance/reliability (not a general platform/SRE role). You’ll work on a small team with fast feedback loops, building critical systems from the ground up rather than inheriting mature infrastructure.

WHAT WE'RE LOOKING FOR

We need someone who:

- Loves distributed systems complexity: Our team builds systems that keeps long training runs stable, debugs training failures across GPU clusters, and improves performance.

- Wants to build: We need builders who find satisfaction in robust, fast, reliable infrastructure.

- Thrives in ambiguity: Our systems support model architectures that are still evolving. We make decisions with incomplete information and iterate quickly.

- Aligns with team priorities and delivers: Our best engineers align with team priorities while pushing back with data when they see problems.

THE WORK

- Design and build core systems that make large training runs fast and reliable

- Build scalable distributed training infrastructure for GPU clusters

- Implement and tune parallelism/sharding strategies for evolving architectures

- Optimize distributed efficiency (topology-aware collectives, comm/compute overlap, straggler mitigation)

- Build data loading systems that eliminate I/O bottlenecks for multimodal datasets

- Develop checkpointing mechanisms balancing memory constraints with recovery needs

- Create monitoring, profiling, and debugging tools for training stability and performance

DESIRED EXPERIENCE

Must-have:

- Hands-on experience building distributed training infrastructure (PyTorch Distributed DDP/FSDP, DeepSpeed ZeRO, Megatron-LM TP/PP)

- Experience diagnosing performance bottlenecks and failure modes (profiling, NCCL/collectives issues, hangs, OOMs, stragglers)

- Understanding of hardware accelerators and networking topologies

- Experience optimizing data pipelines for ML workloads

Nice-to-have:

- MoE (Mixture of Experts) training experience

- Large-scale distributed training (100+ GPUs)

- Open-source contributions to training infrastructure projects

WHAT SUCCESS LOOKS LIKE (YEAR ONE)

- Training throughput has increased

- Overall training efficiency/cost has improved

- Training stability has improved (fewer failures, faster recovery)

- Data loading bottlenecks are eliminated for multimodal workloads

WHAT WE OFFER

- Greenfield challenges: Build systems from scratch for novel architectures. High ownership from day one.

- Compensation: Competitive base salary with equity in a unicorn-stage company

- Health: We pay 100% of medical, dental, and vision premiums for employees and dependents

- Financial: 401(k) matching up to 4% of base pay

- Time Off: Unlimited PTO plus company-wide Refill Days throughout the year

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Liquid AI's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.
  3. We apply on Liquid AI's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.
  4. Every reply, sorted

    Liquid AI's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.
  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.