Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted yesterday

Member of Technical Staff - Pre-Training Evals Infra — Reflection AI

Battery Ventures portfolio1,062 open roles

Pay
$65,000 – $250,000 a year
Where
New York City, New York, United States; San Francisco, California, United States; London, England, United Kingdom; New York, NY; San Francisco, CA; London
Work mode
On site
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowMember of Technical Staff - Pre-Training Evals Infra — Reflection AIBattery Ventures portfolio · New York City, New York, United States; San Francisco, California, United States; London, England, United Kingdom; New York, NY; San Francisco, CA; London

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Battery Ventures portfolio's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

9.5% of postings close within 7 days. Measured by our own scanner across the market. Battery Ventures portfolio postings stay open a median of 29 days.

Share of postings closed within
  1. 1.8%1 day
  2. 4.0%3 days
  3. 9.5%7 days
  4. 15.6%14 days
  5. 33.9%30 days
This job: posted yesterday

Battery Ventures portfolio median: 29 days open

The posting

Our Mission

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.

About the Role

You will build the evaluation infrastructure that researchers use to understand model capabilities, guide training decisions, and assess readiness for release. The role spans evaluation authoring, distributed execution, and results analysis, with a particular focus on pre-training. You will work directly with researchers to make evaluations faster, more reliable, and easier to use, ensuring that the results accurately reflect the models being evaluated.

  • Design and evolve evaluation abstractions. Build clear APIs and data models for evaluation tasks, datasets, configurations, model outputs, and scores, making it straightforward to add new evaluations and extend existing workflows.
  • Build scalable, reliable execution infrastructure. Develop distributed systems for evaluating model checkpoints throughout training and ahead of release, with robust orchestration, observability, and recovery from failures.
  • Optimize evaluation and inference performance. Profile inference servers and distributed execution to improve throughput, GPU utilization, and time to results.
  • Establish evaluation correctness and reproducibility. Build validation and regression tests, version datasets and configurations, and preserve the provenance needed to reproduce results and investigate discrepancies.
  • Build researcher-facing tools. Create intuitive interfaces and dashboards for inspecting model outputs, comparing checkpoints and experiments, and debugging unexpected behavior.
  • Operate and monitor evaluations as training progresses. Track evaluation workloads and results, triage failures, and develop alerts and diagnostics that help distinguish changes in model quality from inference or evaluation issues.
  • Turn research needs into reusable systems. Partner directly with researchers to identify bottlenecks, prioritize high-impact improvements, and turn bespoke workflows into self-service tools that work across teams.

About You

  • Strong proficiency in Python, C++, or Rust, with the ability to reason deeply about concurrency, memory behavior, and performance.
  • Experience designing, building, and operating reliable, maintainable distributed systems, with sound judgment about interfaces, failure modes, and architectural tradeoffs.
  • Working knowledge of model inference and evaluation, including how implementation and configuration choices can affect measured model quality.
  • Ability to investigate complex problems across system boundaries, follow evidence, and identify root causes rather than just address symptoms.
  • Strong product judgment and experience translating the needs of researchers or other technical users into clear APIs and usable tools.
  • Comfort owning ambiguous problems end to end, prioritizing competing research needs, and balancing rapid delivery with long-term reliability and maintainability.
  • Attention to detail and the persistence to investigate subtle discrepancies in model outputs, evaluation results, and system behavior.

What We Offer:

We believe that to make intelligence open and accessible to all, you need to start at the foundation. Joining Reflection means building from the ground up as part of a talent-dense team. You will help define our future as a company, and help define the future of open foundational models.

We want you to do the most impactful work of your career with the confidence that you and the people you care about most are supported.

  • Top-tier compensation: Salary and equity structured to recognize and retain our talent globally.
  • Stock options: Everyone who joins and contributes to Reflection's success gets to share in the upside through stock options.
  • Health & wellness: Comprehensive medical, dental, vision, and life, with an annual wellness allowance.
  • Meals: Lunch and dinner are provided in the office daily.
  • Life & family: 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.
  • Vacation days: Unlimited paid time off in the U.S. and 30 days in the U.K.
  • Sponsorship support: We sponsor visas to help exceptional talent join our team and support long-term immigration pathways where applicable.
  • Team building: We have regular off-sites, happy hours, and team celebrations.

Export Control Notice: This position may require access to technology or source code subject to the U.S. Export Administration Regulations. Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export control laws, which may require the Company to seek government authorization.

Candidate and Employee Privacy Notice: We collect and process personal data about applicants for the purposes described in our Candidate and Employee Privacy Notice

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Battery Ventures portfolio's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Battery Ventures portfolio's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Battery Ventures portfolio's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.