Skip to content

Open nowPosted 130 days ago

Research Engineer - Evals

Firecrawl16 open roles

Pay
$250,000 – $290,000 a year
Where
San Francisco HQ; Toronto Hub
Work mode
Hybrid
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowResearch Engineer - EvalsFirecrawl · San Francisco HQ; Toronto Hub
  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Firecrawl's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

14.3% of postings close within 7 days. Measured by our own scanner across the market. Firecrawl postings stay open a median of 81 days.

Share of postings closed within
  1. 2.2%1 day
  2. 4.7%3 days
  3. 14.3%7 days
  4. 23.1%14 days
  5. 39.5%30 days
  6. 59.2%60 days
This job: posted 130 days ago

Firecrawl median: 81 days open

The posting

RESEARCH ENGINEER - EVALS

You'll build the evaluation systems that tell us whether Firecrawl actually works. That sounds simple. It isn't. Our core promise, convert any URL into clean, structured, LLM-ready data reliably, is hard to measure rigorously across millions of different websites, formats, and edge cases. As the systems we're measuring get more complex, the question "did that work?" gets harder, not easier.

This isn't an eval role where you inherit a framework and run benchmarks. You'll design the metrics, build the pipelines, generate the datasets, and own the feedback loop from output quality back to model and product decisions. If you care about what "good" actually means and have the engineering depth to measure it, this is the role.

Salary Range: $250,000–$290,000 USD/year (SF) / $210,000–$224,000 CAD/year (Toronto)

Equity Range: Competitive equity. Details shared during the process.

Location: San Francisco, CA (SF HQ) or Toronto, ON (Toronto Hub). Hybrid, onsite 3+ days a week.

Job Type: Full-Time

Experience: 4+ years in ML, research engineering, or data-heavy backend, with real evaluation work

Work Authorization: Open to candidates already authorized to work in the US or Canada. In the US, we also support H-1B transfers, F-1 OPT and STEM OPT, TN (Canada and Mexico), and E-3 (Australia). We’re not able to sponsor new H-1B visas for candidates outside the US. For Canada, we’ll consider work permit sponsorship case by case through our Toronto Hub.

ABOUT FIRECRAWL

Firecrawl helps AI agents get the data they need through Alexandria (its library of official data providers), its own indexes, and the live web. Developers first turned to us because getting clean data from the web is harder than it looks. Today, Firecrawl has 200k+ stars on GitHub, and more than 1.5 million people build with it, including teams at 67 of the Fortune 100.

The open web is only part of what agents need. Much of the most useful knowledge, like company data, financial records and research, sits behind paywalls and private APIs. Firecrawl Alexandria brings that into the same API, and we pay the providers whose data agents use. We believe people who share what they know should be compensated.

OUR VALUES

- Bias to action: We move fast, ship hard, and don’t wait to be told.

- Realistic about AI: We’re clear-eyed about what AI can and can’t do, and we keep pushing to use it where it actually works.

- All in: The next few years matter a lot, and we’re all in for them.

- Human connection: As intelligence gets cheaper, relationships with customers, partners, and each other matter more.

WHAT YOU'LL DO

- Design the metrics that define what "good output" actually means across millions of sites, formats, and edge cases

- Build the pipelines and harnesses that measure quality rigorously and at scale

- Generate and curate the datasets that make evaluation trustworthy

- Own the feedback loop from output quality back to model and product decisions

- Turn "did that work?" into an answer the whole team can act on

WHAT WE'RE LOOKING FOR

- You have the engineering depth to build real evaluation systems, not just run existing ones

- You care deeply about what "good" means and how to measure it rigorously

- You're comfortable owning ambiguous problems where the metric itself has to be invented

- You move fast and close the loop. You'd rather ship, measure, and iterate than perfect on paper

WHAT WE'RE NOT LOOKING FOR

- Someone who only wants to run benchmarks someone else designed

- A pure researcher who won't build the systems, or a pure engineer who won't think about methodology

- Someone who needs a fully-specced ticket to start

A NOTE ON PACE

We operate at an absurd level of urgency because the window for what we're building won't stay open forever. If that excites you, keep reading. If it doesn't, no hard feelings, but this role probably isn't for you.

BENEFITS & PERKS

AVAILABLE TO ALL EMPLOYEES

- Salary that makes sense: $250,000–$290,000 USD/year (SF) / $210,000–$224,000 CAD/year (Toronto), based on impact, not tenure

- Own a piece: Gain competitive equity in what you're helping build

- Generous PTO: 15 days mandatory, anything after 24 days, just ask (holidays excluded). Take the time you need to recharge

- Parental leave: 12 weeks fully paid, for all parents

- Wellness stipend: $100 USD/month for the gym, therapy, massages, or whatever keeps you human

- Learning & Development: Expense up to $1,000 USD/year toward anything that helps you grow professionally

- Team offsites: A change of scenery, minus the trust falls

- Sabbatical: 3 paid months off after 4 years, do something fun and new

AVAILABLE TO US-BASED FULL-TIME EMPLOYEES

- Full coverage, no red tape: Medical, dental, and vision (100% for employees, 50% for partner and kids). No weird loopholes, just care that works

- Life & Disability insurance: Employer-paid basic life and AD&D, short-term disability, and long-term disability. Coverage for life's curveballs

- Virtual care and a health guide: Teladoc for the couch doctor visit, plus Rightway to answer coverage questions and fight billing errors for you

- Mental health: Talkspace, therapy and psychiatry on your schedule

- Fertility and family building: Carrot, covering you and your partner

- EAP: Free confidential counseling, legal and financial consults, and online will prep through Guardian

- 401(k) plan: Retirement might be a ways off, but future-you will thank you

- Pre-tax benefits: HSA, FSA, and commuter benefits to help your wallet out a bit

- Supplemental options: Extra life and AD&D, accident, critical illness, hospital indemnity, plus pet, legal, and identity protection through MetLife

AVAILABLE TO CANADA-BASED FULL-TIME EMPLOYEES

- Full coverage, no red tape: Extended health, dental, and vision through Manulife (Diamond, the top tier), 100% employer-paid for you, your partner, and your kids

- Life & Disability insurance: Employer-paid life, AD&D, short-term disability, and long-term disability. Coverage for life's curveballs

- Virtual care: Dialogue Premium, so you can see a doctor or nurse from your couch, any hour

- Mental health: Talkspace Elite, therapy and psychiatry on your schedule

- Fertility and family building: Carrot, covering you and your partner

- Retirement: Group RRSP through Wealthsimple, so future-you can thank you

AVAILABLE TO SF-BASED EMPLOYEES

- SF HQ perks: Snacks, drinks, team lunches, intense ping pong, and peak startup energy

- E-Bike transportation: A loaner electric bike to get you around the city, on us

AVAILABLE TO TORONTO-BASED EMPLOYEES

- Toronto Hub perks: Snacks, drinks, team lunches, glass-walled views down University Avenue, and a home base steps from Union Station

- Transit, covered: A PRESTO card loaded for GO Transit, subway, and streetcar, plus station parking if you drive to the train. Winter-proof, on us

INTERVIEW PROCESS

Application Review: Send us your work and a quick note on why this excites you. Show us what you've built: eval systems, metrics you designed, datasets you created, quality problems you measured. We care about what you've shipped, not where you went to school.

Intro Chat (~25 min): A quick conversation to get to know each other before we go deep. We'll talk about what you've been working on, what drew you to Firecrawl, and what you're looking for in your next role. Time for your questions too.

Technical Chat (~45 min): We'll dig into a real problem from our world: how you'd measure whether messy web output is actually "good," and build the system to prove it. Come ready to think out loud. We care how you reason, not whether you memorized the answer.

Founder Chat (~25 min): Culture, pace, ownership, and how you like to work. Time for your questions too.

Paid Work Trial (~40 Hours): Work with the team on a real, scoped evaluation problem, paid at a contractor rate. It's the truest signal for both sides. You see what building at Firecrawl actually feels like, and we see how you ship. Remote-friendly, and we'll flex around your current commitments.

Decision: We move fast after the trial.

If you care about what "good" actually means and have the engineering depth to measure it, you should join us.

👉 Apply now.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Firecrawl's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.
  3. We apply on Firecrawl's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.
  4. Every reply, sorted

    Firecrawl's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.
  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.