Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 27 days agoWe saw it 4 min after it went up

Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)

Patronus AI, Inc.17 open roles

Pay
$30 – $100 an hour
Where
Remote - India
Work mode
Remote
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowExpert: Software Engineer – AI Evaluation (Task-based Contract, Remote)Patronus AI, Inc. · Remote - India

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Patronus AI, Inc.'s own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

13.9% of postings close within 7 days. Measured by our own scanner across the market. Patronus AI, Inc. postings stay open a median of 19 days.

Share of postings closed within
  1. 2.2%1 day
  2. 4.7%3 days
  3. 13.9%7 days
  4. 22.4%14 days
  5. 38.1%30 days
  6. 59.0%60 days
This job: posted 27 days ago

Patronus AI, Inc. median: 19 days open

The posting

About Patronus AI

Patronus AI is a frontier lab developing simulation research and infrastructure to accelerate progress toward human-aligned AGI. We are on a mission to simulate all of the world’s intelligence.

We are the team behind some of the earliest and most influential research in AI evaluation like FinanceBench, Lynx, SimpleSafetyTests, CopyrightCatcher, Humanity’s Last Exam, and more. We are formerly AI researchers and engineers from companies like Meta AI, Amazon AGI, and Google. Our customers include foundation model labs and Fortune 500 enterprises like Adobe. We are backed by top-tier investors like Lightspeed Venture Partners, Notable Capital, Stanford University, Noam Brown, Gokul Rajaram, and more.

About the role

We are looking for experienced software engineers (task based contract) to design realistic, challenging tasks for frontier AI models and to judge how well those models handle them. You'll apply your day-to-day engineering judgment to show where AI gets code, reasoning and technical instructions right, and where it falls short.

What you’ll do

Depending on the project, you may:

  • Design realistic, multi-step coding and technical tasks that test an AI model's ability to reason, write code and follow precise instructions.
  • Build tasks around realistic workspace materials such as codebases, technical specs, architecture docs and datasets.
  • Evaluate AI-generated code and technical content for correctness, efficiency, clarity and alignment with real-world engineering practice.
  • Grade model outputs against rubrics and ground-truth answers, and write clear, structured feedback.
  • Help improve how AI understands programming, system design, data workflows and engineering best practices.
  • Work independently and asynchronously, meet deadlines, and collaborate with the Patronus team to keep quality high.
  • Test and refine content to reflect real-world complexity.
  • Review other contributors’ work for accuracy, clarity, and practical relevance.
  • Incorporate feedback promptly and flag gaps.

What we’re looking for

  • 2+ years of professional experience in software engineering. (internships excluded)
  • Experience with system design, data pipelines, ML or LLM-based applications.
  • Prior work in code review, technical writing, AI training or model evaluation.
  • Strong coding skills in at least one major language, such as Python, Java, C++, JavaScript, TypeScript or Go.
  • Experience building, reviewing and debugging production-grade code.
  • Ability to follow detailed technical guidelines, work independently, respond to feedback, and meet deadlines.
  • Excellent written communication and close attention to detail.
  • A bachelor's degree in computer science, engineering or a related field, or equivalent experience.

Our Standards

  • All tasks, project files and solutions are human-authored. LLM-generated code, content or text is not permitted.
  • Tasks and solutions must be original. Nothing may be copied from published or easily searchable sources, including tutorials and marketplace content.
  • Never include code, assets or project files from a current or former employer, client or studio, or anything covered by an NDA. All studios, projects and scenarios are fictional, and all code and content is your own.
  • Quality is judged on accuracy, rubric validity, difficulty and reviewer agreement, not volume.

Pay: $50–$100 per hour and some tasks would have a flat rate.

Location: Remote (India)

Type: Hourly, project/task-based contract (not a full-time role)

Commitment: Flexible, typically 15–20 hours per week; can be done alongside a full-time job

Applications are reviewed on a rolling basis. Placement depends on project needs and assessment results.

Patronus AI is an equal opportunity employer. We celebrate diversity in our workplace, and all qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or other legally protected characteristics.

By clicking ‘Apply’, you agree to both of the following:

1) Greenhouse's Terms of Service and Privacy Policy

2) Patronus AI, Inc. Privacy Policy.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Patronus AI, Inc.'s own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Patronus AI, Inc.'s form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Patronus AI, Inc.'s answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.