Skip to content

Open nowPosted 53 days ago

Software Engineer, Agent Platform

Retool25 open roles

Where
New York, San Francisco
Work mode
Hybrid
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowSoftware Engineer, Agent PlatformRetool · New York, San Francisco
  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Retool's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

13.8% of postings close within 7 days. Measured by our own scanner across the market. Retool postings stay open a median of 41 days.

Share of postings closed within
  1. 2.2%1 day
  2. 4.7%3 days
  3. 13.8%7 days
  4. 22.3%14 days
  5. 37.8%30 days
  6. 59.0%60 days
This job: posted 53 days ago

Retool median: 41 days open

The posting

WHY WE’RE LOOKING FOR YOU

We’re building AI-native products where model behavior is part of the product, not just an implementation detail. As LLMs become more capable, the bottleneck is no longer access to models—it’s making non-deterministic systems reliable, evaluable, and trustworthy in production.

We’re hiring AI Engineers to own that problem end to end. This is not a role for someone who simply integrates AI APIs into features. It’s for engineers who take responsibility for how probabilistic systems behave over time, how quality is measured in the presence of variance, and how capabilities improve without regressing.

If model regressions, subtle behavior drift, or edge-case failures keep you up at night—and you enjoy that kind of ownership—we want to talk.

WHAT YOU’LL DO

As an AI Engineer, you’ll own model-driven behavior in production systems, working across product, infrastructure, and evaluation layers. Your work will directly shape what users experience—and how confidently the team can ship. You might:

Own the behavior of AI-powered features across multiple product surfaces, including quality, safety, variance, and failure modes Design and evolve prompting, retrieval, routing, and tool-use strategies that embrace non-determinism while bounding its downside Build and maintain evaluation systems that measure model performance using statistical signals, distributions, and trends—not just pass/fail tests Detect, diagnose, and resolve non-deterministic failures such as hallucinations, partial correctness, instruction drift, or sensitivity to context changes Define and implement guardrails, fallbacks, and degradation paths that keep systems useful even when models behave unexpectedly Partner with product and infra teams to decide when probabilistic behavior is “good enough” to ship—and when it isn’t Influence model selection, model behavior, and tool design to balance quality, cost, latency, and robustness for real user workflows

You’ll work across the stack (e.g., TypeScript, Node.js, React), but your leverage won’t come from code volume alone—it will come from shaping runtime behavior with precision, measurement, and intent.

WHAT THIS ROLE IS (AND IS NOT) This role is:

Accountable for AI behavior, not just system correctness Grounded in evaluation, iteration, and regression prevention under non-determinism Comfortable designing systems where outputs vary, confidence is probabilistic, and correctness is contextual Focused on shipping dependable products on top of imperfect components

This role is not: Adding LLM calls to existing features and moving on Treating models as black boxes with undefined behavior Shipping AI features without owning their long-term reliability, drift, or user trust

THE SKILLSET YOU’LL BRING 6+ years of professional engineering experience, with ownership over complex systems in production Demonstrated experience owning AI/LLM behavior beyond basic integration, including mitigation of variance and failure modes Comfort reasoning about probabilistic systems and tradeoffs (quality vs. cost, recall vs. precision, speed vs. robustness) Experience designing or maintaining evaluation frameworks, golden datasets, regression detection, or human-in-the-loop feedback loops Strong product intuition—you care deeply about what “good” looks like even when outputs are non-deterministic Ability to operate independently in ambiguous problem spaces and set quality standards others rely on Strong opinions, weakly held—you iterate quickly and adjust based on evidence and observed runtime behavior

BONUS POINTS Experience with RAG, agentic systems, or tool-using models in production Familiarity with vector databases, embeddings, or retrieval pipelines Exposure to fine-tuning, model routing, or post-training techniques Experience building shared AI infrastructure used by multiple teams History of mentoring engineers on designing for non-determinism and evaluation-driven development

WHO YOU’LL WORK WITH

You’ll join a small, senior team focused on advancing AI capabilities across the product. You’ll collaborate closely with product engineers, infra engineers, designers, and PMs—often acting as the final owner of AI behavior and quality before features reach users.

Your work will set standards that others build on. If you enjoy being the person teams rely on when AI behavior matters most—and certainty is never guaranteed—you’ll thrive here.

READY TO BUILD RELIABLE AI SYSTEMS?

If you’re excited to move beyond demos and take real ownership of non-deterministic behavior in production—defining quality, preventing regressions, and turning variability into a strength—we’d love to meet you.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Retool's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.
  3. We apply on Retool's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.
  4. Every reply, sorted

    Retool's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.
  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.