Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 121 days ago

Senior QA Engineer, Forward Deployed

ellipsis-health18 open roles

Pay
$120,000 – $140,000 a year
Where
San Francisco - Hybrid
Work mode
Hybrid
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowSenior QA Engineer, Forward Deployedellipsis-health · San Francisco - Hybrid

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on ellipsis-health's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

7.8% of postings close within 7 days. Measured by our own scanner across the market.

Share of postings closed within
  1. 1.7%1 day
  2. 3.5%3 days
  3. 7.8%7 days
  4. 14.6%14 days
  5. 34.1%30 days
This job: posted 121 days ago

The posting

ABOUT THE TEAM

The Forward Deployed Team is our primary customer-facing unit, acting as the crucial bridge between our core product and our enterprise clients. This fast-moving team manages the end-to-end lifecycle of enterprise conversational AI deployments, from pre-sales Statements of Work (SOWs) through go-live.

We maintain daily, high-touch interactions with customers to communicate workflow progress, manage User Acceptance Testing (UAT) deadlines, integrate client feedback, and ensure that deployments are executed rapidly while maintaining rigorous quality standards.

Ellipsis Health is located in the San Francisco Bay Area, but we are open to remote candidates within the United States.

ABOUT THE ROLE

As a Forward Deployed QA Engineer, you will occupy a critical, high-impact role dedicated to ensuring the reliability, stability, and quality of our core conversational AI product, Sage, across diverse client workflows.

This role bridges the gap between Quality Assurance, AI Engineering, and Production Operations. You will focus heavily on automated testing, prompt engineering validation, and rapid root cause analysis (RCA) of Large Language Model (LLM)-driven behaviors in fast-paced, real-world deployments.

RESPONSIBILITIES:

- Workflow Mapping & Test Case Generation: Deeply analyze assigned client workflows to design robust, comprehensive positive and negative test cases that safeguard system stability.

- AI-Driven Test Automation: Build and execute automated test scenarios by configuring shadow agents.

- Prompt Evaluation & Optimization: Apply a strong understanding of prompt awareness to draft, refine, and evaluate prompts used within the testing framework to accurately simulate user behaviors and edge cases.

- End-to-End Testing Execution: Strategically deploy specific testing methodologies including Sanity, Smoke, Regression, and Functional testing - determining the exact environment (staging, pre-production, production) and timing for each execution.

- Deployment Cadence & Cross-Functional Collaboration: Partner closely with engineering teams during release cycles to proactively identify, triage, and unblock technical roadblocks, ensuring the product is continuously deployment-ready.

- Daily LLM Defect RCA: Perform rigorous, daily root cause analysis on LLM-specific failures inherent to generative AI, including hallucinations, high latency, and logic deviations.

- Live Production Call Debugging: Investigate live customer calls and production incidents in real time to unblock critical production use cases.

- Audio & Transcription Validation: Query and analyze historical call transcripts, system behaviors, and audio data pipelines to pinpoint where a conversational workflow broke down.Speech-to-Speech (S2S) Pipeline Monitoring: Monitor and evaluate the end-to-end voice AI pipeline. This involves analyzing Automatic Speech Recognition (ASR) accuracy, managing audio-to-text latency issues, and understanding general Speech-to-Speech mechanics alongside the stability of the core Knowledge Base feeding the AI.

- Advanced Evaluation Frameworks: Maintain a strong conceptual understanding of advanced LLM evaluation paradigms and tools such as LLM-as-a-judge - to remain aware of how AI response quality and accuracy are programmatically graded at scale.

- Telephony & Call Flow Awareness: Possess a foundational understanding of real-world call management and telephony routing concepts, including how the system is expected to navigate warm transfers, blind transfers, and voicemail detection workflows.

QUALIFICATIONS:

- Experience in QA Engineering: Strong background in software quality assurance, with a proven track record of designing, executing, and managing end-to-end test strategies (Smoke, Sanity, Regression, Functional).

- LLM & Generative AI Expertise: Hands-on experience or deep technical familiarity with troubleshooting LLM behaviors, diagnosing hallucinations, managing latency, and using LLM call-tracing tools.

- Technical & Logging Proficiency: Ability to comfortably write SQL queries (specifically PostgreSQL) to pull data logs and navigate cloud infrastructure logs (such as GCP) to perform rapid root-cause analysis.

- Voice & Conversational AI Domain Knowledge: Foundational understanding of Speech-to-Speech pipelines, including Automatic Speech Recognition (ASR), audio-to-text workflows, and core knowledge base integrations.

- Telephony Foundations: Basic familiarity with enterprise telephony routing, call management mechanics (warm/blind transfers), and voicemail detection systems.

- Client-Facing Capability: Strong communication skills and the professional agility required to manage UAT timelines, coordinate with client stakeholders, and support rapid production deployments.

SALARY AND BENEFITS

We offer competitive salary and benefits, including 401(k) matching, health, vision, and dental insurance, and very flexible paid time off.

The typical salary range for this role is $120,000 to $140,000 USD, depending on skills, qualifications, and relevant experience.

BACKGROUND CHECKS

As a health technology company, we reserve the right to run background checks on candidates to whom we extend offers, in compliance with applicable laws. We evaluate candidates holistically and comply with all “ban the box” regulations.

ASSISTANCE

If you have a disability or require accommodations during the application or recruitment process, please contact [email protected].

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against ellipsis-health's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on ellipsis-health's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    ellipsis-health's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.