Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowFirst seen 40 hours ago

AGENTIC AI APPLICATION TESTER

PM Consulting1,469 open roles

Where
Taguig, Metro Manila, Philippines
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowAGENTIC AI APPLICATION TESTERPM Consulting · Taguig, Metro Manila, Philippines

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on PM Consulting's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

11.8% of postings close within 7 days. Measured by our own scanner across the market. PM Consulting postings stay open a median of 1 days.

Share of postings closed within
  1. 2.1%1 day
  2. 4.4%3 days
  3. 11.8%7 days
  4. 18.6%14 days
  5. 35.6%30 days
  6. 45.1%60 days
This job: first seen 40 hours ago

PM Consulting median: 1 days open

The posting

ROLE SUMMARY We are looking for an Agentic AI Application Tester to lead quality assurance and testing for AI-powered workflow systems. This role bridges software quality engineering and AI evaluation — you will need both technical testing proficiency and a practical understanding of how agent-based, LLM-driven systems behave across the full lifecycle, from development through live production.

You will define evaluation strategies, build and maintain automated test infrastructure, monitor production health, and collaborate with engineering teams to continuously improve the safety, reliability, and performance of AI-enabled applications.

KEY ACCOUNTABILITIES • Own the definition and execution of testing and quality assurance strategies for AI-enabled workflows • Continuously evaluate and monitor system behavior in live production environments • Support auditability, risk management, and ongoing quality improvement initiatives across the organization

PRINCIPAL RESPONSIBILITIES Quality Strategy & Evaluation Design • Set quality criteria and testing strategies for agentic workflows, spanning accuracy, response latency, safety, compliance, and operational risk • Design and build automated evaluation harnesses to measure agent performance across key dimensions, including output hallucination rates, tool misuse, policy adherence, and task completion rates • Apply LLM evaluation frameworks to track output quality, detect regressions, and identify system drift over time

Test Automation & Infrastructure • Build and maintain automated test suites covering UI, API, and end-to-end workflow validation • Develop custom evaluation scripts and tooling to support continuous quality assessment of agent behaviors • Create and maintain runbooks for recurring failure modes and actively contribute to incident response processes

Production Monitoring & Reporting • Implement and manage continuous monitoring systems to identify anomalies, quality degradation, and emerging safety issues in production • Design and maintain dashboards and quality reports that surface meaningful metrics and trends for engineering teams and business stakeholders

Collaboration & Compliance • Partner with developers to iterate on prompts, tool configurations, and workflow designs based on test findings and quality data • Ensure all testing, logging, and monitoring activities meet data privacy, audit, and applicable regulatory standards

QUALIFICATIONS Essential Experience • Minimum 3 years of experience in QA, test automation, or DevOps — or at least 2 years with direct, hands-on experience testing AI or ML-enabled systems • Solid Python programming skills applied to test automation, evaluation harness development, and data analysis • Strong attention to detail, with a focus on identifying issues that materially affect system reliability and user trust • Ability to work effectively in an environment where tools, frameworks, and best practices are actively evolving • Collaborative approach, with a track record of using evidence-based findings to inform product and engineering decisions

Required Technical Skills • Programming: Python — used for test automation, evaluation harness development, and data analysis tasks • UI Automation: Playwright — for end-to-end testing of agent-driven workflow interfaces • AI Evaluation: hands-on experience with LLM evaluation frameworks covering quality assessment, drift detection, and regression analysis • Workflow Testing: API and agent workflow validation using custom-built scripts • Monitoring: production-level quality monitoring and anomaly detection implementation

Desirable Skills • Familiarity with Python testing frameworks such as Pytest or equivalent • SQL skills for querying logs, metrics, or evaluation datasets • Experience with observability and monitoring tools such as Prometheus, Grafana, or comparable platforms • Understanding of hallucination detection techniques and AI safety design principles • Exposure to CI/CD pipelines and Git-based development workflows

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against PM Consulting's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on PM Consulting's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    PM Consulting's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.