Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 6 days agoWe saw it 75 min after it went up

AI Testing Engineer, Amazon Bedrock

NTT DATA, Europe & LATAM, Branch in USA, Inc.32 open roles

Pay
$20 – $25 an hour
Where
LATAM
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowAI Testing Engineer, Amazon BedrockNTT DATA, Europe & LATAM, Branch in USA, Inc. · LATAM

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on NTT DATA, Europe & LATAM, Branch in USA, Inc.'s own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

8.0% of postings close within 7 days. Measured by our own scanner across the market. NTT DATA, Europe & LATAM, Branch in USA, Inc. postings stay open a median of 9 days.

Share of postings closed within
  1. 1.6%1 day
  2. 3.6%3 days
  3. 8.0%7 days
  4. 15.0%14 days
  5. 34.2%30 days
This job: posted 6 days ago

NTT DATA, Europe & LATAM, Branch in USA, Inc. median: 9 days open

The posting

Location: 100% remote, seeking candidates located in LATAM.

Contract Length: 1 year, with potential for extension.

Contract Rate: $20-25 per hour in USD. Final rate will be determined based on relevant experience and overall fit for the position.

NTT DATA is seeking an AI Testing Engineer, Amazon Bedrock to define and execute the quality strategy for Generative AI applications and AI agents used in Customer Service and Contact Center scenarios. This role will evaluate AI outputs across deterministic and non-deterministic scenarios using automation, semantic and statistical analysis, human review, and adversarial testing while partnering with AI engineers and functional QA teams.

Responsibilities:

  • Define the testing and evaluation strategy for Generative AI applications and AI agents.
  • Create and maintain evaluation datasets, golden datasets, and regression suites covering representative business scenarios.
  • Design single-turn and multi-turn conversational test cases, including context retention and workflow transitions.
  • Evaluate AI outputs for accuracy, relevance, completeness, groundedness, faithfulness, consistency, tone, safety, and compliance.
  • Build automated evaluation pipelines and integrate AI-quality checks into CI/CD and release processes.
  • Detect regressions introduced by changes to prompts, models, knowledge sources, retrieval strategies, tools, or guardrails.
  • Design negative and adversarial test scenarios covering prompt injection, jailbreaks, hallucinations, PII leakage, unsafe responses, and out-of-scope requests.
  • Validate fallback behavior, uncertainty handling, tool execution, error handling, and escalation to human agents.
  • Perform root-cause analysis for incorrect or undesirable AI outputs and collaborate with AI engineers on remediation.
  • Define measurable quality thresholds and release acceptance criteria for AI-agent capabilities.
  • Produce quality metrics, evaluation reports, dashboards, and trend analysis for project and product stakeholders.
  • Support UAT preparation, defect triage, production monitoring, and continuous improvement of AI behavior.

Required Experience:

  • 5+ years of experience in software testing, quality engineering, or test automation with hands-on automation capabilities.
  • 2+ years of experience with Amazon Bedrock.
  • Practical understanding of Generative AI / LLM behavior and the challenges of testing non-deterministic systems.
  • Experience designing test scenarios for conversational applications, chatbots, virtual assistants, or AI agents.
  • Experience with Python and/or JavaScript / TypeScript for test automation and data processing.
  • Experience testing REST APIs and backend services.
  • Experience working with structured test data in formats such as JSON and CSV.
  • Understanding of evaluation concepts such as groundedness, faithfulness, relevance, hallucination detection, semantic similarity, and human evaluation.
  • Experience with Git, CI/CD, automated regression testing, and defect-management workflows.
  • Strong analytical skills and ability to investigate why an AI output failed rather than only identifying that it failed.
  • Professional proficiency in English.

Nice-to-Have:

  • Experience with AI evaluation frameworks such as RAGAS, DeepEval, Promptfoo, LangSmith, TruLens, or equivalent tools.
  • Experience evaluating RAG systems, retrieval quality, or enterprise knowledge assistants.
  • Experience with Amazon Bedrock and AWS services.
  • Experience testing Amazon Connect, Contact Center, agent-assist, or Customer Service applications.

Education & Certifications:

  • Bachelor’s degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, Software Engineering, or a related technical field.

Required Equipment: This role requires the use of a personal laptop. Candidates must be comfortable with company security software and device management tools being installed on their laptop as part of the onboarding process and for the duration of the engagement.

About NTT DATA

NTT DATA is a Top 5 global IT services provider with more than 190,000 professionals across 50+ countries. We combine industry expertise with capabilities in consulting, technology, AI, cloud, applications, infrastructure, and connectivity to help organizations innovate, optimize, and transform. Through responsible innovation, we deliver meaningful business outcomes, accelerate client success, and create a positive impact on society.

Equal Opportunity Employer

NTT DATA is committed to hiring and retaining a diverse workforce. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, creed, sex, sexual orientation, gender identity, marital status, national origin, age, veteran status, disability, or any other protected class. NTT DATA is an Equal Opportunity Employer Male/Female/Disabled/Veteran and a VEVRAA Federal Contractor.

Req ID: 1355

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against NTT DATA, Europe & LATAM, Branch in USA, Inc.'s own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on NTT DATA, Europe & LATAM, Branch in USA, Inc.'s form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    NTT DATA, Europe & LATAM, Branch in USA, Inc.'s answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.