Skip to content

Open nowPosted 25 days ago

AI Quality Engineer

aviva130 open roles

Where
Poland - Warsaw - ASEC
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowAI Quality Engineeraviva · Poland - Warsaw - ASEC
  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on aviva's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

14.3% of postings close within 7 days. Measured by our own scanner across the market. aviva postings stay open a median of 31 days.

Share of postings closed within
  1. 2.2%1 day
  2. 4.7%3 days
  3. 14.3%7 days
  4. 23.1%14 days
  5. 39.5%30 days
  6. 59.2%60 days
This job: posted 25 days ago

aviva median: 31 days open

The posting

Hi, we're glad you're here! We are hiring to our Aviva Services Excellence Centre! Take a look at our job description - maybe it suits you or one of your friends?

We are looking for an AI Quality Engineer to join Aviva Services Excellence Centre (ASEC) and help build confidence in the quality, safety and reliability of AI-powered products.

This is a hands-on role for someone who enjoys understanding how AI systems behave, identifying weaknesses before they become customer problems, and creating reliable ways of measuring whether AI solutions are actually delivering the outcomes they were designed for. You will work across conversational AI, autonomous agents and other AI-powered systems, partnering closely with engineers, product teams and business stakeholders to ensure solutions are robust, trustworthy and ready for production use.

What will you be doing?

  • Translating business expectations, user outcomes and operational risks into measurable evaluation criteria.
  • Defining what successful, partially successful and unsuccessful system behaviour looks like, including the gradient between clear success and clear failure.
  • Designing evaluation approaches for conversational, multi-turn and non-deterministic AI systems.
  • Building and maintaining representative evaluation datasets covering common, rare, boundary and high-risk scenarios.
  • Assessing dataset quality, including coverage, class balance, duplicates, data leakage, label quality, synthetic-versus-production differences and dataset freshness.
  • Selecting appropriate manual, deterministic, model-based and statistical evaluation methods.
  • Analysing errors by category, user journey, risk level and likely root cause.
  • Distinguishing model failures from failures in retrieval, chunking, ranking, prompts, tools, guardrails, integrations, source data and evaluators.
  • Validating automated evaluators, including LLM-as-a-Judge, against human judgement.
  • Detecting blind spots, noise, bias and misleading aggregation in evaluation results.
  • Creating baselines, controlled comparisons and regression tests that allow meaningful comparison between system versions.
  • Prioritising tests according to the probability and impact of failure rather than coverage alone.
  • Communicating results, uncertainty, limitations and residual risks in a form that supports release and product decisions.
  • Proactively identifying emerging failure modes based on system changes, production observations and research.

What we're looking for:

  • Professional experience in quality engineering, software testing, test automation, data science, software engineering or a related field (we can consider different seniority levels).
  • Experience or strong interest in working with AI-powered solutions, either through developing, testing, evaluating or supporting them.
  • Good Python proficiency and experience writing clean, maintainable code for analysis, automation or testing purposes.
  • Interest in understanding how AI systems behave, how reliable they are, and how their quality can be measured.
  • Strong analytical and problem-solving skills, with the ability to investigate issues and identify root causes.
  • Ability to translate business objectives, customer outcomes and risks into practical evaluation approaches.
  • Strong communication skills and the ability to explain findings to both technical and non-technical stakeholders.
  • A curious, proactive mindset and a genuine interest in AI, emerging technologies and system robustness.

Nice to have:

  • Experience evaluating, testing or monitoring AI-powered systems in production environments.
  • Working with structured data, especially JSON, logs and tabular evaluation results.
  • Understanding of LLM behaviour, tokenisation, context limits, parameters and non-determinism.
  • Understanding of multi-turn context, RAG, grounding, faithfulness, hallucinations, guardrails, attacks (like prompt injection etc.) and tool/function calling.
  • Software engineering practices: Git, modular code, configuration management and automated tests.

In Aviva Services Excellence Centre, we provide:

  • Contract of employment
  • Performance Bonus
  • Private medical care (ENEL-MED)
  • Cafeteria system of benefits (e.g. MultiSport card)
  • Additional parent privileges: - baby leave – 6 weeks leave for second parent - shortened working time after parental leave
  • Employer-funded group insurance
  • Employee Shares Plan
  • Extra day off for birthday
  • Paid time off to volunteer
  • Additional leave acquired over the years in the company (even 5 days more in a year)
  • Wellbeing Leave - additional two weeks paid leave after 10 years of service
  • Psychological support consultations
  • Access to the e-learning platforms
  • Internal employee referral program
  • Hybrid working model from Poland with a few days in Warsaw office per month
  • Attractive office location in Warsaw - Gdański Business Center
  • Additional entitlements for people with disabilities - Work Without Barriers
  • Work environment that promotes diversity - Diversity Charter

Do you see yourself in Aviva Services Excellence Centre? Be sure to apply!

Aviva is for everyone!

We’ve been ranked the #1 Best Workplace in Poland for a second year in a row and a top choice for Millennials by Great Place to Work®!

As a signatory to the Diversity Charter, Aviva applies a policy of equal treatment and creates a work environment that is friendly to people with disabilities. We do not require from candidates the date of birth, marital status and attached photos to your CV. We encourage all individuals regardless of gender to apply.

Most of our people are smart working – spending also time in our office - combining the benefits of flexibility, with time together with colleagues.

We’re inclusive - we welcome applications from people with diverse backgrounds and experiences.

Excited but not sure you tick every box? Research tells us that women, particularly, feel this way. So, regardless of gender, why not apply and try!

To find out more about current job offers at Aviva take a look here.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against aviva's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.
  3. We apply on aviva's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.
  4. Every reply, sorted

    aviva's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.
  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.