Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 6 hours agoWe saw it 29 min after it went up

Senior Software Engineer – LLM Evaluation

Jobgether3,826 open roles

Where
Canada
Work mode
Remote
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowSenior Software Engineer – LLM EvaluationJobgether · Canada

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Jobgether's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

8.0% of postings close within 7 days. Measured by our own scanner across the market. Jobgether postings stay open a median of 5 days.

Share of postings closed within
  1. 1.6%1 day
  2. 3.6%3 days
  3. 8.0%7 days
  4. 15.0%14 days
  5. 34.2%30 days
This job: posted 6 hours ago

Jobgether median: 5 days open

The posting

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer – LLM Evaluation based in Canada.

This part-time consulting opportunity is designed for experienced software engineers interested in shaping the evaluation of advanced large language models. You will curate high-quality code, develop software solutions, and assess AI-generated implementations against professional engineering standards. The role spans multiple programming languages and covers the full software-development lifecycle, from architecture and prototyping to deployment, monitoring, and maintenance. You will also help create automated verification mechanisms and benchmarks that make model evaluation more rigorous and reproducible. Working alongside research and technical teams, you will help identify model strengths, weaknesses, and recurring coding errors. Your engineering judgment will directly contribute to improving coding-focused AI evaluation systems and research. The engagement is fully remote and flexible, with a minimum commitment of 10 hours per week and the potential to work up to 40 hours weekly.

Accountabilities:

  • Curate high-quality software examples and datasets for model training, coding benchmarks, and technical evaluation.
  • Develop precise solutions to software-engineering tasks and correct or improve implementations across multiple programming languages.
  • Evaluate AI-generated code for technical correctness, maintainability, efficiency, scalability, reliability, and alignment with professional engineering standards.
  • Identify implementation weaknesses, recurring model-generated errors, and patterns that reveal gaps in AI coding capabilities.
  • Provide clear, structured rationales supporting technical evaluation decisions.
  • Build agents and automated mechanisms capable of assessing code quality and verifying software solutions consistently.
  • Develop reliable checks that enable reproducible evaluation across repeated engineering assignments.
  • Assess model capabilities across the complete software-development lifecycle, including prototyping, architecture, API design, implementation, experimentation, launch, monitoring, and maintenance.
  • Review AI reasoning and technical decisions against realistic production engineering expectations.
  • Collaborate with research and cross-functional technical teams to define evaluation strategies and improve coding-focused benchmarks.
  • Contribute to datasets used for training and benchmarking while maintaining rigorous standards for technical accuracy and quality.
  • Support iterative improvements to evaluation systems by comparing model performance against real-world software-engineering practices.
  • Complete all project work without using confidential, proprietary, unreleased, employer-restricted, client-restricted, or otherwise protected code, datasets, architecture materials, or technical information belonging to any third party.
  • 3+ years of professional software-engineering experience.
  • Strong full-stack development capabilities and experience building scalable, production-grade software.
  • Strong understanding of software architecture, system design, API design, and production implementation.
  • Deep knowledge of software development, debugging, code review, and code-quality assessment.
  • Demonstrated experience reviewing, troubleshooting, and improving complex software implementations.
  • Proficiency in one or more relevant programming languages, including Python, JavaScript, Java, C++, Rust, or similar technologies.
  • Experience with ReactJS, C, Go, or additional programming languages is valuable depending on project requirements.
  • Familiarity with software monitoring, operational maintenance, and production reliability.
  • Ability to reason across the full software-engineering lifecycle and assess technical decisions from development through ongoing operation.
  • Strong analytical and problem-solving abilities, with a rigorous and detail-oriented approach to technical evaluation.
  • Excellent written and verbal communication skills, including the ability to produce concise, well-structured evaluation rationales.
  • Ability to distinguish between technically correct solutions and implementations that may introduce scalability, reliability, maintainability, or architectural concerns.
  • Comfortable working independently and collaborating remotely with research, engineering, and other technical teams.
  • Based in Canada, the United States, or an eligible Western European country.
  • Willingness to complete a required AI video interview as part of the application process.
  • Fully remote, part-time independent contractor engagement.
  • Flexible workload starting at 10 hours per week, with the potential to work up to 40 hours per week.
  • Approximately one-month initial project duration, with potential extension based on performance and project fit.
  • Opportunity to contribute directly to advanced LLM evaluation, coding benchmarks, and AI-assisted software-engineering research.
  • Exposure to cutting-edge AI evaluation workflows and research involving real-world software-development scenarios.
  • Opportunity to apply professional engineering expertise to improve how AI systems are evaluated across the software lifecycle.
  • Flexible consulting structure suited to experienced engineers seeking project-based work.
  • International remote collaboration with research and technical professionals.
  • No medical insurance or paid-leave benefits are included under the independent contractor arrangement.
  • Compensation is project-specific and was not specified in the available job materials.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Jobgether's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Jobgether's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Jobgether's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.