Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 7 hours agoWe saw it 28 min after it went up

Senior Software Engineer – LLM Evaluation

Jobgether4,270 open roles

Where
US
Work mode
Remote
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowSenior Software Engineer – LLM EvaluationJobgether · US

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Jobgether's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

8.0% of postings close within 7 days. Measured by our own scanner across the market. Jobgether postings stay open a median of 5 days.

Share of postings closed within
  1. 1.6%1 day
  2. 3.6%3 days
  3. 8.0%7 days
  4. 15.0%14 days
  5. 34.2%30 days
This job: posted 7 hours ago

Jobgether median: 5 days open

The posting

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer – LLM Evaluation based in United States.

This part-time consulting opportunity is designed for experienced software engineers interested in advancing the evaluation of large language models. You will curate high-quality code, develop technical solutions, and evaluate AI-generated software against real-world engineering standards. The role spans multiple programming languages and covers the complete software-development lifecycle, from architecture and prototyping through production and maintenance. You will design verification mechanisms and contribute to benchmarks that make AI evaluation more rigorous, consistent, and reproducible. Your engineering expertise will help research teams identify model strengths, weaknesses, and recurring coding errors. You will collaborate remotely with technical and research professionals on projects at the intersection of software engineering and AI. The flexible contractor structure requires a minimum of 10 hours per week, with the possibility of working up to 40 hours depending on project needs.

Accountabilities:

  • Curate high-quality code examples and technical datasets for model training, benchmarking, and evaluation.
  • Develop accurate solutions to software-engineering tasks and correct or improve implementations across multiple programming languages.
  • Work with technologies such as Python, JavaScript, ReactJS, C/C++, Java, Rust, and Go as relevant to project assignments.
  • Evaluate AI-generated code for technical correctness, maintainability, efficiency, scalability, reliability, and adherence to professional engineering standards.
  • Identify implementation weaknesses, recurring coding errors, and patterns that reveal limitations in AI-generated software.
  • Provide clear, structured rationales explaining technical evaluation decisions and assessment outcomes.
  • Build agents and automated mechanisms capable of assessing code quality and verifying software solutions.
  • Design reliable checks that support consistent and reproducible evaluation across repeated engineering tasks.
  • Evaluate AI capabilities across the full software-development lifecycle, including prototyping, architecture, API design, production implementation, experimentation, launch, monitoring, and maintenance.
  • Assess model-generated technical reasoning and decisions against practical software-engineering expectations.
  • Collaborate with research and cross-functional technical teams to define evaluation strategies and improve coding benchmarks.
  • Contribute to datasets used for training and benchmarking while maintaining rigorous standards for quality and technical accuracy.
  • Help improve coding-focused evaluation systems through iterative analysis of model performance.
  • Complete all work without using confidential, proprietary, unreleased, employer-restricted, client-restricted, or otherwise protected code, datasets, architecture materials, or technical information belonging to any third party.
  • 3+ years of professional software-engineering experience.
  • Strong full-stack development capabilities and experience building scalable, production-grade software.
  • Strong understanding of software architecture, system design, API design, and production implementation.
  • Deep knowledge of software development, debugging, code review, and code-quality assessment.
  • Demonstrated ability to review, troubleshoot, and improve complex software implementations.
  • Proficiency in one or more relevant programming languages, including Python, JavaScript, Java, C++, Rust, or related technologies.
  • Experience with ReactJS, C, Go, or additional programming languages is valuable depending on project requirements.
  • Familiarity with software monitoring, operational maintenance, and production reliability.
  • Ability to reason across the complete software-engineering lifecycle and evaluate technical decisions from development through ongoing operation.
  • Strong analytical and problem-solving skills, combined with a rigorous and detail-oriented approach to technical evaluation.
  • Excellent written and verbal communication skills, including the ability to produce concise and well-structured evaluation rationales.
  • Ability to distinguish between technically correct implementations and solutions that may introduce scalability, reliability, maintainability, or architectural concerns.
  • Comfortable working independently and collaborating remotely with research and technical teams.
  • Must be based in the United States, Canada, or an eligible Western European country.
  • Willingness to complete a required AI video interview as part of the application process.
  • Fully remote, part-time independent contractor engagement.
  • Flexible workload starting at 10 hours per week, with the potential to work up to 40 hours per week.
  • Approximately one-month initial project duration, with potential extension based on performance and project fit.
  • Opportunity to contribute directly to advanced LLM evaluation, coding benchmarks, and AI-assisted software-engineering research.
  • Exposure to cutting-edge AI evaluation workflows involving realistic software-development scenarios.
  • Opportunity to apply professional software-engineering expertise to improve how AI systems are evaluated.
  • Flexible consulting structure suited to experienced engineers seeking project-based work.
  • International remote collaboration with research and technical professionals.
  • Application process expected to take approximately 15–30 minutes, followed by a required AI video interview.
  • Compensation is project-specific and was not specified in the available job materials.
  • Medical insurance and paid-leave benefits are not included under the independent contractor arrangement.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Jobgether's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Jobgether's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Jobgether's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.