The posting
Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products.
About the Role:
We are seeking an experienced Computational Life Sciences experts or Medicine expert to lead a team of technical trainers developing realistic, terminal-based scientific and engineering tasks for a frontier AI training and evaluation initiative.
In this role, you will oversee a pod of approximately 5–10 trainers, review technical tasks for accuracy, reproducibility, and evaluation quality, and ensure tasks reflect authentic computer science, software systems, and life science expert workflows. You will also mentor trainers, maintain quality standards, and drive consistent delivery of challenging technical tasks.
Key Responsibilities
- Review trainer tasks end to end, including task statements, biological and clinical datasets, reference files, pipelines, environments, reference solutions, and automated tests.
- Verify that tasks reflect authentic research or clinical analysis workflows and require genuine multi-step scientific reasoning.
- Check that tools, reference genomes, annotation versions, databases, and dependencies are pinned and that pipelines are reproducible.
- Validate scientific correctness, including genome builds, coordinate conventions, normalization, batch-effect handling, statistical assumptions, multiple-testing correction, units, dosing conventions, imaging orientation and spacing, and random-seed control.
- Ensure graders verify substantive outputs such as variant calls, differential expression results, cluster assignments, binding or structural metrics, survival estimates, model performance, segmentation accuracy, and pharmacokinetic parameters rather than formatting.
- Ensure tolerances account for legitimate stochastic and numerical variation without accepting incorrect analyses.
- Ensure tasks cannot be gamed through hard-coded outputs, reading test files, or bypassing the intended analysis.
- Confirm that all data is synthetic, public, appropriately licensed, and de-identified, with no protected health information or restricted-access data.
- Screen tasks for biosafety and dual-use concerns.
- Escalate content involving dangerous pathogens, toxins, or enhancement of harmful biological capabilities.
- Provide clear, actionable written feedback and track revisions through closure.
- Onboard and mentor trainers.
- Allocate work, track quality and throughput, and escalate blockers.
- Maintain pod documentation covering guidelines, recurring issues, and exemplary tasks.
What you'll do:
- Lead and mentor a pod of technical trainers, ensuring consistent quality, productivity, and timely delivery of scientific and engineering tasks.
- Design challenging, realistic agentic scientific workflows across the life sciences, requiring multiple computational steps rather than single-script or simple question-answer solutions.
- Develop realistic input files, scientific datasets, instructions, constraints, and expected deliverables.
- Build tasks requiring agents to inspect data, select appropriate methods, execute analyses, troubleshoot problems, and synthesize results.
- Create reproducible expert solutions and objectively verifiable ground truths, along with robust automated or semi-automated grading criteria.
- Validate scientific assumptions, calculations, code, intermediate outputs, and final answers, ensuring tasks are self-contained and executable in controlled, network-isolated environments.
What we're looking for:
- Ph.D., postdoctoral, or equivalent research experience in the life sciences, with strong demonstrated computational and scientific-programming experience.
- Strong scientific programming experience, particularly in Python.
- Worked as a POD-lead in the previous projects
- Ability to independently validate both scientific reasoning and computational outputs.
- Expertise in one or more areas including bioinformatics, computational genomics, systems biology, computational neuroscience, biostatistics, computational drug discovery, computational biochemistry, structural biology, protein engineering, or computational microbiology.
Preferred Qualifications
- Familiarity with tools such as samtools, bcftools, GATK, BWA, STAR, Salmon, DESeq2, edgeR, Seurat, Scanpy, Biopython, Bioconductor, Nextflow, or Snakemake.
- Familiarity with RDKit, OpenMM, GROMACS, AutoDock Vina, PyMOL, or Biopython structure tools.
- Familiarity with medical imaging and clinical tools such as pydicom, nibabel, SimpleITK, MONAI, FSL, lifelines, statsmodels, or FHIR data formats.
- Familiarity with Docker, Conda, Git, CI systems, automated testing, and HPC environments.
- Experience designing benchmarks, analysis challenges, or computational evaluations. Bonus Points
- Experience evaluating AI coding or terminal agents.
- Experience with data governance, IRB processes, or biosafety review.
- Industry experience in biotech, pharma, clinical research, or health technology.
- Publications or open-source contributions in computational biology, bioinformatics, or biomedical informatics.
- Prior experience in AI training or evaluation programs Total number of positions: 20 Location : Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Nigeria, Turkey
- Commitments Required: At least 4 hours per day and minimum 40 hours per week with overlap of 4 hours with PST
- Employment type : Contractor assignment (no medical/paid leave)
- Compensation: $20-30 per hour
- Duration of contract : 3 weeks
