The posting
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Director of Engineering, IAEF based in the United States.
This executive engineering role leads the architecture, delivery, security, and growth of an AI evaluation platform focused on trustworthy AI. You’ll shape the technical foundation for evaluating AI efficacy, transparency, safety, and reliability in educational contexts. The position combines hands-on technical leadership with team building, platform strategy, operational excellence, and external representation. You’ll lead a distributed engineering organization while working closely with executive, evaluation science, security, and external stakeholders. The role offers significant ownership over platform architecture, evaluation workflows, developer ecosystems, and engineering practices. You’ll also contribute to fundraising, partner engagement, technical due diligence, and the responsible expansion of AI evaluation capabilities. This is a remote-first opportunity with approximately 15% travel and a preference for candidates able to participate in quarterly in-person meetings.
Accountabilities:
- Own the Workbench’s Design → Test → Analyze → Publish (DTAP) workflow, Component Contract, Artifacts API, component registry, and canvas interface.
- Build a persistent, structured, and queryable evaluation history while integrating existing red-teaming, administrative dashboard, and human-in-the-loop data-capture capabilities.
- Evaluate build-versus-buy decisions for authentication, storage, orchestration, LLM-provider services, sandboxing, and other core platform capabilities.
- Recruit, develop, and retain the engineering organization in partnership with the HR function, including Staff/Senior Backend, Senior Frontend, DevOps/Site Reliability, Applied Machine Learning, and Senior Product Management roles.
- Establish effective planning, code review, on-call, incident response, and roadmap review practices.
- Track engineering progress, risks, and delivery outcomes for board and funder reporting.
- Partner with the CEO and Evaluation Science Lead to translate human-in-the-loop red teaming, automated benchmarks, LLM red teaming, LLM-as-judge, and drift detection into a reusable Component Contract.
- Ensure the platform supports the required Test artifact types and planned agentic components.
- Lead vendor risk reviews covering LLM providers, cloud infrastructure, and third-party evaluation tools.
- Coordinate with organizational leadership on required approvals, funder restrictions, and the appropriate separation of resources, data, and intellectual property across related entities and funded programs.
- Escalate unresolved legal, policy, or sponsor-related risks through the appropriate leadership channels.
- Represent the platform in technical discussions with customers, partners, funders, and press.
- Own technical content for investor fundraising materials, roadmaps, demonstrations, and due diligence reviews in collaboration with executive leadership.
- Shape the developer-relations narrative for SDK and ADK capabilities as the Component Contract becomes available to external developers.
- Partner with planned Developer Relations and Ecosystem leadership to support external developer adoption.
- Provide technical leadership across delivery, operational issues, organizational growth, and continuous platform improvement.
- 12+ years of software engineering experience, including at least 4 years in senior technical leadership within a startup, evaluation or audit firm, machine-learning infrastructure company, or research-grade platform team.
- Experience building and operating platforms with plugin, extension, or component models.
- Production experience with AI evaluation, red-teaming, machine-learning observability, model testing, or related systems.
- Proven ability to hire, develop, and retain senior engineers within a distributed organization.
- Experience building or leading teams that include applied machine-learning or agentic-systems expertise.
- Ability to design reusable platform components and work with human-in-the-loop transcripts, benchmark scores, and LLM red-teaming results.
- Strong understanding of platform architecture, engineering delivery, operational reliability, and technical decision-making.
- Ability to communicate complex technical decisions clearly to enterprise security leaders, foundation program officers, investors, customers, partners, and other non-technical stakeholders.
- Strong prioritization skills and the ability to balance competing technical, organizational, and delivery requirements.
- Ability to independently lead distributed engineering teams through delivery, operational challenges, and periods of growth.
- Understanding of the operational separation required between public-benefit organizations, nonprofit entities, and grant-funded programs.
- Strongly preferred experience implementing ISO 42001, the NIST AI Risk Management Framework, or SOC 2 requirements.
- Prior CTO, VP Engineering, or founding-engineer experience at an early-stage company with institutional funding is preferred.
- Experience enabling external developers to build on a platform is preferred.
- Experience developing evaluation methodologies with academic or civil-society partners is a plus.
- Experience working with enterprise security, compliance, vendor risk, or AI governance programs is valuable.
- A PhD or prior experience at a frontier AI research laboratory is not required.
- Strong communication, judgment, collaboration, and team-development skills.
- $215,000–$315,000 annual compensation, commensurate with experience.
- Remote-first, full-time position.
- Health, dental, and vision insurance, with 100% of employee premiums and 85% of dependent premiums covered.
- $365 monthly employer contribution to a pre-tax Health Savings Account.
- Pre-tax Flexible Spending Account.
- Employer-paid AD&D insurance.
- Employer-paid short-term and long-term disability insurance.
- Employer-paid basic life insurance.
- Supplemental voluntary life insurance for employees, spouses, and dependent children.
- Pet insurance options, including Total Pet Plan and Supplemental Wishbone coverage.
- Employee Assistance Program supporting health and wellness.
- Wellness reimbursement program.
- 401(k) retirement plan with 6% employer matching.
- 15 days of paid time off annually, increasing to 20 days after the third anniversary and 25 days after the sixth anniversary.
- 16 paid holidays, including 14 common holidays and 2 flexible holidays.
- Paid bereavement leave.
- Paid parental leave for mothers and fathers, including two weeks after the first year.
- Approximately 15% travel for funder meetings, partner sites, conferences, and off-sites.
- Quarterly in-person meetings, with New York or the San Francisco Bay Area strongly preferred for proximity to key stakeholders.
- Opportunity to lead an impactful AI evaluation platform focused on trustworthiness, transparency, safety, and educational applications.
- High degree of technical ownership across architecture, engineering strategy, security, and platform growth.
- Opportunity to influence AI evaluation practices, external developer ecosystems, and responsible AI initiatives.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
