Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 29 hours ago

Principal Software Engineering Manager - AI Frameworks

Microsoft496 open roles

Where
United States
Work mode
On site
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowPrincipal Software Engineering Manager - AI FrameworksMicrosoft · United States

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Microsoft's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

7.8% of postings close within 7 days. Measured by our own scanner across the market. Microsoft postings stay open a median of 23 days.

Share of postings closed within
  1. 1.7%1 day
  2. 3.5%3 days
  3. 7.8%7 days
  4. 14.6%14 days
  5. 34.1%30 days
This job: posted 29 hours ago

Microsoft median: 23 days open

The posting

Overview

As a Principal Software Engineering Manager – AI Frameworks on the team, you will lead and grow a group of engineers working across the AI software serving stack while remaining technically hands-on. You will set direction and contribute to critical engineering decisions across developer infrastructure, build systems, runtimes, libraries, release pipelines, and application programming interfaces (APIs). You will be responsible for improving engineering velocity and build reliability, establishing effective system and release practices, and ensuring the team ships high-quality software to production for large-scale model training and inference.

In this role, you will guide the team’s work on benchmarking OpenAI and other large language models (LLMs) across GPUs and Microsoft hardware, translating performance findings into production-ready builds and releases. You will drive automation across build, validation, benchmarking, regression detection, and deployment workflows, including advanced AI-assisted automation and closed-loop systems that accelerate diagnosis and remediation. You will partner closely with researchers, product teams, and platform owners to improve developer velocity, reduce time-to-deployment and hardware footprint, and support Microsoft Azure’s capex efficiency goals.

At Microsoft, our mission to empower every person and every organization on the planet to achieve more guides how we partner with customers to deliver trusted, impactful solutions. With a growth mindset culture, we innovate responsibly and measure success by shared progress people, teams, and customers. Join us to do meaningful work that changes the world and helps shape what’s next for everyone.

Responsibilities

  • Lead and develop a team of engineers working across multiple layers of the AI software stack, while staying technically engaged in architecture, design reviews, debugging, and critical implementation decisions that enable large-scale training and inference.
  • Set technical vision and execution strategy for developer infrastructure, build and release systems, model performance benchmarking, optimization, and production deployment across GPUs and Microsoft hardware.
  • Drive performance and production outcomes by prioritizing and overseeing efforts to build, benchmark, profile, debug, validate, and optimize training and inference workloads, from developer workflows through production release.
  • Own build, release, and performance health by establishing best practices for build reliability, system observability, regression monitoring, release quality, impact measurement, developer velocity, time-to-deploy, and hardware efficiency.
  • Partner cross-functionally with research, product, infrastructure, and hardware teams to deliver scalable, production-ready AI performance improvements.
  • Balance short-term delivery and long-term investments by advancing build, release, and validation automation, including AI-assisted workflows and closed-loop systems that identify issues, recommend or implement fixes, and verify outcomes. Ensure these investments align with organizational goals, platform roadmaps, and Azure capex objectives.
  • Build a strong engineering culture through coaching, feedback, hiring, and career development, enabling the team to operate with increasing autonomy and impact.

Qualifications

Qualifications

Minimum/Required Qualifications:

  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Other Requirements:

  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications:

  • Master’s Degree in Computer Science or related technical field AND 10+ years of software engineering experience, including 6+ years in engineering management,OR Bachelor’s Degree in Computer Science or related technical field AND 12+ years of software engineering experience, including 6+ years in engineering management, or equivalent experience.
  • 4+ years people management experience.
  • Strong technical foundation in software engineering principles, computer architecture, GPU architecture, and hardware acceleration for neural networks, combined with hands-on experience in developer infrastructure such as build systems, dependency management, Bazel or comparable tooling, CI/CD, release engineering, and developer productivity.
  • Experience leading teams responsible for end-to-end performance analysis and optimization of LLMs, AI systems, or HPC workloads, including GPU profiling and performance analysis tools, and shipping validated builds through release pipelines into production environments.
  • Demonstrated ability to lead cross-team initiatives, align stakeholders, and translate research or platform capabilities into scalable, production-ready solutions.
  • Proven people leadership skills, including hiring, coaching, performance management, and career development, with a track record of building high-performing, inclusive teams.
  • Working knowledge of AI and ML systems across training, inference, evaluation, and benchmarking, with experience in at least one modern deep learning framework such as PyTorch, TensorFlow, or ONNX Runtime. The candidate should be able to reason about how models, frameworks, runtimes, hardware, and developer infrastructure interact across the end-to-end lifecycle.
  • Familiarity with GPU software stacks and acceleration technologies such as CUDA, ROCm, Triton, or equivalent, sufficient to guide technical direction and evaluate tradeoffs. Experience designing advanced automation for build, test, release, benchmarking, and production validation workflows is preferred, including AI-assisted tooling and closed-loop systems that use observed results to drive subsequent actions.

#AIInfra

Software Engineering M5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Microsoft's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Microsoft's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Microsoft's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.