Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 10 hours ago

Principal Software Engineer - AI Infrastructure

Microsoft1,948 open roles

Pay
$142,800 – $274,800 a year
Where
Redmond, WA, United States
Work mode
On site
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowPrincipal Software Engineer - AI InfrastructureMicrosoft · Redmond, WA, United States

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Microsoft's own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

13.3% of postings close within 7 days. Measured by our own scanner across the market. Microsoft postings stay open a median of 18 days.

Share of postings closed within
  1. 2.1%1 day
  2. 4.6%3 days
  3. 13.3%7 days
  4. 21.3%14 days
  5. 35.7%30 days
  6. 59.1%60 days
This job: posted 10 hours ago

Microsoft median: 18 days open

The posting

Overview

Help build the infrastructure that trains the next generation of agentic Artificial Intelligence (AI) models at Microsoft. The Cloud + AI (C+AI) organization builds the end-to-end AI stack that powers innovation across Azure and Microsoft. The Foundry, Inference and Training (FIT) Training Team is where frontier-scale model training meets production-grade engineering. You will work alongside engineers and researchers to advance the scientific methods and scalable infrastructure used to train Large Language Models (LLMs), Small Language Models (SLMs), and agentic models, spanning both proprietary and open-weight models and building on open-source training frameworks. Your work will directly shape how reliable, enterprise-grade agentic workflows are built across the company and experienced by the customers who depend on them.

As a Principal Software Engineer on this team, you will own the architecture and technical direction of major components of the training platform and drive them from ambiguous problem statements through production operation. You will keep the customer at the center of every decision, partnering directly with the researchers and product teams who rely on the platform to understand their needs, remove friction, and measure success by their outcomes. You will set the technical bar for the team through design leadership, code and test-strategy reviews, and mentorship, and you will partner across C+AI and with partner organizations to align infrastructure roadmaps with model training and evaluation needs. This opportunity will allow you to deepen your expertise in large-scale distributed systems for AI training, develop firsthand understanding of how frontier proprietary and open-weight models are trained and evaluated, and grow your influence as a technical leader who shapes architecture, engineering practices, and the next generation of engineers on a fast-moving, startup-style team.

We are looking for a curious, independent, adaptable, and customer-obsessed problem-solver who thrives on continuous learning, is energized by evolving priorities, and is motivated by the impact your work has on the people who use it. You write efficient, well-tested code, enjoy debugging hard problems in complex distributed systems, and share what you learn. You iterate quickly, adjust course when the evidence calls for it, and collaborate effectively in a dynamic environment.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond

Responsibilities

  • Own the architecture and technical direction of major components of the training platform for Large Language Models (LLMs), Small Language Models (SLMs), multimodal, code-specific, and open-weight models, ensuring designs meet scalability, reliability, security, and cost-efficiency requirements.
  • Partner directly with customers, including researchers and product teams across Microsoft, to understand their training and evaluation needs, translate them into platform requirements, and measure success through customer experience and outcomes.
  • Drive ambiguous, multi-quarter initiatives from proposal to production, breaking work into milestones, identifying dependencies, and negotiating ownership and delivery commitments with partner teams across Cloud + AI (C+AI).
  • Build on and extend open-source training frameworks (for example, PyTorch, DeepSpeed, Megatron-LM, and Hugging Face libraries), adapting them for proprietary and open-weight models and contributing improvements back to the community where appropriate.
  • Design and build services that serve production traffic while meeting Microsoft security, privacy, Responsible AI, and compliance requirements, including auditable evidence of compliance.
  • Lead debugging and resolution of complex issues in distributed training jobs and production services; participate in the on-call rotation as a Designated Responsible Individual (DRI), lead incident retrospectives, and improve troubleshooting guides, testing, safe deployment, and monitoring.
  • Mentor engineers and raise the technical bar through design reviews, code reviews, and ownership of the team's test strategy, while championing AI-assisted engineering tools and practices across the software development lifecycle.

Qualifications

Required Qualifications:

  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Preferred Qualifications:

  • Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
  • 5+ years designing, building, and operating large-scale distributed systems or cloud platforms with end-to-end ownership of production services, including Kubernetes-based containerized platforms on Microsoft Azure, Amazon Web Services (AWS), or Google Cloud, control-plane or backend services that manage compute, storage, and networking, and secure multi-tenant designs covering authentication, authorization, workload identity, quota management, and isolation.
  • 3+ years building machine learning training infrastructure, such as Graphics Processing Unit (GPU) cluster scheduling, data pipelines, and checkpointing and storage systems, using open-source training frameworks such as PyTorch Fully Sharded Data Parallel (FSDP), DeepSpeed, Megatron-LM, Hugging Face Transformers and Accelerate, or Ray Train; contributions to open-source projects are a strong plus.
  • 2+ years training, fine-tuning, or evaluating open-weight models (for example, Llama, Mistral, Phi, Qwen, or DeepSeek model families) at multi-node scale.
  • 2+ years building or operating Reinforcement Learning (RL) post-training pipelines for Large Language Models (LLMs) and agentic models using verl, slime, MILES, or comparable frameworks, with a demonstrated understanding of the end-to-end training stack, including pretraining and post-training frameworks, Reinforcement Learning Environments (RLEs), rollout inference engines (for example, vLLM or SGLang), evaluation, and checkpointing, and how design choices at each layer affect throughput, stability, and cost.
  • 3+ years serving as an on-call or Designated Responsible Individual (DRI) engineer for production services operating against defined availability or latency targets, including incident response, postmortems, structured logging, distributed tracing, dashboards, alerting, automated validation, and incident playbooks.
  • 2+ years partnering directly with internal or external customers to define requirements, instrument customer experience (for example, adoption, satisfaction, or time-to-first-result metrics), and prioritize engineering work by customer impact.
  • Experience leading technical design across 2+ teams, driving ambiguous multi-quarter initiatives to completion, and mentoring senior engineers.

#AIPLATFORM #AIINFRA

Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Microsoft's own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Microsoft's form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Microsoft's answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.