Skip to content
CV Rocket
Get started
How it worksThe enginePricingBlogContactSign in

Open nowPosted 19 hours ago

Senior AI System Engineer - LLM Training System

Huawei Technologies Canada Co., Ltd.189 open roles

Pay
$127,000 – $225,000 a year
Where
Vancouver, British Columbia, Canada
Work mode
On site
Get the CV for this job

From $25 per CV, paid once. No subscription.

Your applicationOpen nowSenior AI System Engineer - LLM Training SystemHuawei Technologies Canada Co., Ltd. · Vancouver, British Columbia, Canada

  1. YouYes, apply to this one.

  2. CV RocketCV written for this posting.

  3. 25 readersRecruiter, hiring manager, skeptic. Round after round.

  4. CV RocketApplied on Huawei Technologies Canada Co., Ltd.'s own form.

The reply lands in your private mailbox

3×more interviews than doing it yourself with ChatGPT.

The clock on this job

Early applications get read.

13.3% of postings close within 7 days. Measured by our own scanner across the market. Huawei Technologies Canada Co., Ltd. postings stay open a median of 13 days.

Share of postings closed within
  1. 2.1%1 day
  2. 4.6%3 days
  3. 13.3%7 days
  4. 21.3%14 days
  5. 35.7%30 days
  6. 59.1%60 days
This job: posted 19 hours ago

Huawei Technologies Canada Co., Ltd. median: 13 days open

The posting

Huawei Canada has an immediate permanent opening for a Senior Engineer.

About the team:

The Computing Data Application Acceleration Lab aims to create a leading global data analytics platform organized into three specialized teams using innovative programming technologies. This team focuses on full-stack innovations, including software-hardware co-design and optimizing data efficiency at both the storage and runtime layers. This team also develops next-generation GPU architecture for gaming, cloud rendering, VR/AR, and Metaverse applications. One of the goals of this lab are to enhance algorithm performance and training efficiency across industries, fostering long-term competitiveness.

About the job:

As a Senior System Engineer, you will drive core compute operator development and distributed scaling performance across foundation model pre-training and post-training workloads on Ascend NPUs. You will identify computational and communication bottlenecks, implement high-performance kernels using AscendC, Triton, and TileLang, and optimize distributed acceleration suites (MindSpeed / open-source training frameworks).

  • Ascend High-Performance Kernel Development: Design, implement, and tune core operators (FlashAttention variants, FP8/MXFP8/W4A8 GEMM, Fused RMSNorm, dynamic MoE routing/dispatch) using AscendC, with explorations in Triton (Triton-Ascend backend) and TileLang.
  • Pre-training & Post-training System Optimization: Architect, scale, and optimize distributed pre-training and RL post-training pipelines using MindSpeed (MindSpeed-LLM / MindSpeed-MM) alongside upstream frameworks (Megatron-LM / Megatron-Core, PyTorch FSDP, veRL, vLLM, Ray).
  • Distributed Parallelism & Training Efficiency: Tune multi-dimensional hybrid parallelism (TP, PP, CP, EP, ZeRO/FSDP) and RL rollout/actor pipelining on Ascend NPU clusters, achieving optimal MFU and compute-communication overlap.
  • Low-Precision & Numerical Parity: Implement low-precision training/inference schemes (FP8, MXFP8, mixed-precision), guaranteeing numerical stability, batch-invariance, and strict precision parity between training and serving runtimes.
  • Profiling & Hardware-Aware Tuning: Conduct deep performance profiling on Ascend hardware using Ascend Profiler (msprof), msprobe, and PyTorch Profiler to eliminate pipeline bubbles, AICore/AIVector stalls, and HCCL communication overhead.
  • Open Source & Ecosystem Enablement: Contribute high-performance operators, bug fixes, and acceleration features to open-source Ascend communities and upstream AI acceleration projects. The total target annual compensation ranges from $127,000 to $225,000 depending on education, experience, and demonstrated expertise.

About the ideal candidate:

  • Bachelor’s, Master’s, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or related technical disciplines, 3+ years of experience in high-performance computing (HPC), AI accelerator operator development, or distributed large model training systems.
  • Hands-on Operator Programming: Solid development experience in AscendC / CANN, or extensive experience in CUDA C/C++ with a proven ability to transition to AscendC. Experience or strong interest in Triton or TileLang for accelerator backends.
  • Distributed Frameworks: Deep practical experience with distributed pre-training and/or post-training frameworks, such as MindSpeed, Megatron-LM / Megatron-Core, PyTorch FSDP, veRL, or vLLM / DeepSpeed. Systems Programming: Strong programming proficiency in C/C++ and modern Python, with hands-on experience building PyTorch custom C++/CANN extensions.
  • Performance Analysis: Proficiency in performance profiling and debugging tools on AI hardware (e.g., Ascend Profiler/msprof, PyTorch Profiler, or Nsight toolchains). Preferred / Standout Qualifications:
  • Direct experience porting or optimizing Triton or TileLang frontends/backends for non-CUDA accelerators (especially Ascend NPU).
  • Deep understanding of Ascend NPU microarchitecture (Cube & Vector compute units, local memory hierarchy/UB, memory movement engines, and HCCL collective communication). Hands-on experience optimizing end-to-end MFU for massive MoE or dense foundation models on multi-node NPU/GPU clusters.
  • Experience in low-precision numerical alignment (FP8, MXFP8, W4A8/W8A8) and eliminating training-inference discrepancy. Active open-source contributions to Ascend repos (MindSpeed, CANN ops, Ascend-vLLM) or upstream projects (Megatron-LM, Triton, TileLang, veRL).

Additional Information：

Huawei Canada is committed to a fair, inclusive, and accessible recruitment process. If you require accommodation during any stage of the hiring process, please let us know and we will work with you to meet your needs.

All applications for this position are reviewed directly by our hiring team, we do not use artificial intelligence tools to screen or select candidates.

From $25, paid onceGet the CV for this job

What happens when you press

One press. We do the rest.

  1. A CV for this posting

    Written against Huawei Technologies Canada Co., Ltd.'s own wording, from every piece of relevant proof in your profile.

  2. 25 readers review it

    Recruiter, hiring manager, skeptic and more read every draft, round after round. You get the best round.

    The review screen in CV Rocket: how each CV was read, round by round.

  3. We apply on Huawei Technologies Canada Co., Ltd.'s form

    Our application engine gets through the hardest forms there are. Where a question needs you, AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

    An application in CV Rocket: every answer filled in on the employer's form.

  4. Every reply, sorted

    Huawei Technologies Canada Co., Ltd.'s answer lands in your private mailbox, and we classify it on arrival: interview, question, rejection.

    The CV Rocket inbox: each employer reply classified as an interview, an action or a rejection.

  5. Reply with AI

    AI helps you write the email, checks it and sends it. We show you whether the recruiter read it.

  6. The interview in your calendar

    Full integration with your calendar. The invitation goes straight in.

    An interview invitation in the CV Rocket inbox, added to the candidate's calendar.
Get the CV for this job

From $25 per CV, paid once. No subscription.

Why it works

3×

more interviews than doing it yourself with ChatGPT.

ChatGPT writes a CV and never learns what happened to it. We see every reply. For each CV we know:

  • How it was written, and how the review scored it
  • When we applied, and how long after the posting went up
  • Which posting, which company, which city
  • Who got the interview, and who heard nothing

That is how we know which CVs get called.

Get the CV for this job

From $25 per CV, paid once. No subscription.

The numbers game

More applications. More interviews.

Every application goes out with its own CV, written for that posting and paid once. Send enough of them and the law of large numbers finds you the job.

Get the CV for this job
By hand5–10
With CV Rocket100
applications a day

Nearby

Live postings like this one

Same employer first, then the same role elsewhere.

Before you press

Straight answers

Get the CV for this job

From $25 per CV, paid once. No subscription.

What if my background isn't good enough?

We make the most of the background you have. The CV uses every piece of relevant proof your profile holds, and one of the 25 readers reads your whole profile and flags what the CV left out.

Do you really apply for me?

Yes, on the employer's own form, the hardest ones included. Where a question needs you, you answer it right there and AI suggests the best answer. Don't want us applying from our IP addresses? Use our Chrome extension: we apply straight from your own browser.

Is it a subscription?

No. You pay once per CV, from $25. Every application goes out with its own CV, written for that posting.

One job. One CV.
Paid once.

Pick the posting you want. We write for it, apply for you and catch the reply.

Get the CV for this job

From $25 per CV, paid once. No subscription.