The posting
Employment Type: Full-Time Work Setting: In-office/Remote Work Location: United States, New York, Mexico, San Francisco, Canada Work Hours: Office hours Find out more here: https://semianalysis.com
About SemiAnalysis
SemiAnalysis is an independent research and analysis firm specializing in the Semiconductor and AI industries. Our in-depth coverage spans the entire supply chain, from semiconductor fabrication processes to cutting-edge AI Models, software, and infrastructure. We are recognized as the leading authority on the semiconductor supply chain, with the highest concentration of industry experts within one team, and a deep-rooted passion for delving into the intricacies.
We’re a global team of over 50 analysts, each with extensive networks across the semiconductor supply chain and AI ecosystem, publishing industry‑shaping articles while participating in 40+ conferences annually.
Our newsletter reaches more than 200 000 subscribers worldwide, including senior management and C‑suite leaders at the leading semiconductor and AI companies.
We also offer three core products:
- Industry Models – we develop and publish industry models on accelerator shipments, datacenter demand and supply, GPU total cost of ownership, and more. We work with hyperscalers, neoclouds, many of the world’s largest hedge funds, and government agencies.
- Core Research – our public equity markets product, geared towards financial investors, distills our deep technical research and knowledge into key insights on technology and product trends.
- Consulting and Technical Due Diligence – We conduct custom research and project work to guide key strategic and investment decisions for the largest private‑equity funds, leading venture‑capital firms, companies across the AI ecosystem, and government agencies.
Position Overview
We are looking for someone with a deep and genuine obsession with hardware, regardless of where that expertise comes from.
Candidates may have experience in chip architecture, SoC design, memory systems, interconnect PHYs, semiconductor manufacturing, advanced packaging, performance engineering, ML systems, HPC, datacenter infrastructure, networking, power, cooling, or large-scale cluster operations.
What matters most is a first-principles understanding of how modern AI hardware works, from silicon and packaging through racks, networks, and full datacenter systems. The ideal candidate is intensely curious about why certain designs outperform others and can translate technical differences into measurable performance and economic outcomes.
The role focuses on quantitative analysis of AI hardware and systems, including:
- Hardware and system performance
- Architecture evaluation
- Silicon and system cost analysis
- Power and cooling constraints
- Total cost of ownership
- AI infrastructure economics
- Technology assessment across the full hardware stack
The work will support SemiAnalysis products and research, including the Inference Simulator, InferenceX, Tokenomics Model, Accelerator & HBM Model, and AI Cloud TCO research.
The scope of the role will flex toward the candidate’s strengths, with strong performers helping define their own coverage areas.
The central question behind the role is:
For a given model, latency target, and system configuration, how many tokens per second does each chip and architecture deliver, and what does each token cost?
Responsibilities
- Build first-principles performance models for LLM inference and training.
- Analyze arithmetic intensity, roofline performance, prefill and decode behavior, KV cache capacity, memory bandwidth requirements, batching dynamics, parallelism strategies, and latency-throughput trade-offs.
- Evaluate AI hardware across the full technology stack, including GPUs, TPUs, custom ASICs, emerging accelerators, HBM, on-chip SRAM, memory tiering, scale-up fabrics, scale-out networks, racks, pods, and datacenters.
- Assess architecture and system design trade-offs, including compute versus memory bandwidth, interconnect topology, power availability, cooling requirements, and workload suitability.
- Connect semiconductor manufacturing decisions to hardware performance, availability, and cost.
- Analyze process-node choices, die size, reticle limits, advanced packaging, 2.5D integration, 3D stacking, HBM integration, yield, and silicon economics.
- Examine new chip, system, rack, and cluster announcements using specifications, die shots, rack layouts, benchmarks, and network diagrams.
- Develop independent and defensible views on real-world performance compared with vendor claims.
- Validate internal models against published benchmarks and independently gathered performance data.
- Investigate and explain differences between theoretical peak performance and delivered application performance.
- Translate technical analysis into economic metrics, including tokens per second per watt, per dollar of capital expenditure, and per megawatt.
- Contribute to TCO, tokenomics, accelerator, memory, networking, and datacenter research.
- Publish detailed research on AI accelerators, systems, infrastructure, and performance.
- Act as a technical authority during client calls, briefings, and discussions with engineering and investment audiences.
- Collaborate with accelerator, memory, networking, semiconductor, and datacenter analysts to connect silicon-level findings with broader system and industry conclusions.
Requirements
- Deep technical understanding of modern compute hardware at one or more levels of the stack, including silicon, systems, networking, clusters, or datacenters.
- Strong interest in learning unfamiliar parts of the hardware and infrastructure stack.
- Strong quantitative reasoning skills and the ability to work from first principles using FLOPs, bytes, bandwidth, latency, joules, watts, and dollars.
- Ability to build structured models that connect hardware design, workload performance, infrastructure requirements, and economic outcomes.
- Working knowledge of how AI workloads stress hardware, including memory bandwidth, memory capacity, interconnect communication, collective operations, and real-world compute utilization.
- Working understanding of semiconductor manufacturing and economics, including process-node trade-offs, die size, yield, die cost, packaging, and HBM integration.
- Ability to explain why an architecture was designed in a particular way and how those decisions affect manufacturing cost, performance, power, and scalability.
- Strong written communication skills, with the ability to explain complex technical findings clearly to both engineering and investor audiences.
- Ability to take an ambiguous technical question from initial definition through analysis, validation, and publication with minimal oversight.
- Self-driven, intellectually rigorous, detail-oriented, and comfortable challenging assumptions.
- Genuine enthusiasm for computer hardware, semiconductor technology, AI systems, and infrastructure research.
Preferred Skills
- Hands-on experience with AI training or inference frameworks such as vLLM, SGLang, TensorRT-LLM, PyTorch, or JAX.
- Experience with GPU programming or kernel development using CUDA, Triton, or HIP.
- Understanding of transformer inference mathematics, including per-token FLOPs, memory traffic, KV cache sizing, MoE routing, and quantization effects.
- Experience designing, deploying, benchmarking, or operating large GPU or accelerator clusters.
- Knowledge of datacenter networking, rack-scale architecture, power distribution, cooling, or large AI infrastructure deployments.
- Familiarity with training performance metrics and concepts such as model FLOPs utilization, parallelism scaling efficiency, communication overhead, and failure recovery.
- Experience benchmarking heterogeneous hardware, including NVIDIA GPUs, AMD GPUs, TPUs, custom accelerators, or emerging non-NVIDIA platforms.
- Hands-on semiconductor process, foundry, product engineering, yield, packaging, or test experience.
- Experience with die cost modeling, semiconductor manufacturing economics, advanced packaging, or teardown analysis.
- Ability to interpret die shots, packaging layouts, physical specifications, and teardown data to infer architecture and process decisions.
- Prior technical publications, research papers, open-source contributions, benchmarks, hardware teardowns, or industry analysis.
- Experience communicating technical findings directly to clients, executives, engineers, or institutional investors.
Growth Areas
The role provides opportunities to expand beyond an existing area of specialization and develop expertise across the complete AI hardware stack.
Potential growth areas include:
- Expanding from chip-level expertise into rack-scale and datacenter system analysis.
- Developing deeper knowledge of AI inference and training workloads.
- Learning first-principles performance and cost modeling.
- Building expertise in accelerator, HBM, networking, power, cooling, and TCO analysis.
- Connecting semiconductor manufacturing and packaging decisions to system-level performance and economics.
- Developing the ability to evaluate new chips, systems, and infrastructure announcements rapidly and independently.
- Strengthening technical writing and publishing skills.
- Becoming a recognized technical authority with engineering, investment, and industry audiences.
- Leading client briefings and translating complex hardware analysis into strategic and financial conclusions.
- Collaborating across accelerator, memory, networking, semiconductor manufacturing, and datacenter research areas.
- Defining and owning a research coverage area based on individual technical strengths.
- Contributing directly to the development of SemiAnalysis models, simulators, datasets, and research methodologies.
SemiAnalysis LLC participates in the E-Verify program to confirm the employment eligibility of all newly hired employees. For more information, visit e-verify.gov.
- E-Verify Poster: https://www.everify.gov/sites/default/files/everify/posters/EVerifyParticipationPoster.pdf
- Right to Work Poster: https://www.justice.gov/crt/case-document/file/1133936/dl?inline=
