The posting
About 100ms
100ms is building AI agents that automate complex patient access workflows in U.S. healthcare — starting with benefits verification, prior authorisation, and referral intake in speciality pharmacy. Our platform combines LLM-based agents, deep healthcare domain expertise, and operational infrastructure to help care teams move faster and get patients on treatment sooner.
Role Summary
Key Responsibilities
- Own the end-to-end architecture of the AI agents platform — services, APIs, data models, async task pipelines, LLM/voice agent orchestration, and integrations with payer systems, EMRs, and telephony.
- Design for reliability, latency, and safety: define patterns for fault tolerance, idempotency, retries, observability, and graceful degradation across agent workflows that run at scale in production.
- Set the architecture for multi-tenancy, data isolation, and PHI handling in partnership with our Security and Compliance Lead, keeping HIPAA and SOC 2 requirements designed in rather than bolted on.
- Make and document high-leverage technical decisions — build vs. buy, service boundaries, storage and queueing choices, LLM provider and inference strategy — through lightweight design reviews and ADRs.
- Stay hands-on: prototype critical paths, write production code on the most ambiguous and highest-risk problems, and unblock teams when they hit hard technical walls.
- Partner with the platform team on how workloads run on our Kubernetes (GKE) infrastructure — autoscaling, deployment topology, cost, and SLOs for agent and voice workloads.
- Raise the engineering bar: mentor senior engineers, drive code and design review culture, and define standards for testing, evaluation, and release safety across teams.
- Translate product and customer needs into a technical roadmap; work with founders and product to sequence architectural investments against business priorities.
- De-risk scale: anticipate where the system breaks at 10x call volume, 10x customers, and 10x team size, and lay the groundwork before it does.
Requirements
- 10+ years of professional software engineering experience, including several years designing and operating large-scale, production distributed systems.
- A track record of owning architecture for a product or platform end to end — you have made foundational technical decisions, lived with their consequences, and evolved them.
- Deep expertise in backend systems: service design, RESTful/async APIs, event-driven architectures, task queues, and data modelling across relational and non-relational stores.
- Strong grasp of reliability engineering — observability, fault tolerance, capacity planning, and incident-informed design — for systems with strict correctness and latency requirements.
- Hands-on proficiency in at least one of our core languages (Python or Go); comfort reading and reviewing code across the stack, including React-based frontends.
- Working knowledge of cloud-native infrastructure (GCP/AWS/Azure), Kubernetes, and infrastructure-as-code, and how architectural choices play out operationally.
- Experience building or integrating AI/LLM-backed systems — or clear evidence you can go deep fast on agent orchestration, evaluation, guardrails, and inference trade-offs.
- Strong security instincts: you design with data isolation, encryption, access control, and auditability in mind from day one.
- Excellent written and verbal communication — you can carry a design from whiteboard to ADR to aligned teams, and explain trade-offs to founders and customers alike.
- Ability to tackle complex, ambiguous technical problems and drive them to crisp decisions.
Nice-to-Haves
- Experience with voice AI, telephony (SIP/WebRTC), or real-time systems.
- Prior work in U.S. healthcare or other regulated domains (HIPAA, SOC 2, HITRUST) — familiarity with EMRs, payer integrations, FHIR/HL7.
- Experience with LLM evaluation frameworks, prompt/fine-tuning pipelines, and hallucination/safety controls in production.
- Experience scaling engineering teams and architecture together — platformization, service extraction, developer experience.
- Open-source contributions, technical writing, or conference talks.
Why This Role Matters
Culture & Growth
- 100ms is an engineering-first startup. Our team includes former entrepreneurs, AI engineering specialists, and healthcare operations professionals, with experience at major technology companies around the world.
- You can grow as a senior individual contributor or into technical leadership — you set your own trajectory, with direct access to the founders.
- We believe in-person collaboration builds better systems and stronger culture: employees work from our Bengaluru office at least three days a week — Tuesday, Wednesday, and Friday. Some overlap with U.S. time zones is expected for customer and partner engagement.
