The posting
What this Job Entails:
We are seeking an experienced AI Solution Architect to design, build, and scale production-grade Generative AI and multi-agent platforms. Mirroring the delivery rigor required in regulated enterprise environments, this role bridges hands-on cloud AI engineering with consultative delivery leadership. You will architect shared, reusable AI platforms on AWS supporting high-stakes business domains—including regulatory compliance, energy contracts, finance, procurement, and operational quality—ensuring strict source grounding, cost efficiency, and measurable business return on investment (ROI).
Your Roles and Responsibilities:
Multi-Agent Platform & Cloud Architecture
- Architect, deploy, and own production readiness for shared, multi-agent Generative AI platforms on AWS (Amazon Bedrock, AWS Lambda, Amazon ECS, Amazon OpenSearch, Terraform) that serve multiple business units from centralized enterprise knowledge bases.
- Design multi-agent orchestration layers using frameworks such as LangGraph and LangChain, establishing reusable modular components and standardized design patterns that accelerate onboarding of new domain assistants.
- Engineer advanced context-management strategies for long-running agent sessions—including on-demand skill and tool loading (progressive disclosure), Model Context Protocol (MCP) integrations, and prompt caching—to optimize latency and control token costs.
Enterprise Retrieval (RAG) & Ingestion Pipelines
- Design and implement scalable document ingestion pipelines and hybrid retrieval architectures that combine semantic vector search, keyword search, and metadata filtering across large enterprise repositories (100+ GiB).
- Establish chunking, reranking, and metadata strategies tailored to complex, heavily versioned utility documentation (such as regulatory filings, energy contract amendments, Standard Operating Procedures, and audit records).
- Build strict guardrails for source grounding, citation traceability, and deterministic behavior within regulated quality and compliance workflows.
AI Evaluation, Observability & Governance
- Develop rigorous, automated evaluation frameworks (including LLM-as-a-judge pipelines) that measure multi-dimensional answer quality, source recall and precision, and statistical confidence intervals across repeated runs.
- Implement step-by-step agent observability and trace-based debugging using OpenTelemetry (OTel) and cloud evaluation tooling (such as AWS AgentCore Evaluations).
- Design and execute structured User Acceptance Testing (UAT) protocols with business owners to validate traceability, accuracy, and compliance against business KPIs prior to production sign-off.
Technical & Delivery Leadership
- Serve flexibly as Solution Architect, Tech Lead, or Delivery Lead depending on engagement needs—guiding cross-functional teams of data scientists and AI engineers through rapid prototyping, pilot validation, and full production rollout.
- Partner directly with utility stakeholders across regulatory, legal/contracts, finance, and operations to translate complex workflows (such as tracing controlling language across contract amendments or automating regulatory filing quality checks) into high-ROI AI solutions.
- Mentor junior engineers and developers on agentic design patterns, production software engineering best practices, and responsible AI governance.
Required Qualifications/Skills:
Required Qualifications
- Experience: 8+ years of overall experience building enterprise data science, machine learning, and AI solutions, including 3+ years architecting and delivering production Generative AI and LLM applications.
- Regulated Industry Delivery: Proven track record leading end-to-end AI delivery in regulated industries (Energy & Utilities strongly preferred; Financial Services, Telecommunications, Manufacturing, or Healthcare also relevant).
- Production Impact: Demonstrated experience taking multi-agent or RAG platforms from prototype to active production adoption with measurable business impact (such as documented hours saved, cost reduction, or operational efficiency gains).
- Cloud & Infrastructure (AWS): Hands-on architecture and deployment experience with Amazon Bedrock, OpenSearch, AWS Lambda, ECS, serverless and containerized AI patterns, cross-account environment promotion, and Infrastructure as Code (Terraform).
- Agentic AI & Orchestration: Deep expertise in LangGraph, LangChain, prompt and context engineering, tool design, multi-agent workflows, prompt caching, and Model Context Protocol (MCP).
- RAG & Evaluation: Practical mastery of hybrid search, metadata filtering, reranking, LLM-as-a-judge evaluation suites, grounding and citation verification, and OpenTelemetry trace debugging.
- Programming & Data Engineering: Strong proficiency in Python, SQL, and FastAPI, with experience building scalable data and document ingestion and cleansing pipelines.
- Education: Master’s or Bachelor’s degree in Information Systems, Data Science, Computer Science, Business Analytics, or a related quantitative discipline.
Preferred Qualifications
- Prior technology or management consulting experience leading client-facing technical delivery, executive demonstrations, and UAT cycles.
- Direct experience building AI assistants for energy contracts, regulatory filings, procurement, or quality and compliance workflows.
- Familiarity with model fine-tuning and transfer learning, fairness metrics and bias mitigation frameworks (such as AIF360), and BI visualization tools (such as Power BI).
- Relevant cloud or AI architecture certifications (such as AWS Certified Solutions Architect, AWS Certified Machine Learning, or Anthropic Certified Architect).
Physical Demand & Work Environment:
- Must have the ability to perform office-related tasks which may include prolonged sitting or standing
- Must have the ability to move from place to place within an office environment
- Must be able to use a computer
- Must have the ability to communicate effectively
- Some positions may require occasional repetitive motion or movements of the wrists, hands, and/or fingers
Salary Range
$114,000.00 - $180,000.00 USD (Salary)
- Please note that the salary information provided herein is base pay only (gross); it does not include other forms of compensation which may or may not apply to this specific position, namely, performance-based bonuses, benefits-related payments, or other general incentives - none of which are guaranteed, may be subject to specific eligibility requirements, and are wholly within the discretion of Astreya to remit.
- Further, the salary information noted above is a range that consists of a minimum and maximum rate of pay for this specific position. Where an applicant or employee is placed on this range will depend and be contingent on objective, documented work-related considerations like education, experience, certifications, licenses, preferred qualifications, among other factors.
Astreya offers comprehensive benefits to all Regular, Full-Time Employees, including:
- Medical provided through UHC (PPO, HSA, Surest options) / Medical provided through Kaiser (HMO option only) for California employees only
- Dental provided through UHC
- Nationwide Vision provided by UHC
- Flexible Spending Account for Health & Dependent Care
- Pre-Tax Account for Commuter Benefit/Parking & Transit (location-specific)
- Continuing Education and Professional Development via various integrated platforms, e.g. Udemy and Coursera
- Corporate Wellness Program provided by Goomi Group
- Employee Assistance Program
- Wellness Days 401k Plan
- Basic and Supplemental Life Insurance
- Short Term & Long Term Disability
- Critical Illness, Critical Hospital, and Voluntary Accident Insurance
- Tuition Reimbursement (available 6 months after start date, capped)
- Paid Time Off (accrued and prorated, maximum of 120 hours annually)
- Paid Holidays
- Any other statutory leaves, paid time, or other ancillary benefits required under state and federal law



