The posting
AI governance & compliance
- Implement and maintain governance controls for AI models and services on the AI Foundation Platform, so AI use is trusted, controlled and compliant.
- Ensure AI services comply with data security, data classification, privacy (PDPA) and Responsible AI requirements.
- Maintain compliance evidence for ICT&SS / IM8, the System Security Plan (SSP), PDPA and Responsible AI requirements for agentic AI, and support security reviews and ARB submissions.
Monitoring & observability
- Implement model monitoring covering accuracy, drift, hallucination, latency, cost and usage across AI applications.
- Implement audit logging and traceability of model inputs, outputs and usage to support accountability and audit requirements.
- Implement end-to-end agent observability and audit trails (traces of prompts, reasoning steps, tool calls, outputs and approvals), with dashboards and alerts mapped to SLIs/SLOs and risk thresholds.
- Develop full-stack monitoring dashboards and reporting tools that give stakeholders visibility of AI performance, risks and compliance status.
- Set up alerts and incident response procedures for AI service anomalies and degradation.
Evaluation & testing
- Develop and run accuracy validation and evaluation processes, including test datasets, benchmarking, regression testing and human-in-the-loop review for model and prompt changes.
- Evaluate agent behaviour, including task success, tool-use correctness, trajectory review, groundedness and regression testing before changes go live.
Guardrails & security
- Configure and maintain AI security and safety guardrails, such as content filtering, PII detection and redaction, prompt injection protection and Amazon Bedrock Guardrails, in line with the agency's AI security governance.
- Define and enforce agent-level guardrails and controls: tool allow-lists, permission and action limits, human approval for high-impact actions, limits on loops, steps and spend, and an emergency stop, aligned with the agency's AI security governance.
- Run red-teaming and security testing for agentic threats, such as prompt injection through tool outputs, excessive agency, data leakage and misuse of tools, and track remediation.
- Perform security checks and support risk assessments for AI components.
MLOps
- Build and maintain MLOps pipelines for model and prompt versioning, deployment, rollback and lifecycle management.
Collaboration & documentation
- Work with the AI Platform Engineer, AI/ML Engineers and DevSecOps Engineers to embed governance controls into the AI development lifecycle.
- Work under the guidance of the Backend Senior Engineer and Solution Architect on guardrail design, risk decisions and the AI governance standards.
- Maintain documentation of AI governance standards, model cards, operating procedures and compliance records.
What we are looking for
Education & experience
- Degree or Diploma in Computer Science, Data Science, Information Technology or related disciplines.
- At least 2 years of experience relevant to software engineering, MLOps, DevOps, data engineering or AI operations.
Programming
- Good programming skills in Python, with working knowledge of JavaScript/TypeScript for building dashboards or web tools.
AI monitoring & evaluation
- Experience with model monitoring, evaluation or observability tools such as Amazon CloudWatch, SageMaker Model Monitor, MLflow, Langfuse or equivalent.
- Understanding of LLM evaluation techniques, such as groundedness and faithfulness metrics, LLM-as-judge and red-teaming.
- Familiarity with Amazon Bedrock features such as Guardrails, Model Evaluation and model invocation logging.
Agentic AI
- Understanding of agentic AI risks and controls, such as excessive agency, prompt injection, tool misuse and human-in-the-loop design, and of agent evaluation and tracing tools (e.g. Langfuse, OpenTelemetry).
DevOps
- Experience with CI/CD pipelines and version control systems such as Git.
- Knowledge of container technologies such as Docker and Kubernetes.
Security, privacy & government policy
- Good understanding of data security, privacy and Responsible AI principles.
- Understanding of Government ICT&SS management policies (IM8), SSP requirements and observability practices, and has applied them to AI guardrails, audit logging and monitoring.
Nice to have
- Familiarity with Singapore's Model AI Governance Framework, AI Verify, IM8 and PDPA.
- Experience with Government Commercial Cloud (GCC) or security assessments for government projects.
- AWS certifications such as Machine Learning Engineer Associate, AI Practitioner or Cloud Practitioner.
Personal attributes
- Strong attention to detail and a risk-aware, quality-driven mindset.
- Comfortable with Agile methodologies and working closely with product development teams.
- Strong on collaboration and communication, including documentation.
