The posting
Overview
We are looking for a Logging & Data Platform Engineer to design, build, and operate the logging and operational data-platform capabilities supporting the future SSOE platform.
You will build the platform that collects, transports, stores, indexes, searches, and serves operational data across MOE's technology environment, spanning on-premise infrastructure, networks, applications, GCC, AWS, Azure, and hybrid environments.
What You Will Be Working On
- As a Logging & Data Platform Engineer, you will build the shared platform capabilities that enable engineering and operations teams to reliably collect and use operational data at scale.
- You will work across logging, telemetry ingestion, data movement, storage, search, retention, and platform integration.
- The role requires an engineer who understands both traditional enterprise infrastructure and modern cloud-native architectures and can design solutions that work securely and reliably across environment boundaries.
- You will work closely with the Observability Engineer on telemetry requirements and with Data Engineering & Analytics on shared data-platform capabilities and integration patterns.
Key Responsibilities
Logging Platform Engineering
- Design, build, and operate logging capabilities across on-premise infrastructure, networks, applications, cloud platforms, and hybrid environments
- Collect logs from servers, network devices, applications, containers, databases, security appliances, cloud services, and other infrastructure sources
- Build scalable log ingestion, routing, enrichment, storage, indexing, search, and retrieval capabilities
- Define structured logging standards, schemas, metadata, tagging, and correlation conventions across services
- Design appropriate retention, archival, lifecycle, and deletion policies for different classes of operational data
- Support correlation between logs, metrics, events, and traces using common identifiers and telemetry standards
- Work with the Observability Engineer to implement platform capabilities supporting end-to-end service observability
Data Collection & Integration
- Design secure and resilient data movement between on-premise environments, GCC, and approved external services
- Implement collection and forwarding patterns appropriate to different infrastructure, application, network, and security environments
- Design for intermittent connectivity, network constraints, buffering, retry, back-pressure, and recovery between environments
- Build event-driven and streaming patterns for moving operational data between producers and consumers
- Integrate legacy and enterprise systems with modern cloud-native platform capabilities
- Define clear interfaces and integration patterns between logging, observability, data engineering, and application platforms
Data Platform Engineering
- Build shared platform capabilities for ingesting, storing, processing, querying, and serving operational data
- Design scalable storage and query architectures appropriate to data volume, access patterns, retention requirements, and cost
- Build ingestion, filtering, enrichment, and transformation pipelines for operational data
- Provide APIs, query interfaces, or other serving mechanisms for authorised downstream consumers
- Support operational datasets consumed by the User Portal, dashboards, reporting, automation, and Data Engineering & Analytics
- Define schemas and data contracts for shared platform interfaces
- Ensure platform changes remain backwards compatible or are coordinated with downstream consumers
Cloud & Platform Engineering
- Design solutions using cloud-native logging, streaming, storage, search, and data capabilities
- Build infrastructure and platform configuration using Infrastructure as Code
- Automate build, test, deployment, configuration, and platform changes through CI/CD
- Design for scalability, resilience, high availability, recoverability, and operational simplicity
- Monitor platform capacity, performance, reliability, and cost
Security & Governance
- Enforce MOE and Government data-classification requirements
- Design secure routing and storage of operational data across security zones and environment boundaries
- Apply appropriate encryption, access controls, authentication, and authorisation
- Ensure logging pipelines do not unnecessarily expose credentials, secrets, or sensitive information
- Implement audit-trail preservation and appropriate retention controls
- Ensure data-residency requirements are considered when routing operational data between on-premise, GCC, cloud, and SaaS environments
- Participate in security, architecture, and operational-readiness reviews
Reliability & Operations
- Define SLOs and operational health indicators for logging and data-platform services
- Build monitoring, alerting, failure detection, retry, and recovery into platform components
- Monitor ingestion health, processing latency, data loss, storage utilisation, search performance, and platform availability
- Participate in incident investigation, root-cause analysis, and post-incident reviews
- Participate in operational support and on-call responsibilities for owned services
- Maintain architecture documentation, operational procedures, and runbooks
What We Are Looking For
Experience
- Minimum 3–5 years of experience in cloud engineering, platform engineering, DevOps, SRE, logging engineering, data platform engineering, or a related discipline
- At least 2 years of hands-on experience building or operating production logging, telemetry, or data-platform capabilities
- Demonstrated experience working with AWS and/or Azure cloud-native services
- Experience integrating on-premise and cloud environments, or operating systems in a hybrid environment
- Experience with production data ingestion, streaming, routing, storage, indexing, or search platforms
- Experience implementing Infrastructure as Code and CI/CD for production environments
- Experience designing systems for scalability, resilience, security, and operational support
