The posting
Job Summary
- We are seeking a Senior Data Engineer to join our Data & Analytics team. The ideal candidate will design, build, and maintain robust on-premise data pipelines, architect flexible lakehouse solutions, and lead data infrastructure initiatives to enable data-driven decision-making across the organization.
Mandatory Skill-set
- Bachelor's degree in Computer Science, IT, Engineering, or related field.
- Must have 5 years of hands-on experience in data engineering and data pipeline development;
- Hands on experience in SQL, with in-depth expertise in Microsoft SQL Server, including query optimization, indexing strategies, performance tuning, and troubleshooting;
- Hands-on experience with at least one NoSQL database, such as MongoDB or Apache Cassandra;
- Practical experience in distributed data processing using Apache Spark and PySpark, including performance optimization and large-scale data processing;
- Strong Python programming skills for data processing, automation, and developing scalable, reliable, production-grade data pipelines;
- Expertise in designing, deploying, and managing containerized data pipelines using Kubernetes in on-premises environments;
- Strong understanding of relational and non-relational data modeling concepts, including schema design and optimization;
- Hands-on experience with Apache Airflow or similar orchestration tools for scheduling, monitoring, and managing batch and real-time data pipelines;
- Knowledge of data security and governance practices, including role-based access control (RBAC), data masking, and compliance with relevant regulatory and organizational standards;
Desired Skill-set
Good to have experience with on-premise data virtualization or logical data warehouse concepts.
Good to have real-time streaming technologies (Kafka, Apache Flink);
Responsibilities
- Design and develop autonomous, production-grade ETL/ELT data pipelines using Python and PySpark that ingest, transform, and deliver high-quality data while maintaining integrity and performance standards.
- Implement and manage flexible lakehouse architecture across raw, curated, and consumption layers, including data partitioning, cataloging, and metadata management;
- Deploy and manage data pipelines using Kubernetes and Docker to ensure scalability, reliability, and efficient resource utilization in on-premise environments;
- Leverage strong SQL Server expertise to design optimal data models, write complex queries, and perform query optimization across the data platform;
- Establish and maintain robust CI/CD practices for data pipeline deployment, including automated testing, version control, and continuous monitoring;
- Enforce security, governance, and role-based access controls across all data layers while ensuring compliance and auditability;
- Mentor junior engineers, conduct code reviews, and establish best practices across the team;
- Collaborate with Data Scientists, Business Analysts, and stakeholders to deliver datasets aligned with operational and analytical needs;
- Provide L3 support and expert consultation for complex data challenges; evaluate and recommend new tools and practices to improve agility and performance.
Should you be interested in this career opportunity, please send in your updated resume to [email protected] at the earliest.
When you apply, you voluntarily consent to the disclosure, collection and use of your personal data for employment/recruitment and related purposes in accordance with the SCIENTE Group Privacy Policy, a copy of which is published at SCIENTE’s
website(https://www.sciente.com/privacy-policy).
Confidentiality is assured, and only shortlisted candidates will be notified for interviews.
EA Licence No. 07C5639



