The posting
This role requires strong technical knowledge of enterprise data platforms, excellent stakeholder management skills, and proven experience delivering complex projects within the banking and financial services industry.
- 10+ years of experience.
- 5+ years managing Enterprise Data Lake, Data Warehouse or Big Data delivery initiatives.
- Proven experience in Hadoop/Cloudera environments.
- Experience managing multiple concurrent project deliveries and BAU portfolios.
- Banking and Financial Services experience preferred.
Data Engineering & Integration
· Design, develop, and maintain scalable data pipelines within the Enterprise Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.
· Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.
· Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.
· Design and implement data ingestion patterns supporting structured, semi-structured, and unstructured datasets.
· Build and maintain batch and near real-time data integration solutions.
· Implement source-to-target mappings and ingestion specifications.
Hadoop & Cloudera Platform Development
· Develop and support solutions within the Cloudera Data Platform ecosystem.
· Build and maintain data pipelines leveraging:
· Hadoop HDFS
· Hive
· Impala
· Spark
· Kafka
· YARN, Oozie
· Perform deployment, troubleshooting, and performance tuning of applications running on the Hadoop ecosystem.
· Partner with platform teams to ensure optimal cluster utilization and workload management.
Data Quality & Governance
· Ensure adherence to Data Lake onboarding standards and governance requirements.
· Investigate and resolve data quality issues and production incidents.
· Ensure compliance with enterprise data governance standards.
DevOps & Platform Automation
· Integrate data engineering solutions into CI/CD pipelines using Git, Jenkins, Azure DevOps, or equivalent tools.
· Automate deployment, testing, monitoring, and operational support activities.
· Develop reusable frameworks and utilities to improve engineering productivity and platform scalability.
Production Support & Incident Management
· Provide L3 support for Data Lake application.
· Perform root cause analysis for production incidents and implement permanent fixes.
· Monitor pipeline execution, cluster health, and workload performance.
· Participate in release implementation, deployment validation, and post-production support activities.
· Strong hands on experience with Hadoop ecosystem and data pipeline orchestration.
· Hadoop/HDFS
· Hive
· Spark
· Unix/Linux scripting
· Shell scripting
· ETL/ELT development experience.
· Data modelling fundamentals.
· Performance tuning and optimization.
· Experience in Data Warehousing and Data Lake environments.
· Strong Java / Python competency with experiences.
· Strong SQL development skills.



