The posting
Principal Data Engineer
## Company Overview (Details about the company are not specified.)
## Job Summary The Principal Data Engineer will be responsible for developing and maintaining data ingestion, transformation, and integration pipelines within the Microsoft Azure ecosystem. This role plays a crucial part in ensuring secure and efficient data storage and processing, contributing to the organization’s overall data strategy and goals.
## Responsibilities - Develop and maintain data ingestion, transformation, and integration pipelines using PySpark, Scala, and SQL within the Microsoft Azure ecosystem. - Manage and maintain Databricks Delta Lake environments for secure and efficient data storage and processing. - Design and implement data models (ER and dimensional) to optimize data access and reporting. - Build pipelines to integrate data from various sources including flat files, SQL Server, Oracle, Informix, JSON, XML, Blob storage, and APIs. - Leverage Azure services such as Azure Data Factory, Azure Storage, Blob Accounts, and Key Vaults to ensure seamless data operations and security compliance. - Ensure best practices for code creation, migration, and deployment across multi-environment setups.
## Qualifications - Strong experience with PySpark, Scala, and SQL. - Proficiency in designing data models and building ETL pipelines. - Familiarity with Databricks Delta Lake and Azure services. - Ability to troubleshoot and resolve data-related issues effectively. - Excellent communication skills for stakeholder engagement. - Experience in providing technical mentorship to junior team members. - Knowledge of data engineering best practices.
### Educational Qualifications - A degree in Computer Science, Data Science, Information Technology, or a related field is preferred.
## Preferred Skills - Experience in handling both traditional and non-traditional data sources. - Knowledge of data governance and security compliance measures. - Familiarity with metrics and data science team collaboration.
## Experience - Minimum of 5-7 years of experience in data engineering or a related role, with a focus on Azure technologies.
## Environment (Details about the work setting, location, or physical conditions are not specified.)
## Salary (Salary details are not specified.)
## Growth Opportunities (Details about potential career advancement opportunities are not specified.)
## Benefits (Details about offered benefits are not specified.)



