The posting
Job Description
Position Summary, Responsibilities and Expectations:
- Design, develop, and optimize scalable ETL/ELT pipelines for enterprise data ingestion, integration, and transformation. - Build and maintain data engineering workflows using Databricks, dbt, PySpark, SQL, and Python. - Develop Data Lake, Data Warehouse, and Lakehouse solutions, including data modeling and incremental processing. - Collaborate with business stakeholders to understand requirements, business processes, and challenges, and translate them into effective data and analytics solutions. - Proactively communicate with business users and technical teams to clarify requirements, provide project updates, address issues, and ensure solutions meet business expectations. - Ensure data quality, consistency, security, and governance through validation and automated testing. - Prepare and integrate enterprise data to support BI, AI/ML applications, and advanced analytics. - Develop AI/LLM solutions, including RAG, AI agents, API integrations, and intelligent workflow automation. - Monitor and optimize data pipeline performance, reliability, resource utilization, and costs. - Implement data orchestration, CI/CD, automated deployment, and monitoring practices. - Collaborate with BI, AI, IT, and business teams to support solution delivery, user testing, documentation, and continuous improvement.
Essential Skills and Experience:
- 2–4 years of hands-on experience in Data Engineering, ETL/ELT, and enterprise data platforms. - Strong SQL and Python skills, with practical experience in Databricks, PySpark, Delta Lake, and dbt. - Experience designing, developing, and optimizing production data pipelines, data models, and Lakehouse/Data Warehouse solutions. - Ability to understand business processes, gather and clarify requirements, and translate business needs into actionable technical solutions. - Strong verbal and written communication skills, including the ability to explain technical concepts to non-technical stakeholders. - Strong analytical, problem-solving, and interpersonal skills, with a proactive attitude and ability to collaborate across business and technical teams. - Familiarity with cloud data platforms (Azure/Microsoft Fabric preferred), Git, CI/CD, and data quality practices. - Understanding of AI/LLM technologies, API integrations, and AI-driven data processing or automation. - Nice-to-Have: Experience with RAG, AI agents, vector databases, Power BI, or enterprise ERP systems.



