The posting
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Data Engineer (Engenheiro de Dados Sênior) based in Brazil.
Join a technology-driven environment where data architecture and engineering play a central role in enabling advanced analytics, streaming, and artificial intelligence initiatives. As a Senior Data Engineer, you will design and develop scalable, secure, and high-performance data platforms using Microsoft Azure and Databricks. You will lead the implementation of Lakehouse architectures, build robust data pipelines, and ensure information remains reliable, accessible, and well-governed. Working closely with technical stakeholders and international teams, you will help transform complex data requirements into efficient, production-ready solutions. Your expertise will contribute to improving data quality, optimizing platform performance, and enabling data-driven decision-making across the organization. This fully remote, long-term opportunity is ideal for a senior professional who combines deep technical expertise with strong communication skills and the ability to collaborate confidently with clients and teams across North America.
Accountabilities:
- Data Architecture and Solution Design: Design, develop, and maintain scalable data architectures using Microsoft Azure and Databricks, ensuring alignment with business objectives, technical requirements, and long-term platform needs.
- Lakehouse Architecture Implementation: Build and optimize modern Lakehouse solutions using Delta Lake and Unity Catalog to support reliable data storage, processing, access management, and analytics.
- Data Pipeline Development: Design, implement, maintain, and optimize robust data pipelines that support efficient ingestion, transformation, integration, and delivery of data across multiple sources.
- Advanced Data Processing: Leverage Apache Spark and PySpark to develop high-performance data processing solutions capable of handling large volumes of structured and unstructured information.
- Streaming and Real-Time Data: Support the development and optimization of streaming data architectures and processing workflows to enable timely insights and data-driven applications.
- Data Quality and Reliability: Establish and implement data quality controls, validation processes, and monitoring mechanisms to ensure accuracy, consistency, completeness, and reliability across data platforms.
- Data Governance and Security: Implement and maintain data governance frameworks, access controls, security policies, and metadata management practices to protect sensitive information and support compliance requirements.
- Platform Performance and Optimization: Monitor and optimize data workloads, infrastructure utilization, processing performance, and platform costs to ensure scalable and efficient operations.
- Analytics and AI Enablement: Develop and maintain data foundations that support business intelligence, advanced analytics, machine learning, and artificial intelligence initiatives.
- Technical Collaboration: Work closely with engineering teams, analytics specialists, business stakeholders, and international clients to translate requirements into effective technical solutions.
- Technical Documentation and Best Practices: Maintain architecture documentation, establish engineering standards, and promote best practices in data modeling, pipeline development, governance, and platform management.
- International Stakeholder Communication: Communicate technical concepts, architectural decisions, project progress, and recommendations clearly in English when collaborating with clients and teams in the United States and Canada.
- Professional Experience: Extensive experience as a Senior Data Engineer and/or Data Architect, with a proven track record of designing and implementing enterprise-scale data solutions.
- Databricks and Microsoft Azure: Advanced knowledge of Databricks and Microsoft Azure, including experience designing, deploying, and managing cloud-based data engineering platforms.
- Lakehouse Architecture: Strong experience with Lakehouse architecture principles, Delta Lake, and Unity Catalog, including data management, governance, and access control.
- Apache Spark and PySpark: Advanced proficiency in Apache Spark and PySpark for large-scale data processing, transformation, and workload optimization.
- Data Pipeline Engineering: Demonstrated experience building, maintaining, and optimizing scalable data pipelines and modern data platforms, with an understanding of performance, reliability, and operational efficiency.
- Data Engineering and Analytics: Strong understanding of data engineering principles, analytics requirements, data integration, and the preparation of reliable datasets for downstream consumption.
- Streaming Technologies: Experience designing or supporting streaming data solutions and processing architectures for real-time or near-real-time applications.
- Data Governance and Security: Knowledge of data quality management, metadata, governance frameworks, access controls, security practices, and data lifecycle management.
- Performance Optimization: Ability to identify bottlenecks, troubleshoot complex data workloads, and improve processing efficiency, scalability, and resource utilization.
- Advanced English Proficiency: Fluent English at an expert professional level, with the ability to communicate technical information effectively and collaborate directly with clients and teams in the United States and Canada.
- Communication and Collaboration: Strong interpersonal and communication skills, with the ability to work effectively across multidisciplinary and geographically distributed teams.
- Proactivity and Ownership: A proactive approach to problem-solving, with the ability to take initiative, anticipate challenges, and drive technical tasks through to completion.
- Prioritization and Adaptability: Strong organizational skills, a sense of urgency, and the ability to prioritize competing demands while maintaining high standards of quality.
- Remote Work Readiness: Ability to work independently in a fully remote environment while maintaining effective communication, accountability, and collaboration with international stakeholders.
- Contract Availability: Willingness to work under a Brazilian service-provider arrangement (Pessoa Jurídica – PJ) for an indefinite-term engagement.
- Fully Remote Work: Work remotely from Brazil, with flexibility in managing your work environment.
- Long-Term Opportunity: Indefinite-term contract offering the potential for continued professional engagement.
- Learning and Development: Access to learning incentive programs, including Udemy resources, to support technical development and continuous learning.
- Corporate English Classes: Access to corporate English lessons at affordable rates to strengthen professional communication skills.
- International Collaboration: Work with clients and technical teams in the United States and Canada, gaining exposure to international projects and engineering practices.
- Technical Growth: Expand your expertise in Databricks, Microsoft Azure, Lakehouse architecture, advanced data processing, and modern data platforms.
- Meaningful Technical Impact: Contribute to scalable data solutions that support analytics, streaming, and AI initiatives across business environments.
- Collaborative Environment: Work with multidisciplinary professionals in an environment that values innovation, knowledge sharing, proactive problem-solving, and technical excellence.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
