The posting
This role is for one of Weekday’s clients
Min Experience: 7+ years Location: Remote (India) JobType: full-time
Work Model: As per project requirements Technical Focus: Databricks, Lakehouse Architecture, Cloud Data Platforms, Presales, and Cost Estimation
About the Role
We are seeking an experienced Databricks Solution Architect to design scalable, secure, and cost-efficient data engineering solutions while bridging the gap between business requirements, technical architecture, presales, and implementation delivery.
The ideal candidate will possess strong hands-on expertise in Databricks, Lakehouse architecture, cloud data platforms, and large-scale data engineering. This role involves engaging with clients to understand business objectives, designing end-to-end data solutions, estimating project effort and infrastructure costs, developing technical proposals, and guiding engineering teams through implementation.
The candidate should combine technical depth with client-facing consulting, commercial awareness, and delivery leadership. They must be comfortable presenting architectural recommendations to stakeholders while actively contributing to proof of concepts (POCs), technical reviews, performance optimization, and implementation challenges.
Requirements
Roles and Responsibilities
A. Solution Architecture and Technical Delivery
- Design end-to-end data engineering and analytics solutions using Databricks Lakehouse architecture.
- Define scalable architectures for data ingestion, transformation, storage, orchestration, analytics, and downstream data consumption.
- Develop architecture patterns using Apache Spark, PySpark, SQL, Delta Lake, Unity Catalog, and relevant Databricks services.
- Design and guide batch and streaming pipelines, ETL/ELT frameworks, data models, and enterprise data integration solutions.
- Evaluate existing data platforms and recommend migration, modernization, and performance optimization strategies.
- Establish architecture standards, reusable design patterns, engineering best practices, and implementation guidelines.
- Guide data engineering teams on technical design, code quality, scalability, reliability, and performance tuning.
- Participate in POCs, architecture reviews, technical troubleshooting, and production deployments as required.
B. Presales and Client Engagement
- Engage with clients to understand business objectives, existing data ecosystems, technical challenges, and project requirements.
- Lead technical discovery sessions, requirement-gathering workshops, and solution architecture discussions.
- Translate business requirements into technical solutions, architecture diagrams, implementation approaches, and delivery roadmaps.
- Prepare technical proposals, solution presentations, RFP/RFI responses, and Statements of Work (SOWs) in collaboration with sales and delivery teams.
- Present architecture recommendations, technology choices, implementation strategies, assumptions, dependencies, and risks to client stakeholders.
- Assess technical feasibility and recommend appropriate technologies based on business needs, scalability, security, and cost.
- Collaborate with sales, business development, engineering, and delivery teams to develop technically sound and commercially viable proposals.
- Support solution handover from presales to implementation teams, ensuring alignment between proposed scope and delivery expectations.
C. Project Estimation and Costing
- Estimate project effort, delivery timelines, resource requirements, and implementation costs based on defined requirements and scope.
- Break down project scope into phases, work packages, technical activities, deliverables, and milestones.
- Determine team composition and resource allocation across architecture, data engineering, cloud engineering, QA, and other relevant functions.
- Estimate Databricks platform consumption and cloud infrastructure costs based on expected workloads and proposed architecture.
- Evaluate key cost drivers, including Databricks Units (DBUs), compute sizing, storage, networking, workload frequency, concurrency, and processing requirements.
- Prepare high-level and detailed estimates for new implementations, migrations, platform modernization, and ongoing support engagements.
- Identify cost optimization opportunities and recommend suitable architecture patterns, compute configurations, and workload-sizing strategies.
- Document assumptions, exclusions, dependencies, risks, contingencies, and potential cost variations.
- Provide technical and commercial inputs for project pricing, budgeting, and proposal development.
D. Cloud Architecture and Cost Optimization
- Design and implement Databricks solutions on Microsoft Azure, AWS, or Google Cloud Platform (GCP), depending on project requirements.
- Evaluate cloud-native services for data ingestion, storage, orchestration, security, monitoring, and system integration.
- Ensure architecture designs meet requirements for scalability, reliability, availability, performance, and maintainability.
- Define data access, security, governance, and compliance considerations across the proposed architecture.
- Optimize compute utilization, workload execution, storage consumption, and overall cloud platform spending.
- Recommend appropriate monitoring, workload management, and operational practices to improve platform efficiency.
- Support cloud data platform migrations and modernization initiatives involving enterprise-scale data environments.
E. Technical Governance and Delivery Leadership
- Review technical designs and implementation plans to ensure alignment with approved architecture and business requirements.
- Establish technical standards for data pipelines, reusable components, data models, and integration patterns.
- Identify architecture risks, technical dependencies, and implementation challenges and recommend mitigation strategies.
- Mentor engineering teams and provide guidance on Databricks development, cloud architecture, and performance optimization.
- Ensure smooth coordination between client stakeholders, presales teams, architects, and engineering teams throughout the project lifecycle.
Must-Have Skills
- Databricks: Strong hands-on experience designing and implementing enterprise-grade solutions on the Databricks platform.
- Lakehouse Architecture: Deep understanding of Lakehouse principles, Delta Lake, data storage, processing, governance, and analytics architecture.
- Apache Spark: Strong expertise in Apache Spark, PySpark, SQL, distributed processing, and performance tuning.
- Experience designing scalable ETL/ELT pipelines, data models, and data integration frameworks.
- Strong understanding of Unity Catalog, data platform architecture, and enterprise data engineering practices.
- Hands-on experience with at least one major cloud ecosystem: Azure, AWS, or GCP.
- Experience with cloud data services, orchestration frameworks, APIs, and enterprise data platforms.
- Ability to estimate project effort, resource requirements, implementation timelines, and platform infrastructure costs.
- Experience with client-facing technical consulting, solution architecture, presales, and proposal development.
- Ability to create architecture diagrams, technical proposals, implementation roadmaps, and cost estimates.
- Strong analytical, presentation, stakeholder-management, and problem-solving skills.
Good-to-Have Skills
- Data Governance: Experience defining data governance frameworks, data ownership, cataloging, lineage, and governance policies.
- Access Controls: Understanding of role-based access control, identity management, permissions, and secure data access.
- Data Security: Knowledge of data protection, encryption, access policies, regulatory compliance, and cloud security best practices.
- Experience with Databricks certifications, including Data Engineer Associate, Data Engineer Professional, or relevant architecture-focused credentials.
- Exposure to enterprise data migration, legacy platform modernization, and large-scale data transformation programs.
- Familiarity with machine learning, Generative AI, and Databricks AI capabilities.
- Experience working with consulting firms, system integrators, cloud service providers, or data engineering service organizations.
- Knowledge of cloud pricing models, platform consumption metrics, FinOps principles, and infrastructure cost optimization.
- Experience supporting proposals from initial client discovery through solution design, estimation, and delivery handover.
Qualifications and Experience
- 8–12+ years of overall experience in Data Engineering, Data Architecture, Cloud Data Solutions, or related disciplines.
- Strong hands-on experience with Databricks architecture, Lakehouse implementations, and enterprise data engineering.
- Proven track record of designing and delivering scalable cloud-based data solutions.
- Demonstrated experience in client-facing solution architecture, technical presales, project estimation, and commercial discussions.
- Practical understanding of workload sizing, cloud infrastructure pricing, and platform cost calculations.
- Relevant Databricks or cloud certifications are preferred.
Ideal Candidate Profile
The ideal candidate is a technically strong Databricks Solution Architect who can independently engage with clients, understand complex business and technical requirements, design scalable data solutions, and develop realistic project and infrastructure cost estimates.
They should be equally comfortable leading presales discussions and validating proposed solutions through hands-on technical involvement. The role requires someone who can build client confidence, make sound architectural decisions, guide engineering teams, and ensure that proposed solutions are technically feasible, commercially viable, secure, and cost-efficient.
This is a hybrid architecture, presales, estimation, and technical delivery role—not a presentation-only or purely commercial presales position.
Application Requirements
Candidates should share an updated resume highlighting:
- Total professional experience and relevant Databricks experience.
- Expertise across Azure, AWS, or GCP.
- Examples of end-to-end data solutions architected and delivered.
- Experience in presales, client engagement, technical proposals, and RFP/RFI responses.
- Experience in project estimation, resource planning, and cloud/platform cost calculations.
- Relevant certifications, notice period, and availability.
