The posting
In most instances, this position requires in-person interviews as part of the hiring process.
In accordance with Washington state law, we are highlighting our comprehensive benefits package, which is available to all eligible US based employees. Benefits for this role include:
- Health, dental, vision, life, disability insurance
- Retirement Benefits: 401(k) with company match
- Paid Time Off: 20 days of vacation per year, accruing at a rate of 6.15 hours per pay period for the first five years of employment
- Sick Time: 40 hours/year (increased to 69 hours/year for Seattle) including 5 discretionary sick days per instance
- Maternity Leave (Short-Term Disability + Baby Bonding): 28-30 weeks
- Baby Bonding Leave: 18 weeks
- Holidays: 13 paid days per year
Minimum qualifications:
- Bachelor's degree in Computer Science or related technical field, or equivalent practical experience.
- 5 years of experience in software development, with experience in C++ or Go.
- 3 years of experience designing and optimizing distributed systems, storage architectures, or database internals (columnar storage formats, serialization, or query execution engines).
Preferred qualifications:
- Experience with next-generation or specialized file formats like Vortex, Lance, Nimble, or FastLanes.
- Experience with vector search (ANN) and building storage layers optimized for retrieval-augmented generation (RAG) and multimodal data.
- Strong understanding of modern machine learning data pipelines, including how PyTorch/TensorFlow consume data and the I/O bottlenecks of AI training.
- History of successful contributions to major open-source data projects (e.g., Apache Arrow, Parquet, Iceberg).
About the job
BigQuery is at a pivotal moment in the evolution of cloud data warehouses. The industry is seeing a massive shift in file format momentum—with next-generation columnar formats like Vortex redefining core analytics performance, and multimodal formats like Lance pulling data gravity toward AI/ML training and vector search workloads.
We are seeking a visionary Tech Lead to own the goal, strategy, and architectural design for BigQuery’s file format. In this role, you will define how BigQuery adapts to these industry shifts, whether by evolving our native Capacitor format, adopting and extending open-source standards like Vortex, or bridging the gap to AI-native lakehouses.Google Cloud accelerates every organization’s ability to digitally transform its business and industry. We deliver enterprise-grade solutions that leverage Google’s cutting-edge technology, and tools that help developers build more sustainably. Customers in more than 200 countries and territories turn to Google Cloud as their trusted partner to enable growth and solve their most critical business problems.Individual pay is determined by factors including job-related skills, experience, and relevant education or training.
US: $174000 - $252000 (USD) + 15% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Own the end-to-end technical strategy for BigQuery's storage formats. Determine the roadmap for our existing proprietary formats (Capacitor) versus adopting or contributing to emerging open standards (Vortex, Lance, BtrBlocks).
- Design storage layer innovations that unlock zero-copy column updates/additions, extremely efficient random access, and seamless integration with deep learning training pipelines.
- Partner with Dremel, BigLake, and GCS teams to ensure our execution engine seamlessly adopts next-generation formats without regressing SQL concurrency.
- Position Google as a thought leader in the open data lakehouse ecosystem, steering the direction of the formats that will define the next decade of data infrastructure.
- Lead technical and ROI evaluations of open-source formats. Determine engineering costs to integrate Arrow-native execution with BigQuery pipelines for wide tables and multimodal AI.
