Design and implement end-to-end data architectures using the Databricks Lakehouse Platform and Medallion Architecture. Oversee the creation of production-ready pipelines, manage data governance via Unity Catalog, and optimize large-scale distributed computing jobs.
At Sequoia Connect, we are a Talent-First Technology Ecosystem that redefines how elite professionals interact with the global digital landscape. We move beyond traditional models to act as a catalyst for the top 1% of global talent, connecting human potential with complex industrial execution. By joining our inner circle, you are not simply taking a position; you are aligning with a strategic partner dedicated to updating your "Human OS" and accelerating your growth through world-class, high-impact projects.
We are currently partnering with a global IT powerhouse that represents the connected world through innovative, customer-centric experiences. As a USD 6 billion organization and one of the top 7 IT service providers globally, our client empowers over 1,200 global customers—including several Fortune 500 companies—to "Rise™." With a massive network of 163,000+ professionals across 90 countries, they are at the absolute forefront of digital transformation, leveraging next-generation technologies such as 5G, AI, Blockchain, and Quantum Computing.
This is your chance to thrive in a workplace recognized as one of the most sustainable corporations in the world. You will join an environment that values innovation and societal impact, working on end-to-end digital transformation projects for global leaders. If you are a driven professional looking for global career opportunities and exposure to high-impact projects within an international network of expertise, this is where you belong.
We are currently searching for an Architect (Databricks Lakehouse):
The Challenge (Responsibilities)
- Design and implement end-to-end data architectures utilizing the Databricks Lakehouse Platform and Medallion Architecture (Bronze, Silver, Gold layers).
- Oversee the reliable creation of production-ready pipelines and data organization patterns.
- Troubleshoot and tune large-scale distributed computing jobs leveraging Apache Spark internals (caching, partitioning, broadcast joins).
- Manage data schemas, ACID transactions, and optimization operations (like Z-ordering) within Delta Lake.
- Design unified governance models, row/column-level security, role-based access control (RBAC), and data lineage using Unity Catalog.
- Implement native cloud infrastructure setups (e.g., ADLS/Entra for Azure, S3/IAM for AWS), including Virtual Network (VNet) configurations and IAM roles.
- Monitor resource usage (DBUs), right-size serverless/multi-node clusters, and implement cost optimization strategies.
Your Profile (Requirements)
- Deep expertise with the Databricks Lakehouse Platform, Apache Spark, and Delta Lake.
- Proven ability to design and implement Medallion data architectures.
- Strong knowledge of Spark internals and large-scale performance tuning.
- Demonstrated experience with Unity Catalog for data governance and AI asset management.
- Strong grasp of native cloud ecosystems (AWS, Azure, or GCP).
- Advanced coding proficiency in SQL and either Python or Scala.
- Demonstrated experience in cloud cost optimization and avoiding bill shock.
- High-Performance Mindset: Resilience, emotional intelligence, and a focus on agile delivery.
- Technologist DNA: A deep understanding of the difference between "coding" and "engineering."
Desired
- Valid Databricks Certified Data Architect or Databricks Certified Data Engineer Professional certifications.
- Experience building and deploying GenAI applications and ML models using Databricks Model Serving, Vector Search, and the Mosaic AI suite.
- Experience automating Databricks workflows using Git (Databricks Repos) and orchestration tools (dbt, Azure Data Factory, or Apache Airflow).
- Familiarity with Databricks Structured Streaming and Auto Loader.
- Understanding of Databricks SQL, Serverless Warehouses, and BI tool integration (e.g., Power BI).
Languages
- Advanced Oral English: For seamless collaboration with global teams.
- Advanced Spanish.
Work Arrangement We value flexibility to support your lifestyle. This position is available as:
If you meet these qualifications and are pursuing new challenges, start your application on our website to join an award-winning employer. Explore all our job openings | Sequoia Career’s Page: https://www.sequoia-connect.com/careers/
Requirements
- Expertise in designing end-to-end data architectures (Bronze, Silver, Gold layers).
- Deep understanding of Databricks & Spark Internals (troubleshooting, tuning, caching, partitioning, broadcast joins).
- Must understand ACID transactions, schema enforcement, time travel, and optimization operations in Delta Lake.
- Proven ability to design unified governance models using Unity Catalog (RBAC, row/column-level security, data lineage).
- Strong grasp of native cloud infrastructure (AWS, Azure, or GCP).
- Advanced SQL skills and fluency in Python or Scala.
- Ability to monitor DBUs and implement cost optimization & performance tuning.