Senior Azure Databricks Architect

PROJECT OVERVIEW

We are looking for a skilled Senior Azure Databricks Architect to design, develop, and optimize scalable data pipelines and cloud-based analytics solutions using Microsoft Azure and Databricks technologies. The ideal candidate should have strong experience in big data processing, ETL/ELT workflows, PySpark, and Azure cloud services.

IN THIS ROLE, YOU WILL

• Design, build, and maintain scalable ETL/ELT pipelines using Azure Databricks and Apache Spark;

• Develop and optimize PySpark/Spark SQL notebooks for large-scale data processing;

• Implement batch and real-time data ingestion pipelines using Azure Data Factory (ADF), Databricks Workflows, and Structured Streaming;

• Manage and optimize Delta Lake/Lakehouse architecture, including Bronze, Silver, and Gold layers;

• Integrate data from multiple sources including APIs, databases, cloud storage, and third-party systems;

• Work with Azure services such as ADLS Gen2, Azure Synapse Analytics, Azure SQL Database, and Azure DevOps;

• Monitor, troubleshoot, and improve pipeline performance, scalability, and cost optimization;

• Implement data governance, security, access controls, and compliance standards using Unity Catalog and RBAC;

• Collaborate with Data Engineers, Data Scientists, Analysts, and Product teams to deliver reliable data solutions;

• Support CI/CD implementation, automation, and infrastructure-as-code practices.

IF YOU ARE

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field;

• 5+ years of experience in Data Engineering or Big Data development;

• Strong hands-on experience with Azure Databricks and Apache Spark;

• Proficiency in PySpark, Spark SQL, and SQL;

• Experience with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS);

• Knowledge of Delta Lake, Lakehouse architecture, and data modeling concepts;

• Experience with distributed data processing and performance tuning;

• Familiarity with CI/CD pipelines, Git, and Azure DevOps.

NICE TO HAVE

• Experience with Kafka, dbt, Airflow, or Snowflake.

• Knowledge of Terraform or Infrastructure as Code (IaC).

• Exposure to MLflow, machine learning pipelines, or MLOps.

AS AN OPINOV8R, YOU WILL HAVE

• Digital-First Approach: Great talent knows no borders! You can work from wherever you are — we hire and collaborate with professionals worldwide.

• Remote Work Model: Balance your professional and personal life with our flexible working conditions, empowering you to deliver your best from anywhere.

• Exciting Projects: Dive into impactful projects across industries that challenge and spark creativity.

• Boost Your Expertise: Grow your career with continuous learning, development opportunities, and hands-on experience.

• Join the Best Team Ever: Collaborate with our diverse and cross-cultural team of passionate technologists and creative thinkers.

HOW’S THE HIRING PROCESS GOING

We strive to make our hiring process smooth and transparent to find the perfect match for both sides. Steps may differ depending on the role, but here’s what to expect:

• Initial Interview: If your background fits the role, we’ll invite you for an interview with a Talent Acquisition Specialist.

• Technical Interview: Depending on the position, you may complete a technical assessment or test task.

• Client Interview

• Final Decision: After all steps, we’ll get back to you with the result and next steps.