Develop and maintain data pipelines within an Azure environment, focusing on Azure SQL Pools and Databricks. The role involves optimizing SQL queries, performing performance tuning, and orchestrating workflows using Synapse Pipelines.
Company Background
Our client is a leading life insurance company, part of a global financial services group. The company offers a wide range of products, including annuities and life insurance, delivered through a large nationwide network of agents.
Project Description
The project focuses on a production system managing clients and life insurance programs. The role involves both support and new feature development, with a strong emphasis on building and optimizing data pipelines and working within a modern Azure-based data platform.
Technologies
- Azure
- Databricks (Architecture, Unity Catalog, Delta, Lineage, Permissions, Databricks Genie, Notebooks)
- Synapse (Pipelines)
- Azure SQL pools
- Python
- SQL
- Performance tuning
What You'll Do
- Develop and maintain data pipelines in the Azure environment;
- Work with Azure SQL Pools for data storage and processing;
- Write and optimize SQL queries;
- Perform performance tuning for queries and data workflows;
- Collaborate with a team of engineers on Databricks-based solutions;
- Build and orchestrate data workflows using Synapse Pipelines / Azure Data Factory;
- Ensure reliability and consistency of data pipelines;
Job Requirements
- 3+ years of experience as a Data Engineer;
- Solid proficiency in SQL;
- Experience working in Azure cloud environment;
- Experience with Azure SQL Pools;
- Strong understanding of performance tuning for data workloads;
- Experience working in Azure cloud environment;
- Excellent communication skills;
- English level: B2 (Upper-Intermediate) or higher;
Nice To Have
- Experience with Azure data integration tools (Synapse Pipelines, Azure Data Factory);
- Experience with Python;
- Experience with Databricks ecosystem (architecture, Unity Catalog, Delta tables, notebooks, lineage, permissions);
- Experience with CI/CD, DevOps practices, Infrastructure as Code (IaC);
- Familiarity with Databricks Asset Bundles;
What Do We Offer
The global benefits package includes:
- Technical and non-technical training for professional and personal growth;
- Internal conferences and meetups to learn from industry experts;
- Support and mentorship from an experienced employee to help you professional grow and development;
- Health insurance;
- English courses;
- Sports activities to promote a healthy lifestyle;
- Flexible work options, including remote and hybrid opportunities;
- Referral program for bringing in new talent;
- Work anniversary program and additional vacation days.