Azure Databricks Data Engineer
Posted an hour ago
Worldwide
⭐
5-10 years experience
Please mention DailyRemote when applying
AI Summary
Design and implement end-to-end data pipelines using Azure-native services and Databricks within a medallion architecture. Focus on building scalable data platforms, implementing data governance via Unity Catalog, and optimizing performance for Azure SQL DBs.
This is a remote position.
Overview :
Design and implement end-to-end data pipeline using Azure-native services and databricks. This role focuses on building scalable, high-performance data platforms, defining robust data frameworks, and partnering with stakeholders to translate business requirements into reliable data solutions.
Implement data governance principles using unity catalog.
Create and Implement data ingestion patterns and data integration patterns in medallion architecture.
Apply best practices in data engineering
Create Pyspark notebooks
Create procedures for advanced data transformation.
Handle performance tuning aspects in Databricks and Azure SQL DBs
Requirements
Required Skills & Qualifications
- Hands-on experience with Azure Databricks
· Hands-on expertise in data ingestion, transformation, and orchestration frameworks.
· Hands on experience in building Insurance specific data models
· Proven ability to optimize data pipelines for performance, scalability, and cost efficiency.
· Solid understanding of distributed data processing concepts.
Preferred Skills & Qualifications
· Hands-on experience with PySpark for large-scale data processing.
· Experience defining and enforcing best practices for Databricks workloads and Spark-based pipelines.
· Strong analytical and problem-solving skills.
· Excellent communication skills to effectively work with cross-functional stakeholders.
· Prior experience in working in Insurance P&C cloud based data & analytics projects
Benefits
Diversity Inclusion: Exavalu also promotes flexibility, adapting to the needs of employees, customers, and the business. It might be part-time work, working outside normal 9-5 business hours or working remotely. We also offer a welcome back program to help individuals return to the mainstream after a prolonged absence due to health or family reasons. At Exavalu, we are committed to building a diverse and inclusive workforce. We welcome applications for employment from all qualified candidates, regardless of race, colour, gender, national or ethnic origin, age, disability, religion, sexual orientation, gender identity or any other status protected by applicable law. We nurture a culture that embraces all individuals and promotes diverse perspectives, where you can make an impact and grow your career
Automatically Apply to the Best Remote Jobs
Stop the endless job search. Our AI finds and applies to the best jobs for you.
