Please mention DailyRemote when applying
AI Summary
Design, build, and stabilize Databricks-based data pipelines while optimizing performance and scalability. Develop end-to-end data workflows from ingestion to reporting to deliver consistent business value.
About the Role:
We are looking for experienced Databricks Engineers / Architects to join a fast-moving data transformation initiative. The client is already live on Databricks with production pipelines and reporting in place, but the platform requires stabilization, optimization, and scaling.
This is an opportunity to work in a high-impact environment, helping transform an evolving data platform into a reliable, production-grade system delivering consistent business value.
Key Responsibilities:
- Design, build, and stabilize Databricks-based data pipelines
- Perform pipeline optimization and production hardening
- Develop and support reporting and data visualization solutions
- Implement best practices for performance, scalability, and cost optimization
- Work on end-to-end data workflows (ingestion → transformation → reporting)
- Identify and resolve performance bottlenecks and system inefficiencies
- Bring structure and consistency to an evolving Databricks environment
- Collaborate with cross-functional teams to improve platform reliability and ROI
Required Skills & Experience:
- Strong hands-on experience with Databricks (must-have)
- Experience in data pipeline development (ETL/ELT)
- Solid understanding of data engineering and data architecture principles
- Experience working with large-scale data processing systems
- Exposure to reporting/BI tools (Power BI, Tableau, or similar)
- Ability to work in fast-paced, evolving environments with minimal handholding
Preferred / Good-to-Have Skills:
- Familiarity with Apache Spark (PySpark / Scala)
- Experience with cloud platforms (AWS / Azure / GCP)
- Knowledge of performance tuning and cost optimization in Databricks
- Exposure to data governance, security, and best practices
- Prior experience in stabilizing or scaling data platforms
Ideal Candidate Profile:
- Can quickly ramp up and contribute from day one
- Strong problem-solver with experience handling unstable or partially built systems
- Comfortable working in a “build while running” environment
- Balances technical expertise with delivery focus
What Success Looks Like:
- Stabilized and optimized Databricks pipelines
- Improved platform reliability and performance
- Scalable and production-ready data workflows
- Consistent and measurable business value delivery
Automatically Apply to the Best Remote Jobs
Stop the endless job search. Our AI finds and applies to the best jobs for you.
