Please mention DailyRemote when applying
AI Summary
Design, develop, and maintain scalable data pipelines using Python, Apache Spark, and Databricks to support large-scale analytics. Collaborate with stakeholders to translate business requirements into technical solutions while ensuring data quality and performance.
This is a remote position.
About the Job
As a Senior Data Engineer at Aspire, you will play a key role in designing and delivering scalable data solutions that enable data-driven decision-making. You will work with modern data platforms and technologies, including Python, Apache Spark, and Databricks, to build high-performance data pipelines and support large-scale analytics initiatives.
What you'll do
- Design, develop, and maintain scalable data pipelines using Python and Apache Spark.
- Build and manage Databricks notebooks, workflows, and data processing solutions.
- Integrate and transform data from multiple sources, including databases, APIs, and cloud storage platforms.
- Optimize data processing workloads to ensure performance, reliability, and scalability.
- Implement data quality, monitoring, and alerting mechanisms across data pipelines.
- Collaborate with solution architects and stakeholders to translate business requirements into technical solutions.
- Support platform configuration, onboarding activities, and best practices for data engineering.
- Participate in code reviews, documentation, and continuous improvement initiatives.
What you'll need
- Bachelor's degree in Computer Science, Engineering, Data Science, or a related field.
- Minimum 5 years of experience in Data Engineering within production environments.
- Strong experience with Python, including PySpark and data processing frameworks.
- Hands-on experience with Apache Spark and distributed data processing.
- Experience with Databricks, including notebooks, clusters, workflows, and repositories.
- Strong SQL skills and experience working with relational databases.
- Experience with cloud platforms, preferably AWS (S3, IAM, Glue, Redshift, or similar).
- Experience integrating data from APIs, databases, and cloud-based systems.
- Familiarity with Git and collaborative software development practices.
- Strong analytical, problem-solving, and communication skills.
- Ability to work effectively with both technical and non-technical stakeholders.
- Databricks Certification (Associate or Professional).
- Experience with Delta Lake or Apache Iceberg.
- Familiarity with Airflow, Databricks Workflows, or similar orchestration tools.
- Knowledge with streaming technologies such as Kafka, Kinesis, or Spark Structured Streaming.
- Knowledge of dbt or similar data transformation frameworks.
- Knowledge in consulting or client-facing environments.
- Awareness or knowledge of IT security best practices as defined by ISO/SOC or similar.
Why Aspire
In addition to a competitive long-term total compensation with salary and performance-based bonus, we have a reward philosophy that expands beyond this.
- Be part of a (Remote is here-to stay) organization.
- Work and learn from great minds.
- Explore new opportunities to learn and grow everyday by attending technical and nontechnical training.
- Get market exposure by working with international tech leaders.
- Nursery reimbursement benefit.
- Attend virtual and onsite international tech conference.
- Exposure to work in an IT environment that adheres to rigorous security and compliance standards defined by ISO/SOC
Automatically Apply to the Best Remote Jobs
Stop the endless job search. Our AI finds and applies to the best jobs for you.
