We are a product team responsible for maintaining and optimizing data pipelines hosted in AWS to support product analytics, KPIs, and data-driven decision-making. Our team ensures seamless data flow, high availability, and performance tuning while keeping costs optimized.
Key Responsibilities:
● Develop, maintain, and optimize scalable data pipelines using Python and AWS-based services.
● Manage and monitor ETL/ELT workflows with tools like Airflow, Lambda, and Glue to ensure data availability and integrity.
● Work with large-scale datasets, improving storage efficiency and query performance.
● Ensure pipeline reliability by implementing logging, monitoring, and alerting mechanisms.
● Collaborate with cross-functional teams, including product managers, analysts, and engineers, to deliver high-quality data for decision-making.
● Identify and resolve bottlenecks, ensuring efficient data transformation and processing.
● Support cost optimization initiatives by evaluating and refining storage and compute resources.
● Enhance data observability by implementing best practices for data quality, validation, and governance.
Required Skills & Experience:
● 3+ years of experience in Data Engineering or Software engineering or a related role.
● Experience with AWS Cloud services
● Strong proficiency in Python for data processing, and automation.
● Experience with cloud services preferably AWS.
● Hands-on experience with basic SQL and data modeling.
● Experience with orchestrating ETL workflows using Apache Airflow or similar tools.
● Familiarity with CI/CD pipelines (Terraform/Github Actions), and version control (Git).
● Strong problem-solving skills and the ability to work in a fast-paced, agile environment.
Preferred Qualifications:
● Experience with data lake architectures.
● Knowledge of data governance, observability, and cost management best practices.
● Prior experience working with IoT data pipelines.
EEO Statement
Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.
Ace Your Job Interview
Read our advice on how to answer the most common interview questions.
Exclusive remote jobs daily.
