Senior Software Engineering Manager - Data Platform

Overview

Instacart is a leading online grocery platform in North America, enabling customers to order from their favorite retailers and have their groceries delivered directly to their doorsteps. We are dedicated to transforming the way people shop for groceries by providing convenient and innovative solutions.

About the Team

The Data Platform team at Instacart is responsible for building and maintaining a robust, cutting-edge platform that processes petabytes of data daily. We collaborate closely with Product, Data Science, and Machine Learning teams to understand their needs and provide high-quality, scalable solutions. As we continue to grow, we're looking for a seasoned Engineering Manager to lead this team and drive our data strategy forward.

About the Job

As a Senior Engineering Manager for the Data Platform team, you will:

• Lead and Mentor: Manage a team of 10+ talented engineers, fostering a culture of excellence, collaboration, and continuous learning.
• Drive Data Strategy: Shape the direction of Instacart's data infrastructure, ensuring it meets the needs of various stakeholders and supports our rapidly growing data lake.
• Collaborate Cross-functionally: Work closely with other teams (Product, Infrastructure, Data Science, ML, etc.) to identify pain points and translate them into self-service, reliable solutions.
• Champion Engineering Excellence: Promote best practices in engineering and data processing, ensuring high-quality and scalable solutions.
• Own Projects End-to-End: Oversee projects from design and technical decisions to execution and delivery, managing timelines and resources effectively.
• Innovate and Influence: Suggest and drive organization-wide initiatives that improve our data platform and overall engineering processes.

About you

Minimum Qualifications:

• Experience: 10+ years in software engineering, with significant experience in data infrastructure and at least 3+ years in a leadership or management role.
• Technical Expertise: Strong knowledge of data infrastructure technologies such as Python, Scala, Kafka, Airflow, Spark, Iceberg, Delta Lake.
• Datastores and Warehouses: Proficiency with industry-standard databases and warehouses like Postgres, Elasticsearch, Cassandra, DynamoDB, Snowflake.
• SQL Skills: Strong command of SQL and experience with large codebases.
• Leadership Abilities: Proven track record of mentoring team members and fostering an environment of engineering excellence.
• Ownership and Drive: A strong sense of ownership with self-motivation and the ability to thrive in a fast-paced startup environment.
• Collaboration: Excellent communication skills with the ability to work effectively across cross-functional teams.
• Problem-Solving: Ability to balance urgency with delivering high-quality, pragmatic solutions.

Preferred Qualifications:

• Education: Bachelor’s or advanced degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent work experience.
• Large-Scale Data Handling: Experience working with petabyte-scale data and optimizing data processing pipelines.
• Machine Learning and Data Science: Familiarity with data science and machine learning workflows and requirements.
• Strategic Influence: Experience driving organization-wide initiatives and influencing technical strategies.