Lead Data Engineer

Key Responsibilities

• Design end-to-end data architectures for client engagements, including ingestion, transformation, modeling, warehousing, and analytics delivery.

• Lead the technical execution of client-facing and internal data projects, maintaining accountability for timelines, scope, and quality.

• Provide subject-matter expertise across the data engineering lifecycle, including hands-on support for complex technical issues and performance optimization.

• Facilitate architecture and design reviews, articulating options, trade-offs, and recommendations to technical and business audiences.

• Establish and enforce engineering standards, controls, and audit procedures that safeguard data accuracy, integrity, and timeliness.

• Define and promote AI-augmented development practices, including structuring projects for effective use of coding agents and coaching team members in these workflows.

• Partner with data scientists, analysts, business stakeholders, and practice leadership to align technical delivery with client objectives and the team’s broader technology roadmap.

• Mentor engineers, encourage knowledge sharing, and contribute to a culture of continuous improvement across the practice.

• Identify and implement improvements to delivery processes, tooling, and methodology.

• Design and implement DevOps and SDLC strategies and best practices for customers.

   

  Qualifications

  Required

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical discipline.

• 8–10 years of professional experience in data engineering, including demonstrated success designing and delivering production data solutions.

• Expertise in data modeling, ETL/ELT development, and data integration.

• Hands-on experience with at least one major cloud platform (AWS, Azure, or GCP) and modern data warehousing technologies.

• Advanced proficiency in SQL and Python; experience with Spark or comparable distributed processing frameworks.

• Prior technical leadership experience, with the ability to direct and develop engineering teams.

• Excellent analytical, problem-solving, and communication skills, with the ability to engage effectively with clients, executives, and technical staff.

• Ability to balance technical rigor with business priorities, budgets, and client constraints.

• Experience architecting and implementing SDLC and DevOps best practices, including but not limited to version control, deployment pipelines, Infrastructure as Code (IaC), Azure DevOps, GitHub Actions, GitLab, Terraform, and Declarative Automation Bundles

  Preferred

• Active, day-to-day use of AI-assisted development tools (e.g., Cursor, Claude Code, Codex) as part of a professional engineering workflow.

• Experience building agentic pipelines, retrieval-augmented generation (RAG) systems, or LLM-powered data and analytics products.

• Prior experience in a consulting or professional services environment.

Datavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner.