Please mention DailyRemote when applying
See how much of this job your resume covers, and what’s missing.
Want a recruiter to go through it line by line?
Get professional reviewQuestions interviewers often ask for this role, with sample answers.
One click makes a copy of your resume for this role, scores it against the posting, and proposes edits line by line. Your default resume is never changed.
Upload your resume and we draft a letter for this exact role, tailored to what it asks for.
The Lead Data Engineer will architect and execute complex data migration plans, including schema mapping and automated ETL pipeline development. They are responsible for overseeing migration dry runs and certifying data integrity through final reconciliation reports.
Seniority / Level: | Senior (7+ Years) |
Location / Work Mode: | Offshore |
| |
Primary Deliverables / Focus: | Data Migration Plan (D-03), Schema Mapping Specs,
Pipeline Architecture & Final Reconciliation Report (D-08) |
Role Objective & Summary
The Lead Data Engineer commands the data migration
architecture, transformation logic, and reconciliation framework. Starting in
November for legacy schema discovery, this senior engineer maps source data
structures to the modern Azure SQL schema, designs automated ETL pipelines in
Azure Data Factory, governs multi-cycle dry runs, and authors the final
audit-grade reconciliation report.
Core Key Responsibilities
· Author the Data Migration Plan (Deliverable D-03), establishing
migration stages, transformation rules, validation thresholds, and rollback
plans.
· Conduct deep legacy schema analysis, mapping historical
entities, fields, constraints, and data dictionaries to the target Azure SQL
schema.
· Architect scalable, automated ETL/ELT pipelines using Azure Data
Factory (ADF), PySpark, and SQL stored procedures.
· Design and oversee multi-cycle migration dry runs, tracking
extraction throughput, transformation accuracy, and load performance.
· Establish automated field-level checksum, hash comparison, and
record reconciliation routines to ensure zero critical data loss.
· Author the Final Data Reconciliation Report (Deliverable D-08),
certifying 100% data integrity and financial ledger reconciliation.
· Lead the technical execution of cutover data migration during
the production go-live window in September.
Required Technical Skills & Competencies
· 7+ years of data engineering and database migration experience
leading complex, large-scale enterprise data migrations.
· Extensive expertise with Microsoft Azure Data Factory (ADF),
Azure Data Lake Storage (ADLS Gen2), and Azure SQL Database / Managed Instance.
· Proficiency in SQL data manipulation, stored procedure
optimization, indexing, and bulk loading utilities (BCP, SSMA).
· Strong programming skills in Python / PySpark for complex data
transformations and normalization.
· Deep understanding of data quality governance, schema
translation, data cleansing, and audit reconciliation techniques.
· Experience working with relational (Oracle, SQL Server) and
legacy data formats (copybooks, fixed-width files, VSAM extracts).
Preferred Qualifications & Certifications
· Microsoft Certified: Azure Data Engineer Associate.
· Prior experience migrating legacy IBM or Unisys mainframe data
into modern relational cloud databases.
Stop the endless job search. Our AI finds and applies to the best jobs for you.
Featuring 218,429+ Jobs in Data Engineer
Answer easy questions
218,429+ jobs across 15+ categories
Get your best job matches
Only hand-screened, legit jobs
Find a remote job faster
No ads, scams, or junk
“I was the first applicant for a remote marketing position that got listed on the company website the same day I applied. Had an interview within 48 hours!”