Automation Test Lead – Data & Platform Engineering

This is a remote position.

• Test Automation Architecture & Strategy: Own a scalable, modular, metadata-driven framework covering data pipelines (batch/streaming), APIs/backend services, and end-to-end data product validation; enable plug-and-play components, parallel execution, environment isolation, deterministic runs
  
• Data Testing Framework Engineering: SQL-based assertions/reconciliation, schema validation, data contracts, lineage/freshness validation, config-driven test definitions (YAML/JSON)
  
• Destructive Testing: Schema drift, backward incompatibility, late-arriving data, partial failures, duplicate/missing/out-of-order events, stress, concurrency, retries, DLQ handling, backpressure
  
• ETL/Streaming Validation at Scale: Row/aggregate/hash-based reconciliation, incremental/backfill validation, delivery semantics validation, window/time-based correctness
  
• Data Quality & Observability: Integrate/extend Great Expectations or Soda; custom validations for accuracy, completeness, uniqueness, timeliness; quality dashboards
  
• CI/CD & DataOps Enforcement: Pre-merge gates, release blockers, selective/parallel test execution, GitHub Actions/Jenkins integration
  
• Test Data Management: Synthetic data generation, masking/anonymization, deterministic datasets, edge case simulation
  
• Performance & Reliability Testing: Pipeline/query benchmarks, concurrency/stress testing, data skew analysis, cost/time optimization
  
• Security & Compliance: PII/PHI exposure checks, encryption/access control, retention, audit requirements; support GxP/SOX/ISO frameworks
  
• Cross-Functional Quality Leadership: Work with data engineers, platform teams, architects; mentor engineers
  
• Incident Analysis & Prevention: Root-cause analysis of production data issues, reduce flaky tests
  

Requirements

Mandatory Skills:

• Strong Python (test frameworks, libraries, CLI tools); advanced SQL
  
• Hands-on DBT, Airflow, Snowflake or similar; ETL/ELT and data modeling
  
• Great Expectations/Soda; lineage and catalog systems
  
• Kafka/Kinesis testing; delivery semantics
  
• Git workflows; CI/CD (Jenkins, GitHub Actions)
  
• AWS; IaC (Terraform/CloudFormation)
  

Desired Skills: API contract testing (PACT); basic UI automation; data mesh/data product exposure; Prometheus/Grafana; regulated domain experience (healthcare, life sciences, finance)

Experience: 9+ years in automation/QE; 5+ years building frameworks for data platforms at scale; track record improving reliability/reducing incidents