Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark while optimizing ETL/ELT workflows. Collaborate with stakeholders to deliver data-driven solutions and manage cluster performance and data governance.
The Data QA Engineer will perform ETL testing and source-to-target data validation within a Databricks environment. They are responsible for designing test cases, tracking defects, and ensuring data integrity and quality throughout the transformation process.
Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark while optimizing ETL/ELT workflows. Collaborate with stakeholders to deliver data-driven solutions and manage Databricks clusters for performance and cost efficiency.
