Data Engineer - Quality
Posted 2 hours ago
Worldwide
⭐
2-5 years experience
Please mention DailyRemote when applying
AI Summary
Design and implement a Python-based tool using LLMs and machine learning to automate quality control checks for vendor data sets. Collaborate with Data Managers and Software Engineers to improve data quality coverage and develop compliance dashboards.
This is a remote position.
Alternative Path is seeking skilled Data Engineer to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company department to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but you won't be isolated; the support of the Engineering team leads, the Product team leads, and every other technology team member is behind you. This is an opportunity to join a team-first meritocracy and help grow an entrepreneurial group inside Alternative Path. You will be asked to contribute, given ownership, and will be expected to make your voice heard.
Role Summary
Design and implement a Python-based tool to automate the configuration and definition of quality control checks for vendor data sets. The tool will leverage LLM and data set documentation to understand the critical fields and leverage machine learning algorithms to define a threshold for quality checks. The checks are configured in a proprietary data quality engine. The tool will be used to set up new data feeds and review and adjust thresholds for existing checks on a periodic basis.
You will work closely with Data Managers and Software Engineers to understand data sets and their existing checks and develop techniques to improve the coverage of checks across our platform.
Role Summary
Design and implement a Python-based tool to automate the configuration and definition of quality control checks for vendor data sets. The tool will leverage LLM and data set documentation to understand the critical fields and leverage machine learning algorithms to define a threshold for quality checks. The checks are configured in a proprietary data quality engine. The tool will be used to set up new data feeds and review and adjust thresholds for existing checks on a periodic basis.
You will work closely with Data Managers and Software Engineers to understand data sets and their existing checks and develop techniques to improve the coverage of checks across our platform.
Location - Remote
Requirements
Role Overview
The Data Guardian Team is seeking a creative and detail-oriented developer to contribute to projects. The team develops a data quality engine that runs data quality assertions on a vast array of data and orchestrates operational alerting workflows to our Data Operations team for review and remediation. We are seeking someone who is self-motivated, self-sufficient, with a passion for tinkering and a love for automation.
In your role, you will:
➢ Collaborate with other software engineers to understand and resolve requirements
➢ Design and implement a data quality automation tool using LLMs and machine learning algorithms
➢ Develop dashboards for data quality coverage and compliance
Qualifications
Must have
The Data Guardian Team is seeking a creative and detail-oriented developer to contribute to projects. The team develops a data quality engine that runs data quality assertions on a vast array of data and orchestrates operational alerting workflows to our Data Operations team for review and remediation. We are seeking someone who is self-motivated, self-sufficient, with a passion for tinkering and a love for automation.
In your role, you will:
➢ Collaborate with other software engineers to understand and resolve requirements
➢ Design and implement a data quality automation tool using LLMs and machine learning algorithms
➢ Develop dashboards for data quality coverage and compliance
Qualifications
Must have
➢ Bachelor’s or Master’s degree in computer science or engineering
➢ 3-5 years of experience in Data Engineering
➢ Strong experience building enterprise solutions in Python
➢ Pandas and data science transformation logic
➢ Exposure to SQL
Preferred
➢ AWS experience including Kubernetes
➢ S3 experience
➢ Trino experience
➢ 3-5 years of experience in Data Engineering
➢ Strong experience building enterprise solutions in Python
➢ Pandas and data science transformation logic
➢ Exposure to SQL
Preferred
➢ AWS experience including Kubernetes
➢ S3 experience
➢ Trino experience
Automatically Apply to the Best Remote Jobs
Stop the endless job search. Our AI finds and applies to the best jobs for you.
