Please mention DailyRemote when applying
See how much of this job your resume covers, and what’s missing.
Want a recruiter to go through it line by line?
Get professional reviewQuestions interviewers often ask for this role, with sample answers.
Upload your resume and we draft a letter for this exact role, tailored to what it asks for.
Design and refine evaluation rubrics for preference data while building pipelines to support data generation and collection workflows. Collaborate with researchers to translate qualitative judgments into scalable processes and synthesize findings into actionable technical updates.
Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products.
About the Role:
We're looking for a high-taste engineer with a strong eye for quality — someone who's worked hands-on with preference data and rubric creation, and knows what separates a good example from a great one. You communicate clearly and can synthesize complex technical work the way a strong MAI dev would: concise, structured, and easy for others to act on. You're comfortable in ambiguous, fast-moving environments and default to shipping over deliberating.
What does day-to-day look like:
Design, refine, and iterate on rubrics and evaluation criteria for preference data
Review and label data with a high bar for quality, catching subtle issues others might miss
Build and maintain pipelines/infra that support data generation, collection, and evaluation workflows
Synthesize findings from data work into clear write-ups, updates, or recommendations for the team
Collaborate closely with researchers/engineers to translate qualitative judgment into scalable processes
Required Skills:
5+ yrs of proficiency in Python (especially building/maintaining pipelines and infra) or Typescript/Golang/JavaScript (experience contributing to or navigating large codebases like VSCode a plus)
Hands-on experience with preference data collection and/or rubric creation
Strong written and verbal communication — able to distill technical work into clear, actionable summaries
Demonstrated high-taste judgment — able to articulate why something is good, not just that it is
Perks of Freelancing With Turing:
Work in a fully remote environment.
Opportunity to work on cutting-edge AI projects with leading LLM companies.
Engagement Details:
Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
Employment type : Contractor assignment (no medical/paid leave)
Duration of contract : 3 month; [expected start date is next week]
Location : North America
Evaluation Process (approximately 30 mins) :
One round of interview (30 mins of technical & cultural discussion)
5+ years of professional experience
Total number of positions: 320
Engagement Length: 4 weeks, Full-time (8 hours/day) 40 hours per week. Overlap 4 hours with PST
6-week project, with a preferred commitment of up to 40 hours per week.
Fully remote, with payment in USD.
Start as soon as you successfully pass the assessment.
Stop the endless job search. Our AI finds and applies to the best jobs for you.
Featuring 215,067+ Jobs in Python Developer
Answer easy questions
215,067+ jobs across 15+ categories
Get your best job matches
Only hand-screened, legit jobs
Find a remote job faster
No ads, scams, or junk
“I was the first applicant for a remote marketing position that got listed on the company website the same day I applied. Had an interview within 48 hours!”