Please mention DailyRemote when applying
See how much of this job your resume covers, and what’s missing.
Want a recruiter to go through it line by line?
Get professional reviewOne click makes a copy of your resume for this role, scores it against the posting, and proposes edits line by line. Your default resume is never changed.
Upload your resume and we draft a letter for this exact role, tailored to what it asks for.
You will design reinforcement learning environments, build evaluation systems, and develop reward functions to assess agent performance. Additionally, you will integrate REST APIs, lead demonstration programs, and mentor contributors on evaluation practices.
Join a network of experienced technical contributors working on paid AI human-data projects focused on agents and reinforcement learning. You will design training environments, build evaluation systems, and assess agent behavior as projects become available.
Design reinforcement learning environments, including Python and Docker setups, state spaces, tool integrations, and scoring mechanisms.
Build verifiers and reward functions that reliably evaluate agent performance and resist gaming.
Translate real-world business workflows in areas such as ERP, CRM, finance, and operations into measurable scenarios.
Integrate tools and REST APIs, and build test harnesses using frameworks such as pytest.
Lead agent demonstration and computer-use programs, setting quality standards for trace collection.
Grade and repair agent trajectories, and mentor contributors on evaluation practices.
At least 5 years of experience in software engineering, QA automation, machine learning, reinforcement learning, business systems, or operations paired with hands-on agent or RL data work.
Strong Python skills and experience building tools or services that others rely on.
Experience designing environments, verifiers, reward functions, or test harnesses to evaluate software or agent behavior, using tools such as pytest, Playwright, Selenium, or Gymnasium-style environments.
Hands-on experience with agent or RL data, such as demonstrations, trajectory grading, trace collection, or computer-use demos.
Experience integrating REST APIs and using Docker.
Experience leading or reviewing work on data, evaluation, or demonstration programs is valued, as is expertise modeling workflows in business systems such as ERP or CRM platforms.
A degree in computer science, engineering, or a related field is relevant.
Compensation is $80 to $200 per hour, depending on project, field, and experience level. Pay is based on hours worked. Visa sponsorship information was not provided.
Remote. Projects typically last from one week to several months.
Stop the endless job search. Our AI finds and applies to the best jobs for you.
Featuring 216,691+ Jobs in Others
Answer easy questions
216,691+ jobs across 15+ categories
Get your best job matches
Only hand-screened, legit jobs
Find a remote job faster
No ads, scams, or junk
“I was the first applicant for a remote marketing position that got listed on the company website the same day I applied. Had an interview within 48 hours!”