See how much of this job your resume covers, and what’s missing.
Want a recruiter to go through it line by line?
Get professional reviewUpload your resume and we draft a letter for this exact role, tailored to what it asks for.
You will develop reproducible, containerized coding environments and benchmarks to train and evaluate advanced AI models. This involves creating tasks from real-world software engineering data and building robust verifiers to ensure model performance and safety.
Join an early-stage AI company as a Coding RL Engineer, working with the reinforcement learning team to build coding environments and tasks for training and evaluating advanced models. You will turn real-world software engineering work into reproducible benchmarks that help improve coding AI.
Convert licensed production codebases into reproducible, containerized coding environments.
Use issues, pull requests, commit histories, and incident records to create tasks covering bug fixes, features, refactors, testing, and multi-file agent work.
Build and improve verifiers with hidden tests, rubric-based or model-based grading, and safeguards against reward hacking and answer leakage.
Run model rollouts, review traces, and repair or remove tasks that are broken, ambiguous, too easy, or exploitable.
Contribute tasks and analysis to coding benchmarks and build tools that speed up task creation and validation.
Work with research teams to calibrate task difficulty and benchmark coverage.
At least 2 years of professional software engineering experience delivering software systems.
Very strong Python skills and fluency in another language such as TypeScript, Go, Java, or Rust.
Hands-on Docker experience and familiarity with reproducible build and test environments.
Experience navigating unfamiliar, complex codebases and getting them building, running, and tested.
Working knowledge of coding model training and evaluation, including SWE-bench-style tasks, code evaluations, RLVR, and agent harnesses.
Experience creating coding tasks from real engineering work and hardening verifiers against flaky tests, unclear requirements, and leaked solutions.
Relevant experience with frontier AI labs, evaluation teams, RL environments, coding agents, or large enterprise and legacy codebases is helpful.
Remote within the United States or in person in San Francisco, California.
Stop the endless job search. Our AI finds and applies to the best jobs for you.
Featuring 222,714+ Jobs in Software Development
Answer easy questions
222,714+ jobs across 15+ categories
Get your best job matches
Only hand-screened, legit jobs
Find a remote job faster
No ads, scams, or junk
“I was the first applicant for a remote marketing position that got listed on the company website the same day I applied. Had an interview within 48 hours!”