Senior Software Engineer, AI Evaluation Infrastructure

This is a remote position.

• Typically 5+ years building and operating production backend systems, with ownership of at least one service from design through on-call.
  
• Strong Python. Experience with Django (or a similar framework), PostgreSQL, and a task queue such as Celery. What matters most is having run a real system, not knowing a particular framework.
  
• Hands-on experience in LLM evaluation, beyond calling a model API. You understand how a benchmark task is built, how an agent trial is scored, and how results are aggregated and compared. Experience with Harbor, Terminal-Bench, SWE-bench, Inspect, or a similar harness is a strong plus.
  
• Judgment about task quality. You can spot ambiguous instructions, weak or gameable verifiers, leaked answers, and checks that pass for the wrong reason.
  
• Good tradeoffs. You reach for the smallest design that solves the problem, extend what already exists before adding something new, and can explain what you chose not to build.
  
• Ownership. You take a fuzzy request, find the relevant code, ask questions only when the answer changes the design, and finish the job, including migrations, tests, and edge cases users will hit.
  
• Quick adoption of team conventions. You read the written guidelines and your first pull requests already match them in naming, structure, tests, and error handling.
  
• Comfort with a TypeScript and React frontend when a change is visible there. You don’t need to be a frontend specialist.
  
• Clear writing. Your pull request descriptions and design notes explain why, not just what.