Founding AI Quality & Release Engineer

About the Role

This is a founding-level software engineering role focused on building reliable, AI-powered verification and release infrastructure. You will develop the systems that evaluate agent-based products, strengthen CI signals, and help ensure safe, predictable releases.

What You'll Do

• Build and maintain test harnesses that use agents to verify product behavior.

• Develop evaluation logic, including LLM graders and regression coverage for agentic systems.

• Create automated release gates that make readiness visible and prevent unsafe releases.

• Own CI health, release pipelines, artifact publication, and deployment automation.

• Improve the reliability of staging and demonstration environments.

• Investigate quality gaps and incidents, and improve systems to prevent recurring issues.

What We're Looking For

• Seven to fifteen years of experience in quality engineering, release engineering, or platform engineering.

• Recent hands-on experience using AI-native methods to test or evaluate LLM-based or agentic systems.

• Practical experience with LLM APIs, tool calling or MCP, and evaluation methods.

• Strong familiarity with GitHub Actions and GitHub-based CI/CD, including container builds and pipeline automation.

• Experience with Python, TypeScript, Docker, Kubernetes, and Helm.

• Working knowledge of at least one major cloud or data platform, such as AWS, Azure, Databricks, or Snowflake.

• A track record of taking independent ownership of engineering systems, ideally in a high-velocity startup or while modernizing established engineering practices.

• Authorization to work in the United States or Canada without visa sponsorship.

Compensation & Benefits

Base salary: $170,000 to $200,000 USD annually. Visa sponsorship is not available.

Location

Remote within the United States or Canada. Primary location: New York, New York, United States.