Evaluate AI coding agents by assigning real software engineering tasks, running and reviewing their output, and identifying where their work fails. Work through real problems in familiar codebases alongside AI and provide feedback that helps improve frontier AI models.
The Strategic Account Executive will focus on acquiring new Fortune 500 and 100 enterprise clients through proactive outbound prospecting and strategic account planning. They will manage the entire sales cycle, from initial meetings and executive alignment to complex contract negotiations and deal closure.
You will define and improve benchmarking methodologies for frontier coding agents while partnering with top AI researchers and engineers. Your work involves creating high-signal feedback systems, developing evaluation environments, and contributing to open-source research products.
You will stress-test large language models by designing adversarial prompts to expose vulnerabilities such as bias, hallucinations, and broken guardrails. Additionally, you will document experiments, score model responses against harm taxonomies, and collaborate with researchers to strengthen AI defenses.
You will design, build, and publish coding benchmarks to measure and improve frontier coding agents. This involves creating realistic software-engineering tasks and developing reliable evaluation methodologies to diagnose agent capabilities and failure modes.
You will design adversarial prompts and multi-turn interaction chains to test AI models for vulnerabilities related to cyberattacks and malware generation. Additionally, you will evaluate model-generated technical output for functional correctness and document findings within a structured cybersecurity framework.
Design and execute adversarial prompts to test AI model safety against CBRNE threats while documenting failure points. Collaborate with researchers and policy teams to refine evaluation frameworks and improve model defenses.
