Principal Researcher, Benchmarks, Evaluation, and Standards

ROLE SUMMARY 

The Partnership on AI is seeking a Principal Researcher to lead the technical backbone of PAIs’ work, with emphasis on benchmarking, evaluation, and technical standards that strengthen PAI's impact. This person will report to the Chief Programs and Insights Officer and work closely with colleagues leading PAI’s focus areas: AI, Human Connection, and Well-Being; Trustworthy AI Value Chains; and AI & Shared Prosperity

PAI's approach is sociotechnical by design: we don't treat "technical" and "social" as separate tracks, but as a single problem to be solved coherently. This role will direct how PAI orients its work around that philosophy. The Principal Researcher will bring deep technical fluency to questions that are also social: for instance, how multi-turn safety degradation in chatbot design affects psychological well-being, how agent identification and authentication systems can support authentic interaction without compromising privacy, and how societally agreed norms actually get built into model specs and evaluation practice. The job is to make sure PAI's guidance across the AI value chain--model, product, and policy layers, for example--is principled in the public interest and grounded by stakeholders, technically implementable and measurable, and transferred to the appropriate body responsible for benchmarks, evaluations, and standards.

WHAT YOU’LL DO

Strategic Planning & Vision

• Develop PAI's point of view on the evaluation and benchmarking ecosystem, including assessing the infrastructure required for PAI to successfully develop and run its own evaluations and directing PAI’s strategy for integrating into the evaluation ecosystem, in collaboration with supervisor and executive team
• Advance the adoption of shared industry guidance through development of and integration into benchmarks, evaluations, and technical standards, in collaboration and partnership with relevant organizations in the auditing and certification space
• Offer technical leadership and shape the strategic direction of PAI’s focus areas (AI, Human Connection, and Well-Being, AI and Shared Prosperity, and Trustworthy AI Value Chains) in collaboration with those leading those focus areas and with the Chief Programs and Insights Officer.
• Support technical efforts from other PAI teams, including Policy and International
• Work with PAI’s Partnerships team to structure work to support fundraising efforts
• Consider which questions across PAI can be addressed most effectively and urgently with sociotechnical solutions that PAI is well placed to develop, and design and build implementation pathways.

Research & Subject Matter Expertise 

• Work closely across PAI projects in order to ensure that guidance is technically feasible and implementable, e.g., as evaluations or standards, and supporting testing and sandboxing exercises related to PAI guidance where relevant
• Design approach for maintaining PAI’s benchmarking, evaluation, and standards work over time, including quality control for established partnerships
• Author/co-author peer-reviewed research papers, policy briefs, and accessible reports that translate PAI’s insights into actionable recommendations for industry, policymakers, and civil society, with an emphasis on bridging technical and social considerations. 

Stakeholder & Relationship Management 

• Initiate and lead collaborative research partnerships with academic institutions, civil society organizations, and industry Partners to advance programmatic work.
• Represent PAI at select conferences (e.g., NeurIPS) and other venues, presenting findings and methodological innovations.
• Run point on key relationships with benchmarking, technical standards, and evaluation-oriented organizations and settings, including AISIs and assurance bodies
• Collaborate with Partnerships team to test approaches to earned-revenue models, including but not limited to licensing or content contributions to credentialing and training programs

Strategic Communication and Impact

• Translate rigorous findings into clear, compelling narratives for diverse audiences, including industry and civil society leaders, policymakers, and affected communities.
• Support strategic communications efforts to amplify and increase the adoption of PAI's guidance

• Represent work to standards bodies, policymakers, media, funders, and the public, and connect open research questions to the resourcing that can answer them.

REQUIRED QUALIFICATIONS

• Eight or more years of relevant technical experience across some combination of AI, human-computer interaction, trust and safety, or another relevant interdisciplinary field.
• Fluency in both technical and sociotechnical domains, with priority given to those with advanced degrees in a technical discipline like computer science.
• Strong, accessible writing.  A large share of this role's output is documents that other organizations will adopt or reject on the strength of how they are written. Your ability to write in a compelling way for different audiences is key to the success of our goal. 
• Experience translating guidance into evaluations, benchmarks, standards, audit criteria, or informing certifications and professional training
• Demonstrated ability to interface with and convene stakeholders with genuinely conflicting interests and varied disciplinary training, and to produce something from that convening that the cohort can get behind. 
• Experience representing and speaking on behalf of an organization at events featuring executives and leaders from across domains and experiences.
• Demonstrated capacity to set your own objectives and drive multistakeholder strategy with a high degree of autonomy

PREFERRED QUALIFICATIONS

• PhD in computer science, AI, or a comparably technical discipline. 
• Experience actively participating in and working with implementation partners on standards development and implementation
• Professional emphasis on the social impact of technology and experience working to apply their technical skillset to social questions. 
• Familiarity and experience with how AI systems are actually built and evaluated, including model behavior specification, red-teaming, and automated evaluation.
• Experience working on questions related to psychological well-being and/or content authenticity.
• Comfort with domestic and international travel, at times substantial.
• Comfort working in a small, evolving cross-disciplinary organization, including contributing to priorities beyond your immediate focus.

QUALITIES THAT ARE IMPORTANT TO US:

• Justice: Notices and addresses inequities, includes diverse perspectives, and acts fairly in decisions and collaboration.
• Trust: Communicates openly, follows through on commitments, and demonstrates reliability and integrity.
• Respect: Listens fully, values others’ time and contributions, and collaborates thoughtfully.
• Compassion: Shows empathy, patience, and care, especially when supporting colleagues or navigating challenges.

ADDITIONAL INFORMATION

We encourage candidates from all backgrounds to apply, even if you don't meet every qualification listed. What matters most is your commitment to PAI’s mission and the potential you bring. We are interested in building a diverse pool of candidates and welcome applications from people of all backgrounds, identities, and experiences.

Compensation 

PAI's compensation philosophy reflects our commitment to fairness and equity across our distributed team. Compensation is based on role level, experience, skills, geographic location, and internal equity.

Benefits 

• PAI offers a generous paid leave and benefits package, currently including:
• • Twenty vacation days, three personal reflection days, and sick and family leave above industry standards
  • High-quality PPO and HMO health insurance plans, many 100% covered by PAI
  • Dental and vision insurance, 100% covered by PAI
  • Up to a 7% 401(k) match, vested immediately
  • Automatic cell phone reimbursement ($75/month)
  • Annual funds to support professional development
  • $150 per month for co-working space access
  • Regular team lunches and focused work days
  • Opportunities to attend AI-related conferences and to collaborate with our partners across industry, academia, and civil society

Please refer to our careers page for an updated list of benefits.

Location and Work Authorization

• PAI is a remote-first organization with a global membership base and scope. This role is eligible for remote work in the United States, Canada, London (UK), or Brussels (Belgium).
• Candidates must be authorized to work in the country where they will be based. We are unable to sponsor visas at this time.

PAI is proud to be an equal opportunity employer.  We celebrate diversity and we are committed to creating an inclusive environment in all aspects of employment, including recruiting, hiring, promoting, training, professional development, social and recreational programs, compensation, benefits, transfers, discipline, and all privileges and conditions of employment.  Employment decisions at PAI are based on business needs, job requirements, and individual qualifications.

PAI will consider for employment qualified applicants with criminal histories, in a manner consistent with applicable federal, state, and local fair chance hiring laws.

The Partnership on AI may become subject to certain governmental record keeping and reporting requirements for the administration of civil rights laws and regulations. We also track diversity in our workforce for the purpose of improving over time. In order to comply with these goals, the Partnership on AI invites employees to voluntarily self-identify their gender and race/ethnicity. Submission of this information is voluntary and refusal to provide it will not jeopardize or adversely affect employment or any consideration you may receive for employment or advancement. The information obtained will be kept confidential.

HOW TO APPLY

Applications will be reviewed on a rolling basis until the position is filled. To apply, please submit a cover letter and resume.

Only complete applications will be reviewed.

In your cover letter, we would welcome your view on one question the field has not resolved: how can normative guidance be better integrated with technical approaches to support impact and adoption by AI developers and deployers? Please keep your cover letter/expression to no more than 750 words. We seek to understand your orientation towards the issue as well as how you express your thinking, in your own voice.

The compensation range for this role depends on where you are based:

United States: $133,900 – $197,800 USD
Canada: C$170,200 – C$204,000 CAD
London, UK: £109,100 – £130,800 GBP
Brussels, Belgium: €135,600 – €162,600 EUR

In Belgium, the required 13th-month pay and double holiday pay are paid on top of this base salary range.

PAI uses a geographic compensation structure with three U.S. pay zones. The posted US range reflects compensation across all three zones, and an individual candidate’s salary range will depend on their geographic location. Final compensation is determined based on role level, experience, skills, and internal equity considerations.