Design challenging tasks focused on reaction mechanisms, synthesis routes, and analytical interpretation to evaluate AI agents. Develop reference solutions while ensuring chemical correctness, feasibility, and adherence to lab safety standards.
Design and develop audit tasks and reference solutions to evaluate AI agents on their ability to perform complex financial audit workflows. Assess agent performance in inspecting documents, testing transactions, and identifying unsupported conclusions or procedural errors.
You will create realistic tasks and reference solutions to assess AI agents on financial reporting workflows, including statement preparation and consolidation. Additionally, you will evaluate agent performance in mapping trial balances, drafting disclosures, and identifying errors in financial reporting.
Design and build financial tasks and reference models to evaluate AI agents on business performance analysis. Identify formula errors, validate assumptions, and define criteria for model integrity and analytical depth.
Create realistic accounting tasks and reference solutions to evaluate AI agents performing client accounting workflows. Define scoring criteria for technical accuracy and assess the quality of agent-generated client deliverables.
The accountant will create realistic tasks and reference solutions to assess AI agents' performance in accounting workflows. They will evaluate agent accuracy in classifying transactions, reconciling accounts, and preparing financial reports.
Design and develop challenging tax-related tasks and reference solutions to evaluate AI agents' performance. Assess agent accuracy in tax research, calculation, compliance workflows, and the identification of errors or outdated information.
You will design challenging tasks involving observational astronomy and astrophysics to test AI agents' scientific reasoning. Additionally, you will develop reference solutions and evaluate agent performance using astronomical data and simulation tools.
The Admissions Advisor will manage the full enrollment pipeline, from qualifying inbound leads to conducting consultative interviews and closing enrollments. They will also collaborate with academic and marketing teams to improve processes and ensure candidates are well-prepared for the fellowship.
You will manage a regional portfolio of fellows to recover outstanding payments by reviewing contracts and conducting professional negotiations. This role involves documenting all collection activities in the CRM and coordinating with finance and legal teams to resolve complex cases.
Design challenging tasks and reference solutions involving observational astronomy, astrophysics, and cosmology to test AI agents. Evaluate AI performance using astronomical data, simulation tools, and rigorous scoring criteria for numerical and physical accuracy.
Design challenging tasks and reference solutions to test AI agents in biological research workflows. Evaluate agent performance using scientific tools, databases, and literature while identifying methodological weaknesses.
Design and develop realistic tasks to assess AI agents' ability to investigate ecological and evolutionary questions. Evaluate agent performance using scientific datasets, analytical methods, and evidence-based criteria.
Design and develop tasks that test AI agents' ability to reason through complex neuroscience research and analysis. Evaluate agent performance in interpreting neural data, scientific literature, and experimental methodologies.
Design realistic marketing tasks and evaluation criteria to challenge and assess the performance of AI agents. Review AI-generated outputs to identify errors, gaps in execution, and strategic weaknesses.
The designer will create high-quality visual assets for social media, websites, and marketing campaigns while maintaining brand consistency. They will collaborate with cross-functional teams to translate concepts into engaging digital content and UI components.
Own the AI fellowship curriculum and learning design, ensuring its materials, projects, and teaching remain rigorous, high-quality, and aligned with current industry practices and hiring demand. Build and calibrate the instructor network, assess learner outcomes and delivery feedback, and partner with program operations to protect the learning experience.
You will review and evaluate real-world software engineering tasks derived from open-source repositories to ensure they are technically sound and reproducible. This involves assessing bug fixes, feature implementations, and unit tests while providing clear feedback on task quality and complexity.
You will be responsible for reviewing CVE reproduction environments for technical accuracy and evaluating the effectiveness of proposed security fixes. Additionally, you will verify that security patches address root causes without introducing regressions or breaking application functionality.
Interact with various AI models to compare and evaluate the quality, accuracy, and helpfulness of their responses. Provide clear, well-reasoned written explanations to support evaluation decisions based on detailed rating guidelines.
Interact with various AI models to compare and evaluate the quality, accuracy, and helpfulness of their responses. Provide clear, well-reasoned written explanations to support evaluation decisions based on detailed guidelines.
Interact with and compare AI-generated responses to assess accuracy, helpfulness, and instruction-following. Write clear, well-reasoned explanations to support evaluation decisions based on detailed guidelines.
You will interact with AI models to compare responses side by side and assess them based on accuracy, helpfulness, and instruction-following. Your evaluations and written explanations will directly contribute to improving the reliability and quality of frontier AI systems.
Interact with AI models to compare and evaluate their responses based on accuracy, helpfulness, and instruction-following. Provide clear, well-reasoned written explanations to support evaluation decisions and improve AI system reliability.
Interact with AI models to compare and evaluate their responses based on accuracy, helpfulness, and instruction-following. Write clear, well-reasoned explanations to support evaluation decisions and help improve AI system performance.
Interact with various AI models to compare and evaluate the quality, accuracy, and helpfulness of their responses. Write clear, well-reasoned explanations to support evaluation decisions based on detailed rating guidelines.
Evaluate AI assistant performance by conducting natural, multi-turn conversations and comparing responses based on context and helpfulness. Provide clear, detailed feedback and explanations to support your evaluations of the AI's accuracy and safety.
Conduct live, real-time roleplay scenarios with conversational AI agents and capture technical logs. Label conversation transcripts and evaluate interactions against structured rubrics to improve AI performance.
Evaluate AI-generated technical content and customer service responses for accuracy, clarity, and quality. Validate and troubleshoot JSON structures while providing reasoned feedback on model outputs.
Interact with various AI models to compare and evaluate the quality, accuracy, and helpfulness of their responses. Write clear, well-reasoned explanations to support evaluation decisions based on detailed rating guidelines.
