Review screen-recorded Adobe Premiere Pro workflows to ensure they meet professional standards and project guidelines. Provide clear, constructive written feedback to contributors and log review outcomes in the project tracker.
Research frontier AI systems and develop datasets, experiments, prototypes, tools, and evaluation frameworks across areas such as reinforcement learning, post-training, synthetic data, and AI agents. Analyze results rigorously and collaborate across teams to turn validated research into scalable products and share findings with the research community.
Lead end-to-end delivery of large-scale multimodal AI annotation projects, managing scope, resources, timelines, budgets, clients, and stakeholder communication. Lead and mentor Team Leads, partner with Quality Managers, monitor performance and quality metrics, address risks, and scale teams and processes as needed.
Lead QA teams across shifts and manage quality operations for multimodal AI annotation projects, including review priorities, sampling, audits, calibration, and performance tracking. Investigate quality issues, drive corrective actions, document guideline decisions, share delivery insights, and report results to the Quality Manager.
Lead annotators and Pod Leads in managing annotation batches from launch through delivery, setting targets, maintaining staffing, and monitoring productivity and quality. Coach and retrain team members, collaborate with QA and tooling teams to improve processes, develop future leaders, and report progress, risks, and blockers to the Project Lead.
The Quality Lead will manage review queues and oversee QA reviewers to ensure the accuracy and consistency of annotated data. They are responsible for providing actionable feedback, tracking performance metrics, and training new team members on guidelines and tools.
Plan and deploy resources across a large-scale AI training program, aligning contributor capacity with client demand and monitoring utilization, throughput, and operational risks. Maintain and improve resource tracking systems, lead process improvements, coordinate with stakeholders, and manage Supply Coordinators to support continuous operations.
Allocate and redeploy contributors across AI training projects, coordinating access, training, instructions, and resource requests. Maintain workforce and capacity data, monitor project throughput and onboarding, escalate risks or data issues, and communicate pending actions across shifts.
Design challenging, realistic agentic scientific workflows that require AI agents to independently navigate data, execute code, and troubleshoot complex tasks. Ensure all tasks are reproducible, self-contained, and objectively verifiable within a controlled computational environment.
The Student Project Ambassador is responsible for recruiting university students to participate in an AI video collection project. They will manage community outreach, track referral progress, and provide guidance to candidates throughout the application process.
The role involves analyzing complex datasets and research-based questions to improve the performance of large language models. Responsibilities include summarizing content, validating information accuracy, and creating training scenarios to enhance model responses.
Design challenging, agentic genomics tasks and reference solutions for AI model evaluation. Develop reproducible scientific workflows and robust grading criteria to assess AI performance on complex biological data analysis.
Review AI-generated medical text and images for dermatological accuracy, safety, and clinical completeness. Collaborate with researchers to align model performance with evidence-based dermatology standards.
The role involves reviewing AI-generated medical text and images for dermatological accuracy, safety, and completeness. You will provide expert feedback to improve the medical integrity of LLM outputs and collaborate with researchers to align model performance with clinical standards.
Create original research tasks that evaluate AI systems by synthesizing clues from reputable open web sources. Document definitive answers, verify all sources, and refine tasks based on model performance and reviewer feedback.
Review AI conversations to evaluate context gathering and provide structured feedback to improve conversational quality. Collaborate with research teams to ensure consistent assessments and identify missed opportunities for clarification.
Design and refine evaluation rubrics for preference data while building pipelines to support data generation and collection workflows. Collaborate with researchers to translate qualitative judgments into scalable processes and synthesize findings into actionable technical updates.
Create research tasks that evaluate AI systems by synthesizing clues from reputable open web sources. Document definitive answers, verify sources, and refine tasks to ensure appropriate difficulty for frontier models.
Record 4-5 screen captures demonstrating professional Adobe Premiere Pro editing workflows while narrating the process. Perform tasks such as cutting raw footage, color matching, audio cleanup, and captioning for social media platforms.
Record 4-5 screen captures of professional Adobe Lightroom Classic editing workflows. Provide brief narration while demonstrating tasks such as importing, masking, and exporting.
The recruiter will be responsible for sourcing, screening, and engaging a high volume of US-based candidates for remote and project-based roles. They will manage the full recruitment lifecycle, including interview coordination, onboarding, and maintaining accurate candidate records to meet hiring targets.
The recruiter will source and engage a high volume of Spanish-speaking candidates across Latin America for remote project-based roles. Responsibilities include screening applicants, managing candidate pipelines, and coordinating the full recruitment and onboarding process.
The Project Lead will manage day-to-day operations, including recruiting and onboarding contributors for an AI video data collection project. They are responsible for maintaining project trackers, ensuring quality requirements are met, and providing ongoing support to contributors in both English and Spanish.
Record 4-5 screen captures demonstrating real-world Salesforce workflows such as lead conversion, pipeline reporting, and data management. Provide clear audio narration while performing these tasks in your own Salesforce environment.
Record 4-5 screen captures demonstrating professional workflows in Sprout Social, including content scheduling, inbox management, and performance reporting. Provide clear audio narration while performing these tasks to create realistic professional workflow examples.
Record 4-5 screen captures demonstrating real-world Windows administration workflows such as user management, policy configuration, and security settings. Narrate the actions taken during these recordings to provide clear professional guidance.
Record 4-5 screen captures demonstrating real-world QuickBooks Online workflows while narrating the actions taken. Perform routine financial operations such as bank reconciliation, invoice management, and financial reporting.
Record 4-5 screen capture videos demonstrating real-world Siemens Desigo CC workflows with accompanying audio commentary. Perform typical building automation tasks such as alarm triage, trend log review, and equipment overrides.
Record 4-5 screen captures demonstrating professional Cloudflare workflows such as DNS management, firewall configuration, and Zero Trust policy setup. Provide clear audio narration to explain the technical actions being performed during these routine administration tasks.
Record 4-5 screen capture videos demonstrating real-world IBM Maximo workflows including work orders, asset registration, and inventory management. Provide brief audio commentary to explain the professional processes being performed.
Record 4-5 screen captures demonstrating professional workflows using AI meeting-note tools. Narrate your actions while organizing notes, creating templates, and retrieving information from previous meetings.
Design and validate complex engineering simulation tasks to train and evaluate AI agents. Analyze agent execution logs to identify systemic reasoning gaps and refine benchmark difficulty.
You will review and annotate ITSM workflows to ensure they are realistic for training AI agents in incident resolution and service desk procedures. This involves validating triage, escalation, and closure steps against enterprise standards to create high-fidelity simulation data.
The role involves designing multi-turn conversational prompts to evaluate the quality, grounding, and integration of personalized AI responses. Analysts must perform side-by-side comparisons of model outputs and provide detailed, defensible rationales for their rankings.
Design and implement realistic multi-step scenarios for AI agents to execute on a Linux desktop environment. Write Python scripts to set up these environments and develop automated scoring mechanisms to evaluate agent performance.
The role involves creating, editing, and evaluating audio-related tasks for AI models to ensure technical accuracy. You will also provide detailed feedback and documentation on AI-generated audio outputs while maintaining consistency across tasks.
Design and implement realistic multi-step scenarios for AI agents to perform tasks on a Linux desktop. Write Python scripts to set up environments and automatically score the agent's performance.
The role involves recording short, scripted conversations in Bengali that simulate natural sales and customer service interactions. These recordings are intended to support the development and improvement of voice-based technology.
Record short, natural scripted conversations and dialogues for sales and customer service scenarios. Ensure clear, consistent audio quality and re-record lines based on feedback to support voice technology development.
The role involves recording short, scripted conversations in Marathi to simulate sales and customer service interactions. These recordings are intended to support the development and improvement of voice-based technology.
Record short, natural scripted conversations for sales and customer service scenarios to improve voice-based technology. Follow provided scripts and tone guidelines while ensuring clear and consistent audio delivery.
The voice actor will record short, natural scripted dialogues simulating sales and customer service interactions. These recordings are intended to assist in the development and improvement of voice-based technology.
The role involves recording short, natural scripted conversations for sales and customer service scenarios to improve voice-based technology. You will be responsible for following scripts, maintaining a consistent tone, and re-recording lines based on quality feedback.
Record short, scripted conversations in a natural, conversational style for sales and customer service scenarios. Ensure clear, consistent audio delivery and perform re-recordings based on quality feedback.
The role involves recording short, scripted conversations in Odia to simulate sales and customer service interactions. These recordings will be used to improve voice-based technology.
The role involves recording short, scripted conversations in Assamese to simulate sales and customer service interactions. You will be responsible for delivering clear, natural-sounding audio files while following specific tone and style guidelines.
The role involves recording short, scripted conversations in Malayalam that simulate sales and customer service interactions. These recordings are intended to help develop and improve voice-based technology.
The voice actor will record short, scripted conversations in Punjabi for sales and customer service scenarios. These recordings are intended to assist in the development and improvement of voice-based technology.
The role involves recording short, scripted conversations in Hindi that simulate natural sales and customer service interactions. You will be responsible for delivering clear, consistent audio files while following specific tone and style guidelines.
Review and validate ITSM workflows for realism and fidelity to ensure they effectively train AI agents. Annotate incorrect workflows, suggest improvements, and define standard outcomes for common IT service desk incidents.
Participants are responsible for exporting financial data from at least seven personal accounts and creating two financial artifacts. They must also use a provided tool to redact personally identifiable information before submitting their data for quality review.
Participants are required to export and submit financial data from various personal apps and services while ensuring all personal identifiers are redacted. They must also provide self-created financial artifacts and ensure all submissions meet quality and recency requirements.
Participants will export and submit structured financial data from various personal apps and services while ensuring all personal identifiers are redacted. They must also provide self-created financial artifacts and adhere to strict quality and timeline requirements.
Participants are responsible for exporting financial data from various personal apps and services across five mandatory categories. They must also redact personally identifiable information using a provided tool and submit self-created financial artifacts.
Participants will export and submit structured financial data from multiple personal apps and services while ensuring all PII is redacted. They must also provide self-created financial artifacts and ensure all submissions meet quality and recency requirements.
The project manager will export financial data from various personal apps and services across five mandatory categories. They are also responsible for redacting personally identifiable information and submitting self-created financial artifacts according to strict quality requirements.
Participants will export and submit financial data from various personal apps and services while ensuring all personal identifiers are redacted. They must also provide self-created financial artifacts and follow strict quality and timeline requirements.
Participants will export historical financial data from various personal apps and services while ensuring all personally identifiable information is redacted using a provided tool. They must submit these redacted files along with self-created financial artifacts for quality review and potential rework.
Contributors will export financial data from their personal accounts across multiple categories and use a provided tool to redact personally identifiable information. They must submit these cleaned files along with self-created financial artifacts for quality review.
Document a recurring professional sales task in detail to help train AI systems. This involves outlining specific thresholds, scoring criteria, and workflows used in your daily sales operations.
Record high-quality Hindi speech samples based on provided scripts while maintaining accurate pronunciation and natural fluency. You will be required to express various emotions as instructed and submit recordings within project timelines.
The role involves documenting a recurring professional task from your own work in detail to help train AI systems. You will provide structured documentation covering thresholds, scoring criteria, and escalation paths for your specific sales workflow.
The recruiter will document a recurring, long-term professional task from their own experience in structured detail. This involves outlining triggers, decision rules, tools, and edge cases to help train AI systems.
Document a recurring professional task from your experience in structured detail to help train AI systems. You will provide specific insights into decision rules, thresholds, and workflows used in your practice.
Document a recurring professional travel task in a structured online form with high detail for AI training purposes. Provide clear instructions on thresholds, escalation paths, and tool usage to enable others to replicate the process.
Document a recurring professional task in extreme detail to assist in training AI systems. This involves outlining specific thresholds, escalation paths, and tool access requirements for complex workstreams.
Review and redline commercial contracts on a legal AI platform using track changes and detailed commentary. Calibrate evaluations against golden references and participate in team communications via Slack.
Participants are responsible for exporting their own historical data from at least five eligible applications using native export features. They must then upload these ZIP files through a secure submission form after ensuring PII is masked.
Review and assess AI training problems and research environments for a frontier AI lab. Ensure problems are scientifically realistic, well-designed, and correctly graded without solving them personally.
Develop and validate CAD models and simulation workflows for aerospace, mechanical, and electrical components. Optimize designs based on simulation results and collaborate with cross-functional engineering teams.
Evaluate Gemini's personalization features by designing multi-turn prompts and assessing model responses for grounding, integration, and helpfulness. Perform side-by-side rankings of model outputs and provide detailed rationales for the evaluations.
Evaluate the quality and personalization of AI-generated responses using a personal Gmail account. Provide detailed, objective feedback on the relevance and accuracy of AI interactions based on email history.
Evaluate the quality, relevance, and personalization of AI-generated responses using a personal Gmail account. Provide detailed objective feedback and compare multiple AI responses to improve user experience.
Evaluate the quality and personalization of AI-generated responses based on professional business email history. Provide detailed feedback and compare multiple AI outputs to improve model performance and user experience.
Evaluate the quality and personalization of AI-generated responses based on professional email history. Provide detailed feedback to improve model accuracy and user experience while adhering to privacy guidelines.
Evaluate the quality and personalization of AI-generated responses based on professional business email history. Provide detailed feedback to improve model accuracy, relevance, and user experience.
Evaluate the quality and personalization of AI-generated responses based on professional business email history. Provide detailed feedback and compare multiple AI outputs to improve model performance and user experience.
Record high-quality voice-overs for characters and stories across various domains using a professional home or in-studio setup. Interpret scripts and revise performances based on creative direction and feedback.
Record high-quality voice-overs and perform scripts with appropriate emotion, personality, and pacing. Revise performances based on creative direction and deliver audio files in the correct format.
Record high-quality voice-overs for characters and stories across various domains using a professional home studio. Interpret scripts and revise performances based on creative direction and feedback.
Record high-quality voice-overs for characters and stories across various domains using a professional studio. Interpret scripts and revise performances based on creative direction and feedback.
Record high-quality voice-overs for characters and stories across various domains using a professional home studio. Interpret scripts and revise performances based on creative direction and feedback.
Record high-quality voice-overs and perform scripts with appropriate emotion and pacing. Interpret creative direction and revise performances based on feedback for AI projects.
Record high-quality voice-overs for characters and stories across various domains using a professional home studio. Interpret scripts and revise performances based on creative direction and feedback.
Record high-quality voice-overs and perform scripts with appropriate emotion and pacing. Revise performances based on feedback and deliver audio files in the correct format.
Perform high-quality voice-overs for characters and stories across various domains using a professional home studio. Interpret scripts and revise performances based on creative direction and feedback.
The specialist will analyze and annotate logistics documentation to help train AI systems in supply chain reasoning. They will also evaluate workflows and provide detailed feedback on process optimization and industry compliance.
The engineer will translate style guides and screenshots into pixel-perfect, responsive web interfaces. Additionally, they will provide domain expertise to help train and develop advanced AI models.
The role involves reviewing, labeling, and validating data to support the development and improvement of AI models. Candidates must ensure accuracy and consistency across datasets while identifying and escalating edge cases.
Create and narrate short 'walk-and-talk' videos of public locations to help train AI systems. Provide conversational commentary and submit footage with accurate location details following creative guidelines.
Create short 'walk-and-talk' videos of public locations to help train AI systems in understanding the world. Provide natural, conversational commentary while recording steady footage and submitting accurate location data.
The specialist will research tax form layouts and use Label Studio to draw precise bounding boxes and define field relationships. They are responsible for exporting and validating JSON/YAML annotations to ensure accurate data overlays on source PDFs.
Analyze financial data to provide accounting guidance and advisory-based recommendations. Handle complex bookkeeping and finance-related queries for SMB clients.
Evaluate Gemini's personalization features by designing multi-turn prompts based on personal experiences. Analyze model responses for grounding, integration, and helpfulness while providing detailed rationales for rankings.
Evaluate and improve Gemini's personalization features by designing multi-turn prompts based on personal experiences. Analyze model responses for grounding, integration, and helpfulness while providing detailed rationales for rankings.
Design original, competition-level mathematics problems to evaluate and stress-test large language models. Review model-generated solutions to identify logical fallacies and diagnose reasoning failures.
Provide omnichannel chat support to SMB owners by answering general and complex accounting queries. Analyze financial data and provide bookkeeping guidance to ensure accurate responses.
The role involves applying expert software engineering knowledge to train next-generation AI systems through bug fixing, feature implementation, and codebase optimization. You will collaborate with multidisciplinary teams to provide technical insights and document best practices for scalable AI training workflows.
The role involves training and evaluating AI systems by solving real-world software engineering tasks across backend, full-stack, and infrastructure projects. Responsibilities include reviewing code, designing practical technical solutions, and documenting technical reasoning and tradeoffs.
Design and implement robust checkers for competitive programming problems to validate diverse outputs and edge cases. Collaborate with teams to fine-tune problem requirements and provide feedback to problem setters.
Evaluate Gemini's personalization features by designing multi-turn prompts based on personal experiences. Analyze model responses for grounding, integration, and helpfulness while providing detailed rationales for rankings.
Evaluate Gemini's personalization features by designing multi-turn prompts based on personal experiences. Analyze model responses for grounding, integration, and helpfulness while providing detailed rationales for rankings.
Evaluate Gemini's personalization features by designing multi-turn prompts based on personal experiences. Analyze model responses for grounding, integration, and helpfulness while providing detailed rationales for rankings.
Create stylized illustrations and replicate diverse artistic styles based on project briefs. Ensure high-quality output in lighting and texture while documenting style choices and collaborating with teams.
Review audio files using an online tool to rate their technical quality. Consistently apply guidelines to ensure accurate data submission across large batches.
Evaluate Gemini AI personalization features by designing multi-turn prompts and assessing responses for quality, relevance, and grounding. Identify hallucinations and provide clear rationales for scoring decisions to improve AI output.
Evaluate AI-generated and real-world images and videos for quality, realism, and composition using structured rating frameworks. Provide detailed written justifications for evaluation decisions to help improve cutting-edge AI systems.
The role involves designing multi-turn prompts to stress-test Gemini's personalization features and evaluating AI responses for accuracy, grounding, and natural integration. The analyst will perform side-by-side rankings of model outputs and provide detailed rationales for their quality assessments.
Review and annotate Indonesian language content, including articles and advertisements, to train AI systems. Tasks include drawing bounding boxes, answering structured questions, and writing short summaries.
Train and improve AI systems by reviewing AI-generated financial analyses and providing structured feedback. Work on real-world finance use cases including investment memos, valuation, and accounting workflows.
Create and evaluate advanced Physics problems to enhance AI reasoning and evaluation systems. Collaborate with LLM researchers to develop benchmarks and analyze model struggles in symbolic reasoning.
Design and solve complex mathematical problems to test and fine-tune large language models. Create high-quality, step-by-step solutions and collaborate with researchers to define new evaluation benchmarks.
Solve advanced postgraduate-level chemistry problems and create detailed textual and visual explanations to fine-tune large language models. Analyze complex reaction mechanisms and chemical equilibrium using a variety of diagrams and simulations.
Design and solve complex biology problems to evaluate and improve AI model performance. Create detailed step-by-step scientific solutions and contribute to the development of evaluation benchmarks.
Review and annotate Korean language content by drawing bounding boxes and answering structured questions. Write short summaries of content to help train AI systems.
Review and annotate Danish language content, including articles and advertisements, to train AI systems. Tasks include drawing bounding boxes, answering structured questions, and writing short summaries.
Review and annotate Norwegian text and image content to help train AI systems. Tasks include drawing bounding boxes, answering structured questions, and writing short summaries.
Review and annotate Turkish language content, including articles and advertisements, to train AI systems. Tasks include drawing bounding boxes, answering structured questions, and writing short summaries.
You will develop and maintain Model Context Protocol (MCP) tools to connect AI models with complex database environments. Additionally, you will refactor legacy code, expand test coverage for agentic coding assistants, and create technical documentation to support community adoption.
You will design and execute multi-turn conversational prompts to evaluate the personalization quality of AI models. Additionally, you will perform side-by-side comparisons of model responses and provide detailed rationales for your rankings.
Record first-person household task videos using a head-mounted device to generate training data for robotics. Ensure consistent weekly output by following strict capture protocols and managing file uploads.
Review images in an online tool to rate their technical quality based on specific guidelines. Identify and document image defects such as blur, noise, poor exposure, and compression artifacts.
Review images in an online tool to rate their technical quality based on specific guidelines. Identify and document image defects such as blur, noise, poor exposure, and compression artifacts.
