Evaluate AI-generated content for safety, reliability, and quality using structured rubrics. Provide high-quality human feedback and identify model failure modes to improve frontier AI systems.
mpathic
20 Remote Job Openings at mpathic
Evaluate and improve AI models supporting clinical operations and administrative workflows to ensure accuracy and practical application. Review real-world healthcare scenarios and provide expert feedback on AI-generated responses regarding EHRs and regulatory standards.
Design and evaluate clinically realistic tasks to benchmark multimodal AI systems in the radiology domain. Provide expert reference answers and detailed feedback to improve AI diagnostic reasoning and instruction-following.
Design expert-level cybersecurity prompts and evaluate AI-generated responses for technical accuracy and safety. Provide actionable feedback to improve the reliability and security of frontier AI systems.
Evaluate AI-generated prior authorization recommendations and specialty medication requests for accuracy and alignment with payer requirements. Provide actionable feedback to improve AI performance on complex pharmacy authorization workflows.
Design realistic clinical scenarios and evaluate AI-generated responses to ensure clinical accuracy and safety. Provide actionable feedback to improve the reliability and trustworthiness of healthcare AI models.
Design clinically realistic psychiatric scenarios and prompts to test and improve the reasoning of large language models. Evaluate AI-generated responses for accuracy and safety while providing structured feedback to researchers.
Lead the end-to-end execution of AI safety, evaluation, and red teaming programs for leading AI companies. Act as the primary operational link between data services, engineering, and customer-facing teams to ensure high-quality delivery.
Responsibilities include designing and executing red-teaming scenarios across diverse user behaviors, reviewing AI responses for safety and policy compliance, and identifying failure modes and behavioral risks in sensitive interactions. The role also involves documenting edge cases, contributing to evaluation rubrics, and participating in exercises to surface system vulnerabilities.
The specialist will review AI-generated customer support responses for quality, accuracy, tone, and safety, identifying issues like incorrect information or poor customer experience. Responsibilities include evaluating AI handling of customer intent, frustration, and escalation, while documenting failure patterns and risks.
Responsibilities focus on reviewing AI-generated financial content and conversations to identify risks such as misleading guidance, overconfidence, and inappropriate agreement, while also participating in adversarial probing exercises to surface failure modes.
This is a general application where candidates are asked to submit their resume and specify their areas of expertise for future consideration. The Talent Acquisition team will review submissions and reach out directly if a suitable opening arises.
Responsibilities include designing and testing AI chat scenarios, conducting qualitative conversation analysis, developing behavioral taxonomies, and providing expert feedback on internal mental health policy. The expert will also collaborate with engineers on safety metrics and support testing through structured scenarios.
Responsibilities include roleplaying clinical scenarios with AI agents, red teaming, identifying behavioral edge cases, and supporting accurate identification and response to psychological distress. This role may also involve developing psychometrics, rubrics, behavioral taxonomies, and evaluations.
Responsibilities include designing and testing AI chat scenarios, conducting qualitative conversation analysis, developing behavioral taxonomies, and providing expert feedback on internal mental health policy. The expert will also collaborate with engineers on safety metrics and support testing and fine-tuning through structured scenarios.
Mental Health Expert, Part-Time (Mexico, Spanish Speaking)
mpathic
·
Full Time
·
5 months ago
mpathic
Responsibilities include designing and testing AI chat scenarios, conducting qualitative conversation analysis, and developing behavioral taxonomies and evaluation rubrics for internal mental health policy. The expert will also collaborate with engineers on safety metrics, document edge cases, and support model fine-tuning through structured scenarios.
Mental Health Expert, Part-Time (Brazil, Portuguese Speaking)
mpathic
·
Full Time
·
5 months ago
mpathic
Responsibilities include designing and testing AI chat scenarios, conducting qualitative conversation analysis, developing behavioral taxonomies, and providing expert feedback on internal mental health policy. The role also involves collaborating with engineers on safety metrics and documenting edge cases and model failures.
Responsibilities include designing and testing AI chat scenarios, conducting qualitative conversation analysis, developing behavioral taxonomies, and providing expert feedback on internal mental health policy. The role also involves collaborating with engineers on safety metrics and supporting model fine-tuning through structured scenarios.
Responsibilities involve supporting confidential initiatives focused on AI safety, health information quality, and medical risk mitigation by roleplaying medical scenarios and evaluating AI-generated health information for accuracy and safety. This includes red teaming for edge cases, identifying patient distress, rating model responses, and developing evaluation frameworks.
Responsibilities involve supporting confidential initiatives focused on AI safety for children and adolescents, which includes roleplaying youth-facing scenarios and red teaming for safety edge cases. Experts will also evaluate AI responses for age-appropriateness, tone, and harm prevention, and contribute to policy development.
