Delivery Manager - Multimodality LLM Engagement

Job Title: Delivery Manager – Multimodality LLM Engagement

Location: Remote (USA)

Position Overview:

As a Delivery Manager, you will lead the end-to-end execution of LLM training projects involving Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and Reinforcement Learning from Execution Feedback (RLEF). You will manage a cross-functional team of AI Trainers, Leads and Engineering Managers to ensure delivery of high-quality data and model improvements.

You will operate as the senior-most project owner, accountable for strategic client alignment, throughput, quality, and operational efficiency across multiple streams and modalities (Vision, Video, Audio/Speech, Computer Use Agents, etc).

This role requires hands-on delivery leadership, team scaling, stakeholder management, and a clear understanding of both technical and operational dimensions of LLM development.

Key Responsibilities:

1. Project Leadership & Cross-Functional Management

• Own full project lifecycle from kickoff and scoping to delivery and stabilization.
• Manage multiple concurrent LLM training streams (Evals, SFT, RLHF, RLEF, etc.,) across languages and domains.
• Lead and coordinate distributed remote teams including AI trainers, team leads and engineering managers.
• Maintain strong alignment with Engineering Managers, ensuring delivery of technically sound and review-compliant datasets.
• Monitor throughput, quality, rework, review coverage and staffing requirements.
  
  

2. Client Engagement & Operational Oversight

• Act as the strategic point of contact for clients - gathering requirements, aligning on expectations, and managing feedback loops.
• Build and maintain detailed project trackers, dashboards, and delivery health reports.
• Proactively flag risks and drive resolution to ensure uninterrupted, high-quality delivery.
• Set up scalable processes, SOPs, and review systems to mature project operations.
  
  

3. Quality Governance & Continuous Improvement

• Own delivery-level quality KPIs across all roles from trainers to engineers.
• Ensure clarity, accuracy, and completeness in outputs: code, responses, explanations, and evaluations.
• Work closely with team leads to implement quality review loops and resolve systemic quality gaps.
• Identify inefficiencies and continuously optimize workflows and operational structure.
• Ensure all output meets the highest standards expected by AI researchers and clients.

Required Qualifications:

• 4–8 years of experience in a Delivery Manager, Program Manager, or similar role within a technical or data-driven environment.
• Proven track record in managing end-to-end project lifecycles, scaling teams, and optimizing delivery pipelines.
• Strong experience in client-facing roles involving requirements gathering, delivery tracking, and stakeholder alignment.
• Experience managing diverse and distributed teams.
• Skilled in driving team performance, managing escalation workflows, and balancing speed, quality and cost.
• Working knowledge of multimedia modalities and the ability to interact with engineering and product managers to monitor and maintain output quality.
• Proficient in using project management and tracking tools (e.g., Airtable, Notion, JIRA, Asana, Google Sheets).

• Exceptional communication and documentation skills - comfortable leading async and live updates across technical and non-technical audiences.
• Strong decision-making, prioritization, and conflict-resolution abilities in dynamic, high-stakes environments.

Preferred/Bonus Qualifications:

• Background in Machine Learning or Data Science is a plus.
• Familiarity with LLM concepts, training cycles, or evaluation methods (e.g., RLHF, SFT, RAG).
• Hands-on experience with LLM APIs (GPT, Gemini, Claude, etc.) and RAG workflows.
• Hands-on experience with multimodality paradigms, including image generation, audio recording, speech generation, and/or content creation.

Ideal Candidate Profile:

• Brings a blend of technical fluency and operational leadership, enabling you to lead delivery in AI/LLM training projects without needing to code.
• Take initiative to unblock teams, address quality issues, and streamline workflows.
• A systems thinker, able to design processes and tracking mechanisms that scale with team growth and delivery complexity.
• Are client-obsessed, with strong stakeholder instincts and the ability to proactively manage expectations, risks, and feedback loops.
• Have a continuous improvement mindset, always looking for ways to evolve team structure, workflows, tooling, and delivery KPIs.
• An empathetic leader, able to manage global teams in async environments while building trust and accountability.