Technical Solution Management Specialist - AI Evaluation, Research, and Technical Architecture Func.

Strategic Planning & Vision

• Define and communicate a future-state vision for AI evaluation, research quality, and technical architecture across the XIO AI Innovation Lab.
• Develop and maintain technical blueprints for research, validation, evaluation, experimentation, and Test Kitchen-ready handoff capabilities.
• Define and maintain related processes and playbooks for scaling pre 0-to-1 motions using the lab flow: Signal -> Research -> Validation -> Research Paper -> PoC Candidate -> Test Kitchen.
• Influence technical roadmaps for consistent product evaluation frameworks and tooling.
• Track best practices, Microsoft product updates, frontier AI research, and emerging AI trends to inform lab strategy and publication direction.
• Ensure alignment of evaluation frameworks with business priorities and Responsible AI standards.

Technical Roadmapping & Execution

• Build and scale multi-dimensional evaluation approaches for diverse lab-stage AI workloads, including chatbots, customer self-service agents, predictive models, and generative tools.
• Develop automated testing frameworks, including synthetic scenario generation, adversarial stress-testing, and batch evaluation.
• Implement lab-appropriate experimentation patterns, including offline evaluation, synthetic scenario testing, feature-flagged trials where appropriate, and monitoring plans for downstream PoC owners.
• Partner with engineering, data science, and prototyping teams to integrate evaluation tools into prototype scaffolds, CI/CD pipelines where relevant, and Test Kitchen handoff plans.
• Develop reference architectures, data needs assessments, secure data pipeline considerations, hosting-model options, and reusable solution patterns for candidate AI solutions.
• Package validated research into PoC candidate artifacts with value proposition, evidence, architecture, evaluation plan, governance considerations, and owner model.

Cross-Functional Program Management

• Coordinate across business, technical, and governance teams to ensure successful delivery of evaluation programs.
• Help establish and operate the XIO AI Innovation Lab as a lightweight research and validation capability, acting as a consultant and trainer for project teams.
• Drive organizational adoption of best practices, shared infrastructure, common metrics, research publication standards, and Test Kitchen handoff discipline.
• Support focused leadership alignment, research reviews, Demo Showcase updates, and handoffs to prototyping or product teams without slowing early-stage research autonomy.

Customer Experience & UX Research

• Apply UX research principles to understand lived experiences and translate insights into actionable improvements for AI solutions.
• Collaborate with domain experts, support engineers, CSU/account managers/CSAMs, and business owners to define success criteria and evaluation rubrics.
• Lead direct customer and field-team engagements and collect performance metrics to inform continuous improvement and readiness for PG engagement.

Governance & Responsible AI

• Integrate Responsible AI evaluations, safety, ethics, and compliance checks into all testing and experimentation processes.
• Manage gate reviews, documentation, risk findings, and RAI oversight appropriate to lab-stage research and Test Kitchen handoff readiness.
• Foster a culture of "fail fast, learn fast" with accountability and continuous improvement.

Required/Minimum Qualifications  

• Bachelor's Degree AND 4+ years experience in technical program management, preferably in AI, machine learning, research, data science, technical architecture, or customer experience domains.

  • OR equivalent experience.

• 2+ years of experience managing cross-functional and/or cross-team projects.  

Additional or Preferred Qualifications  

• Bachelor's Degree AND 8+ years experience in technical program management, preferably in AI, machine learning, research, data science, technical architecture, or customer experience domains
  • OR product development
  • OR equivalent experience.
• 6+ years of experience managing cross-functional and/or cross-team projects.
• 1+ year(s) of experience reading and/or writing code (e.g., sample documentation, product demos).
• Proven track record of AI evaluation, testing methodologies, and performance metrics.
• Experience with automated testing pipelines, synthetic data generation, and A/B experimentation.
• Familiarity with Responsible AI frameworks, governance, and compliance standards.
• Demonstrated ability to produce research papers, decision-grade technical artifacts, reference architectures, or reusable solution patterns.
• Ability to adapt strategy based on emerging technology trends, frontier AI research, customer signals, and industry research.
• Comfort operating in ambiguous pre 0-to-1 environments and moving work from research to prototype-ready handoff.