AI Operations & QA Analyst

About the Role

Apply Minds is seeking a detail-oriented AI Operations & QA Analyst to support projects involving artificial intelligence, document processing, data extraction, quality assurance, and data validation.

You will play a key role in ensuring that AI-generated outputs are accurate, complete, and reliable by comparing system results against source documents, identifying inconsistencies, and documenting findings. Working alongside Product, Engineering, and AI teams, you will help improve the quality and performance of AI-powered solutions through structured testing, validation, and continuous feedback.

This role is ideal for someone who enjoys problem-solving, has excellent attention to detail, and is passionate about delivering high-quality results. While exposure to software testing or AI technologies is beneficial, this is not primarily a software development role.

Key Responsibilities

• Review AI-generated summaries, extracted data, classifications, and responses for accuracy and completeness.

• Compare AI outputs against original source documents and identify discrepancies.

• Validate information extracted from PDFs, scanned documents, forms, and structured datasets.

• Execute manual functional, regression, exploratory, and user acceptance testing (UAT) where required.

• Create, maintain, and execute structured test cases and test scenarios.

• Identify, document, and report defects with clear reproduction steps and supporting evidence.

• Verify defect fixes and perform regression testing before releases.

• Maintain QA trackers, validation logs, and testing documentation.

• Support document classification, data validation, and quality control activities.

• Review AI confidence scores and investigate low-confidence or inconsistent outputs.

• Escalate complex or high-risk cases to the appropriate stakeholders.

• Collaborate with Product, Engineering, and AI teams to improve system quality and performance.

• Contribute to improving testing processes, validation procedures, and QA documentation.

• Ensure deliverables meet quality, accuracy, and compliance standards.

Required Skills & Experience

• Excellent attention to detail and a methodical approach to work.

• Strong written and verbal communication skills in English.

• Experience in quality assurance, software testing, data validation, document review, or a related analytical role.

• Ability to analyse information and identify inconsistencies or anomalies.

• Experience using Microsoft Excel or Google Sheets.

• Ability to follow structured processes, testing procedures, and documentation standards.

• Strong organisational and time-management skills.

• Comfortable working with large volumes of documentation and digital platforms.

• Ability to work independently in a remote environment while collaborating effectively with distributed teams.

Technical Skills

Experience with one or more of the following is advantageous:

• Microsoft Excel or Google Sheets

• Jira, Azure DevOps, or similar issue tracking tools

• Microsoft Office or Google Workspace

• AI platforms such as ChatGPT, Claude, Gemini, Microsoft Copilot, or similar

• Browser Developer Tools

• Test management tools and QA documentation platforms

Desirable Experience

• Experience testing AI applications, chatbots, or Large Language Models (LLMs).

• Experience with OCR (Optical Character Recognition) or document-processing systems.

• Exposure to workflow automation or low-code platforms.

• Understanding of Agile or Scrum development methodologies.

• Experience with API testing or SQL is advantageous.

• Knowledge of Playwright, Selenium, Cypress, or other test automation frameworks is beneficial.

• Previous experience within insurance, legal, financial, real estate, or other regulated industries.

Personal Attributes

We're looking for someone who is:

• Naturally curious with an investigative mindset.

• Highly organised and dependable.

• Detail-oriented and committed to accuracy.

• Comfortable questioning AI-generated results rather than accepting them at face value.

• A proactive communicator who collaborates well with cross-functional teams.

• Adaptable and eager to learn emerging AI technologies and quality assurance practices.

Success in This Role

Success will be measured by your ability to:

• Deliver accurate and consistent validation of AI-generated outputs.

• Identify defects and quality issues before release.

• Produce clear, well-documented findings and recommendations.

• Improve AI output quality through detailed analysis and feedback.

• Maintain high standards of accuracy, consistency, and compliance.

• Contribute to the continuous improvement of QA processes and AI validation workflows.

Why Join Apply Minds?

At Apply Minds, you'll work on innovative AI-driven projects that solve real business challenges across multiple industries. You'll have the opportunity to develop your expertise in artificial intelligence, quality assurance, and document intelligence while working with a collaborative, forward-thinking team in a flexible remote environment.