Senior / Principal QA

We're growing our team and looking for a Senior/Principal QA Engineer to own quality across our AI voice assistant: from prompt and agent behavior testing to mobile, TV, and backend.

Requirements

• Senior / Principal, 5+ years of manual QA - a high level of ownership and self-direction: the ability to take on the project as a whole, be fully accountable for quality, and get up to speed on new things independently.
• Experience testing AI agents / assistants: checking scenario and prompt following (prompt / instruction following), response quality and relevance.
• Experience testing "under the hood" via AI observability / tracing tools (LangSmith and analogs): analyzing an agent's traces - tool calls, memory handling, the model that ran, inputs/outputs, token consumption, spotting bottlenecks.
• Experience with mobile and/or TV-app testing; experience specifically with TV devices is less critical.
• Experience testing backends and integrations: API testing (REST / gRPC), verifying the seams between services, reading logs and traces (Kibana / Grafana / Loki, etc.).
• Solid QA fundamentals: test design, test documentation, defect handling.
• Hands-on experience using AI tools in the day-to-day QA process.

Nice to have

• Experience testing speech technologies (ASR / TTS) and working with audio.
• Evals - building and running AI-assistant quality evaluations: scenario-based self-play, LLM-as-judge, manual validation as human-in-the-loop.
• CI/CD / deployment / test environments: configuring stands, building and deploying applications (GitHub / GitLab, etc.).
• Device testing experience / ADB.
• Willingness to move into test automation over time.

Responsibilities

AI assistant:

• Assessing assistant quality: prompt / instruction following, compliance with requirements; catching regressions between prompt and model versions.
• Testing the assistant "under the hood" via observability / tracing tools: correctness and parameters of tool calls (MCP / integrations with external APIs), memory handling, the bot's reasoning, traces, etc.

Devices and applications:

• End-to-end manual testing of the assistant in the mobile app and on the devices (TV, speaker).
• Testing voice interaction with the assistant, recognition (ASR) and synthesis (TTS) quality, voice UX.
• Testing the assistant's product scenarios: agentic e-commerce (voice order -> cart -> confirmation on TV via remote), media content and a movie showtimes guide, flight tickets, basic scenarios (weather, currency rates, time, timers, Q&A).

Platform:

• Testing the platform / backend part the assistant runs on - a distributed backend of many services: dialog orchestration, LLM agent, ASR / TTS, etc.
• Verifying the request's end-to-end path through these components, the correctness of integrations and the seams between services, working with backend logs and traces.

Process:

• Maintaining test cases and bug reports.
• Close collaboration with the team: TPM, Prompt Engineer, DEV, DevOps, ML.
• First-line QA of critical scenarios ahead of demos.

What we offer

• The team has built award-winning AI products for tech corporations - devices, voice assistants, products that are actually in the world 
• Cutting-edge tech stack: Speech Technologies, NLP, Generative AI (LLMs, diffusion models), voice-first agentic architecture with privacy-first and on-premises deployment
• High engineering bar and real ownership - the team cares about what actually works in production, not what looks good in a demo, and you'll see the impact of your work directly 
• Fast career progression - a senior-heavy team and a high volume of real problems means you grow faster than you would anywhere else 
• Startup pace with enterprise stability - real clients, real revenue, no bureaucracy 
• Fully remote across Europe
• 21 vacation days + public holidays + 5 sick days 
• Private English lessons via Preply