Senior QA Engineer ID66279

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

We are looking for a Senior QA Engineer to perform manual and automation testing of web applications, validate AI-generated test cases and LLM outputs, and contribute automation scripts to a stable existing framework under the guidance of a dedicated Automation Lead. You will design and execute test plans in TestRail, conduct API testing with Postman, document defects in Jira, and collaborate across Agile sprint cycles. The role suits an engineer with a strong manual background who is actively growing their automation skills using tools including Claude and GitHub Copilot.

- Perform manual and automation testing of web applications to identify bugs, glitches, and inconsistencies;

- Take ownership of automation tasks within the team and proactively develop your automation skills;

- Develop, maintain, and execute detailed manual test cases based on product requirements and specifications;

- Perform thorough cross-browser UI testing to ensure consistent behavior across Chrome, Firefox, Safari, Edge, and other browsers;

- Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs to ensure accuracy, completeness, and alignment with project requirements;

- Design and execute test scenarios to verify AI behavior, including prompt-response consistency, contextual relevance, and deterministic versus non-deterministic outputs;

- Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate;

- Conduct manual API testing to validate request/response accuracy, error handling, and integration;

- Use TestRail to create, manage, and track test plans, test cases, and test execution results;

- Work with Jenkins CI/CD pipelines;

- Report and document defects clearly using Jira;

- Participate in daily stand-ups, sprint planning, and other Agile team meetings;

- Review user stories and acceptance criteria to understand the scope of testing;

- Estimate own tasks and report risks to the team lead;

- Collaborate with developers and other QA team members to reproduce issues and verify fixes;

- Maintain and update test documentation as needed.

- 4+ years of experience in software testing;

- Strong manual and automation testing knowledge;

- Basic understanding of SDLC and STLC (Software Development and Testing Life Cycle) models;

- Knowledge of Testing and Quality Assurance processes;

- Strong experience with Test Design activities to cover a set of product requirements;

- Knowledge of Test Design techniques;

- Knowledge of testing methodologies;

- Defect analysis skills and the ability to write detailed and useful defect reports with expected defect-fixing outcomes;

- Experience with Postman (API testing) and browser developer tools;

- Experience with UI testing;

- Experience with TestRail as a Test Management system;

- Basic understanding of CI/CD processes;

- Intermediate English or higher.

- Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles;

- Familiarity with AI/ML concepts, APIs, and frameworks such as OpenAI API, Azure OpenAI, and LangChain;

- Knowledge of data protection and compliance standards for AI systems (GDPR, zero-data retention, PII anonymization);

- Experience with SSO testing;

- Experience with Performance or Load testing.

- Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.

- Competitive compensation: USD-based pay with education, fitness, and team activity budgets.

- Exciting projects: Modern solutions with Fortune 500 and top product companies.

- Flextime: Flexible schedule with remote and office options.

Application → Coding Challenge → Video Interview → Technical Interview or Hiring Manager Interview

Each step helps us understand your skills and overall fit.

If it’s a match, you’ll receive an offer.