See how much of this job your resume covers, and what’s missing.
Want a recruiter to go through it line by line?
Get professional reviewUpload your resume and we draft a letter for this exact role, tailored to what it asks for.
Design and maintain evaluation frameworks for LLM outputs and agentic workflows while integrating these tests into CI/CD pipelines. Collaborate with engineering teams to ensure high-quality, reliable AI features through rigorous testing and observability.
Novara provides safety, operational risk management, and sustainability software that empowers organizations to identify and resolve issues before they become incidents. Through the Flex and Risk Management Center platforms, Novara helps organizations manage operational risk proactively with a system they can configure to fit their operations, data that unifies risk and safety intelligence, and tools that increase workforce participation. Novara’s combination of software, training, and AI-powered tools put people and safety first while protecting critical operations.
AI is core to where our product is heading, and the quality bar for it has to be as high as anything else we ship. This role owns that bar. You will define what "working" means for non-deterministic systems, build the tooling to prove it, and give our teams the confidence to move faster on AI-powered features and agent-based products.
You will partner closely with product, engineering, and the broader QA team to bring rigor to how we test LLM-driven and agentic workflows, while also contributing to traditional automation coverage where it matters.
This is a hands-on IC role reporting into the QA Manager, with high visibility into AI initiatives across the company.
\nNot required, but they will stand out:
TypeScript, Playwright, vitest, Postman, GitHub Actions, Jira/Xray, Datadog, AWS, plus emerging AI evaluation tooling.
As a growing company, Novara values its employees by supporting them with a full benefits package including Medical, Dental, Vision, Flexible Spending Accounts, PTO, Paid and Floating Holidays, 401k with Company match and immediate vesting, Company-funded Life Insurance, Employee Assistance Programs, and No-cost Mental Health Benefits.
About Novara
Novara provides safety and operational risk management software that empowers organizations to identify and resolve issues before they become incidents. Through the Flex and Risk Management Center platforms, Novara helps organizations address operational risk proactively by unifying data, increasing workforce engagement, and proactively managing risk. Novara’s combination of training, software, and tools puts people and safety first while protecting critical operations.
Novara, a Providence Equity portfolio company, provides safety and operational risk management software that empowers organizations to identify and resolve issues before they become incidents. Through the Flex and Risk Management Center platforms, Novara helps organizations address operational risk proactively by unifying data, increasing workforce engagement, and proactively managing risk. Novara’s combination of training, software, and tools puts people and safety first while protecting critical operations.
Novara launched January 1 2026, as an independent company, a spin-off of the Flex and RMC software businesses formerly part of KPA.
Don’t meet every job requirement? At Novara, we are dedicated to building a diverse, inclusive, and authentic workplace. Studies have shown that women and people of color are less likely to apply unless they meet every requirement. If you’re excited about the role but your past experience doesn’t align perfectly with every qualification, we still encourage you to apply! You might just be the right candidate for this or other roles.
Please note that we may use AI tools to assist in the initial screening of resumes to help identify qualified candidates more efficiently. All decisions are reviewed by a human recruiter, and no hiring determination is made solely by automated means.
Novara is committed to providing equal opportunity in all of our employment practices, including selection, hiring, promotion, transfer, and compensation, to all qualified applicants and employees without regard to race, religion, religious dress/grooming, color, ethnicity, sex (including sex stereotyping), sexual orientation, gender identity or gender expression, national origin, ancestry, citizenship status, creed, uniform service member status, military or veteran status, marital status, pregnancy, breast-feeding and/or pregnancy-related conditions, age, protected medical condition, leave status, physical or mental disability, genetic characteristics, or any other legally-protected status in accordance with the requirements of all federal, state and local laws. In compliance with federal law, all persons hired will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification document form upon hire.
If you need assistance or an accommodation due to a disability, you may contact us at hr@novara.com.
Stop the endless job search. Our AI finds and applies to the best jobs for you.
Featuring 214,355+ Jobs in Software Development
Answer easy questions
214,355+ jobs across 15+ categories
Get your best job matches
Only hand-screened, legit jobs
Find a remote job faster
No ads, scams, or junk
“I was the first applicant for a remote marketing position that got listed on the company website the same day I applied. Had an interview within 48 hours!”