AI/LLM Safety Engineer

Description

AI/LLM Safety Engineer

Own Your Impact. At Propio, we don't believe careers happen to people. We believe people create them.

Here, you're trusted to make decisions, challenge assumptions, drive innovation, and shape outcomes. Your success is not limited by hierarchy or tenure. It's fueled by your ambition, your curiosity, and your willingness to own your impact. If you're looking for a role where you can simply maintain the status quo, this probably isn't it, but if you're looking for a place where your ideas matter, your growth is accelerated, and your work creates meaningful impact across the world, we'd love to talk.

Why Propio?

Every day, communication changes lives. A patient receives care they otherwise couldn't access. A family gains critical information. A business connects with a customer. A community becomes more inclusive. These moments happen because barriers are removed. And behind those moments are Propio team members who show up every day to solve problems, innovate, and build the future. This isn't just work. This is world impact.

As a AI/LLM Safety Engineer, you'll have the opportunity to make a meaningful contribution to the continued growth and transformation of Propio.

You'll be empowered to:

• Take ownership of important initiatives and outcomes.
• Drive meaningful business results.
• Influence decisions and contribute new ideas.
• Partner with talented, high-performing team members.
• Challenge yourself through continuous learning and growth.
• Help shape the future of a rapidly growing organization.

What You'll Own

The AI/LLM Safety Engineer will join our AI team and take ownership of how safely our models and agents behave in production; with a focus on AI Safety, Trust & Safety, and Responsible AI. You will design the evaluations that catch unsafe behavior, build the guardrails that stop it, and lead the red-teaming that finds the gaps before our users—or attackers—do. Agent safety is the primary focus of this role: you will help ensure that as our systems gain the ability to call tools and take actions, they do so within well-defined, well-tested boundaries.

LLM Safety Evaluation & Red Teaming

• Design and maintain a safety evaluation framework—adversarial prompt sets, scenario-based test suites, and regression suites—so that every model and agent update is validated before it ships.
• Lead structured red-teaming exercises covering jailbreaks, prompt injection, tool misuse, and data exfiltration; document findings and drive each issue through to remediation and closure.

Guardrails & Runtime Controls

• Build and iterate on guardrail logic, including input/output filtering, tool-boundary constraints, action validation, sensitive-data redaction, and policy prompting.
• Integrate safety checks into CI/CD and runtime so that unsafe behavior is intercepted before it reaches users.

Agent Safety (primary focus of this role)

• Perform threat modeling for agentic scenarios: tool-call boundaries, sandbox isolation, and least-privilege access, with particular attention to preventing agents from exfiltrating data or executing irreversible actions through chained tool calls.
• Conduct safety reviews of reinforcement-learning (RL) environments and trajectory data, partnering with environment and agent engineering teams to embed safety constraints directly into the environments themselves.

Monitoring & Observability

• Instrument AI features for safety with structured logging, tracing, and metrics, enabling detection of unsafe patterns and regressions in production.

Governance & Collaboration

• Prepare evidence for governance reviews—test reports, evaluation summaries, and mitigation validation—aligned with internal Responsible AI standards.
• Collaborate with Product and UX to improve safety interactions (warnings, confirmations, refusal messaging, and feedback collection), and align evaluation goals with the Research and Data teams.

What Makes Someone Successful Here

The most successful people at Propio aren't necessarily the ones with the longest resumes. They're the people who:

• Take ownership instead of waiting for direction.
• Embrace challenges as opportunities to grow.
• Continuously seek better ways of working.
• Turn ideas into action.
• Hold themselves and others accountable to high standards.
• Are driven by making a measurable impact.

Requirements

What You'll Bring

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, or a related technical field—or equivalent practical experience.
• 4+ years building production software, with direct experience working on—or securing—ML/LLM systems.
• Strong software engineering skills with the ability to write production-grade code (primarily Python), beyond scripting or notebook prototyping.
• Solid understanding of LLMs and ML: how models work, prompt engineering, and the safety implications of fine-tuning and RAG (e.g., unsafe retrieval, tool misuse, and data exfiltration).
• A security mindset with demonstrated threat-modeling ability; able to threat-model AI workflows and familiar with the fundamentals of access control, data retention, and incident response.
• Familiarity with the LLM attack surface—prompt injection, jailbreaks, data poisoning, and supply-chain risk—and working knowledge of the OWASP LLM Top 10.
• Hands-on experience with at least one of safety evaluation or red teaming, with the ability to walk through a real finding and how it was remediated.

Preferred Qualifications

• Hands-on experience with industry safety tooling such as garak, PyRIT, promptfoo, Giskard, and NeMo Guardrails, and the ability to articulate the trade-offs between them.
• Visible output in AI safety or security: publications at relevant venues (e.g., the NeurIPS AI Safety Workshop, USENIX Security, or DEF CON AI Village), open-source contributions, or responsible disclosures on frontier models with public write-ups.
• Familiarity with AI governance and compliance frameworks (NIST AI RMF, ISO/IEC 42001, EU AI Act) and the ability to translate compliance requirements into concrete engineering tasks.
• Engineering experience with agents, RL environments, and/or tool use.
• Practical experience with threat-modeling methodologies such as MITRE ATLAS and STRIDE/PASTA.

Even if your experience doesn't perfectly match every qualification, we encourage you to apply. We're looking for potential, drive, and a commitment to growth as much as experience.

What You'll Gain

Own Your Growth: We invest in people who invest in themselves. You'll have opportunities to learn, develop, and expand your capabilities while building a meaningful career.

Own Your Impact: You'll see the connection between your work and our success. We believe great people deserve the opportunity to make a real difference.

Own Your Innovation: The best ideas can come from anywhere. We encourage curiosity, creativity, and challenging the way things have always been done.

Own Your Success: Whether you're building expertise, pursuing leadership opportunities, or expanding your career path, we'll give you room to grow and the support to get there.

At Propio, your work doesn't just move a company forward. It helps connect people, communities, and opportunities across the world. Ready to Build Something Bigger? Apply today and discover what happens when you own your success.