AI Engineer

Team (Project) Introduction
The Department of Veterans Affairs (VA) Cybersecurity Operations Systems Engineering (COSE) project serves as an overarching technical engine that unifies Architecture and Engineering Services throughout the VA Enterprise. This program implements cohesive organizational security architecture and underlying engineering components that utilize national security standards, guidelines, and frameworks. COSE bridges the gap between high-level business requirements and technical structures, enabling the consistent deployment of secure technologies through implementation guidance and the establishment of an analytic library. Systems security engineering within COSE contributes a holistic perspective to the systems engineering effort, ensuring that stakeholder protection needs are addressed throughout the entire system life cycle from concept and development to production, support, and decommissioning. By drawing upon well-established systems engineering and security principles, COSE adapts and supplements practices to protect intellectual property, data, and the methods used to create VA systems. These activities improve the security posture of VA applications to prevent, deter, or detract from cyberattacks by nefarious adversaries or insider threats.

9th Way Insignia is looking for an Engineer, 3, Artificial Intelligence Engineer to join this team.

Professional Level Information:
An Engineer, 3 typically plans and directs research or development work on complex projects, along with engaging various parties in design and development. Costs and recommendations of new components may also involve part of the job scope. Performs multiple engineering-related tasks in various assignments within the project and firm. An Engineer, 3 oversees the design, development, implementation, and analysis of technical products and systems.  An Engineer, 3 has broad knowledge of engineering procedures and assists in the resolution of complex problems.  An Engineer, 3 has strong technical skills and background, a knack for learning new technologies, and a blend of good problem-solving and innovation needed to resolve a wide variety of technical production challenges.

Functional Job (LCAT) Information:
The Artificial Intelligence Engineer will design, develop, implement, secure, evaluate, and maintain Artificial Intelligence (AI), Machine Learning (ML), and Large Language Model (LLM) solutions supporting the VA COSE program. The engineer will work closely with cybersecurity engineers, architects, data scientists, software developers, and Government stakeholders to deliver secure, reliable, responsible, and production-ready AI capabilities.

Responsibilities:

• Design, develop, train, test, deploy, and maintain AI and machine learning models for enterprise-scale and Government applications.
• Develop and operationalize production AI/ML solutions, ensuring models and supporting systems are reliable, scalable, maintainable, and appropriate for VA mission requirements.
• Collect, clean, transform, analyze, and prepare structured and unstructured data for AI/ML model development, training, testing, and evaluation.
• Evaluate potential AI technologies, models, tools, frameworks, and architectures and recommend solutions appropriate for complex Federal Government and cybersecurity use cases.
• Apply AI and machine learning techniques to solve real-world business, cybersecurity, engineering, analytics, and operational problems across a large enterprise environment.
• Develop and support Generative AI and Large Language Model (LLM) solutions, including model configuration, system prompts, classifiers, fine-tuning, content moderation, safety filters, and other model controls.
• Evaluate AI-generated outputs for accuracy, factuality, grounding, reliability, bias, helpfulness, honesty, and overall model performance before outputs are incorporated into VA efforts.
• Design and execute AI/ML model evaluation and validation methodologies, including benchmark testing and comparative evaluation of model outputs.
• Perform and support AI red-team testing to identify weaknesses, unintended model behaviors, bias, security vulnerabilities, and other risks associated with AI-generated outputs.
• Implement mechanisms and controls to improve the factuality and grounding of AI/LLM outputs, including system-level instructions, source attribution, content controls, and appropriate response behavior when information is incomplete, contradictory, or uncertain.
• Develop, implement, and maintain processes for continuous AI model monitoring, including evaluation of model outputs and identification of material deviations from applicable AI requirements.
• Investigate AI performance, compliance, security, or model-behavior issues and develop corrective actions and mitigation strategies, including identification of responsible parties and resolution timelines.
• Manage and assess AI/LLM model changes throughout the system lifecycle, including retraining, fine-tuning, model version changes, new features, classifiers, prompts, filters, controls, and architectural modifications.
• Maintain detailed technical documentation supporting the development and operation of AI/LLM solutions, including Model Cards, System Cards, Data Cards, evaluation results, training activities, model configurations, enterprise controls, and system documentation.
• Document pre-training and post-training activities, including actions affecting model factuality and grounding, system prompts, safety controls, content moderation, red-team activities, and other model configuration decisions.
• Support development and maintenance of AI acceptable-use policies, end-user guidance, feedback mechanisms, monitoring plans, and governance documentation.
• Ensure AI solutions protect VA-sensitive information, personally identifiable information (PII), protected health information (PHI), and other sensitive Government data from unauthorized access, disclosure, use, or incorporation into external AI training datasets.
• Ensure AI solutions comply with applicable VA cybersecurity requirements, Federal AI requirements, security policies, privacy requirements, and responsible/trustworthy AI governance requirements.
• Incorporate secure-by-design, Zero Trust, least-privilege, defense-in-depth, and risk-management principles into the architecture and engineering of AI-enabled systems.
• Support security and architectural reviews of AI-enabled solutions and identify security risks, vulnerabilities, control gaps, attack paths, misuse cases, and appropriate mitigations prior to implementation.
• Collaborate with data scientists, cybersecurity architects and engineers, software developers, DevSecOps engineers, domain SMEs, program leadership, and Government stakeholders throughout the AI system development lifecycle.
• Translate mission, cybersecurity, business, and technical requirements into practical AI/ML architectures, models, engineering solutions, and implementation approaches.
• Participate in technical design reviews, pilots, proofs of concept, use-case development, modernization initiatives, engineering working groups, and other activities involving AI and emerging technologies.
• Produce technical documentation, analysis, recommendations, briefings, and other materials that clearly communicate AI architecture, model performance, risks, controls, and recommended courses of action to both technical and non-technical stakeholders.
• Deliver Government-owned AI-related technical artifacts developed under the program, including applicable models, model weights, algorithms, configurations, documentation, data models, source code, and supporting technical components.
• Other responsibilities as assigned
• May require up to two onsite travel visits per year

Requirements:

• Minimum of 10 years of experience in relevant fields including IT security, data science, or software engineering, of which at least 5 years involve the design, development, or deployment of AI or machine learning systems in enterprise or Government environments A PhD in a related field (IT security, data science, or software engineering) may substitute for up to 5 years of the required experience 
• Expertise working on data science, machine learning, or AI projects, either through internships, research projects, or professional roles. 
• Expertise building, training, and deploying machine learning models and AI systems in a production environment. 
• Expertise collaborating with cross-functional teams, including data scientists, software developers, and domain experts. 
• Expertise in data collection, cleaning, and analysis techniques to prepare datasets for AI modeling. 
• Expertise applying AI solutions to real-world problems in various enterprises and domains such as large corporations and Government agencies similar in size/scope to GSA, IRS, DoD or VA

Preferred/Desired:

• Master’s degree in Artificial Intelligence, Business Administration, Business Management, Cybersecurity, Computer Science, Information Systems, Information Assurance, Information Security, Information Resource Management, or related fields. 
• CASP+ (SecurityX), CCISO, CISA, CISM, CISSP, CISSP-ISSAP, CISSP-ISSEP, GCED, GCIH, GSLC, CCNP Security