About the Role
You will lead end to end AI risk assessments for generative AI and LLM use cases, embedding in GenAI review processes, coordinating cross functional SMEs (Legal, Compliance, InfoSec, Data Governance, MRM, ERM, BRC, TPRM, Financial Crimes), and managing timelines to ensure reviews are completed within SLA. You will review AI system design and documentation, including data sources, assumptions, limitations, fallback plans, guardrail configurations, and change management procedures across use cases such as fraud detection, BSA AML compliance, credit decisioning, and customer facing applications, ensuring governance controls match risk level. You will assess pre deployment testing for adequacy including output integrity, hallucination detection, boundary and edge case testing, ethical and safety guardrails, bias testing, A/B testing, volume testing, and UAT, designing and conducting independent testing as needed. You will evaluate ongoing monitoring plans for comprehensiveness including accuracy, hallucination rates, drift detection, sensitive data controls, reliability metrics, CSAT, acceptable performance ranges, and documented remediation procedures. You will develop and maintain templates, tools, and procedures to support the effectiveness and scalability of the AI Risk Governance Program. You will monitor the evolving regulatory landscape for AI in banking including FFIEC IT Examination Handbook standards, FDIC Financial Institution Letters, interagency statements, and the anticipated RFI on AI model risk management referenced in SR 26-2, and incorporate emerging guidance into the AI risk governance program, supporting regulatory inquiries as needed.
Requirements
- A minimum of 5 years of related experience with a Bachelor’s degree in a quantitative field or 3 years and a Master’s degree or PhD without experience or equivalent work experience in risk management model risk management or AI risk management
- Proficiency in Python or similar languages for evaluating AI system behavior writing test scripts or analyzing model outputs
- Strong understanding of generative AI architectures including LLMs transformer models RAG systems and agentic AI
- Knowledge of interagency model risk management principles including SR 26-2
- Experience with AI testing methodologies including functional testing bias testing adversarial testing and performance monitoring plus familiarity with data privacy and security principles
- Excellent written and verbal communication and the ability to translate complex technical AI concepts for non-technical stakeholders senior management and regulators
- Strong analytical judgment with the ability to manage multiple concurrent assessments prioritize effectively and drive risk-based decisions
Responsibilities
- Lead end to end AI risk assessments for generative AI and LLM use cases across the enterprise embedding in GenAI review process coordinating cross functional SMEs and managing timelines to ensure reviews are completed within SLA
- Review AI system design and documentation including data sources assumptions limitations fallback plans guardrail configurations and change management procedures across banking use cases such as fraud detection BSA AML compliance credit decisioning and customer facing applications
- Assess pre deployment testing for adequacy including output integrity hallucination detection boundary and edge case testing ethical and safety guardrails bias testing A/B testing volume testing and UAT designing and conducting independent testing as needed
- Evaluate ongoing monitoring plans for comprehensiveness including accuracy hallucination rates drift detection sensitive data controls reliability metrics CSAT acceptable performance ranges and documented remediation procedures
- Develop and maintain templates tools and procedures to support the effectiveness and scalability of the AI Risk Governance Program
- Monitor the evolving regulatory landscape for AI in banking including FFIEC IT Examination Handbook standards FDIC Financial Institution Letters interagency statements and the anticipated RFI on AI model risk management referenced in SR 26 2 and incorporate emerging guidance into the AI risk governance program supporting regulatory inquiries as needed
Benefits
- Remote work
- Medical insurance
- Flexible time off
- Retirement savings plans
- Modern family planning
- Reasonable accommodations during recruitment process