Agenda

Chair’s opening remarks

REGULATION

A supervisory perspective on bank’s use of AI

• Key considerations and questions about sound and responsible AI usage
• Thoughts on ways to manage risks presented by AI

RISING SCRUTINY & AI REGULATION IMPACT - PANEL DISCUSSION INSURANCE VS. BANKS

Impact of new AI regulations on governance and model risk management

• Evaluating global, regional, and state-level AI/ML regulations and their effects on model risk management (e.g., GDPR, CCPA, IFRS, California, New York)
• Assessing regulatory expectations for model risk teams and how they must adapt to new requirements
• Analyzing increased regulatory scrutiny's influence on compliance, credit risk, and liquidity management
• Enhancing model governance to detect credit risk trends early and ensure board accountability
• Exploring alternative and customer data impacts on regulations in the EU/US and financial products like auto loans
• Aligning risk management practices with global, national, and state regulations

GEOPOLITICAL RISK IMPLICATIONS

Integrating MRM Programs for Small Banks with AI Applications (crossing the $10b mark)

• Changes in oversight
• MRM as a strategy tool: Improving the effective challenge
• Typical new models for small banks
• Reducing the budget, bringing validations from external to internal
• Education with model owners and training staff
• Collaboration with other departments (audit, third party, ERM, info security, data governance)
• Adjusting to AI developments
• Testing black-box AI applications (tools)
• Framework of MRM review for AI applications vs AI models
• What can be reviewed in a black-box AI application
• Examples of MRM reviews of AI applications

Morning refreshment break and networking

ADAPTING MRM FOR GROWTH

Integrating MRM Programs for Small Banks with AI Applications (crossing the $10b mark)

• Changes in oversight
• MRM as a strategy tool: Improving the effective challenge
• Typical new models for small banks
• Reducing the budget, bringing validations from external to internal
• Education with model owners and training staff
• Collaboration with other departments (audit, third party, ERM, info security, data governance)
• Adjusting to AI developments
• Testing black-box AI applications (tools)
• Framework of MRM review for AI applications vs AI models
• What can be reviewed in a black-box AI application
• Examples of MRM reviews of AI applications

ADOPTING AND ENHANCING METHODS & PRACTICE

Adjusting practices to modern challenges and technologies

• Identifying gaps in frameworks for assessing risks in AI and Gen AI models
• Addressing new failure modes from model experimentation
• Determining tools and technologies for regulatory scrutiny and operational scaling
• Resolving collaboration challenges between model development and risk/compliance teams
• Incorporating AI tools into model development and validation without increasing risk

ADVANCED RISK METRICS – JOINT PRESENTATION

Enhancing strategies for leveraging advanced risk models in risk tolerance assessment, stress testing, and climate risk management

• Examining AI's role in advancing risk metrics and stress testing methodologies
• Explaining the application of extreme VaR to mitigate model risks in volatile markets
• Utilizing innovative risk metrics, including extreme VaR, to AI and machine learning models for improved risk management
• Discussing how including Covid-era data affects advanced risk metrics and AI model management
• Exploring strategies for updating AI models to incorporate insights from pandemic data

Lunch

TALENT MANAGEMENT - PANEL DISCUSSION

Futureproofing talent and training needs for AI models

• Identifying current and future training needs for AI and Generative AI
• Addressing skill gaps for effective AI model management
• Anticipating evolving training requirements in the next 1-2 years
• Comparing skill requirements for AI versus traditional model validation
• Reinforcing the need for continuous training to keep pace with AI advancements
• Examining successful training programs and comparing centralized versus decentralized solutions
• Training AI teams to identify and mitigate fraud risks from AI-powered bad actors

AI MODEL MANAGEMENT

AI Model Risk Evaluation and Validation Practice in the Era of Advanced Technology

• Model risk assessment, model risk appetite, model risk mitigation
• Digitalization of IV activity and use of GEN AI to enhance validation capabilities
• Example of validation of a credit risk model that applies ML/AI techniques

Afternoon refreshment break and networking

ETHICS AND BIAS

Addressing GEN AI ethical considerations and minimizing bias in model risk

• Developing strategies to mitigate risks such as model hallucinations
• Reducing hallucinations, bias, and toxicity in LLMs
• Analyzing implications in automated decision-making
• Achieving socially responsible outcomes through AI
• Promoting transparency and accountability in machine learning models
• Implementing AI governance frameworks to ensure fairness and accountability
• Minimizing ethical risks and biases in financial AI models

GEN AI VALIDATION – PANEL DISCUSSION

Ensuring model validation for Gen AI through governance and automation

• Reviewing governance practices for validating generative AI models
• Enhancing validation efficiency through automation and risk-tier categorization
• Exploring best practices for evaluating and validating AI models
• Addressing hallucinations to ensure reliable model outputs
• Establishing governance frameworks for generative AI use
• Managing new failure modes emerging from generative AI experimentation

AUTOMATED RISK MONITORING

Strategizing automation in monitoring for enhanced compliance and risk management

• Aligning with regulatory expectations and setting performance tolerance levels
• Integrating automation into risk identification and monitoring processes
• Formulating strategies for integrating automation into ongoing monitoring systems
• Optimizing model ops and efficiency gains

Chair’s closing remarks and end of day