Responsible AI Governance Workshop

Responsible AI Governance Workshop

Learn, self-assess, and win prizes.
Welcome!

To access the all materials, use the QR code or go to https://www.techbetter.ai/cincy-ai-week
This workshop is an educational event and a competition.

You will learn how to evaluate and improve your AI governance using a framework based a US regulatory standard (the NIST AI RMF). This freamwork is based on academic research (Dotan et al., 2024), and you can read more about it here.

You will evaluate your own products or vendors

Winners get prizes!
- Full participants are eligible for the following. Which ones are you interested in?
  
  A completion certificate with my name on itOffice hour with Ravit later todayA PDF summary of my responsesA discounted consultation session with RavitA copy of the report summarizing findings from the workshop
- Winners are eligible for the following. Which ones are you interested in?
  
  Be publicly celebrated at the conference media reception and leadership summitA free consultation session with RavitA free consultation session with ETA
- Full participation criteria:
  - Answer all questions, including the case study and self-assessment. Make sure to complete the rankings as well as answer the open questions
  - Be respectful
  Winner criteria:
  - Full participation criteria
  - Best answers to the open questions, based on their level of clarity, how well the reasoning supports the numerical score they accompany, and how detailed they are
  - The numerical scores themselves do not afffect winner selection. This is a competition of the best assessment, regardless of the company's maturity level in AI responsibility

Risk Intro

Definitions and case study
- Air LLM is an airline that added a chatbot to its website. The chatbot was build by a third-party, Chatbots-R-Us, and it acts as a travel agent and customer support: It helps customers find and book flights, answers questions about company policies, and addresses customer-specific issues such as questions about previous flights or frequent flyer special offers.
- Rank
  
  Reminder: 0="not a priority", 4="top priority"
- Reflection
- Explain your top three choices. Why are they your top choices? Please provide an explanation in terms of the impact on stakeholders.

Risk Triage

Evaluate your product or vendor
- Briefly describe the product you are assessing
- What is your relation to this product?
  
  My company develops itMy company uses itMy company is considering buying itMy company has bought itOther
- Rank
  
  Reminder: 0="not a priority", 4="top priority"
- Explain your top three choices. Why are they your top choices? Please provide an explanation in terms of the impact on stakeholders.

Governance Intro

Instructions, definitions, and case study
- Air LLM is an airline that added a chatbot to its website. The chatbot was build by a third-party, Chatbots-R-Us, and it acts as a travel agent and customer support: It helps customers find and book flights, answers questions about company policies, and addresses customer-specific issues such as questions about previous flights or frequent flyer special offers.
- Recall the three top risks you identified.
  
  Answer the following questions with regard to these risks.
- A. Which AI responsibility actions should Air LLM prioritize?
- B. What minimal level of responsibility should Air LLM require from Chatbots-R-Us?
- C. How should Chatbots-R-Us demonstrate their AI responsibility?
- Reflection
- Explain your rationale for each of the questions. Feel free to add other comments!

Developer/User Governance

Evaluate a product your company is developing or using
- Briefly describe the system you are assessing
- What is your relation to this product?
  
  My company develops itMy company uses itOther
- Recall the three top risks you identified.
  
  Answer the following questions with regard to these risks.
- Part 1
  
  Statements relevent for planning phases and on
- 1. Mapping impacts - We clearly define what the AI is supposed to do and its impacts, including scope, goals, methods, and the negative and positive potential impacts of these activities.
- 2. Identify Requirements - We identify the requirements the AI must meet, including compliance, certifications, and human oversight needs.
- 3. Responsibility Mindset - We facilitate a mindset of responsibility, for example, by providing AI ethics training to relevant personnel, clearly defining relevant roles, establishing policies, and implementing practices for critical thinking.
- Part 2
  
  Statements relevent for products in data collection, model building, and on
- 4. Measuring Risks - We measure potential negative impacts.
- 5. Transparency - We document information about the system, including explaining how it works, limitations, and risk controls.
- 6. Management Plan - We plan how to respond to risks, including setting priorities and documenting residual risks.
- 7. Risk Mitigation - We act to minimize risks, including addressing our prioritized risks and tracking incidents.
- Part 3
  
  Statements relevent for deployed products
- 8. Pre-Deployment Checks - We only release versions that meet our AI ethics standards.
- 9. Monitoring - We monitor and resolve issues as they arise.
- Explain your rationale. How did you decide how to rank yourself?
- What did you learn from did process? What stood out to you? And did the process affect how you think of your work?
- Optional - Do you have other comments?

Buyer Governance

Evaluate a product your company is considering buying or has bought
- Briefly describe the product you are assessing
- What is your relation to this system?
  
  My company is considering buying itMy company bought itMy company uses it for freeOther
- Recall the three top risks you identified.
  
  Answer the following questions with regard to these risks.
- A. Which AI responsibility actions should your company prioritize?
- Explain your rationale for question A. For example: What are your top choices and why?
- B. What is the minimally acceptable level of responsible governance activities?
- Explain your rationale for question B. For example: What are your top choices and why?
- C. Does your company address AI risks during procurement?
- Are you happy with your company's approach? What, if anything, would you change?
- D. How should the vendor demonstrate their AI responsibility?
- Explain your rationale for question D. For example: What are your top choices and why?
- What did you learn from did process? What stood out to you? And did the process affect how you think of your work?
- Optional - Do you have other comments?

Finish Line

Next steps and preferences
- Full participation criteria:
  - Answer all questions, including the case study and self-assessment. Make sure to complete the rankings as well as answer the open questions
  - Be respectful
  Winner criteria:
  - Full participation criteria
  - Best answers to the open questions, based on their level of clarity, how well the reasoning supports the numerical score they accompany, and how detailed they are
  - The numerical scores themselves do not afffect winner selection. This is a competition of the best assessment, regardless of the company's maturity level in AI responsibility
- You are welcome to share any additional comments or requests
- Should be Empty:

Responsible AI Governance Workshop

Welcome!

About You

Risk Intro

Risk Triage

Governance Intro

Developer/User Governance

Buyer Governance

Finish Line