Responsible AI Governance Workshop

Responsible AI Governance Workshop

Learn, self-assess, and win prizes.
Welcome!

To access the all materials, use the QR code or go to https://www.techbetter.ai/cincy-ai-week
This workshop is an educational event and a competition.

You will learn how to evaluate and improve your AI governance using a framework based a US regulatory standard (the NIST AI RMF). This freamwork is based on academic research (Dotan et al., 2024), and you can read more about it here.

You will evaluate your own products or vendors

Winners get prizes!

Prizes and rewards
Full participants are eligible for the following. Which ones are you interested in?

A completion certificate with my name on itOffice hour with Ravit later todayA PDF summary of my responsesA discounted consultation session with RavitA copy of the report summarizing findings from the workshop
Winners are eligible for the following. Which ones are you interested in?

Be publicly celebrated at the conference media reception and leadership summitA free consultation session with RavitA free consultation session with ETA

Case study description
Air LLM is an airline that added a chatbot to its website. The chatbot was build by a third-party, Chatbots-R-Us, and it acts as a travel agent and customer support: It helps customers find and book flights, answers questions about company policies, and addresses customer-specific issues such as questions about previous flights or frequent flyer special offers.

Risk Triage
Rank

Reminder: 0="not a priority", 4="top priority"
Reflection
Explain your top three choices. Why are they your top choices? Please provide an explanation in terms of the impact on stakeholders.

Describe the product
Briefly describe the product you are assessing
What is your relation to this product?

My company develops itMy company uses itMy company is considering buying itMy company has bought itOther

Risk Triage
Rank

Reminder: 0="not a priority", 4="top priority"
Explain your top three choices. Why are they your top choices? Please provide an explanation in terms of the impact on stakeholders.

Case Study
Air LLM is an airline that added a chatbot to its website. The chatbot was build by a third-party, Chatbots-R-Us, and it acts as a travel agent and customer support: It helps customers find and book flights, answers questions about company policies, and addresses customer-specific issues such as questions about previous flights or frequent flyer special offers.

Assess
Recall the three top risks you identified.

Answer the following questions with regard to these risks.
A. Which AI responsibility actions should Air LLM prioritize?
B. What minimal level of responsibility should Air LLM require from Chatbots-R-Us?
C. How should Chatbots-R-Us demonstrate their AI responsibility?
Reflection
Explain your rationale for each of the questions. Feel free to add other comments!

Developer/User Governance

Evaluate a product your company is developing or using

Describe the product
Briefly describe the system you are assessing
What is your relation to this product?

My company develops itMy company uses itOther

Assess and prioritize
Recall the three top risks you identified.

Answer the following questions with regard to these risks.
Part 1

Statements relevent for planning phases and on
1. Mapping impacts - We clearly define what the AI is supposed to do and its impacts, including scope, goals, methods, and the negative and positive potential impacts of these activities.
2. Identify Requirements - We identify the requirements the AI must meet, including compliance, certifications, and human oversight needs.
3. Responsibility Mindset - We facilitate a mindset of responsibility, for example, by providing AI ethics training to relevant personnel, clearly defining relevant roles, establishing policies, and implementing practices for critical thinking.
Part 2

Statements relevent for products in data collection, model building, and on
4. Measuring Risks - We measure potential negative impacts.
5. Transparency - We document information about the system, including explaining how it works, limitations, and risk controls.
6. Management Plan - We plan how to respond to risks, including setting priorities and documenting residual risks.
7. Risk Mitigation - We act to minimize risks, including addressing our prioritized risks and tracking incidents.
Part 3

Statements relevent for deployed products
8. Pre-Deployment Checks - We only release versions that meet our AI ethics standards.
9. Monitoring - We monitor and resolve issues as they arise.

Reflect
Explain your rationale. How did you decide how to rank yourself?
What did you learn from did process? What stood out to you? And did the process affect how you think of your work?
Optional - Do you have other comments?

Buyer Governance

Evaluate a product your company is considering buying or has bought

Describe the product
Briefly describe the product you are assessing
What is your relation to this system?

My company is considering buying itMy company bought itMy company uses it for freeOther

Assess
Recall the three top risks you identified.

Answer the following questions with regard to these risks.
A. Which AI responsibility actions should your company prioritize?
Explain your rationale for question A. For example: What are your top choices and why?
B. What is the minimally acceptable level of responsible governance activities?
Explain your rationale for question B. For example: What are your top choices and why?
C. Does your company address AI risks during procurement?
Are you happy with your company's approach? What, if anything, would you change?
D. How should the vendor demonstrate their AI responsibility?
Explain your rationale for question D. For example: What are your top choices and why?

Reflect
What did you learn from did process? What stood out to you? And did the process affect how you think of your work?
Optional - Do you have other comments?

Welcome!