One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI - - PowerPoint PPT Presentation

▶
one explanation does not fit all a toolkit and taxonomy
SMART_READER_LITE
LIVE PREVIEW

One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI - - PowerPoint PPT Presentation

Mar 19, 2023 139 likes •600 views

One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Ronny Luss* IBM Research AI *Speaker Joint Work with AIX360 Team at IBM Research. Data Council NYC, November 2019. 1 2019 IBM Corporation Agenda Why

slide-1
SLIDE 1 1 × 2019 IBM Corporation

One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability

Ronny Luss* IBM Research AI *Speaker Joint Work with AIX360 Team at IBM Research. Data Council NYC, November 2019.

slide-2
SLIDE 2 2 × 2019 IBM Corporation

Agenda

  • Why Explainable AI?
  • Types and Methods for Explainable AI
  • AIX360
  • CEM-MAF Example
  • FICO Example (BRCG and

Protodash)

slide-3
SLIDE 3 3 × 2019 IBM Corporation

AI IS NOW USED IN MANY HIGH-STAKES DECISION MAKING APPLICATIONS

Credit Employment Admission Sentencing

slide-4
SLIDE 4 4 × 2019 IBM Corporation

WHAT DOES IT TAKE TO TRUST A DECISION MADE BY A MACHINE (OTHER THAN THAT IT IS 99% ACCURATE)

Is it fair? “Why” did it make this decision? Is it accountable?

slide-5
SLIDE 5 5 × 2019 IBM Corporation

THE QUEST FOR “EXPLAINABLE AI”

slide-6
SLIDE 6 6 × 2019 IBM Corporation

BUT WHAT ARE WE ASKING FOR?

Paul Nemitz, Principal Advisor, European Commission Talk at IBM Research, Yorktown Heights, May, 4, 2018

slide-7
SLIDE 7 7 × 2019 IBM Corporation

WHY EXPLAINABLE AI?

Understanding what’s truly happening can help build simpler systems. Simplification Insight Check if code has comments

slide-8
SLIDE 8 8 × 2019 IBM Corporation

WHY EXPLAINABLE AI? (CONTINUED)

Can help to understand what is wrong with a system. Debugging Self driving car slowed down but wouldn’t stop at red light???

slide-9
SLIDE 9 9 × 2019 IBM Corporation

WHY EXPLAINABLE AI? (CONTINUED)

Can help to identify spurious correlations. Existence of Confounders Pneumonia Diabetes

slide-10
SLIDE 10 10 × 2019 IBM Corporation

WHY EXPLAINABLE AI? (CONTINUED)

Is the decision making system fair?

Fairness

Is the decision making system fair? Is the system basing decisions on the correct features?

Wide Spread Adoption

Robustness and Generalizability

Is the system basing decisions on the correct features?

slide-11
SLIDE 11 11 × 2019 IBM Corporation

Agenda

  • Why Explainable AI?
  • Types and Methods for Explainable AI
  • AIX360
  • CEM-MAF Example
  • FICO Example (BRCG and

Protodash)

slide-12
SLIDE 12 12 × 2019 IBM Corporation

Toolkit Data Explanations Directly Interpretable Local Post-hoc Global Post-hoc Custom Explanation Metrics IBM AIX360 2 2 3 1 1 2 Seldon Alibi ✓ ✓ Oracle Skater ✓ ✓ ✓ H2o ✓ ✓ ✓ Microsoft Interpret ✓ ✓ ✓ Ethical ML ✓ DrWhyDalEx ✓

AIX360: COMPETITIVE LANDSCAPE

All algorithms of AIX360 are developed by IBM Research AIX360 also provides demos, tutorials, and guidance on explanations for different use cases. Paper: One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. https://arxiv.org/abs/1909.03012v1

slide-13
SLIDE 13 13 × 2019 IBM Corporation

THREE DIMENSIONS OF EXPLAINABILITY

One explanation does not fit all: There are many ways to explain things. Decision rule sets and trees are simple enough for people to understand. Supervised learning of these models is directly interpretable. Probe a black-box with a companion model. The black box model provides actual predictions while the interpretation is thru the companion model Shows the entire predictive model to the user to help them understand it (e.g. a small decision tree, whether obtained directly or in a post hoc manner). Only show the explanations associated with individual predictions (i.e. what was it about this particular person that resulted in her loan being denied). The interpretation is simply presented to the user. The user can interact with interpretation. directly interpretable post hoc interpretation vs. Global (model-level) Local (instance-level) vs. static interactive (visual analytics) vs.

slide-14
SLIDE 14 14 × 2019 IBM Corporation

data model samples features local global direct Understand data or model? Explanations as samples, distributions or features? distributions tabular image text ProtoDash

(Case-based reasoning)

DIP-VAE

(Learning meaningful features)

Explanations for individual samples (local) or overall behavior (global)? A directly interpretable model or posthoc explanations? BRCG or GLRM post-hoc A surrogate model or visualize behavior? surrogate visualize ProfWeight

(Learning accurate interpretable model) (Easy to understand rules)

interactive Explanations based on samples or features?

? ? ?

ProtoDash

(Case-based reasoning)

CEM or CEM-MAF

(Feature-based explanations)

TED

(Persona-specific explanations)

features samples One-shot static or interactive explanations? static A directly interpretable model or posthoc explanations? self-explaining post-hoc

  • EXPLAINABILITY

TAXONOMY

slide-15
SLIDE 15 15 × 2019 IBM Corporation

EXPLANATION METHOD TYPES

Decision rule sets and trees are simple enough for people to understand. Directly (global) interpretable Decision Tree Rule List

(Wang and Rudin 2016) (Quinlan 1987)

slide-16
SLIDE 16 16 × 2019 IBM Corporation

EXPLANATION METHOD TYPES

Boolean Decision Rules via Column Generation (BRCG):

  • DNF formulas (OR of ANDs) with small clauses to predict a binary target.
  • Exponential number of possible clauses
  • Fitting with DNFs as a Mixed Integer Program.
  • Column Generation
  • Use few clauses to start with – solve the MIP.
  • Use a Pricing Problem on dual variables to identify the best clauses

that still increase prediction accuracy – efficient step.

  • Iterate - stop when nothing more can be added.
  • Scales to datasets of size ~ 10000 samples.

Directly (global) interpretable

(Dash et. al. 2018)

A variant is in AIX360. This technique won the NeurIPS ‘18 FICO xML Challenge !!

slide-17
SLIDE 17 17 × 2019 IBM Corporation

EXPLANATION METHOD TYPES (CONTINUED)

Start with a black box model and probe into it with a companion model to create interpretations. Post hoc interpretation (Deep) Neural Network Ensembles

slide-18
SLIDE 18 18 × 2019 IBM Corporation

Post hoc (global) interpretation

EXPLANATION METHOD TYPES (CONTINUED)

Can you transfer information from a pre-trained neural network to this simple model ?

Simple Model (Decision Tree, Random forests, smaller neural network) Complex Model (Deep Neural Network)

slide-19
SLIDE 19 19 × 2019 IBM Corporation

Post hoc (global) interpretation

EXPLANATION METHOD TYPES (CONTINUED)

(Hinton et. al. 2015)

Knowledge Distillation

Re-train a simple model with temperature scaled soft scores of complex model.

Prof-Weigh t

(Dhurandhar et. al. 2018)

Re-train a simple model by weighing samples. Weights

  • btained by looking at inner layers of Complex Model.

Logistic Probe Logistic Probe Logistic Probe Logistic Probe p

1

p

2

p

3

p

4

Weight= (p1+ p2+p3+p4)/4 High -> Easy sample Low->Difficult sample Works Well

When Simple Model’s complexity is comparable to Complex Model –ideal for compression When Simple Model complexity is very small compared to Complex Model.

slide-20
SLIDE 20 20 × 2019 IBM Corporation

Post hoc (local) interpretation Saliency Maps

EXPLANATION METHOD TYPES (CONTINUED)

(Sinmoyan et. al. 2013)

slide-21
SLIDE 21 21 × 2019 IBM Corporation

Post hoc (local) interpretation Contrastive Explanations – “Pertinent Negatives” (CEM-MAF):

EXPLANATION METHOD TYPES (CONTINUED)

(Dhurandhar et. al. 2018)

slide-22
SLIDE 22 22 × 2019 IBM Corporation

ONE EXPLANATION DOES NOT FIT ALL – DIFFERENT STAKEHOLDERS

Different stakeholders require explanations for different purposes and with different objectives. Explanations will have to be tailored to their needs.

End users

“Why did you recommend this treatment?”

Who: Physicians, judges, loan officers, teacher evaluators Why: trust/confidence, insights(?)

Affected users

“Why was my loan denied? How can I be approved?”

Who: Patients, accused, loan applicants, teachers Why: understanding of factors

Regulatory bodies

“Prove that your system didn't discriminate.”

Who: EU (GDPR), NYC Council, US Gov’t, etc. Why: ensure fairness for constituents

AI system builders/stakeholders

“Is the system performing well? How can it be improved?“

Who: EU (GDPR), NYC Council, US Gov’t, etc. Why: ensure or improve performance

slide-23
SLIDE 23 23 × 2019 IBM Corporation

Agenda

  • Why Explainable AI?
  • Types and Methods for Explainable AI
  • AIX360
  • CEM-MAF Example
  • FICO Example (BRCG and

Protodash)

slide-24
SLIDE 24 24 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0)

slide-25
SLIDE 25 25 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0)

slide-26
SLIDE 26 26 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0)

slide-27
SLIDE 27 27 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0)

slide-28
SLIDE 28 28 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CONTRASTIVE EXPLANATIONS VIA CEM-MAF

slide-29
SLIDE 29 29 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CONTRASTIVE EXPLANATIONS VIA CEM-MAF

slide-30
SLIDE 30 30 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CONTRASTIVE EXPLANATIONS VIA CEM-MAF

slide-31
SLIDE 31 31 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CONTRASTIVE EXPLANATIONS VIA CEM-MAF

slide-32
SLIDE 32 32 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CONTRASTIVE EXPLANATIONS VIA CEM-MAF

slide-33
SLIDE 33 33 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CONTRASTIVE EXPLANATIONS VIA CEM-MAF

slide-34
SLIDE 34 34 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CONTRASTIVE EXPLANATIONS VIA CEM-MAF

slide-35
SLIDE 35 35 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

slide-36
SLIDE 36 36 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

Sample of FICO HELOC data

slide-37
SLIDE 37 37 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

BRCG requires data to be binarized

slide-38
SLIDE 38 38 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

Run Boolean Rule Column Generation

slide-39
SLIDE 39 39 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

slide-40
SLIDE 40 40 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

1. Process and Normalize HELOC dataset for training 2. Define and train a Neural Network classifier (loan approval model to be explained) 3. Obtain similar samples as explanations for a HELOC applicant predicted as “Good”

slide-41
SLIDE 41 41 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

slide-42
SLIDE 42 42 × 2019 IBM Corporation

AI EXPLAINABILITY 360 (V0.1.0): CREDIT APPROVAL TUTORIAL

Display similar users and give explanation as to why they are similar Most prototypes have no debt

slide-43
SLIDE 43 43 × 2019 IBM Corporation

AI Explainability 360 Causal Inference 360

Trusted AI Toolkits

Adversarial Robustness 360 ✔ ✔ AI Fairness 360 ✔

AIX360: IBM RESEARCH AI EXPLAINABILITY 360 TOOLKIT

Goals

  • Support a community of users and contributors who will

together help make models and their predictions more transparent.

  • Support and advance research efforts in explainability.
  • Contribute efforts to engender trust in AI.

IBM Research AIX360 Explainability Algorithms 8 innovations to explain data and AI models Repositories github.ibm.com/AIX360 github.com/IBM/AIX360 Interactive Experience aix360.mybluemix.net API aix360.readthedocs.io Tutorials 13 notebooks (finance, healthcare, lifestyle, Attrition, etc.) Developers > 15 Researchers + Software engineers across YKT, India, Argentina

slide-44
SLIDE 44 44 × 2019 IBM Corporation

Q&A

YOU HAVE QUESTIONS, WE HAVE ANSWERS

slide-45
SLIDE 45 45 × 2019 IBM Corporation

Thank you