Building Interpretable & Secure AI Systems using PyTorch

BUILDING INTERPRETABLE & SECURE AI SYSTEMS USING PYTORCH GEETA CHAUHAN AI PARTNER ENGINEERING FACEBOOK AI

AGENDA 01 INTERPRETABLE AI 02 SECURE & PRIVACY PRESERVING AI 03 REFERENCES

I N T E R P R E T A B I L I T Y

WHAT IS MODEL INTERPRETABILIT Y ? “THE ABILIT Y TO DESCRIBE AI MODEL INTERNALS AND PREDICTIONS IN HUMAN UNDERSTANDABLE TERMS*” * LH Gilpin, et. al., Explaining explanations: An overview of interpretability of machine learning in IEEE 5th International Conference on data science and advanced analytics (DSAA), 2018

I N C R E A S E D T R A N S P A R E N C Y D E B U G G I N GB E T T E R U N D E R S TA N D I N G MODEL INTERPRETABILIT Y Screenshot of the tool Attributing to dog Attribution Magnitudes

MODEL INTERPRETABILIT Y LIBRARY FOR PY TORCH M U LT I M O D A L E A S Y T O U S EE X T E N S I B L E class MyAttribution(Attribution): def attribute(self, input, ...): attributions = self._compute_attrs(input, ... ) # <Add any logic necessary for attribution> return attributions visualize_image_attr(attr_algo.attribute(input), ...) captum.ai

GradientSHAP DeepLiftSHAP SHAP Methods Integrated Gradients Saliency GuidedGradCam Attribute model output (or internal neurons) to input features LayerGradientSHAP LayerDeepLiftSHAP SHAP Methods LayerConductance InternalInfluence GradCam Attribute model output to the layers of the model DeepLift NoiseTunnel (Smoothgrad, Vargrad, Smoothgrad Square) LayerActivation LayerGradientXActivationLayerDeepLiftFeatureAblation / FeaturePermutation GuidedBackprop / Deconvolution AT TRIBUTION ALGORITHMS Input * Gradient LayerFeatureAblation LayerIntegratedGradients Occlusion Shapely Value Sampling Gradient Perturbation Other

attributions = Attribution(forward_func, ...).attribute(inputs, ...)* * Check out our Getting Started docs and API: https://github.com/pytorch/captum https://captum.ai/api/ GradientAttribution PerturbationAttribution IntegratedGradients FeatureAblation ... ...

attributions = Attribution(forward_func, ...).attribute(inputs, ...)* * Check out our Getting Started docs and API: https://github.com/pytorch/captum https://captum.ai/api/ the importance of inputs to forward_func model's forward function or any modification of it

EXPL AINING WITH INTEGRATED GRADIENTS FEATURE 0 FEATURE 1 FEATURE 2 TARGET 0 TARGET 1 from captum.attr import IntegratedGradients attr_algo = IntegratedGradients(model) input = torch.rand(1, 3) attributions = attr_algo.attribute(input, target=0)

EXPL AINING WITH INTEGRATED GRADIENTS from captum.attr import IntegratedGradients attr_algo = IntegratedGradients(model) input = torch.rand(1, 3) attributions = attr_algo.attribute(input, target=0) FEATURE 0 FEATURE 1 FEATURE 2 TARGET 0 TARGET 1

EXPL AINING WITH INTEGRATED GRADIENTS from captum.attr import IntegratedGradients attr_algo = IntegratedGradients(model) input = torch.rand(1, 3) attributions = attr_algo.attribute(input, target=0) FEATURE 0 FEATURE 1 FEATURE 2 TARGET 0 TARGET 1 OUTPUT attributions: tensor([[-0.41, 0.54, 0.88]])

EXPL AINING WITH INTEGRATED GRADIENTS from captum.attr import IntegratedGradients attr_algo = IntegratedGradients(model) input = torch.rand(1, 3) attributions, delta = attr_algo.attribute(input, target=0, return_convergence_delta=True) FEATURE 0 FEATURE 1 FEATURE 2 TARGET 0 TARGET 1 OUTPUT attributions: tensor([[-0.41, 0.54, 0.88]]) delta: 0.0190

EXPL AINING WITH INTEGRATED GRADIENTS from captum.attr import IntegratedGradients attr_algo = IntegratedGradients(model) input = torch.rand(1, 3) baseline = torch.rand(1, 3) attributions, delta = attr_algo.attribute(input, target=0, return_convergence_delta=True, n_steps=5000, baselines=baselines) FEATURE 0 FEATURE 1 FEATURE 2 TARGET 0 TARGET 1 OUTPUT attributions: tensor([[0.0, 0.88, -2.45]]) convergence delta: 1.5497e-06

ORIGINAL IMAGE ATTRIBUTING* TO DOG ATTRIBUTING* TO CAT * MATTHEW D ZEILER, ROB FERGUS, OCCLUSION: VISUALIZING AND UNDERSTANDING CONVOLUTIONAL NETWORKS, IN SPRINGER INTERNATIONAL PUBLISHING SWITZERLAND, 2014 VISUALIZATIONS USING RESNET152 MODEL

VISUALIZING EXPL ANATIONS OF A TEXT CL ASSIFICATION MODEL USING IMDB DATASET WITH CAPTUM INSIGHTS

VISUALIZING EXPL ANATIONS OF MULTIMODAL VQA MODELS WITH CAPTUM INSIGHTS

VISUALIZING EXPL ANATIONS OF A 3-L AYER MLP MODEL USING TITANIC DATASET WITH CAPTUM INSIGHTS

EXPL AINING BERT MODELS + Fine-tuning BERT model for Question Answering on SQUAD dataset + Evaluating on Dev Set Exact Match: 78% F1-Score: 86% + Understanding the importance of different types of word tokens, layers and neurons + Already existing research in understanding and visualizing attention heads + What Does BERT Look At? An Analysis of BERT's Attention, Clark, et. al. 2019, BlackBoxNLP@ACL + ExBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models, Hoover, et. al., 2019,

EXPL AINING BERT MODELS FOR QUESTION ANSWERING text = 'It is important to us to include, empower and support humans of all kinds.' question = 'What is important to us?' [CLS] tokens what [SEP]to isimportant ?is us it important to us to include em, and support humans of all kinds .##power P(Start Position) = 0.72 P(End Position) = 0.73 [SEP]

# explaining layers for i in range(model.config.num_hidden_layers): lc = LayerConductance(squad_pos_forward_func, model.bert.encoder.layer[i]) layer_attributions_start = lc.attribute( input_embed, baselines=ref_emb, ..., 0)) layer_attributions_end = lc.attribute( input_embed, baselines=ref_emb, ..., 1)) EXPL AINING BERT MODELS FOR QUESTION ANSWERING

AT TRIBUTION HEAT MAP OF ALL TOKENS ACROSS ALL 12 BERT L AYERS FOR START POSITION PREDICTION

AT TRIBUTION HEAT MAP OF ALL TOKENS ACROSS ALL 12 BERT L AYERS FOR END POSITION PREDICTION

THE LIMITATIONS OF AT TRIBUTIONS + Attributions do not capture feature correlations and interactions + Finding good baselines is challenging + They are difficult to evaluate + Attributions do not explain the model globally

FUTURE DIRECTIONS + captum.robust + adversarial robustness and attacks + studying the connections between model robustness and interpretability + captum.metrics + model interpretability, sensitivity, trust, infidelity and robustness related metrics + captum.benchmarks + benchmarks for different datasets and methodologies + sanity checks + captum.optim + optimization-based visualizations ...

S E C U R E & P R I V A C Y P R E S E R V I N G A I

IS IT POSSIBLE TO: answer questions using data we cannot see?

What do handwritten  digits look like? ◆ Step 1: Download data ◆ Step 2: Download SOTA training script ◆ Step 3: Run script.

Source: Wikipedia Commons What do tumors look like in humans? ◆ Step -1: Persuade a VC. ◆ Step 0: Buy a dataset from a hospital. ◆ Step 1: Download millions of tumor images.

Getting access to private data is HARD!

We SOLVE tasks which are accessible: ✓ ImageNet ✓ MNIST ✓ CIFAR-10 ✓ Librispeech ✓ WikiText-103 ✓ WMT ◆ Cancer ◆ Alzheimers ◆ Dementia ◆ Depression ◆ Anxiety ◆ … Covid-19 Cure? … but what about?

TOOLS + Remote Execution + OpenMined PySyft + Search and Example Data + OpenMined PyGrid + Differential Privacy + OpenMined PyDP + Secure Multi-Party Communication + CrypTen.ai

CRYPTEN import crypten import torch crypten.init() # sets up communication x = torch.tensor([1.0, 2.0, 3.0]) x_enc = crypten.cryptensor(x) # encrypts tensor x_dec = x_enc.get_plain_text() # decrypts tensor assert torch.all_close(x_dec, x) # this passes! y_enc = crypten.cryptensor([2.0, 3.0, 4.0]) xy_enc = x_enc + y_enc # adds encrypted tensors xy_dec = xy_enc.get_plain_text() assert torch.all_close(xy_dec, x + y) # this passes! z = torch.tensor([4.0, 5.0, 6.0]) xz_enc = x_enc + z # adds FloatTensor to CrypTensor xz_dec = xz_enc.get_plain_text() assert torch.all_close(xz_dec, x + z) # this passes! K E Y F E AT U R E S : • Tensors and CrypTensors coexist and can be mixed and matched • Uses standard eager execution — No compilers! Easy debugging and learning • Support for Secure multi-party computation (MPC) A platform for research in machine learning using secure-computation techniques

HELLO CRYPTENSOR 1. CrypTensor wraps an implementation that does: 1. Arithmetic secret sharing. 2. XOR secret sharing. 3. Conversions between both secret sharings. 4. A large number of operations. 2. CrypTensor exposes these via a PyTorch-like API. PyTorch LongTensor Binary (XOR) Sharing CrypTensor Arithmetic Sharing Trusted Party Numerical Library Secure Computation Primitives Secure Computation Protocol Protocol-Independent Layer uses abstracts uses AutogradCrypTensor MPCTensor B2A/A2B Conversion uses Automatic di erentiation User-level code Neural networks, etc. uses Parties

1. Make a CrypTen Model. 2. Encrypt Data 3. Train! ENCRYPTED TRAINING import crypten crypten.init() # sets up communication class LogisticRegression(crypten.nn.Module): def __init__(self): super().__init__() self.linear = crypten.nn.Linear(28 * 28, 10) def forward(self, x): return self.linear(x) model = LogisticRegression().encrypt() # encrypts tensor

1. Join Encrypted Data 2. Encrypt Model 3. Train! Training Across Par ties import crypten crypten.init() # sets up communication alice_images_enc = crypten.load("/tmp/data/alice_images.pth", src=ALICE) bob_labels_enc = crypten.load("/tmp/data/bob_labels.pth", src=BOB) model = LogisticRegression().encrypt() train_model(model, alice_images_enc, bob_labels_enc)

1. Create a PyTorch or ONNX model. 2. Import model into CrypTen. 3. All computations are now encrypted. PY TORCH / ONNX INTEGRATION import torchvision.datasets as datasets import torchvision.models as models # download and set up ImageNet dataset: transform = transforms.ToTensor() dataset = datasets.ImageNet( imagenet_folder, transform=transform, ) # download pre-trained ResNet-18 model and encrypt it: model = models.resnet18(pretrained=True) encrypted_model = crypten.nn.from_pytorch( model, dataset[0], ) # do inference on encrypted images with encrypted model: encrypted_image = crypten.cryptensor(dataset[1]) encrypted_output = encrypted_model(encrypted_image) output = encrypted_output.get_plain_text() # this works

USE CASES + COVID-19 Sols + Cancer Research + Integrity (eg PhotoDNA project) + Federated AI across Enterprise Silos + What problems will you solve?

• Captum: https://captum.ai/ • Captum Blog: https://bit.ly/2vHBxJI • Captum Algorithms Matrix: https://captum.ai/docs/algorithms_comparison_matrix • Interpreting MultiModal models: https://captum.ai/tutorials/Multimodal_VQA_Interpret • Interpretable ML Book: https://christophm.github.io/interpretable-ml-book/ • Crypten: https://crypten.ai/ • CrypTen Tutorials: https://github.com/facebookresearch/CrypTen#how-crypten-works • OpenMined: https://www.openmined.org/ • OpenMined for Covid-19 Apps: https://blog.openmined.org/providing-opensource-privacy-for-covid19/ • Udacity Course: https://www.udacity.com/course/secure-and-private-ai--ud185 • Active Federated Learning Paper: https://arxiv.org/pdf/1909.12641.pdf • Microsoft PhotoDNA Project: https://www.microsoft.com/en-us/photodna REFERENCES

QUESTIONS? Contact: Email: gchauhan@fb.com Linkedin: https://www.linkedin.com/in/geetachauhan/

Building Interpretable & Secure AI Systems using PyTorch

More Related Content

What's hot

Similar to Building Interpretable & Secure AI Systems using PyTorch

More from geetachauhan

Recently uploaded

Building Interpretable & Secure AI Systems using PyTorch