> For the complete documentation index, see [llms.txt](https://deeplearningpytorch.gitbook.io/dlpt/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://deeplearningpytorch.gitbook.io/dlpt/interpretability.md).

# Interpretability

## Visualizing and Understanding Convolutional Networks

<https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf>

## Learning Deep Features for Discriminative Localization (Class activation mapping, CAM)

<http://cnnlocalization.csail.mit.edu/Zhou_Learning_Deep_Features_CVPR_2016_paper.pdf>

Class activation mapping (CAM)

* A weakly supervised localization method.
* If the last three layers are "convolution + global average pooling+ full connection), then you can apply this method to visualize object location.
* Just use fc's weight and calculate weighted sum of convolution feature maps (without global average pooling).

![](/files/-LJ4u9Ay_rDyrkni1-ul) *Class activation mapping*

Let $$a\_1,..., a\_k$$ be the k feature maps of the last convolution layer and followed by GAP and FC layers, and $$M$$is the size of a feature map, and $$w\_n$$ is the weight which connected n-th filter to one of the scores called "s".

$$
s = \sum\_{n=1}^k (w\_n \frac{\sum\_{ij} (a\_n)*{ij}}{M})=\frac{1}{M}\sum*{ij} (\sum\_{n=1}^kw\_na\_n)\_{ij}
$$

It changed the layers' order from "conv->GAP->FC" to "conv->FC->GAP". The following simple equation is called the "class activation mapping" of class "&#x73;**"**.

$$
\text{CAM}=\sum\_{n=1}^kw\_na\_n
$$