## Issue

Hi everyone i came across an example of how to use shap on lstm Time-step wise feature importance in deep learning using SHAP. I’m curious why the author chose to use

```
e = shap.DeepExplainer((regressor.layers[0].input,
regressor.layers[-1].output),data)
```

instead of just

```
e = shap.DeepExplainer(regressor,data)
```

I suspect the reason is very important but I cannot be sure. Anyone can shed some light on this?

Partial code below

```
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from keras.models import load_model
import shap
regressor = load_model(‘lstm_stock.h5’)
pred_x = regressor.predict_classes(X_train)
random_ind = np.random.choice(X_train.shape[0], 1000,
replace=False)
print(random_ind)
data = X_train[random_ind[0:500]]
e = shap.DeepExplainer((regressor.layers[0].input,
regressor.layers[-1].output),data)
test1 = X_train[random_ind[500:1000]]
shap_val = e.shap_values(test1)
shap_val = np.array(shap_val)...
```

## Solution

The solution is quite simple. Let’s look at the DeepExplainer documentation. This is the `__init__`

function:

```
__init__(model, data, session=None, learning_phase_flags=None)
```

Your confusion is about the first argument, that is, `model`

. According to the documentation, for Tensorflow, `model`

is:

a pair of TensorFlow tensors (or a list and a tensor) that specifies

the input and output of the model to be explained.

So that’s it. The first argument is just a pair indicating the input and the output of the model. In this case:

```
(regressor.layers[0].input, regressor.layers[-1].output)
```

**Update:**

In the example Front Page DeepExplainer MNIST Example it is however shown this piece of code:

```
import shap
import numpy as np
# select a set of background examples to take an expectation over
background = x_train[np.random.choice(x_train.shape[0], 100, replace=False)]
# explain predictions of the model on three images
e = shap.DeepExplainer(model, background)
# ...or pass tensors directly
# e = shap.DeepExplainer((model.layers[0].input, model.layers[-1].output), background)
shap_values = e.shap_values(x_test[1:5])
```

From this it seems that, besides the pair of tensors, it is also possible to use the `model`

as input, just like it is possible for Pytorch. Indeed in Pytorch, instead of the pair, you can also use a `nn.Module`

object.

At this point then I guess that the one for Tensorflow is just an undocumented feature. It should then be equivalent to use directly the model or the pair of tensors.

Answered By – claudia