Skip to content

Choosing a module

Since 0.4.0 there is one entry point per kind of data. Each is a thin Interpreter with sensible defaults and the same methods, and it gives the same numbers as fa.Interpreter with the same options:

data / model module class guide
PyTorch model on tabular rows fannar.tabular TabularInterpreter Tabular data
PyTorch CNN or ViT on images fannar.image ImageInterpreter Images
PyTorch transformer on tokens (Hugging Face included) fannar.text TextInterpreter Text
scikit-learn decision tree fannar.tree TreeInterpreter Decision trees
anything else fannar Interpreter(extractor=...) below
import fannar as fa
fa.tabular.TabularInterpreter    # the modules load on first access; no extra import needed

fa.Interpreter still picks the sources from the type of the model, as in earlier versions:

model sources "layers"
torch.nn.Module TorchSources: activations, gradients, parameters the modules in layers
scikit-learn tree (has tree_ and decision_path) TreeSources: path, slack, classes each depth, depth1 … depthD
anything else your extractor function whatever your function returns

PyTorch options

These options apply to TabularInterpreter, ImageInterpreter, TextInterpreter and to fa.Interpreter with a PyTorch model. Any classifier that returns logits (N, K) works. Name the modules to probe as they appear in model.named_modules(); with layers=None, every direct child that has parameters is probed.

interp = fa.Interpreter(model, X_test, y_test, layers=["layer1", "layer2", "layer3"],
                        reference_model=initial)

X can be an array, a tensor or a DataLoader (the first element of each batch is used). The keyword arguments that Interpreter does not know are passed to TorchSources:

option default meaning
target "fixed" class of the contrastive margin: "fixed", "predicted" or an int
fixed_class 0 the class used by target="fixed"
grid 4 (8 in TextInterpreter) positions kept per spatial or token axis (None keeps all)
layout "channels" ("tokens" in TextInterpreter) "tokens" for outputs shaped (N, L, C)
sources all three a subset of ("parameters", "activations", "gradients")
weight_modules automatic {layer: module or name} whose weight defines the parameter source
margin "contrastive" or a callable fn(logits, target) -> (N,)
device, batch_size model's device, 256

The parameter source

The parameter source needs the weight W that produces the probed output. By default it is the probed module's own weight or, inside a block, the last module up to it whose weight has as many rows as the output has channels (conv2 in a residual block, the Linear before a probed activation). When none fits, the parameter source is skipped with a warning; point to the right module with weight_modules={"layer": "path.to.module"}.

Any other model

Give extractor, a function that returns the sources of a batch of objects and the model's predictions:

def sources(X):
    layers = {"output": {"proba": model.predict_proba(X), "leaves": model.apply(X)}}
    return layers, model.predict(X)                 # optionally a third item: probabilities

interp = fa.Interpreter(model, X_test, y_test, extractor=sources,
                        pipeline=fa.KernelPipeline(["proba", "leaves"]),
                        baselines={})

The function can return (layers, predictions), (layers, predictions, probabilities) or a Sources object. layers maps each layer name to {source name: array}, with the objects on the first axis. The probabilities feed the soft confusion of the confusion_agreement reading; without them the confusion is counted.

The default baseline is the activations source alone. It is skipped in the layers where your function provides no activations, so pass baselines={} to say so explicitly, or baselines built from the sources you do provide.

Pair explanations with your own extractor

explain_pair, explain_errors and swap_test call your function on new rows (the object with one feature swapped), so it must work on any (m, F) float32 array, not only on X_test. swap_test also needs predict=, a function returning labels, for models that are neither PyTorch nor scikit-learn trees.