Quick start¶
This page interprets a small PyTorch network trained on scikit-learn's digits dataset, layer by
layer. The full script is
examples/quickstart_digits.py.
1. Keep a copy of the untrained model¶
Every reading needs a reference to be meaningful. The natural one is the same network before training, so copy it before the first optimisation step:
import copy
import numpy as np
import torch
import torch.nn as nn
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
X, y = load_digits(return_X_y=True)
X = (X / 16.0).astype(np.float32)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, stratify=y, random_state=0)
torch.manual_seed(0)
model = nn.Sequential()
model.add_module("block1", nn.Sequential(nn.Linear(64, 128), nn.ReLU()))
model.add_module("block2", nn.Sequential(nn.Linear(128, 128), nn.ReLU()))
model.add_module("block3", nn.Sequential(nn.Linear(128, 64), nn.ReLU()))
model.add_module("head", nn.Linear(64, 10))
initial = copy.deepcopy(model) # the floor: the same network before training
2. Train as usual¶
opt = torch.optim.Adam(model.parameters(), 1e-3)
xt, yt = torch.from_numpy(X_tr), torch.from_numpy(y_tr)
for epoch in range(60):
for idx in torch.randperm(len(xt)).split(64):
opt.zero_grad()
nn.functional.cross_entropy(model(xt[idx]), yt[idx]).backward()
opt.step()
model.eval()
3. Build the interpreter¶
Each 64-pixel image is a row of 64 features here, so the entry point is
fannar.tabular:
import fannar as fa
from fannar.tabular import TabularInterpreter
interp = TabularInterpreter(
model, X_te, y_te, # the trained model and the objects to interpret
layers=["block1", "block2", "block3"], # names from model.named_modules()
reference_model=initial, # optional, but recommended: the floor
feature_names=[f"pixel{t}" for t in range(64)],
)
TabularInterpreter is an fa.Interpreter with defaults for tables. fa.Interpreter takes the
same arguments and gives the same numbers. For images, token sequences and trees, see
Choosing a module.
Nothing is computed yet. The first call that needs the spaces runs interp.fit(), which
extracts the sources of every layer in one pass over X_te and builds one
GramSpace per layer.
4. Read the report¶
report = interp.evaluate()
print(f"test accuracy: {report.meta['accuracy']:.3f}")
print(report.to_markdown()) # also report.to_frame() and report.to_json("r.json")
interp.plot_report(report) # readings × layers
For each layer the report holds the readings of three representations:
joint: the method, with the three sources combined;activations: a baseline with the activations alone;floor: the same pipeline on the untrained network, measured against the trained network's behaviour.
A reading only says something about training when joint is above floor. What each reading
asks is described in Readings and reports.
5. Look at the spaces¶
interp.plot_embedding() # each layer on its two dominant modes
interp.plot_gram("block2") # component Grams, Hadamard product, final Gram
interp.plot_spectrum("block2") # eigenvalues and participation rank
space = interp.space("block2") # the GramSpace itself
space.neighbors(0, 5)
space.energies()["E_ratio"]
6. Explain decisions¶
expl = interp.explain_pair(0) # object 0 vs. its nearest object with another prediction
for name, vi, vj, share in expl.top(5):
print(f"{name:8s} i = {vi:.2f} j = {vj:.2f} share {share:.0%}")
expl.plot()
errs = interp.explain_errors() # which features keep the errors away from their class
errs.top(5)
errs.plot()
interp.swap_test(30, layer="block3") # validation outside the geometry
See Explanations for what these numbers mean and how they are validated.
Saving figures¶
Every plot_* method returns a Matplotlib figure:
Next steps¶
- Examples: one runnable script per kind of data, with its output.
- The method: what the three sources are and why the defaults are what they are.
- Choosing a module:
fannar.tabular,fannar.image,fannar.text,fannar.tree, and any model through a function.