Interpreter¶
Interpreter
¶
Interpreter(
model,
X,
y=None,
*,
layers: Sequence[str] | None = None,
pipeline: KernelPipeline | str = "paper",
reference_model=None,
baselines: Sequence[KernelPipeline | str] | dict = ("activations",),
extractor: Callable | None = None,
feature_names: Sequence[str] | None = None,
class_names: Sequence[str] | None = None,
k: int = 5,
**source_options,
)
Builds the representation space of a trained model over chosen objects, per layer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
A trained PyTorch classifier, a fitted scikit-learn decision tree, or anything else when
|
required | |
X
|
The objects to interpret (array, tensor or, for PyTorch, a DataLoader). |
required | |
y
|
True labels of |
None
|
|
layers
|
Sequence[str] | None
|
Layers to probe (PyTorch module names); ignored for trees and custom extractors. |
None
|
pipeline
|
KernelPipeline | str
|
A |
'paper'
|
reference_model
|
Optional model of the same architecture used as the floor (typically the same network with its initial weights). Its geometry is measured against the trained model's behaviour. |
None
|
|
baselines
|
Sequence[KernelPipeline | str] | dict
|
Other pipelines or preset names evaluated alongside (default: activations alone). |
('activations',)
|
extractor
|
Callable | None
|
|
None
|
**source_options
|
Passed to |
{}
|
Source code in src\fannar\interpreter.py
feature_names
instance-attribute
¶
class_names
instance-attribute
¶
extractor
instance-attribute
¶
reference_extractor
instance-attribute
¶
reference_extractor = (
None
if reference_model is None
else self._make_extractor(reference_model, None, layers, source_options)
)
fit
¶
Extracts the sources and builds the representation space of every layer.
Source code in src\fannar\interpreter.py
floor_space
¶
floor_space(layer: str) -> GramSpace
The same pipeline on the reference (e.g. untrained) model.
Source code in src\fannar\interpreter.py
space_from_grams
¶
space_from_grams(
grams: dict[str, ndarray],
pipeline: KernelPipeline | None = None,
name: str = "custom",
) -> GramSpace
A space from Gram matrices you built yourself ({component: (n, n)}), composed by
pipeline (default: the interpreter's). Components not in grams must be sources.
Source code in src\fannar\interpreter.py
evaluate
¶
evaluate(
layers: Sequence[str] | None = None,
include_baselines: bool = True,
include_floor: bool = True,
) -> Report
Behaviour readings per layer for the joint space, the baselines and the floor.
The floor uses the reference model's geometry against the trained model's behaviour (the "cross" floor), so the only difference between a reading and its floor is training.
Source code in src\fannar\interpreter.py
explain_pair
¶
explain_pair(
i: int, j: int | None = None, layers: Sequence[str] | None = None
) -> PairExplanation
Which features distinguish object i from j for the model (j defaults to the
nearest object with a different prediction). Vector inputs only.
Source code in src\fannar\interpreter.py
explain_errors
¶
explain_errors(
layers: Sequence[str] | None = None,
against: str = "correct",
max_errors: int | None = None,
reference: int = 100,
seed: int = 0,
) -> ErrorExplanation
Which features drive the model's errors, aggregated over the misclassified objects.
Each error i is explained as a pair against a partner j: with against="correct"
(default) the nearest object correctly predicted as the true class of i (falling back
to the nearest unlike object when that class has none), so the shares say which features
keep i away from its class in the representation space; with against="unlike", the
nearest object with another prediction, as in explain_pair. max_errors caps the
number of errors and reference the number of correctly predicted objects explained the
same way as a baseline (both sampled with seed; reference=0 skips it).
Vector inputs only.
Source code in src\fannar\interpreter.py
swap_test
¶
swap_test(
pairs: int | Sequence[tuple[int, int]] = 50,
layer: str | None = None,
predict: Callable | None = None,
seed: int = 0,
) -> dict
Swap-test area per ranking (method, |Delta x|, random among differing features),
averaged over pairs. pairs is a number of random correctly-predicted objects (each with
its nearest unlike neighbour) or explicit (i, j) pairs.
Source code in src\fannar\interpreter.py
energy_map
¶
Retained spectral energy of every position of one input (e.g. an image), as a map.
Positions are the objects; components use linear kernels without the angular transform (which would give every position the same energy).
Source code in src\fannar\interpreter.py
plot_embedding
¶
plot_embedding(
layers: Sequence[str] | None = None,
color_by: str = "prediction",
include_floor: bool = True,
include_baselines: bool = True,
)