Skip to content

Your own Gram matrices

The construction does not depend on neural networks. Any set of objects with one or more sources of information can go through a KernelPipeline, and any component can be your kernel, your transform, or a matrix you already computed. The full script is examples/custom_gram.py.

A pipeline

import numpy as np
import fannar as fa

pipe = fa.KernelPipeline(
    {
        "state":  fa.Component(fa.Linear(), fa.angular() >> fa.shift(1.0)),
        "text":   fa.Component(fa.RBF(gamma=0.1), fa.identity),
        "mine":   fa.Component(lambda A, B: (A @ B.T + 1) ** 2),   # any function k(X, Y)
    },
    final=fa.center >> fa.trace_normalize,
)

space = pipe(sources={"state": S, "text": T, "mine": M}, labels=y)

Each source is an array whose first axis indexes the objects; the other axes are flattened. NumPy arrays, PyTorch tensors and lists are accepted.

There are three shorter ways to declare the components:

fa.KernelPipeline(["a", "b"])                     # default component for each name
fa.KernelPipeline({"a": "linear", "b": "cosine"}) # a kernel name per source
fa.KernelPipeline({"a": fa.Polynomial(degree=3)}) # a kernel object per source

The default component is the one of the paper: linear kernel, then angular() >> shift(1.0). The default final transform is center >> trace_normalize.

Matrices you already have

A raw Gram passed in grams takes precedence over the kernel applied to sources, so you can replace any component, or all of them:

space = pipe(sources={"state": S, "text": T}, grams={"mine": K_mine})   # mix
space = pipe(grams={"state": K1, "text": K2, "mine": K3})               # matrices only

The component transform is still applied to a matrix given in grams. To skip the construction altogether, build the space directly from any PSD matrix:

space = fa.GramSpace(my_psd_matrix, labels=y)

With an Interpreter, interp.space_from_grams({...}) does the same with the model's labels and predictions attached.

Kernels

kernel \(\kappa(x, y)\) notes
Linear() \(\langle x, y \rangle\) with angular(), the cosine
Cosine(eps=1e-12) \(\langle x, y \rangle / (\lVert x \rVert \lVert y \rVert)\) zero vectors get similarity 0 (1 with themselves)
RBF(gamma=None) \(\exp(-\gamma \lVert x - y \rVert^2)\) gamma=None uses the median heuristic
Polynomial(degree=2, coef0=1, gamma=None) \((\gamma \langle x, y \rangle + c_0)^{p}\) coef0 >= 0; gamma defaults to 1/D
FunctionKernel(fn, name=None) fn(X, Y) any callable; positive definiteness is up to you

A plain function fn(X, Y) -> (n, m) array is wrapped in FunctionKernel automatically. For a reusable kernel, subclass Kernel and implement compute on (n, D) and (m, D) arrays:

class Laplacian(fa.Kernel):
    name = "laplacian"

    def __init__(self, gamma=1.0):
        self.gamma = gamma

    def compute(self, X, Y):
        d = np.abs(X[:, None, :] - Y[None, :, :]).sum(-1)
        return np.exp(-self.gamma * d)

Tip

Prefer Linear or Cosine when you want the interaction between sources: under the Hadamard product, Gaussian components add their metrics instead of multiplying the similarities. See The method.

Transforms

Transforms compose with >>, from left to right: fa.center >> fa.trace_normalize centres and then normalises.

transform effect cost
identity \(G\)
center \(HGH\), with \(H = I - \frac1n J\) \(O(n^2)\)
trace_normalize \(G / \operatorname{tr} G\) \(O(n)\)
frobenius_normalize \(G / \lVert G \rVert_F\) \(O(n^2)\)
max_eig_normalize \(G / \lambda_{\max}\) \(O(n^3)\)
angular(eps=1e-12) \(D^{-1/2} G D^{-1/2}\) (unit diagonal; 0 for null objects) \(O(n^2)\)
shift(c=1.0) \((G + cJ)/(1 + c)\), c >= 0 \(O(n^2)\)
whiten(tol=1e-10) projector onto the range of \(G\) \(O(n^3)\)
spectral(f) \(V f(\Lambda) V^\top\), with \(f \ge 0\) on \(\lambda \ge 0\) \(O(n^3)\)
psd_project() nearest PSD matrix (negative eigenvalues set to 0) \(O(n^3)\)

identity, center, trace_normalize, frobenius_normalize and max_eig_normalize are transform objects; the others are functions that return one, so they take parentheses.

A transform of your own is any function of a symmetric matrix wrapped in Transform. It is your responsibility that it is admissible (PSD in, PSD out, independent of the order of the objects):

sqrt_spectrum = fa.spectral(np.sqrt, name="sqrt")         # V Λ^{1/2} Vᵀ
square = fa.Transform(lambda G: G * G, "hadamard_square")  # PSD by the Schur product theorem

component = fa.Component(fa.Linear(), fa.angular() >> square)

Entry-wise functions are the usual trap: most of them (np.log1p, np.sqrt entry by entry) do not keep a matrix PSD.

Presets

preset what it is
fa.paper_pipeline(sources=("parameters", "activations", "gradients"), shift_by=1.0) the instantiation of the paper
fa.single_source("activations", angular_shift=True) one source with the paper's component transform, for ablations
fa.single_source(..., angular_shift=False) the plain linear centred Gram: Euclidean distances, the geometry of linear CKA and RSA
fa.positions_pipeline(sources) positions of one input as objects (used by the energy maps)

Wherever a pipeline is expected, the names in fa.PRESETS also work: "paper", "activations", "gradients", "parameters", "euclidean".

The Hadamard root

KernelPipeline(..., root=True) takes the \(d\)-th Hadamard root of the product of \(d\) components before the final transform: the entry-wise geometric mean of the components. Fractional Hadamard powers do not preserve positive semidefiniteness, so the result is projected back on the PSD cone, and the fraction of spectral mass that projection discarded is stored in space.info["root_discarded"].

Inspecting a pipeline

print(pipe)                       # kernels and transforms of every component
grams = pipe.component_grams(sources={...})   # T_l(K_l) for each component
space.components                  # the same, kept on the space (unless keep_components=False)
space.info["pipeline"]