Skip to content

Kernels

Positive-definite kernels that turn one source of information into a raw Gram matrix.

Every kernel maps a set of objects X (any array-like whose first axis indexes the objects; the remaining axes are flattened) to the raw Gram matrix K[i, j] = kappa(x_i, x_j). The method only requires each kernel to be positive definite; which kernel to use is part of the instantiation, not of the method.

A word on composition. The joint Gram is the Hadamard product of the component Grams. With linear-type kernels this realises the tensor product of the feature spaces, so the joint similarity keeps the multiplicative interaction between sources. With Gaussian kernels it does not: the product of two Gaussians is the Gaussian of the summed exponents, and the composition collapses into an additive combination of the source metrics. RBF is available, but Linear/Cosine are the default for this reason.

Linear

Bases: Kernel

kappa(x, y) = <x, y>. Combined with the angular transform it becomes the cosine.

name class-attribute instance-attribute

name = 'linear'

compute

compute(X, Y)
Source code in src\fannar\kernels.py
def compute(self, X, Y):
    return X @ Y.T

Cosine

Cosine(eps: float = 1e-12)

Bases: Kernel

kappa(x, y) = <x, y> / (|x| |y|); zero vectors get similarity 0 (1 with themselves).

Source code in src\fannar\kernels.py
def __init__(self, eps: float = 1e-12) -> None:
    self.eps = eps

name class-attribute instance-attribute

name = 'cosine'

eps instance-attribute

eps = eps

compute

compute(X, Y)
Source code in src\fannar\kernels.py
def compute(self, X, Y):
    nx = np.linalg.norm(X, axis=1)
    ny = np.linalg.norm(Y, axis=1)
    k = (X @ Y.T) / np.maximum(np.outer(nx, ny), self.eps)
    if X is Y:
        np.fill_diagonal(k, 1.0)
    return k

RBF

RBF(gamma: float | None = None)

Bases: Kernel

Gaussian kernel exp(-gamma |x - y|^2). gamma=None uses the median heuristic.

See the module docstring: under Hadamard composition, Gaussian components add their metrics instead of multiplying the similarities.

Source code in src\fannar\kernels.py
def __init__(self, gamma: float | None = None) -> None:
    self.gamma = gamma

name class-attribute instance-attribute

name = 'rbf'

gamma instance-attribute

gamma = gamma

compute

compute(X, Y)
Source code in src\fannar\kernels.py
def compute(self, X, Y):
    sx = (X**2).sum(1)[:, None]
    sy = (Y**2).sum(1)[None, :]
    d2 = np.maximum(sx + sy - 2 * X @ Y.T, 0.0)
    gamma = self.gamma
    if gamma is None:
        med = np.median(d2[d2 > 0]) if np.any(d2 > 0) else 1.0
        gamma = 1.0 / med
    return np.exp(-gamma * d2)

Polynomial

Polynomial(degree: int = 2, coef0: float = 1.0, gamma: float | None = None)

Bases: Kernel

kappa(x, y) = (gamma <x, y> + coef0) ** degree with coef0 >= 0.

Source code in src\fannar\kernels.py
def __init__(self, degree: int = 2, coef0: float = 1.0, gamma: float | None = None) -> None:
    if coef0 < 0:
        raise ValueError("coef0 must be >= 0 for the kernel to be positive definite")
    self.degree, self.coef0, self.gamma = degree, coef0, gamma

name class-attribute instance-attribute

name = 'polynomial'

compute

compute(X, Y)
Source code in src\fannar\kernels.py
def compute(self, X, Y):
    gamma = self.gamma if self.gamma is not None else 1.0 / X.shape[1]
    return (gamma * (X @ Y.T) + self.coef0) ** self.degree

FunctionKernel

FunctionKernel(fn: Callable[[ndarray, ndarray], ndarray], name: str | None = None)

Bases: Kernel

Wraps a user function fn(X, Y) -> (n, m) array. Positive definiteness is the user's.

Source code in src\fannar\kernels.py
def __init__(self, fn: Callable[[np.ndarray, np.ndarray], np.ndarray], name: str | None = None) -> None:
    self.fn = fn
    if name:
        self.name = name

name class-attribute instance-attribute

name = 'function'

fn instance-attribute

fn = fn

compute

compute(X, Y)
Source code in src\fannar\kernels.py
def compute(self, X, Y):
    return np.asarray(self.fn(X, Y), dtype=np.float64)

Kernel

Bases: ABC

Base class. Subclasses implement compute on (n, D) and (m, D) arrays.

name class-attribute instance-attribute

name = 'kernel'

compute abstractmethod

compute(X: ndarray, Y: ndarray) -> ndarray

Raw Gram block K[i, j] = kappa(X[i], Y[j]).

Source code in src\fannar\kernels.py
@abstractmethod
def compute(self, X: np.ndarray, Y: np.ndarray) -> np.ndarray:
    """Raw Gram block ``K[i, j] = kappa(X[i], Y[j])``."""

as_kernel

as_kernel(obj) -> Kernel

Accepts a Kernel, a name ("linear", "cosine", "rbf", "polynomial") or a callable fn(X, Y).

Source code in src\fannar\kernels.py
def as_kernel(obj) -> Kernel:
    """Accepts a :class:`Kernel`, a name (``"linear"``, ``"cosine"``, ``"rbf"``, ``"polynomial"``)
    or a callable ``fn(X, Y)``."""
    if isinstance(obj, Kernel):
        return obj
    if isinstance(obj, str):
        try:
            return _BY_NAME[obj.lower()]()
        except KeyError as exc:
            raise ValueError(f"unknown kernel {obj!r}; choose from {sorted(_BY_NAME)}") from exc
    if callable(obj):
        return FunctionKernel(obj)
    raise TypeError(f"cannot interpret {obj!r} as a kernel")