Contractivity of Neural ODEs: An Eigenvalue Optimization Problem

Contractivity is a key stability property for neural ordinary differential equations: nearby trajectories should move closer together rather than amplify small perturbations. Establishing this property leads to a challenging eigenvalue-optimization problem involving the model’s weight matrices and activation derivatives.

This work introduces a two-level numerical method that searches for the largest activation-derivative interval over which a neural ODE remains contractive. The approach extends from one-layer, weight-tied systems to multilayer and time-dependent neural ODEs.

Numerical experiments illustrate the method’s behaviour and show its stabilising effect on a neural ODE used for image classification.

Paper: https://doi.org/10.1090/mcom/4059