online learning using ssm

online linear regression

Sequential Bayesian inference for the parameters of a linear regression model. Treat the parameters of the model as the unknown hidden states (${\mathbf{z}}_t={\theta }_t^T$). The parameters are updated with each new measurement.

This is a linear gaussian ssm with $\mathbf{F}=\mathbb{I}$, $\mathbf{Q}=0$, ${\mathbf{H}}_t={\mathbf{x}}_t^T$, $\mathbf{R}={\sigma }^2$.

$$\begin{array}{rl}{\theta }_t& ={\theta }_{t-1}\\ y_t& ={\mathbf{x}}_t^T{\theta }_t+r_t,r_t\sim N(0,{\sigma }^2)\end{array}$$

online learning for a neural network

Sequential Bayesian inference for the parameters of a multi-layer dense neural network (perceptron) for regression. Treat the parameters (weights and biases) of the network as the unknown hidden states.

$$\begin{array}{rl}{\theta }_t& ={\theta }_{t-1}\\ y_t& =h({\theta }_t,x_t)+r_t,r_t\sim N(0,{\sigma }^2)\end{array}$$

This is a nonlinear gaussian ssm, where $h$ is the nonlinear observation model defined by a neural network. Add a small amount of Gaussian drift for numerical stability ${\mathbf{q}}_t\sim N(0,0.01\mathbb{I})$. Inference can be done using the extended Kalman filter.

The parameters are updated with each new measurement (video of training).

The model can be generalised to other likelihoods, such as Bernoulli for classification, which is then a generalised Gaussian SSM.