23.linearized-population-equation.md

layout	title	date	author	summary	references	weight
notes	23.Linearized Population Equation and Transients	2016-11-11	OctoMiao	The population equation is quite complicated to solve, hence we linearize it and inspect the perturbation theory.	name Spiking Neuron Models, Section 7.1, 7.2	23

This is the first section of chapter 7, which is about signal transmission and neuronal coding, i.e., reaction of population of neurons to different inputs.

Linearized Population Equation

In this section, we linearize the population equation. More specifically, we develop a perturbation theory of the population equations.

1. Input: $I(t)=I_0+\Delta i(t)$. The input is restrained to be a small perturbation upon a constant input.
2. Population activity: $A(t) = A_0 + \Delta A(t)$. We expect the output to be also not changing dramatically. This might not be true if the equation is non-linear, especially for integro-differential equations. In the examples given in Sec. 6.3, this perturbative output assumption is always true. HOWEVER, WE SHOULD DEFINITELY TAKE NOTE ON THIS CAVEAT.

From Eq. 6.75, we can derive a "equation" that shows how this method works.

$$ \begin{align} & A(t) = \int_{-\infty}^t P_I(t|\hat t) A(\hat t) d\hat t\\ \Rightarrow & A_0 + \Delta A(t) = \int_{-\infty}^t P_I(t|\hat t) A_0 d\hat t + \int_{-\infty}^t P_I(t|\hat t) \Delta A(\hat t) d\hat t\\ \Rightarrow & \Delta A(t) = A_0 ( -1 + \int_{-\infty}^t P_I(t|\hat t) d\hat t ) + \int_{-\infty}^t P_I(t|\hat t) \Delta A(\hat t) d\hat t. \label{eqn-perturbation-1} \end{align} $$

We would also expect the interval distribution is perturbative under small perturbations of input current, i.e.,

$$ \begin{equation} P_I(t\vert \hat t) = P_{I_0}(t\vert \hat t) + \Delta P_{I_0}(t\vert \hat t). \end{equation} $$

For a perturbation method, we drop all terms with high orders. Thus Eq. ($\ref{eqn-perturbation-1}$) becomes

$$ \begin{equation} \Delta A(t) = A_0 \left( -1 + \int_{-\infty}^t P_I(t|\hat t) d\hat t \right) + \int_{-\infty}^t P_{I_0}(t|\hat t) \Delta A(\hat t) d\hat t. \label{eqn-perturbation-2} \end{equation} $$

Some Simplifications

How to calculate the integral $\int_{-\infty}^t P_I(t\vert\hat t) d\hat t$? For models without noise, we should have

$$ P_I(t\vert \hat t) = P_I(t-\hat t), $$

which means the spikes depends on the time interval since last spike. The integral $\int_{-\infty}^t P_I(t\vert\hat t) d\hat t$ becomes

$$ \begin{align} &\int_{-\infty}^t P_I(t\vert\hat t) d\hat t\\ =& - \int_{\hat t = -\infty}^{\hat t=t} P_I(t - \hat t) d (t - \hat t ) \\ =& - \int_{s = \infty}^{s=0} P_I(s) ds\\ =& \int^{\infty}_{0} P_I(s) ds. \end{align} $$

Eq. ($\ref{eqn-perturbation-2}$) reads

$$ \begin{equation} \Delta A(t) = A_0 \left( -1 + \int^{\infty}{0} P_I(s) ds \right) + \int{-\infty}^t P_{I_0}(t|\hat t) \Delta A(\hat t) d\hat t. \label{eqn-perturbation-3} \end{equation} $$

More Simplifications

The problem is reduced to the calculation of $P_I(t\vert\hat t)$. Recall that interval distribution $P_I$ is in fact the time derivative of the survivor function (Sec. 5.2.3). The subscript ${}_I$ indicates the current.

Some review of the definitions.

• Interval distribution is defined as

  $$ \begin{equation} \int_{t_1}^{t_2} dt P_I(t\vert \hat t) = \text{Probability of firing during time interval $[t_1,t_2]$ when the neuron fired at $\hat t$}. \end{equation} $$

• Survivor function is

  $$ \begin{equation} S(t\vert \hat t) = 1 - \int_{\hat t}^t P_I(t\vert \hat t) dt. \end{equation} $$

  We also derived

  $$ \begin{equation} P_I(t\vert \hat t) = - \frac{d}{dt} S_I(t\vert \hat t). \end{equation} $$

• Hazard function is

  $$ \begin{equation} \rho(t\vert\hat t) = -\left(\frac{d}{dt} S(t\vert \hat t)\right)/S(t\vert \hat t) = - P_I(t\vert \hat t)/S_I(t\vert \hat t). \end{equation} $$

  Meanwhile

  $$ \rho(t\vert \hat t) S(t\vert \hat t) = P_I(t\vert \hat t). $$

I have no idea how to proceed from the Eq. ($\ref{eqn-perturbation-3}$).

Anyways, the book postulated a solution with multiple integrals,

$$ \begin{equation} \Delta A(t) = \int_{-\infty}^t P_{I_0}(t-\hat t) \Delta A(\hat t)d\hat t + A_0 \frac{d}{dt}\int_0^\infty \mathcal{L}(x) \Delta h(t-x) dx, \label{eqn-perturbation-final} \end{equation} $$

with

$$ \Delta h(t) = \int_0^\infty \kappa(s) \Delta I(t-s)ds. $$

Interpretations

• The first term in Eq. ($\ref{eqn-perturbation-final}$) describes the dependence on the previous population activity perturbation.

• The second term is the dependence on the input variations.

• Low noise limit: kernel $\mathcal L(x) \to \delta(x)$ (Eq. 6.75 in the book)

  $$ \begin{equation} \Delta A(t) = \cdots + A_0 \frac{d}{dt} \Delta h(t) = \cdots + A_0 \int_0^\infty ds , \kappa (s) \frac{d}{dt} \Delta I(t-s). \end{equation} $$

  This amazing result tells us that any fast change in the input current leads to a large jump in the population activity, regardless of the amplitude of the input current.

• Large noise limit: noise is critical.

  1. Slow noise (adiabatic limit): the response of the system is fast enough to keep track on the noise at any time. This is similar to noise-free limit with noise as the effective input or something. Fast changing input means large change in population activity.
  2. Fast noise: the system is too slow to track the noise. The noise is ambient so that the kernel $\mathcal{L}(x)$ becomes broad. In a limit that the kernel is flat, the population activity depends on the amplitude of potential $\Delta h$.

1. Sec. 7.1.1 proves that the noise-free system indeed depends on the time derivative of the input.
2. Sec. 7.1.2 proves that the escape noise, aka fast noise, indeed provides a broad kernel $\mathcal{L}(x)$.

Rapid change in input might indicate danger for the host of the neural system. This feature might be useful for the survivability of animals.

Transients

The problem to solve in this section is the time course of population activity under a rapid change of input.

Consider step-like input

$$ \begin{equation} I(t) = \begin{cases} I_0 &\qquad t\leq t_0\\ I_0+\Delta I & \qquad t>t_0, \end{cases} \end{equation} $$

which generates step like input potential

$$ \begin{equation} \Delta h(t) = \begin{cases} h_0 & \qquad t\leq t_0 \\ h_0 + \Delta I \int_0^{t-t_0} \kappa_0(s),ds &\qquad t> t_0. \end{cases} \end{equation} $$

As long as the kernel $\kappa$ is defined, we can obtain the time course of every quantity.

We assume the population activity has reached equilibrium (asynchronous firing) before $t_0$. The population activity would also obtain a sudden change

$$ \begin{equation} A(t) = \begin{cases} A_0 &\qquad t\leq t_0\\ A_0+\Delta A(t) &\qquad t>t_0. \end{cases} \end{equation} $$

This change is called transient. During a short time interval $[t_0,t_0+\Delta t]$, we expect the first term in Eq. ($\ref{eqn-perturbation-final}$) doesn't contribute because the network has not react to the change at time $t_0$, i.e., $\Delta A(\hat t)=0$. We have to solve

$$ \begin{equation} \Delta A(t)\sim A_0 \frac{d}{dt}\int_0^\infty \mathcal L(x) \Delta h(t-x) dx, \end{equation} $$

for $t-t_0\ll T$, where $T=1/A_0$ is the mean interspike interval.

As we expected, the short time interval $[t_0,t_0+\Delta t]$ is short enough if $\Delta t\ll 1/A_0$.

Noise-free Network

For noise-free network, averaging over time is equivalent to averaging over populations (ensembles).

We can inspect a single neuron. A neuron fired at $t_0$ will fire again at $t_0+T$ with $T$ being determined by $u(t_0+T)=\theta$. Meanwhile the theory of population gives us the population activity change

$$ \Delta A(t) = A_0 \frac{d}{dt}\left( \Delta I \int_0^{t-t_0} \kappa_0(s),ds \right) \propto A_0\Delta I \kappa_0(t-t_0), $$

for $t_0< t<t_0+T$.

The solution is written as

$$ \Delta A(t) = a A_0 \kappa_0(t-t_0), $$

where $a$ can be derived for different models.

Does it come from the differentiation of time? Not sure how it is derived.

1. SRM0: $a=R\Delta I/\eta'$, where $\eta$ is the intrinsic response;
2. Integrate-and-fire: $a=R\Delta I/u'$.

Kernel $\kappa$.

• Integrate and fire model:

  $$ \kappa_0(s) = \frac{1}{\tau_m} \exp\left( -\frac{s}{\tau_m}\right) \Theta(s). $$

For integrate-and-fire model, we can show that the input potential is

$$ h(t) = h_0 + R \Delta I \left( 1 - \exp\left( -\frac{t-t_0}{\tau_m} \right) \right), $$

after the input current change, i.e., $t>t_0$. Within a short period, $h(t)\sim h_0$. However, as we have derived previously, the population activity change drastically,

$$ \Delta A(t) = a\frac{A_0}{\tau_m} \exp\left( - \frac{t-t_0}{s}\right)\Theta(t-t_0), $$

for $t_0<t<t_0+T$.

Fig. 7.5

Transients with Noise

For slow noise, refer to Fig. 7.6A.

Some random questions:

1. How to explain the abrupt transient? Before the change of input, there are always neurons at sub-threshold in the population. Increase the input triggers these neurons immediately.
2. Why does the population activity approach another asynchronous state after a long time? Donno.

For fast noise (standart rate mode, or Wilson-Cowan based model), refer to Fig. 7.6B.

For SRM0 with escape noise, refer to Fig. 7.7.

For diffusive noise, refer to Fig. 7.8.