17.homogeneous-network.md

layout	title	date	author	summary	references	weight
notes	17.Homogeneous Network	2016-09-02	OctoMiao	Review of population activity; Homogeneous network.	name Section 6.1, Section 6.2.1	17

Review of Population Activity

Information from spatio-temporal pattern:

1. Temporal average: mean firing rate of the network:

   $$ \begin{equation} A = \frac{\text{total number of spikes } n_{\mathrm{act}}(t;t+\Delta t) \text{ during a time window } \Delta t}{\text{time widow } \Delta t \times \text{ number of neurons } N }, \end{equation} $$

   where $\Delta t$ is the time binning. We can take the limit $\Delta t\to 0$. For spiking neuron models,

   $$ \begin{align} A =& \lim_{\Delta t\to 0}\frac{n_{\mathrm{act}}(t;t+\Delta t)}{\Delta t} \frac{1}{N} \ =& \frac{1}{N} \text{total number of active neurons at time } t. \end{align} $$

   Assume the $j$th neuron fires at $t_j^{(f)}$ (where $f$ means neuron $j$ can fire at $t_j^{(1)}$, $t_j^{(2)}$, etc), we know which neuron would fire at any time $t$ by calculating

   $$ \delta(t-t_j^{(f)}). $$

   If the argument is zero for neuron $j$, we know this neuron fires at time $t$. Thus the total number of active neurons at time $t$ is

   $$ \sum_{j=1}^N \sum_{f} \delta(t-t_j^{(f)}). $$

   The summation of $f$ counts all the contributions from neuron $j$ since it can fire multiple times.

2. Temporal correlations between pulses of different neurons;

Define Homogeneous Network

Homogeneous:

1. Each and every neurons are the same;
2. Neurons take the same external input;
3. Weight between arbitrary neuron $i$ and arbitrary neuron $j$ are the same $w_{ij}=J_0/N$.

Modeling Using Leaky Integrate-and-fire Model

$$ \tau_m \frac{d}{dt} u_i = - u_i + R I_i(t), $$

where the input on neuron $i$ at time $t$ ($I_i(t)$) includes both external and internal signal

$$ I_i(t) = \sum_j \sum_f w_{ij} \alpha(t-t_j^{(f)}) + I^{\mathrm{ext}}(t), $$

where $\alpha(t-t_j^{(f)})$ is the time course after a $f$th spike in neuron $j$.

QUESTION HERE:

Assuming that we start from a homogeneous initial condition that all neurons have the same input will the network evolve to an inhomogeneous state under tiny perturbation? This should be called the stability of the network.

The method I can think of to check the instability is to add a field of perturbation, i.e., a perturbation to each input and check the evolution of these perturbations. If all the perturbations increase with time, we obtain a hint that the system could be unstable. Then we can investigate using numerical methods or linear stability analysis to find out exactly how.

COMMENT: This perturbation method is similar to the one discussed in Section 6.2.1 as stochastic input.

Regardless of the question above, we assume that no perturbation is included so that the system evolve homogeneously. Input current on arbitrary neuron is the same as each other

$$ I(t) = \frac{1}{N} \sum_j \sum_f J_0 \alpha(t-t_j^{(f)}) + I^{\mathrm{ext}}(t). $$

Due to homogeneity, we can plug in the definition of population activity

$$ A = \frac{1}{N} \sum_j \sum_f \delta(t-t_j^{(f)}) $$

for every neurons. At time $s$, we know the activity is calculated as $$A(s)$$, the input current at time $t$ which is later than $s$ depends on all the history of population activity, which means we include all the history to find the current input current, i.e.,

$$ I(t) = \frac{1}{N} \int_0^\infty ds \alpha(s) \left[N A(t-s)\right] + I^{\mathrm{ext}}(t). $$

Modeling Using SRM

SRM

$$ \begin{equation} u_i(t) = \eta(t-\hat t_i) + h_{\mathrm{PSP}}(t\vert \hat t_i)\label{srm}. \end{equation} $$

Using an argument similar to integrate-and-fire model, we can plug in the population activity to equation (\ref{srm}).

Recall that spikes occur as the potential $u_i$ reaches the threshold $\theta$.

Density Equations

Intergrate-and-fire and stochastic spike arrival

Each neuron is described by its membrane potential. We can find the percentage of neurons within a range of membrane potential $[u,u+du]$, which is

$$ \begin{equation} \frac{dN}{N} = p(u,t) du, \end{equation} $$

where $p(u,t)$ is the membrane potential (distribution) density.

1. Due to the normalization $1/N$, we expect integral over all the possible potentials gives us 1, i.e.,

   $$ \begin{equation} \int_{-\infty}^\theta p(u,t),du = 1. \end{equation} $$

2. Define the flux of neutrons going through the threshold by inspecting the population increase at potential $u_{\mathrm{reset}}$.

About the stochastic input, we assume that a total number of M types of synapses are introduced. For each type $k$ we have a input rate $\nu_k$. The equation for the $p(u,t)$ is written down in Section 6.2.1 as equation (6.14). In general, we can define the flux going across a potential by checking the time derivative of $p(u,t)$,

$$ \begin{equation} \partial_t p(u,t) ,du = dJ + dJ_{\mathrm{reset}}, \end{equation} $$

where

$$ dJ = dJ_{\mathrm{drift}} + dJ_{\mathrm{jump}}, $$

where $dJ_{\mathrm{drift}}$ rate of neurons that has a potential continuously evolved into $u$ during time $[t,t+dt]$, while $dJ_{\mathrm{jump}}$ stands for the rate of neurons jumping from all different potential $u-w_k$'s into $u$ due to stochastic input. READ Fig. 6.2

For a better understanding of $dJ$, please refer to Eq. (\ref{continuity-equation}).

Flux means the percentage of neurons that goes across a specific potential $u$ at time $t$ per unit potential. In other words, it's the changing rate of the membrane potential density at potential $u$ during time interval $[t,t+dt]$.

Equation 6.18 shows the population activity.

Continuity Equation

Continuity equation describes the rate of change. In this case, continuity is the rate of change in percentage of neurons at time $t$ within a range of potentials $[u_0,u_1]$, so that

$$ \begin{equation} \partial_t \int_{u_0}^{u_1} p(u,t),du = J(u_0,t) - J(u_1,t). \label{continuity-equation} \end{equation} $$

An example of continuity equation is the conservation of charge. The change of charge equals to the charge coming in and the charge going out.

The we know the change in the percentage of neurons within potential range $[u_0,u_1]$ is the percentage of neurons moving within the potential range $J(u_0,t)$ and the percentage of neurons going out $J(u_1,t)$. In the language of density, we have

$$ \begin{equation} \partial_t p(u,t) = - \partial_u J(u,t), \end{equation} $$

where $u\neq u_{\mathrm{reset}}$ and $u\neq \theta$.

Diffusion Approximation

Expand equation 6.14 in terms of $w_k$ and keep only $\mathscr{O}(w_k^2)$,

$$ \begin{align} p(u-w_k,t) =& p(u,t) + \partial_u p(u-w_k,t)\vert_{u-w_k=u} (-w_k) + \frac{1}{2}\partial_u^2 p(u-w_k,t)\vert_{u-w_k=u} w_k^2 + \mathscr{O}(w_k^3) \\ =& p(u,t) - \partial_u p(u,t) w_k + \frac{1}{2} \partial_u^2 p(u,t) w_k^2 + \mathscr{O}(w_k^3). \end{align} $$

Plug into equation 6.14

$$ \begin{align} \tau_m \partial_t p(u,t) =& - p(u,t) - \left[ -u + R I^{\mathrm{ext}}(t) + \tau_m \sum_k v_k(t) w_k \right] \partial_u p(u,t) \\ &+ \frac{1}{2} \tau_m \sum_k v_k(t) w_k^2 \partial_u^2 p(u,t) + \tau_m A(t) \delta(u-u_r) + \mathscr{O}(w_k^3). \end{align} $$

The first two terms are perfect derivatives

$$ \begin{align} \phantom{=}&- p(u,t) - \left[ -u + R I^{\mathrm{ext}}(t) + \tau_m \sum_k v_k(t) w_k \right] \partial_u p(u,t) \\ =& -\partial_u \left[\left(-u + R I^{\mathrm{ext}}(t) + \tau_m \sum_k v_k(t) w_k \right)p(u,t) \right]. \end{align} $$

The equation we need to deal with is

$$ \begin{align} \tau_m \partial_t p(u,t) =& -\partial_u \left[\left(-u + R I^{\mathrm{ext}}(t) + \tau_m \sum_k v_k(t) w_k \right)p(u,t) \right]\\ & + \frac{1}{2} \tau_m \sum_k v_k(t) w_k^2 \partial_u^2 p(u,t) + \tau_m A(t) \delta(u-u_r) + \mathscr{O}(w_k^3). \end{align} $$

Fokker–Planck equation

$A(t)$ is the flux at threshold $\theta$. To solve it we simply set $u=\theta$, $$ \begin{equation} A(t) = \frac{1}{\tau} (- \theta + R I^{\mathrm{ext}}(t)) p(\theta,t) + \sum_k v_k \int_{\theta - w_k}^\theta p(u,t) du. \end{equation} $$

To expand $A(t)$ around $u=\theta$, we need

$$ \begin{equation} p(u,t) = p(\theta,t) + \partial_u p(u,t)\vert_{u=\theta} (u-\theta) + \frac{1}{2} \partial_u^2 p(u,t) \vert_{u=\theta} (u-\theta)^2 + \mathscr{O}((u-\theta)^3),\label{eq-p-u-t-expand-theta} \end{equation} $$

so that $$ \begin{equation} \int_{theta-w_k}^\theta(u,t) du = p(\theta,t) w_k + \frac{1}{2} \partial_u p(u,t)\vert_{u=\theta} w_k^2,\label{eq-int-p-u-t-expand-theta} \end{equation} $$ where $p(\theta,t)=0$ since this is the threshold and no neurons can stay at this potential. Eq.(\ref{eq-p-u-t-expand-theta}) and Eq.(\ref{eq-int-p-u-t-expand-theta}) are plugged back into the express for $A(t)$. The simplified threshold flux is

$$ \begin{equation} A(t) = \frac{1}{2}\sum_k v_k w_k^2\partial_u p(u,t)\vert_{u=\theta}. \end{equation} $$

I obtained a different sign. In the text book $A$ is negative.

Stationary Solution

Why stationary solution? Experiments.

Define

$$ \begin{equation} \sigma^2 = \tau_m \sum_k v_k w_k^2. \end{equation} $$

Stationary state means $p(u,t)$ is time translation invariant, i.e., $p$ only depends on $u$. The Fokker-Planck equation becomes

$$ \begin{equation} 0= -\partial_u \left[ (-u + h)p - \frac{1}{2}\frac{\sigma^2}{\tau_m} \partial_u p \right] + A\delta(u-u_r), \end{equation} $$ where $h=R I^{\mathrm{ext}}+\tau_m \sum_k v_k w_k$.

$A$ is the change of percentage of neurons at threshold (some kind of flux), dimension analysis shows the terms left is of dimension $[\partial_u J]$.

Dimension of $\delta(u-u_r)$ is $[1/u]$.

Indeed we define a flux

$$ \begin{equation} J = \frac{-u+h}{\tau_m} p - \frac{1}{2} \frac{\sigma^2}{\tau_m} \partial_u p. \end{equation} $$

Educated guess $$ \begin{equation} p = \begin{cases} \frac{c_1}{\sigma^2} \exp \left( - \frac{(u-h)^2}{\sigma^2} \right) & \qquad u<u_r\ \frac{c_2}{\sigma^2} \exp \left( - \frac{(u-h)^2}{\sigma^2} \right) \int_u^\theta \exp \left( \frac{(x-h)^2}{\sigma^2} \right) dx & \qquad u_r<u<\theta\ \end{cases} \end{equation} $$