Deploying to main from @ lava-nc/lava-docs@4b668c6 🚀

lava-lib-dl/slayer/notebooks/pilotnet/train.html

@@ -824,7 +824,7 @@ <h2>What are SDNNs?<a class="headerlink" href="#What-are-SDNNs?" title="Permalin
 <table><tr><td><p><img alt="Drawing" src="../../../../_images/delta_encoder.png" /></p>
 </td><td><p><img alt="Drawing" src="../../../../_images/sigma_decoder.png" /></p>
 </td></tr></table><p><strong>Sigma-delta neural networks</strong> consists of two main units: <em>sigma</em> decoder in the dendrite and <em>delta</em> encoder in the axon. Delta encoder uses differential encoding on the output activation of a regular ANN activation, for e.g. ReLU. In addition it only sends activation to the next layer when the encoded message magnitude is larger than its threshold. The sigma unit accumulates the sparse event messages and accumulates it to restore the original value.</p>
-<table><tr><!-- <td> <img src="images/sdnn.png" alt="Drawing" style="height: 300px;"/> </td> --><td><p><img alt="e14975ee1cce44e5a76831004644964a" class="no-scaled-link" src="../../../../_images/sdnn.png" style="height: 300px;" /></p>
+<table><tr><!-- <td> <img src="images/sdnn.png" alt="Drawing" style="height: 300px;"/> </td> --><td><p><img alt="7407cb930aed4352aa8fc77c6ecd90f0" class="no-scaled-link" src="../../../../_images/sdnn.png" style="height: 300px;" /></p>
 </td></tr></table><p>A sigma-delta neuron is simply a regular activation wrapped around by a sigma unit at it’s input and a delta unit at its output.</p>
 <p>When the input to the network is a temporal sequence, the activations do not change much. Therefore, the message between the layers are reduced which in turn reduces the synaptic computation in the next layer. In addition, the graded event values can encode the change in magnitude in one time-step. Therefore there is no increase in latency at the cost of time-steps unlike the rate coded Spiking Neural Networks.</p>
 <table><tr><td><p><img alt="Drawing" src="../../../../_images/temporal_redundancy.png" /></p>

lava/notebooks/in_depth/clp/tutorial01_one-shot_learning_with_novelty_detection.html

@@ -841,24 +841,24 @@ <h4>Online Continual Learning with Open-set Recognition<a class="headerlink" hre
 <li><p>Finally, we want our system to be able to perform open-set recognition, i.e. rather than performing recognition purely on the known classes (close-set recognition), we also want to recognize instances from unknown classes (Figure 2, <a class="reference external" href="https://www.nature.com/articles/s41598-020-63649-6">Shu et al. 2020</a>). For this purpose, we will use prototypes with a radius (or margin) of recognition defined around them (Figure 3). In terms of the spiking (LIF) neurons, this margin translates to the
 voltage threshold for spiking.</p></li>
 </ul>
-<center><p><img alt="6f37d3d670dd4adb8d9be92e10b98021" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/continual_learning.png" /></p>
+<center><p><img alt="9130784cdee24c3f98a0a5a723c0d856" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/continual_learning.png" /></p>
 <center><center><figcaption align="center"><p>Figure 1. Continual Learning (image credit: Matthias De Lange)</p>
-</figcaption><center><p><img alt="852d1264cfcb4c5c94759c814a789e6b" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/open_set_recognition.png" /></p>
+</figcaption><center><p><img alt="3bbc35a4f7ee415496dec8b77ec89a2d" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/open_set_recognition.png" /></p>
 <center><figcaption align="center"><p>Figure 2. Open-set recognition</p>
-</figcaption><center><center><p><img alt="d82e4401b8354767b604da2362d3df16" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/knn_clp.jpg" /></p>
+</figcaption><center><center><p><img alt="38147e66865143da9d90c6cbc12e2d1e" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/knn_clp.jpg" /></p>
 <center><center><figcaption align="center"><p>Figure 3. The concept of prototypes in the brain. (Left) Learning by hard all examples is akin to k-NN. (Right) Learning an abstract concept from the seen examples is the idea behind the prototypes and what CLP uses</p>
 </figcaption><center></section>
 <section id="Similarity-Measure:-Why-Cosine-Similarity?">
 <h4>Similarity Measure: Why Cosine Similarity?<a class="headerlink" href="#Similarity-Measure:-Why-Cosine-Similarity?" title="Permalink to this heading"></a></h4>
 <p>The prototype-based methods generally use Euclidean distance as the similarity measure. However, in high-dimensional space, the Euclidean distance starts to lose its meaning because of the curse of dimensionality. In addition, calculating the Euclidean distance requires types of operations (vector length, squaring) that are not supported on Loihi or are just not local computations. The cheapest operation available in Loihi is the dot product between the input spikes and the weights. We can
 efficiently compute this dot product, especially with the graded spikes in Loihi 2. However, the dot product is not a reliable similarity measure as long as the vectors have different lengths. Therefore, we will normalize all input vectors and inject them as graded spikes. As the weights are learned from these inputs, they are also normalized. Consequently, the dot product between inputs and weights corresponds to the cosine similarity between the non-normalized original input and the normalized
 weights. These stored weights are the learned prototypes.</p>
-<center><p><img alt="b93cf862c6594ca8af12409b4aa76654" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/cosine_similarity_formula.png" /></p>
+<center><p><img alt="8fe731d1bb49490f897fe1abbcc01a13" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/cosine_similarity_formula.png" /></p>
 <center><p>From the literature, we know that cosine similarity works well for high-dimensional spaces (<a class="reference external" href="https://openaccess.thecvf.com/content/CVPR2022/papers/Hersche_Constrained_Few-Shot_Class-Incremental_Learning_CVPR_2022_paper.pdf">Hersche et al. 2022</a>). As our main problem domain is visual processing, which generally has high-dimensional input, implicitly using cosine-similarity fits our purposes. Implicitly because we arrive at cosine similarity using the dot-product of normalized vectors. From the
 CLP’s perspective, it still performs a dot product similarity. This will play an essential role in deriving the learning rules and interpreting the similarity measure.</p>
 <p>In Figure 4, we see prototypes with some recognition margin around them. On the left panel, a prototype with Euclidean similarity defines a point in Euclidean space and a circle in 2D (or a sphere in 3D) with a given radius (i.e., recognition threshold) around it. On the middle panel, we the prototype vectors with cosine similarity. In this case, both input and prototype vectors can have different lengths: the similarity is measured as the angle between the input vector and the weight vectors.
 Hence the recognition margins are circular and spherical sectors in 2D and 3D, respectively. On the right panel, we see our case: all input and weight vectors are normalized, i.e., they fall on the surface of a hyper-sphere. The recognition margins are defined solely on this surface: an arc of a circle in 2D and a piece (cap) on the sphere’s surface. Therefore, CLP will perform its computation always on this kind of surface.</p>
-<center><p><img alt="fec979988ba54fdf989b6f4faeb6c609" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/sim_measure_and_protos.png" /></p>
+<center><p><img alt="c56f71cb27d54c5f9194d40f73ddcaa6" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/sim_measure_and_protos.png" /></p>
 <center><center><figcaption align="center"><p>Figure 4. Prototypes with different similarity measures in 2D and 3D. (a) Euclidean distance/similarity, (b) cosine similarity, (c) Dot product similarity on normalized vectors. Figures are adopted from [Liu et al. 2017].</p>
 </figcaption><center><p>References: <a class="reference external" href="https://link.springer.com/article/10.1007/s11432-021-3316-x">Zhu et al. 2022</a>, <a class="reference external" href="https://openaccess.thecvf.com/content_cvpr_2017/papers/Liu_SphereFace_Deep_Hypersphere_CVPR_2017_paper.pdf">Liu et al. 2017</a></p>
 </section>
@@ -1011,15 +1011,15 @@ <h2>Data generation and visualization<a class="headerlink" href="#Data-generatio
 <img alt="../../../../_images/lava_notebooks_in_depth_clp_tutorial01_one-shot_learning_with_novelty_detection_10_1.png" src="../../../../_images/lava_notebooks_in_depth_clp_tutorial01_one-shot_learning_with_novelty_detection_10_1.png" />
 </div>
 </div>
-<center><p><img alt="b377fb1b7ec9416e8a45d6ae348b7718" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/neural_diagram.png" /></p>
+<center><p><img alt="60a67ff3bab84ad7bd0c0409f5919dd1" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/neural_diagram.png" /></p>
 <center><center><figcaption align="center"><p>Figure 5. CLP Diagram. The main components of the CLP algorithm and their connections are depicted.</p>
 </figcaption><center><p>This first version of CLP includes the features like novelty detection, one-shot learning, and supervised and unsupervised labeling of the learned prototypes. The normalized input vectors are one-by-one injected into the prototype neurons through prototype weights/synapses. Suppose none of the prototype neurons spike after the input injection. In that case, this is detected by the novelty detector via coincidence detection between the presence of the input and the absence of any output. Once
 this is detected, the novelty detector sends a strong third-factor signal to the prototype population. It aims explicitly at the unallocated neuron in this population. As a result, this prototype neuron learns the input in one shot, effectively memorizing the input as a prototype. The outputs of prototype neurons are sent to the Readout module, which interprets the prototype index as a prediction label. Note that one class may have more than one prototype. Therefore, the Readout module keeps a
 dictionary that maps the prototype ids to the labels. If a user provides a label to the Readout module during the one-shot allocation process, this label will label the newly allocated prototype id. If the user provides no label, the newly allocated neuron will get a unique pseudo-label so that the system can later identify other instances from this class as “unknown class x”. This is precisely what open-set recognition tries to achieve (Figure 2). If alter the actual label is provided together
 with an instance from this unknown class, the Readout module automatically changes the pseudo-label to this actual label.</p>
 <p>All these modules are implemented as processes in Lava (Figure 6): PrototypeLIF, NoveltyDetector, and Readout. The input process is connected to the PrototypeLIF process via a LearningDense process - a dense connection of plastic synapses. To understand the learning setup and connections, we refer the reader to the following tutorial on <a class="reference internal" href="../three_factor_learning/tutorial01_Reward_Modulated_STDP.html"><span class="doc">the reward-modulated learning rule</span></a>.</p>
 <p>The system described in Figure 6 runs on a CPU, though with the Loihi protocol. The Loihi 2 implementation will be coming soon.</p>
-<center><p><img alt="53bb33f846cf467cac7af699a145d5fc" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/lava_diagram.png" /></p>
+<center><p><img alt="2cd69a218dbc4c2db0965fade47f84c8" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/lava_diagram.png" /></p>
 <center><center><figcaption align="center"><p>Figure 6. CLP Lava Diagram.</p>
 </figcaption><center><p>Let’s first set the parameters for all the processes.</p>
 <div class="nbinput nblast docutils container">
@@ -1422,7 +1422,7 @@ <h2>Network Outputs: Novelty Detection, Prototypes, Predictions, Labeling<a clas
 <p>The 3rd input pattern is from cluster 1, and as expected, the novelty detector spikes again, allocating the <code class="docutils literal notranslate"><span class="pre">Prototype</span> <span class="pre">Neuron</span> <span class="pre">1</span></code>this time. However, the label is provided together this time, so the labeling of the newly allocated neuron happens immediately (with the label <code class="docutils literal notranslate"><span class="pre">2</span></code>). The next pattern is also from cluster 1, so the newly allocated neuron spikes and the correct label is predicted.</p>
 <p>Finally, the fifth input pattern is from an unsee cluster (cluster 2). Hence, a novelty spike follows it after some time and allocates the <code class="docutils literal notranslate"><span class="pre">Prototype</span> <span class="pre">Neuron</span> <span class="pre">2</span></code> and has been assigned the pseudo-label <code class="docutils literal notranslate"><span class="pre">-3</span></code>. All the next input patterns from this cluster are recognized as <code class="docutils literal notranslate"><span class="pre">-3</span></code>. After one of those input patterns, the actual label (<code class="docutils literal notranslate"><span class="pre">3</span></code>) is provided so that the label of the <code class="docutils literal notranslate"><span class="pre">Prototype</span> <span class="pre">Neuron</span> <span class="pre">2</span></code> is updated to <code class="docutils literal notranslate"><span class="pre">3</span></code>. After that, the patterns from this cluster are predicted with the label <code class="docutils literal notranslate"><span class="pre">3</span></code>.
 One can validate the correct prototype neuron spikes for each input by comparing it to the table above of the accurate cluster ids and labels.</p>
-<center><p><img alt="31f1145f7b8a4fb991f15e6150b57c51" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/readout_labeling_analysis.png" /></p>
+<center><p><img alt="cc7b7a363dcc4856ad9007fcb70e124e" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/readout_labeling_analysis.png" /></p>
 <center><center><figcaption align="center"><p>Figure 7. Readout and labeling analysis.</p>
 </figcaption><center></section>
 <section id="Visualization-of-Learning">

lava/notebooks/in_depth/clp/tutorial02_clp_on_coil100.html

@@ -867,7 +867,7 @@ <h2>Overview<a class="headerlink" href="#Overview" title="Permalink to this head
 </div>
 <section id="Dasaset:-COIL-100">
 <h3>Dasaset: COIL-100<a class="headerlink" href="#Dasaset:-COIL-100" title="Permalink to this heading"></a></h3>
-<center><p><img alt="f6bdc61cf51a44a8a081b75a7d51eb41" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/coil_100.png" /></p>
+<center><p><img alt="c41831dd921d4b74a9e46f02a4d5050e" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/coil_100.png" /></p>
 <center><center><figcaption align="center"><p>Figure 1. COIL-100 dataset. The dataset includes 72 frames for each of the 100 objects rotated on a turntable.</p>
 </figcaption><center></section>
 <section id="Data-preparation-and-visualization">
@@ -1046,7 +1046,7 @@ <h4>Visualization of the similarity between the training and testing samples<a c
 </section>
 <section id="Lava-diagram-of-CLP-for-COIL-100-experiment">
 <h3>Lava diagram of CLP for COIL-100 experiment<a class="headerlink" href="#Lava-diagram-of-CLP-for-COIL-100-experiment" title="Permalink to this heading"></a></h3>
-<center><p><img alt="43d342307ba84d51941ac966ddab4981" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/clp_coil100_lava_diagram.png" /></p>
+<center><p><img alt="77b51c84b39142aca7649133b85f32e1" src="https://raw.githubusercontent.com/lava-nc/lava-nc.github.io/main/_static/images/tutorial_clp/clp_coil100_lava_diagram.png" /></p>
 <center><center><figcaption align="center"><p>Figure 1. Lava process diagram of the CLP that is capable of both unsupervised and supervised learning</p>
 </figcaption><center><p>Compared to the introductory CLP tutorial, we are using an upgraded version of the algorithm here. Specifically, this version is concurrently capable of both unsupervised and supervised learning. This is possible thanks to error feedback from the <code class="docutils literal notranslate"><span class="pre">Readout</span></code> process to the <code class="docutils literal notranslate"><span class="pre">Allocator</span></code> process. Basically, when there is a mismatch between the predicted label and the user-provided true label, the <code class="docutils literal notranslate"><span class="pre">Readout</span></code> process generates an error signal, which triggers the allocation of a new prototype
 neuron for this mistaken pattern. As we will see later, this will improve the performance. This error feedback can be turned on or off, and based on this the CLP performs either supervised or unsupervised learning.</p>