deploy: 870722c

_sources/resources/exploring-performance.ipynb

@@ -343,13 +343,17 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "video_html = f\"\"\"\n",
-    "<video width=\"640\" height=\"480\" controls>\n",
-    "  <source src=\"{vid_out_path}\" type=\"video/mp4\">\n",
-    "  Your browser does not support the video tag.\n",
-    "</video>\n",
-    "\"\"\"\n",
-    "ipd.display(HTML(video_html))"
+    "#video_html = f\"\"\"\n",
+    "#<video width=\"640\" height=\"480\" controls>\n",
+    "#  <source src=\"{vid_out_path}\" type=\"video/mp4\">\n",
+    "#  Your browser does not support the video tag.\n",
+    "#</video>\n",
+    "#\"\"\"\n",
+    "#ipd.display(HTML(video_html))\n",
+    "\n",
+    "from IPython.core.display import Video\n",
+    "\n",
+    "Video(vid_out_path, embed=True)"
    ]
   },
   {
@@ -515,8 +519,53 @@
    },
    "outputs": [],
    "source": [
-    "# Predict raga using subset of features corresponding to beginning of audio for faster prediction\n",
-    "predicted_raga = deepsrgm.predict(feat[:2])"
+    "# Predict raga using subset of features from the very beginning of audio for faster prediction\n",
+    "predicted_raga = deepsrgm.predict(feat[:8])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(f'Raga: {predicted_raga}')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "remove-output"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "from compiam import load_model\n",
+    "from compiam.melody.pitch_extraction import Melodia\n",
+    "\n",
+    "melodia = Melodia() \n",
+    "ftanet_carnatic = load_model(\"melody:ftanet-carnatic\")\n",
+    "PITCH_EXTRACTION_SR = 44100\n",
+    "\n",
+    "freqs = ftanet_carnatic.predict(audio_mix)[:, 1]\n",
+    "tonic = tonic_multipitch.extract(audio_mix)\n",
+    "\n",
+    "k = 5\n",
+    "N = 200\n",
+    "new_feat = []\n",
+    "\n",
+    "feature = np.round(1200 * np.log2(freqs / tonic) * (k / 100)).clip(0)\n",
+    "\n",
+    "if len(feature) <= 5000:\n",
+    "    raise ValueError(\"Audio signal is not longer enough for a proper estimation. Please provide a larger audio.\")\n",
+    "for i in range(N):\n",
+    "    c = np.random.randint(0, len(feature) - 5000)\n",
+    "    new_feat.append(feature[c : c + 5000])\n",
+    "new_feat = np.array(new_feat)\n",
+    "\n",
+    "raga = deepsrgm.predict(new_feat)"
    ]
   },
   {
@@ -631,7 +680,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The f0 pitch track of the predominant vocal melody has proved useful for range of computational analysis tasks in Indian Art Music. We can extract this for our performance using Melodia, a broadly used knowledge-based method. We will also test a recently published DL model to achieve the same goal: FTA-Net model, which has been trained specifically for the Carnatic Music use case and included in compIAM as well."
+    "The f0 pitch track of the predominant vocal melody has proved useful for range of computational analysis tasks in Indian Art Music. We can extract this for our performance using Melodia {cite}`salamon_pitch_2012`, a broadly used knowledge-based method. We will also test a recently published DL model to achieve the same goal: FTA-Net model, which has been trained specifically for the Carnatic Music use case and included in compIAM as well {cite}`plaja_pitch_2023`."
    ]
   },
   {
@@ -652,6 +701,13 @@
     "PITCH_EXTRACTION_SR = 44100"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To extract an example pitch track, we first load 30s of the mixture recording, and run prediction with both methods. Once the methods are initialized, running pitch extraction is easily done in one line of code."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -675,6 +731,13 @@
     "ftanet_pitch_track = ftanet_carnatic.predict(prediction_input, input_sr=PITCH_EXTRACTION_SR)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's now plot both pitch tracks and compare!"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -852,6 +915,10 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "Pitch curves are a really important feature for the computational analysis of Carnatic Music. The important and very much present ornamentation of notes and transitions between notes, it is better represented with continuous pitch value arrays. However, since the lead melodic instruments are normally found mixed with accompanying instruments, we require methods that are able to capture the melodies in the presence of background music. \n",
+    "\n",
+    "Pitch curves, or tracks, can be used for a list of tasks that aim at extracting relevant melodic information from the performed melodies. Also, several classification and tagging tasks (e.g. raga classification), build on top of melodic features, normally including pitch information, whether explicitly, or embedded in other representations.\n",
+    "\n",
     "For further reference, please visit the [pitch extraction page](melody-extraction)."
    ]
   },
@@ -917,6 +984,24 @@
     ");"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "See some demos of related beat tracking and percussion pattern research done within the context of the CompMusic project."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from IPython.display import YouTubeVideo\n",
+    "\n",
+    "YouTubeVideo(\"wvrGhXFXtv8\", width=800, height=450)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

introduction/compiam.html

@@ -732,19 +732,22 @@ <h5>The <code class="docutils literal notranslate"><span class="pre">.load_datas
 <div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.00B [00:00, ?B/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  1%|▏         | 8.00k/543k [00:00&lt;00:35, 15.4kB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  1%|▏         | 8.00k/543k [00:00&lt;00:35, 15.3kB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  9%|▉         | 48.0k/543k [00:00&lt;00:05, 91.3kB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  3%|▎         | 16.0k/543k [00:00&lt;00:20, 26.5kB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 24%|██▎       | 128k/543k [00:00&lt;00:01, 223kB/s]  
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 10%|█         | 56.0k/543k [00:00&lt;00:05, 97.8kB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 94%|█████████▍| 512k/543k [00:00&lt;00:00, 949kB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 24%|██▎       | 128k/543k [00:00&lt;00:01, 213kB/s]  
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>544kB [00:00, 576kB/s]                           
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 55%|█████▍    | 296k/543k [00:01&lt;00:00, 487kB/s]
+</pre></div>
+</div>
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>544kB [00:01, 461kB/s]                           
 </pre></div>
 </div>
 <div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>

melodic_analysis/melodic-pattern-discovery.html

@@ -775,21 +775,16 @@ <h3>2.2 Visualising predominant pitch<a class="headerlink" href="#visualising-pr
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>{54.997834212038505: &#39;R2&#39;,
- 109.99566842407701: &#39;R2&#39;,
- 219.99133684815402: &#39;R2&#39;,
- 439.98267369630804: &#39;R2&#39;,
- 879.965347392616: &#39;R2&#39;,
- 65.40381575469627: &#39;M1&#39;,
- 130.80763150939254: &#39;M1&#39;,
- 261.6152630187851: &#39;M1&#39;,
- 523.2305260375701: &#39;M1&#39;,
- 1046.46105207514: &#39;M1&#39;,
- 48.9975: &#39;S&#39;,
- 97.995: &#39;S&#39;,
- 195.99: &#39;S&#39;,
- 391.98: &#39;S&#39;,
- 783.96: &#39;S&#39;,
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>{87.30361988416256: &#39;N2&#39;,
+ 174.60723976832512: &#39;N2&#39;,
+ 349.21447953665023: &#39;N2&#39;,
+ 698.4289590733005: &#39;N2&#39;,
+ 1396.857918146601: &#39;N2&#39;,
+ 73.4133009992652: &#39;P&#39;,
+ 146.8266019985304: &#39;P&#39;,
+ 293.6532039970608: &#39;P&#39;,
+ 587.3064079941216: &#39;P&#39;,
+ 1174.6128159882435: &#39;P&#39;,
  58.268175617345825: &#39;G2&#39;,
  116.53635123469165: &#39;G2&#39;,
  233.0727024693833: &#39;G2&#39;,
@@ -800,16 +795,21 @@ <h3>2.2 Visualising predominant pitch<a class="headerlink" href="#visualising-pr
  329.614576851151: &#39;D2&#39;,
  659.229153702302: &#39;D2&#39;,
  1318.458307404604: &#39;D2&#39;,
- 73.4133009992652: &#39;P&#39;,
- 146.8266019985304: &#39;P&#39;,
- 293.6532039970608: &#39;P&#39;,
- 587.3064079941216: &#39;P&#39;,
- 1174.6128159882435: &#39;P&#39;,
- 87.30361988416256: &#39;N2&#39;,
- 174.60723976832512: &#39;N2&#39;,
- 349.21447953665023: &#39;N2&#39;,
- 698.4289590733005: &#39;N2&#39;,
- 1396.857918146601: &#39;N2&#39;}
+ 65.40381575469627: &#39;M1&#39;,
+ 130.80763150939254: &#39;M1&#39;,
+ 261.6152630187851: &#39;M1&#39;,
+ 523.2305260375701: &#39;M1&#39;,
+ 1046.46105207514: &#39;M1&#39;,
+ 48.9975: &#39;S&#39;,
+ 97.995: &#39;S&#39;,
+ 195.99: &#39;S&#39;,
+ 391.98: &#39;S&#39;,
+ 783.96: &#39;S&#39;,
+ 54.997834212038505: &#39;R2&#39;,
+ 109.99566842407701: &#39;R2&#39;,
+ 219.99133684815402: &#39;R2&#39;,
+ 439.98267369630804: &#39;R2&#39;,
+ 879.965347392616: &#39;R2&#39;}
 </pre></div>
 </div>
 </div>
@@ -884,7 +884,7 @@ <h3>3.1 Melodic feature extraction<a class="headerlink" href="#melodic-feature-e
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 11:22:44,034] INFO [compiam.melody.pattern.sancara_search.complex_auto.cqt.load_audio:44] loading file ../audio/pattern_finding/Koti Janmani.multitrack-vocal.mp3
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 17:28:03,473] INFO [compiam.melody.pattern.sancara_search.complex_auto.cqt.load_audio:44] loading file ../audio/pattern_finding/Koti Janmani.multitrack-vocal.mp3
 </pre></div>
 </div>
 <div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>tensor([[0.6483, 1.1698, 0.7740,  ..., 0.2975, 0.2547, 0.9335],

melodic_analysis/tonic-identification.html

@@ -635,16 +635,22 @@ <h2> Contents </h2>
 <div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.00B [00:00, ?B/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  2%|▏         | 8.00k/473k [00:00&lt;00:22, 21.5kB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  2%|▏         | 8.00k/473k [00:00&lt;00:35, 13.5kB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 10%|█         | 48.0k/473k [00:00&lt;00:03, 121kB/s] 
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  7%|▋         | 32.0k/473k [00:00&lt;00:08, 52.8kB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 29%|██▊       | 136k/473k [00:00&lt;00:01, 276kB/s] 
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██        | 96.0k/473k [00:00&lt;00:02, 157kB/s] 
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>480kB [00:00, 635kB/s]                           
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 47%|████▋     | 224k/473k [00:01&lt;00:00, 358kB/s] 
+</pre></div>
+</div>
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 98%|█████████▊| 464k/473k [00:01&lt;00:00, 724kB/s]
+</pre></div>
+</div>
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>480kB [00:01, 407kB/s]                           
 </pre></div>
 </div>
 <div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>

rhythmic_analysis/meter_analysis.html

@@ -536,7 +536,7 @@ <h2> Contents </h2>
 <div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.00B [00:00, ?B/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 11:49:39,704] ERROR [root.download_from_remote:258] 
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 17:58:39,946] ERROR [root.download_from_remote:258] 
                             mirdata failed to download the dataset from hhttps://zenodo.org/records/14007971/files/compmusic_carnatic_rhythm_full_index_1.0.json?download=1!
                             Please try again in a few minutes.
                             If this error persists, please raise an issue at

structure_analysis/music-segmentation.html

@@ -552,49 +552,37 @@ <h2>Dhrupad Bandish segmentation<a class="headerlink" href="#dhrupad-bandish-seg
 <div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  0%|          | 0.00/1.59M [00:00&lt;?, ?iB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  1%|          | 16.4k/1.59M [00:00&lt;00:12, 127kiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  2%|▏         | 24.6k/1.59M [00:00&lt;00:10, 143kiB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  7%|▋         | 106k/1.59M [00:00&lt;00:04, 353kiB/s] 
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  3%|▎         | 49.2k/1.59M [00:00&lt;00:09, 154kiB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 16%|█▋        | 262k/1.59M [00:00&lt;00:01, 753kiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>  8%|▊         | 131k/1.59M [00:00&lt;00:04, 328kiB/s] 
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 22%|██▏       | 352k/1.59M [00:00&lt;00:02, 609kiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 19%|█▊        | 295k/1.59M [00:00&lt;00:02, 624kiB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 31%|███▏      | 500k/1.59M [00:00&lt;00:01, 833kiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 38%|███▊      | 606k/1.59M [00:00&lt;00:00, 1.14MiB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 43%|████▎     | 688k/1.59M [00:00&lt;00:01, 885kiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 77%|███████▋  | 1.23M/1.59M [00:00&lt;00:00, 2.39MiB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 52%|█████▏    | 819k/1.59M [00:01&lt;00:00, 984kiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 1.59M/1.59M [00:00&lt;00:00, 1.72MiB/s]
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 65%|██████▍   | 1.03M/1.59M [00:01&lt;00:00, 1.02MiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 18:01:50,909] INFO [compiam.utils.download.download_zip:95] Download complete: /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/compiam/models/structure/dhrupad_bandish_segmentation/dhrupad_bandish_segmentation.zip
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 81%|████████  | 1.29M/1.59M [00:01&lt;00:00, 1.36MiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 18:01:50,913] INFO [compiam.utils.download.extract_zip:103] Extracting /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/compiam/models/structure/dhrupad_bandish_segmentation/dhrupad_bandish_segmentation.zip...
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 96%|█████████▌| 1.52M/1.59M [00:01&lt;00:00, 1.30MiB/s]
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 18:01:50,929] INFO [compiam.utils.download.extract_zip:106] Extraction complete: Files extracted to /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/compiam/models/structure/dhrupad_bandish_segmentation
 </pre></div>
 </div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 1.59M/1.59M [00:01&lt;00:00, 1.05MiB/s]
-</pre></div>
-</div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 11:52:55,983] INFO [compiam.utils.download.download_zip:95] Download complete: /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/compiam/models/structure/dhrupad_bandish_segmentation/dhrupad_bandish_segmentation.zip
-</pre></div>
-</div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 11:52:55,987] INFO [compiam.utils.download.extract_zip:103] Extracting /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/compiam/models/structure/dhrupad_bandish_segmentation/dhrupad_bandish_segmentation.zip...
-</pre></div>
-</div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 11:52:56,005] INFO [compiam.utils.download.extract_zip:106] Extraction complete: Files extracted to /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/compiam/models/structure/dhrupad_bandish_segmentation
-</pre></div>
-</div>
-<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 11:52:56,006] INFO [compiam.utils.download.download_remote_model:71] Files downloaded and extracted successfully.
+<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[2024-12-01 18:01:50,930] INFO [compiam.utils.download.download_remote_model:71] Files downloaded and extracted successfully.
 </pre></div>
 </div>
 </div>