Slow to load with large music collection

[rustygreen]

If I point the music app to a large music collection (~500GB), to load the music app takes about ~30 seconds. I noticed one of the requests returns 17MB of data (in one request). I expected this app to be scalable. Is there a max practical limit for this app?

Once the music app loads (after a while), the app functions well.

Any suggestions?

[paulijar]

"500 GB" doesn't really tell much about the size of the library, except that it is large. But the significant figures for the Music app are the number of songs and number of albums. It doesn't matter if each song takes 1 KB or 1 GB of disk space.

It's true that the architecture of this app is more geared towards less than humongous music libraries. However, something like 50k songs on 5000 albums should still be tolerable. 100k songs might be too much although I haven't tested that. But at least it would take ages to scan those to even get started.

The 17 MB file you saw was probably the collection json. The good thing is that if your music library rarely changes, then this file doesn't need to be loaded on each page load because it will be cached on your browser. However, there is a small catch: The first time when each album cover gets loaded, the contents of the collection change slightly and this forces the file to be recreated and reloaded from the server on the next page load. But on static music library, the contents will eventually stabilize and caching can be utilized. Of course, if you obtain new music on daily basis then the file needs to be recreated each time and you don't really benefit from the caching.

[JoshuaPettus]

Ftr, if you find the web interface is too cumbersome, the music app is also an easy to use subsonic and apache server. You can always try one of those apps instead. Mileage may very depending on the app, but worth a shot. I'm sure will have similar long wait for initial loading, but an app's cache is generally less volatile than a web browser's. Wiki has these subsonic apps tested for windows.
https://github.com/jeffvli/sonixd
https://www.strawberrymusicplayer.org/

I personally use strawberry, but it's subsonic interface is quite limited. Clementine is also on the list but it's quite defunct, and Strawberry seems to be the continuation of that project.

[djtulan]

I have around 6000 Folders / 36000 Music Files in my Collection and the first Load of the Music App is very slow.
Strange thing is, that the Nextcloud server has nearly no processor or io load.
Is there any way to speed up loading?

Edit:

scanstate takes a big amount of the time.

[djtulan]

Albums & Folder View needs lots of time to load.
Genres, All tracks taking only a fraction of that time.

[paulijar]

scanstate takes a big amount of the time.

The UI should be shown and usable even if the scanstate request has not yet completed. This API endpoint is used to check if all audio files within the music library path are already scanned. If not, the user is shown a banner "New music available. Click here to start the scan". But it will inevitably take some time when there are lots of files.

[djtulan]

The banner is not showing up, everything is scanned already.
I assume the loading time is not depending on the nextcloud-"server" side. I assume, this is depending on the browser side. As I read the source code correctly I uses kind of Angular JS?

Maybe the nested "ng-repeat" slow down the build up of the view?

Genres view and All Tracks view are responding very fast.

[paulijar]

Albums & Folder View needs lots of time to load.
Genres, All tracks taking only a fraction of that time.

The whole collection is loaded from the server when you first enter the application, usually through the Albums view. That will take some time if the collection is huge and not yet cached. Building the folders tree structure also takes quite a bit of time for huge collections (*). That operation is postponed until the time when the user firsts navigates to the Folders view to not have to do it in case the user never enters the view.

Genres and All tracks views, on the other hand, are basically built out of the same data which was already loaded when you entered the application. That's why they are faster to load.

(*) And lot of magic has been applied to not have the folder tree building be insanely slow. If we were using the file system APIs of ownCloud/Netxcloud the way they are designed to be used, this operation could be slower by factor of 10 or more.

[djtulan]

1. I build up a cache already, by navigating through all the albums (Covers after showing Albums are also very quick)
2. loading Folder & Albums view always take much longer then than Genres & All tracks view, if I navigate through it.

Album View ~11sec
Folder View ~7sec
Genres View: immediately
All Tracks View: immediately

[paulijar]

The banner is not showing up, everything is scanned already.

Yes, that's the usual case. But to know not to show the banner, we need to check this anyway and it will take some time. But as stated, that's something happening on the background and shouldn't affect your user experience.

Maybe the nested "ng-repeat" slow down the build up of the view?

Yes, this is possible and even probable. However, there is a kind of lazy loading logic which ensures that only the currently visible albums are fully built. The most of the nested elements are just empty placeholders until when the view is scrolled to make them visible. Without this, the view building would be considerably slower.

Folder View ~7sec

Is this only the initial load of the view or does it take a similar amount of time if you navigate to another view and then back to the Folders view?

[djtulan]

Is this only the initial load of the view or does it take a similar amount of time if you navigate to another view and then back to the Folders view?

Yes, it always takes that amount, when navigating. During that time the icon-loading is shown.

Edit: I noticed, the "foldersFlatLayout" is faster. Half the time of the other view. 3,5Sec

[paulijar]

Do you have loads of folders (like thousands) directly under the root of your music library? Or did you expand lots of folders before doing the test? Because that's the only case where it would make sense that the "flat" layout is faster than the "tree" layout. I have a generated test library where I have around 60000 tracks in around 8000 folders. But on the main level, there's only a few folders. With this library, opening the Folders view in tree mode is instant. But if I expand lots of folders so that there are ~2000 folders visible, then activating the view takes 4-5 seconds.

[djtulan]

I have about 6000 directories in the root folder.

[paulijar]

Okay, so basically you have only one level of folders then. In that case, using the flat layout makes more sense. The flat view gives you the alphabet navigation on the right which makes it easier to navigate the long list. And it can utilize the lazy loading which is why you see it open up quicker. But it was awkward to navigate for those people who actually have organized their library using multiple levels of nested folders; that's why the tree layout alternative was added in the first place.

[Jayd603]

Same story here - 86,000 songs , each album has its own folder.. ~30-40 seconds to load, every load. Not really surprised, this is a lot of songs. Results could be cached then only loaded incrementally etc. ..or just have an option to adjust the number of albums to pre-load and the rest only show while scrolling or if a letter is clicked to seek