Better WebDAV-file-server module as the current one is dirt-slow

[Ozymandias42]

Context

I'm using Nextcloud on multiple devices, only some of which I can or am willing to use the client on.
Some of them being VMs for which it's easier to just mount via Webdav for the only very occasional need for some file, others are mobile devices on which I use the Cryptomator Client to access individual encrypted folders on my Nextcloud.

As such having accessed Nextcloud from all manner of WebDAV clients I was able to eliminate the clients itself as source for sometimes unreasonably long load times.

Problem

WebDAV access is dirt slow. Especially on reading the directory structure upon entering/loading a folder and then again upon loading or uploading a file.

Test Environment

Production Nextcloud Server with around 5000 files hosted in k3s with traefik and the community docker image
From those accessed are around 1600 in a Cryptomator container.
Subsequent test-environments those 1600 are the only ones put there.

WebDAV Clients used over time

Clients included Plasma's Dolphin's KIO framework, the Cryptomator Android app, rclone (thanks to vfs-caching the fastest out of all clients), davfs2, macOS' Finder, Microsoft's Explorer.
WebDAV Clients used in the test:

• Dolphin's KIO framework
• Cryptomator on Android

Other WebDAV servers or Nextcloud configurations tested with the same files on the same storage backend as comparison and results:

Protocol	Client	Result
webdav	seafile's seafdav	blazingly fast.
webdav	Nextcloud test instance with primary storage on s3 via minio	just as slow as the main instance
webdav	External Storage in Nextcloud on the same minio server accessed via nextcloud's webdav export	just as slow as the main instance
s3	direct access to the bucket for the external storage test	blazingly fast

Suggestions and Feature request

Use a different module/library/framework for the webdav file server module or change it's implementation.
I don't know if this is only the case with the s3 primary storage backend but the database should hold the directory structure and metadata too so why not just query that instead of doing whatever slow process it is currently doing to load files and directory-structure data?

Of the reasons I assume seafile is so fast with webdav, one is it doing just that. Seafile chunks all files into it's own pseudo-s3 object structure (or actual s3 objects in the pro version) and -like nextcloud- stores all metadata and structure data in the database.

With seafile I can scroll through image galleries via webdav and have the images load pretty much as quickly as my connection allows. With Nextcloud each image, even ones just a few kilobytes in size, at the very least a two or more seconds and that would be surprisingly fast. Certainly, however a lot more than would be due to connection limits.

Furthermore even without that chunking, the access times for the external-storage bucket are identical to that of seafile and here the full files and structure is merely transferred into the object tree and not also chunked like seafile does. As S3 and webdav both are pretty much identical on pure read-only access in terms of their REST API I think the comparison is a fair one to make at this point.

[SpirioSpirio]

+1, I have the same need !

[joshtrichards]

See WebDAV Client Small Files Performance Testing for some testing I did last year.

[synthfi]

I find that the browser interface is consistently faster than using the webdav interface with dolphin KIO.

Additionally with webdav access: the first time I access a folder takes the longest (couple of seconds). Navigating to another folder and then going back to the folder is sub-second.