Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide fallback storage when using tus in Web Workers #664

Open
dumitrascuSorin opened this issue Jan 31, 2024 · 8 comments
Open

Provide fallback storage when using tus in Web Workers #664

dumitrascuSorin opened this issue Jan 31, 2024 · 8 comments

Comments

@dumitrascuSorin
Copy link

Is your feature request related to a problem? Please describe.
Following PR #660, uploading files from a Web Worker has been a breeze.
However, resuming an interrupted upload is no longer possible because of the hard dependency on localStorage, which is not supported in Web Workers.

Describe the solution you'd like
Would be nice to use a storage mechanism supported by Web Workers, such as the IndexedDB API.

Can you provide help with implementing this feature?
Yes, if the tus team is open to this idea.

Looking forward to your reply.

@Acconut
Copy link
Member

Acconut commented Feb 1, 2024

Good idea, we would be happy to accept a PR adding an IndexedDB-based URL storage to tus-js-client.

@geekypradip
Copy link

geekypradip commented Mar 16, 2024

Good idea, we would be happy to accept a PR adding an IndexedDB-based URL storage to tus-js-client.

I've implemented a new feature for my personal project. Previously, I encountered issues when uploading more than 30,000 files at once. The problem stemmed from local storage limitations, causing it to fill up quickly and ultimately crash the browser. To address this, I integrated indexDB-based storage into the system. Now, the application checks whether the client's browser supports indexDB. If it does, the system utilizes indexDB; otherwise, it falls back to local storage or noopStorage.

@Acconut
Copy link
Member

Acconut commented Mar 25, 2024

@geekypradip Thank you for the PR. Anybody who is interested can have a look at #677

@kesavkolla
Copy link

@geekypradip better option would be using OPFS so that we can get better I/O than IndexDB.

@Acconut
Copy link
Member

Acconut commented Nov 21, 2024

@kesavkolla Would you mind elaborating how OPFS could be better than IndexedDB for this use case? tus-js-client doesn't read and write much, so I/O performance is not a huge concern.

@kesavkolla
Copy link

If we write the file to OPFS then tus can guarantee even browser restart will continue upload.

In general OPFS read/write are faster than Indexdb. Indexdb again depends on the async loop and also has to do serialize and deserialize the data.

OPFS is native file I/O so read write doesn't involve any serialization.

@Acconut
Copy link
Member

Acconut commented Nov 21, 2024

Thanks for the insights!

If we write the file to OPFS then tus can guarantee even browser restart will continue upload.

Do you know if that's not the case for IndexedDB? Does data in IndexedDB get erased when the browser closes?

@kesavkolla
Copy link

Indexdb will keep data after browser restarts. But yeah performance of file I/O is better than Indexdb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants