Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow copying from cache to output folder #1109

Open
ForestQ opened this issue Oct 30, 2024 · 1 comment
Open

Slow copying from cache to output folder #1109

ForestQ opened this issue Oct 30, 2024 · 1 comment

Comments

@ForestQ
Copy link

ForestQ commented Oct 30, 2024

I have 3 machines with 12 GPU workers, 6 CPU workers, and 12 health checkers. There's a 1 Gbit network between the sources, output folder, and cache folder.
When I upload files, - currently testing with 100 files totaling 20 gigabytes — the conversion takes about 3 minute. The cache folder fills up with files totaling around 2 gigabytes, and then slow copying to the output folder begins. Uploading 100 files takes around 7 minutes, even though the 1 Gbit speed is available.
I understand this is due to single-threaded copying from the cache to the output folder. I ran a test on a local machine using a single SSD without networks, and copying is still slow in a single thread through the health check. Is it possible to increase the number of concurrent threads from the cache to the output folder?
I want to say that the software is excellent! Thank you!
Screenshot_1

@HaveAGitGat
Copy link
Owner

If you use flows then each worker does the copy/move itself so multiple will happen in parallel depending on how many workers a running.

You can optimise this further by using the node tags feature (pro) so you can create a node that is just for moving the file at the end and have e.g. 5 workers for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants