You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have 3 machines with 12 GPU workers, 6 CPU workers, and 12 health checkers. There's a 1 Gbit network between the sources, output folder, and cache folder.
When I upload files, - currently testing with 100 files totaling 20 gigabytes — the conversion takes about 3 minute. The cache folder fills up with files totaling around 2 gigabytes, and then slow copying to the output folder begins. Uploading 100 files takes around 7 minutes, even though the 1 Gbit speed is available.
I understand this is due to single-threaded copying from the cache to the output folder. I ran a test on a local machine using a single SSD without networks, and copying is still slow in a single thread through the health check. Is it possible to increase the number of concurrent threads from the cache to the output folder?
I want to say that the software is excellent! Thank you!
The text was updated successfully, but these errors were encountered:
If you use flows then each worker does the copy/move itself so multiple will happen in parallel depending on how many workers a running.
You can optimise this further by using the node tags feature (pro) so you can create a node that is just for moving the file at the end and have e.g. 5 workers for that.
I have 3 machines with 12 GPU workers, 6 CPU workers, and 12 health checkers. There's a 1 Gbit network between the sources, output folder, and cache folder.
When I upload files, - currently testing with 100 files totaling 20 gigabytes — the conversion takes about 3 minute. The cache folder fills up with files totaling around 2 gigabytes, and then slow copying to the output folder begins. Uploading 100 files takes around 7 minutes, even though the 1 Gbit speed is available.
I understand this is due to single-threaded copying from the cache to the output folder. I ran a test on a local machine using a single SSD without networks, and copying is still slow in a single thread through the health check. Is it possible to increase the number of concurrent threads from the cache to the output folder?
I want to say that the software is excellent! Thank you!
The text was updated successfully, but these errors were encountered: