Add infer request queue for tokenizers and allow for optional plugin_config in tokenizer #1010
Job | Run time |
---|---|
12m 19s | |
10m 27s | |
10m 38s | |
18m 28s | |
17m 25s | |
10m 45s | |
1h 20m 2s |
Job | Run time |
---|---|
12m 19s | |
10m 27s | |
10m 38s | |
18m 28s | |
17m 25s | |
10m 45s | |
1h 20m 2s |