YOLOx eval has been enabled #1
Conversation
added data_num_workers parameter performance metrics' list has been expanded enabled --fuse Updated README enabled eager mode added --cpu-post-processing option added --warmup_steps added total time metric added Postprocess class with optimized postproc implementation added mark_step before tensor coping Change-Id: Iee48da7cbe3862a4f50795f2edefe7973f991be3
PyTorch/computer_vision/detection/yolox/yolox/evaluators/coco_evaluator.py
Show resolved
Hide resolved
PyTorch/computer_vision/detection/yolox/yolox/evaluators/coco_evaluator.py
Outdated
Show resolved
Hide resolved
PyTorch/computer_vision/detection/yolox/yolox/evaluators/coco_evaluator.py
Outdated
Show resolved
Hide resolved
PyTorch/computer_vision/detection/yolox/yolox/evaluators/coco_evaluator.py
Outdated
Show resolved
Hide resolved
|
|
||
| random.seed(input_shape_seed) | ||
| # torch.set_num_interop_threads(7) | ||
| # torch.set_num_threads(7) |
There was a problem hiding this comment.
remove comments, but before checking this
PyTorch/computer_vision/detection/yolox/yolox/evaluators/coco_evaluator.py
Show resolved
Hide resolved
PyTorch/computer_vision/detection/yolox/yolox/evaluators/coco_evaluator.py
Outdated
Show resolved
Hide resolved
| @@ -216,7 +255,7 @@ def convert_to_coco_format(self, outputs, info_imgs, ids): | |||
| for (output, img_h, img_w, img_id) in zip( | |||
| outputs, info_imgs[0], info_imgs[1], ids | |||
| ): | |||
| if output is None: | |||
| if output.size(0) == 0: | |||
There was a problem hiding this comment.
will it work with original function for voc dataset?
| cls = output[:, 5] | ||
| scores = (output[:, 4]).float() |
There was a problem hiding this comment.
will it work with original function for voc dataset?
|
@Luca-Calabria |
| **Run validation on 1 HPU:** | ||
| * FP32 data type: | ||
| ```bash | ||
| $PYTHON tools/eval.py -n yolox-s -c path/to/yolox_s.pth --data_dir path/to/data/COCO -b 256 -d 1 --conf 0.001 --data_num_workers 4 --hpu --fuse --cpu-post-processing --warmup_steps 4 |
There was a problem hiding this comment.
Instead of path/to/data/COCO use /data/COCO, in agreement to the training section above
| **Run validation on 1 HPU:** | ||
| * FP32 data type: | ||
| ```bash | ||
| $PYTHON tools/eval.py -n yolox-s -c path/to/yolox_s.pth --data_dir path/to/data/COCO -b 256 -d 1 --conf 0.001 --data_num_workers 4 --hpu --fuse --cpu-post-processing --warmup_steps 4 |
There was a problem hiding this comment.
You are mixing - and -- args, please use only -- (e.g., --name, --devices, --batch-size)
There was a problem hiding this comment.
This applies to all your command lines
| self.cpu_post_processing = cpu_post_processing | ||
| self.warmup_steps = warmup_steps | ||
|
|
||
| self.post_proc_device = None |
There was a problem hiding this comment.
What are the options? if torch.cuda.is_available() maybe you want to force it to gpu, else, I would directly go on cpu. Or is there an option in which is worth sending it to hpu?
There was a problem hiding this comment.
If not, I would remove the option, and freeze the code. That one was useful to test at dev time
There was a problem hiding this comment.
yes, we can do it on hpu if user set --hpu but not set --cpu-post-processing.
This script was not validated on gpu (also there's no strict requirements for that), so this way prevents usage of not validated scenario.
There was a problem hiding this comment.
yes, we can do it on hpu if user set --hpu but not set --cpu-post-processing.
This is clear from code, but it is not convenient to set --hpu and at the same time to keep the post processing on hpu (wasn't it slow?). If this is not convenient, then why we don't remove the --cpu-post-processing side, and send the post processing either on cuda, if avail, or to cpu directly? My question is more on whether it makes sense to keep the option for user, or just force the code on the most convenient direction
There was a problem hiding this comment.
So, from my point of view post-processing should work on HPU with good performance, but we can't achieve it now even with eager mode (but we never tried torch.compile option), so my opinion is if we set --hpu option whole pipeline should work on --hpu. This behavior is expected by user who usually don't review the code.
However, we can achieve better performance with CPU fallback for post-processing. And it is optimization option so, my opinion is it should stay as an option in command line.
There was a problem hiding this comment.
I see your point. Still not convinced that this is useful at the moment, as the implicit message is CPU is better than HPU for postprocessing. Which is true, but maybe there is no need to highlight it with an option. Please, go ahead in creating the PR for the model garden to collect other feedbacks
added data_num_workers parameter
performance metrics' list has been expanded
enabled --fuse
Updated README
enabled eager mode
added --cpu-post-processing option
added --warmup_steps
added total time metric
added Postprocess class with optimized postproc implementation added mark_step before tensor coping