Fix performance regression in EspNet E2E with csj recipie

[take-cheeze]

relates to: #289

[take-cheeze]

GetXXXXX() could make some slow down

[shinh]

As I chatted locally, this regression was caused in this 3 months or so and #289 is not the culprit. So, let me highjack this issue to handle the perf regression.

Copy and paste of how to run the test:

$ PYTHONPATH=ch2o python3 ch2o/tests/model/EspNet_E2E.py --recipe csj_medium --gen csj_medium --gpu
$ ./build/tools/run_onnx --test csj_medium_backprop --backprop -d cuda -I 10 --fuse_operations --use_nvrtc
Average elapsed: 192.045 msec

The following command runs the same network by Chainer for reference:

$ PYTHONPATH=ch2o python3 ch2o/tests/model/EspNet_E2E.py --recipe csj_medium --run --gpu
Elapsed: 4776.919364929199 msec
Elapsed: 203.45425605773926 msec
Elapsed: 209.4278335571289 msec
Elapsed: 212.3851776123047 msec
Elapsed: 209.69533920288086 msec
Elapsed: 223.52290153503418 msec
Elapsed: 209.55896377563477 msec
Elapsed: 211.23909950256348 msec
Elapsed: 208.7113857269287 msec
Elapsed: 209.62882041931152 msec
Average elapsed: 210.84708637661404 msec

[shinh]

It looks like OneHot is the bottleneck, but it might not. I it's just forcing GPU to sync due to AsScalar or something. Maybe returning Scalar type from IntScalarConstantOp and FloatScalarConstantOp and using Scalar as an argument of OneHot would remove the bottleneck. I'm not sure just removing the GPU sync helps, but it would be a good change anyway.