image_transforms preprocess quite slow when run large image with qwen2vl

[zhjunqin]

System Info

• transformers version: 4.45.2
• Platform: Linux-5.4.0-132-generic-x86_64-with-glibc2.31
• Python version: 3.12.7
• Huggingface_hub version: 0.25.1
• Safetensors version: 0.4.5
• Accelerate version: 1.0.0
• Accelerate config: not found
• PyTorch version (GPU?): 2.4.0+cu121 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?:
• Using GPU in script?:
• GPU type: NVIDIA GeForce RTX 3090

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

funcitons in image_transforms, rescale, normalize quite slow when preprocess large image.
https://github.com/huggingface/transformers/blob/main/src/transformers/image_transforms.py

here is benchmark

please refer to vllm-project/vllm#9238

Expected behavior

how to improve performance?

[zucchini-nlp]

Hey @zhjunqin !

Can be related to #28847, where we enabled image processing with torchvision but that only is supported in ViT model. Also @yonigozlan is working on optimizing image processing time in #33810, so he might be your point of contact :)

[yonigozlan]

Hey @zhjunqin !
Thanks a lot for raising this issue. Indeed I'm currently working on adding fast image processors to Transformers, and I'll try to address the Qwen one shortly. I'll ping this issue once a PR is opened!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[SinanAkkoyun]

@yonigozlan Hey :) Did you find time to address the qwen preprocessor?

[yonigozlan]

Not yet, but it is still planned :). I will ping here when it's done

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.