qwen2vl support

[eaidova]

What does this PR do?

Enable qwen2vl models export and inference with openvino.
Example of usage:

import requests
from PIL import Image
from transformers import AutoProcessor

from optimum.intel.openvino import OVModelForVisualCausalLM


model_id = "Qwen/Qwen2-VL-2B-Instruct"
# Load the model in half-precision on the available device(s)
model = OVModelForVisualCausalLM.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id)

# Image
url = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"
image = Image.open(requests.get(url, stream=True).raw)

conversation = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]


# Preprocess the inputs
text_prompt = processor.apply_chat_template(conversation, add_generation_prompt=True)
# Excepted output: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>Describe this image.<|im_end|>\n<|im_start|>assistant\n'

inputs = processor(text=[text_prompt], images=[image], padding=True, return_tensors="pt")

# Inference: Generation of the output
output_ids = model.generate(**inputs, max_new_tokens=10)
generated_ids = [output_ids[len(input_ids) :] for input_ids, output_ids in zip(inputs.input_ids, output_ids)]
output_text = processor.batch_decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True)
print(output_text)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[AlexKoff88]

I don't have major comments. Thanks @eaidova!

@echarlaix, @IlyasMoutawwakil, the PR is ready for your review.

[echarlaix]

Thanks for the addition @eaidova !

[eaidova]

@echarlaix could you please check one more time?
Thanks!

[echarlaix]

Awesome work, thanks a lot @eaidova