Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import OVQuantizer only if it is required in VLM #1048

Merged
merged 4 commits into from
Dec 5, 2024
Merged

import OVQuantizer only if it is required in VLM #1048

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3181535
import OVQuantizer only if it is required in VLM
eaidova Dec 4, 2024
e26f516
do not use TaskManager for getting task constant
eaidova Dec 4, 2024
97e9141
avoid import onnx in modeling_base and stateful
eaidova Dec 4, 2024
6f6b245
reduce onnx dependency
eaidova Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion optimum/intel/openvino/modeling_visual_language.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@
from ...exporters.openvino import main_export
from ...exporters.openvino.stateful import ensure_stateful_is_available, model_has_input_output_name
from ...exporters.openvino.utils import save_config
from .. import OVQuantizer
from .configuration import OVConfig, OVWeightQuantizationConfig
from .modeling_base import OVBaseModel, OVModelPart
from .modeling_decoder import CausalLMOutputWithPast, OVModelForCausalLM
Expand Down Expand Up @@ -549,6 +548,8 @@ def _from_pretrained(
)

if to_quantize:
from optimum.intel.openvino.quantization import OVQuantizer

quantization_config_copy = copy.deepcopy(quantization_config)
quantization_config_copy.tokenizer = quantization_config.tokenizer or model_id
potential_processor_id = config.mm_vision_tower if isinstance(model, _OVNanoLlavaForCausalLM) else model_id
Expand Down
Loading