Partly finetune of OpenCLIP

[Kimsure]

Does anyone know how to partly finetune the CLIP model? For example,

1. only the attention layer, such as, query and key projection layers are finetuned while value projection layers is frozen.
2. only several attention layers are finetuned while other layers are frozen.
   Some works implemented the above operation based on the original CLIP code. OpenCLIP uses timm model so help is needed