SnapKV_Cache support added

[shirinyamani]

What does this PR do?

This PR adds the implementation of SnapKV Cache paper to cache_utils.py file. Also adds the changes in flash_attention2 in llama_modeling to reflect the initialization of the SnapKV cache.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[LysandreJik]

Pinging the cache gang @zucchini-nlp @ArthurZucker @gante

[ArthurZucker]

Hey! super interested in having new kv cache implementations, but we need to make them compatible with the current API! Tha kv length is contained in the cache_positions which is what should be used! No modeling changes should be required!

[shirinyamani]

@ArthurZucker Thanks alot for your response! So do you think if I remove the modeling changes then the SnapKV class is fine to be added to the cache_utils.py ? , couldn't find the cache_positions you are referring to in the files (cache n llama)!

[ArthurZucker]

Yes !

transformers/src/transformers/models/llama/modeling_llama.py

Line 282 in 0ade1ca

cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position}

cache positions are here.
Overall we should not need modeling changes!

[ArthurZucker]

But with #35235 you can maybe find a better way to support this by creating a new attn_implementation and add it to the mapping? 🤗