Add Gemma2 GGUF support

[yijun-lee]

What does this PR do?

Add GGUF support for Gemma2
#33260

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[bekzod]

@yijun-lee really hoping this gets merged soon 🙏 really need this on our project

[yijun-lee]

@yijun-lee really hoping this gets merged soon 🙏 really need this on our project

I’m working on it bit by bit whenever I have time🙂. I’ll make sure to wrap it up soon! Thank you for your interest !

[bekzod]

@yijun-lee thank you for your work, this helps a lot of people !

[SunMarc]

Feel free to tag me when this is ready !

[yijun-lee]

@SunMarc
I realize I’ve left this task unattended for quite some time now 😅
But it’s finally complete—I’d greatly appreciate your review!

[SunMarc]

Thanks for the PR @yijun-lee ! @Isotr0py have a PR that refactor quite a bit the GGUF integration, so that we don't need to specify the tensor mapping. However, we still need the GGUFGemmaConverter that you coded. I'll let him review and decide if we merge this before or after his PR gets merged

[Isotr0py]

I tested the Gemma2 support introduced in this PR on my PR, and tests can still pass on that branch as well.

I'm fine to have this merged before my PR, because the only difference is the removal of GGUF_TENSOR_MAPPING, and there won't be too many conflicts.

[SunMarc]

LGTM then ! Thanks for iterating

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.