Add Mamba2

[zzhhjjj]

What does this PR do?

Fixes #31204 (issue)

[amyeroberts]

cc @ArthurZucker

[vasqu]

This doesn't look like Mamba2 at all, just a copy-paste of Jamba. Is it meant to be a placeholder for now?

One can easily see it as mamba2 doesn't use attention (except for some ablation studies) and the core (trition) functions mamba_chunk_scan_combined and mamba_split_conv1d_scan_combined are not even imported. Not to be rude but what's the purpose behind this PR? 😅

[Adibvafa]

@vasqu I agree. Do you wanna work on a PR together?

[vasqu]

@Adibvafa look into the issue #31204 , you can help me there or I can help here / a different PR if this is not being continued.

[ArthurZucker]

Feel free to open a PR and maybe leverage the new diff conversion! #30868

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.