-
Notifications
You must be signed in to change notification settings - Fork 27.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RWKV] Add RWKV6 model #34918
base: main
Are you sure you want to change the base?
[RWKV] Add RWKV6 model #34918
Conversation
Hi @uniartisan, this looks good, but can we rework the PR to use the new Modular style? It should make the PR much smaller and easier to write/review, since you can just import most of it from other models, including earlier RWKV models. |
I can have a try later:) |
What does this PR do?
This is currently a draft. I will supplement the documentation and related tests as soon as possible. RWKV6 and RWKV5 are derived from the same source, and I believe it is necessary to skip RWKV5 and only support RWKV6. After the RWKV7 pre-trained model is released, I will add support for RWKV7.
Fixes # (issue)
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.