-
Notifications
You must be signed in to change notification settings - Fork 27.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
timm to pytorch conversion for vit model fix #26908
timm to pytorch conversion for vit model fix #26908
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the cleanup, LGTM but I need a second look from @rwightman 🤗
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
Overall looking much better, should be more robust now. A few things, there are a number of vit configurations supported in timm that are not, to my knowledge, supported in transformers. Should there be an attempt to detect? Thinking of some examples
|
The git history is completely messed up!
Hey! Make sure to rebase to only have your changes! 😉 |
44432e3
to
330eaf2
Compare
I have reset the branch's history and left only my changes which fix the issue here. |
I have tried to add some checks before trying to convert the model from timm to huggingface. I have tested the script on the pre-trained ViTs and only the following give errors:
|
@staghado looking good, those hybrid resnet-vit models should be possible to catch (see if below) with a meaningful error .. other than looks ready to go
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good from the timm perspective
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding this and making the script more general!
Just a small question on an outstanding to-do. Otherwise LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for improving this script! 🚀
This PR fixes this issue #26219 with timm to PyTorch conversion. It removes the need for hard coded values for model dims by using the attributes of the timm model without needing the model name.
It does the following things :
This works for a large number of models in the ViT family.
@ArthurZucker, @amyeroberts, @rwightman