Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ersi lig 3912 refactor mae to use timm vit #1461

Merged
merged 59 commits into from
Mar 6, 2024
Merged

Ersi lig 3912 refactor mae to use timm vit #1461

merged 59 commits into from
Mar 6, 2024

Conversation

ersi-lightly
Copy link
Contributor

No description provided.

guarin and others added 25 commits May 30, 2023 14:25
@guarin
Add MAE evaluation
951d43e
@guarin
Add stochastic depth dropout
503cc44
@guarin
Add MAE
ac43499
@guarin
Drop assertion
15bfe3a
@guarin
Fix smooth cross entropy loss and mixup
49c85c0
@guarin
Update comments
9d95783
@guarin
Add layer lr decay and weight decay
0bb601d
@guarin
Update comment
d7d69af
@guarin
Add test for MAE images_to_tokens
ec05437
@guarin
Disable BN update
923a606
@guarin
Add BN before classification head
bdce8a6
@guarin
Format
316f918
@guarin
Fix BN freezing
a6943fd
@guarin
Cleanup
1a2b454
@guarin
Use torch.no_grad instead of deactivating gradients manually
bc066ae 
* This is required as torch.no_grad doesn't change the model configuration
  while manual gradient deactivation/activation can have unintended
  consequences. For example, MAE ViT positional embeddings are parameters
  with requires_grad=False that should never receive an update. But if
  we use activate_requires_grad for finetuning we break those
  parameters.
@guarin
Create new stochastic depth instances
d56e340
@guarin
Add mask token to learnable params
5ed6803
@guarin
Add sine-cosine positional embedding
4f0baf1
@guarin
Initialize parameters as in paper
9c4a8cf
@guarin
Merge branch 'master' into guarin-lig-3056-add-mae-imagenet-benchmark
9904c10
@guarin
Fix types
83edd1c
@guarin
Format
e27946e
@ersi-lightly
Merge branch 'guarin-lig-3056-add-mae-imagenet-benchmark' of github.c…
0672b0a 
…om:lightly-ai/lightly into guarin-lig-3056-add-mae-imagenet-benchmark
@ersi-lightly
adjusted to existing interface
45433c5
@ersi-lightly
draft
c5cab9e
@codecov Codecov
Copy link

codecov bot commented Dec 19, 2023 •
edited
Loading

Codecov Report

Attention: Patch coverage is 23.68421% with 203 lines in your changes are missing coverage. Please review.

Project coverage is 82.07%. Comparing base (4dede59) to head (d89808f).

Files Patch % Lines
...y/models/modules/masked_vision_transformer_timm.py 0.00% 94 Missing ⚠️
...s/modules/masked_vision_transformer_torchvision.py 30.43% 48 Missing ⚠️
lightly/models/modules/masked_autoencoder_timm.py 0.00% 45 Missing ⚠️
lightly/models/modules/masked_autoencoder.py 84.84% 5 Missing ⚠️
lightly/models/utils.py 16.66% 5 Missing ⚠️
...ightly/models/modules/masked_vision_transformer.py 75.00% 4 Missing ⚠️
lightly/models/modules/__init__.py 33.33% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #1461      +/-   ##
==========================================
- Coverage   84.35%   82.07%   -2.28%     
==========================================
  Files         140      144       +4     
  Lines        5802     6065     +263     
==========================================
+ Hits         4894     4978      +84     
- Misses        908     1087     +179

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

guarin
guarin requested changes Feb 26, 2024
View reviewed changes
Copy link
Contributor

@guarin guarin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Left some comments :)

examples/pytorch/mae.py Outdated Show resolved Hide resolved
docs/source/getting_started/install.rst Outdated Show resolved Hide resolved
examples/pytorch/mae.py Outdated Show resolved Hide resolved
examples/pytorch/mae.py Outdated Show resolved Hide resolved
examples/pytorch_lightning/msn.py Show resolved Hide resolved
examples/pytorch_lightning_distributed/pmsn.py Outdated Show resolved Hide resolved
lightly/models/modules/__init__.py Outdated Show resolved Hide resolved
lightly/models/modules/__init__.py Outdated Show resolved Hide resolved
guarin
guarin reviewed Feb 29, 2024
View reviewed changes
lightly/models/modules/__init__.py Outdated Show resolved Hide resolved
lightly/models/modules/masked_autoencoder_timm.py Outdated Show resolved Hide resolved
lightly/models/modules/masked_vision_transformer_timm.py Outdated Show resolved Hide resolved
lightly/models/utils.py Show resolved Hide resolved
lightly/utils/dependency.py Outdated Show resolved Hide resolved
tests/models/modules/test_masked_autoencoder.py Outdated Show resolved Hide resolved
tests/models/modules/test_masked_autoencoder.py Outdated Show resolved Hide resolved
tests/models/modules/test_masked_autoencoder.py Outdated Show resolved Hide resolved
@ersi-lightly ersi-lightly reopened this Feb 29, 2024
@ersi-lightly
Ersi lig 4471 cleanup and merge mae branch (#1510)
d89808f 
* renamed test class

* fixed imports

* ficed imports

* fixed import

* fixed imports and decreased batch size

* format

* removed comments

* use function defined in utils

* added docstrings

* added doctrings

* added docstring

* formatted

* formatted

* import Tensor
@ersi-lightly ersi-lightly merged commit 053327d into master Mar 6, 2024
8 of 10 checks passed
@ersi-lightly ersi-lightly deleted the ersi-lig-3912-refactor-mae-to-use-timm-vit branch March 6, 2024 08:45
This was referenced Jul 26, 2024
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guarin guarin guarin approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ersi-lightly @guarin