Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: if nolint files change, do a full rebuild #19275

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

eric-wieser
Copy link
Member

@eric-wieser eric-wieser commented Nov 20, 2024

Otherwise CI will succeed when removing entries from the file, but fail later when someone changes something unrelated.


Open in Gitpod

Copy link

github-actions bot commented Nov 20, 2024

PR summary f706345d1a

Import changes for modified files

No significant changes to the import graph

Import changes for all files
Files Import difference

Declarations diff

No declarations were harmed in the making of this PR! 🐙

You can run this locally as follows
## summary with just the declaration names:
./scripts/declarations_diff.sh <optional_commit>

## more verbose report:
./scripts/declarations_diff.sh long <optional_commit>

The doc-module for script/declarations_diff.sh contains some details about this script.


No changes to technical debt.

You can run this locally as

./scripts/technical-debt-metrics.sh pr_summary
  • The relative value is the weighted sum of the differences with weight given by the inverse of the current value of the statistic.
  • The absolute value is the relative value divided by the total sum of the inverses of the current values (i.e. the weighted average of the differences).

@eric-wieser eric-wieser changed the title fix: if nolints-style.txt changes, do a full rebuild fix: if nolint files change, do a full rebuild Nov 20, 2024
@adomani
Copy link
Collaborator

adomani commented Nov 20, 2024

Thanks!

bors d+

@mathlib-bors
Copy link
Contributor

mathlib-bors bot commented Nov 20, 2024

✌️ eric-wieser can now approve this pull request. To approve and merge a pull request, simply reply with bors r+. More detailed instructions are available here.

@eric-wieser
Copy link
Member Author

Before merging, we should locally test that changing the file causes lake build to rebuild everything

@adomani
Copy link
Collaborator

adomani commented Nov 20, 2024

You may need to re-add the removed lemma. For instance, you could cherry-pick 2858e09.

@adomani
Copy link
Collaborator

adomani commented Nov 20, 2024

That may also serve as a test that the rebuilding will happen.

Copy link
Collaborator

@grunweg grunweg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me also: one question and one documentation suggestion.

lakefile.lean Outdated
@[default_target]
lean_lib Mathlib where
leanOptions := mathlibLeanOptions
-- Mathlib also enforces these linter options, which are not active by default.
moreServerOptions := mathlibOnlyLinters
extraDepTargets := #[
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: add a comment why this is here? (Perhaps this is sufficiently obvious... I'm not sure. Up to you.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

lakefile.lean Outdated
@@ -62,11 +62,21 @@ package mathlib where
## Mathlib libraries
-/

target nolints_style (pkg) : System.FilePath :=
inputTextFile <| pkg.dir / "scripts" / "nolints-style.txt"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, you could probably remove this one, right? nolints-style.txt is only used for lake exe lint-style, which is run as a separate executable by CI. (This is not related to syntax linters and caching their results.)

Leaving this in for paranoia is fine with me --- how about adding a comment that this is not defence in depth, and currently not actually required?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can; I removed it and added a comment explaining a little more about why.

In principle someone could start calling the function that reads nolints-style.txt from another linter, then suddenly we would need this again.

Probably it would be best to refactor things such that the script reads the file, and passes the result into the library function, to make it clear that the library can't read the file at import-time; but that's out of scope for this PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thanks!
I'm adding the lint-style refactoring to my list of things; expect a PR within 1-2 weeks. (Currently, some more urgent things have to be finished first. But this one should be very quick, once I start it.)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, I had some time earlier - #19384 implements this.

@grunweg
Copy link
Collaborator

grunweg commented Nov 20, 2024

Also: does nolints.json need the same treatment? Or is it just as superfluous as nolints-style.txt?

@adomani
Copy link
Collaborator

adomani commented Nov 20, 2024

You may need to re-add the removed lemma. For instance, you could cherry-pick 2858e09.

Btw, master is "fixed" now, without this commit. I am not completely sure about what happened there...

@adomani
Copy link
Collaborator

adomani commented Nov 21, 2024

Something funny may be going on with the cache: in #19322, the step that checked the cache failed and re-running the jobs rebuilt all of mathlib again, even though the upload step had apparently worked the first time around.

First attempt

Second attempt

@eric-wieser
Copy link
Member Author

Perhaps relevant is that the lake exe cache get step does not respect this new dependency, so it will claim the cache is legal but then discover that it is invalid.

@eric-wieser
Copy link
Member Author

@adomani, I take it your comment above means this PR should not be merged?

@adomani
Copy link
Collaborator

adomani commented Nov 21, 2024

I like that CI will rebuild everything if the no_lint file is modified, but I want to make sure that this change will not create issues for cache. Would you mind trying to modify the no_lints file and seeing if the cache gets uploaded correctly?

Simply sorting the file could be enough? Or adding an extra line break somewhere.

grunweg added a commit that referenced this pull request Nov 21, 2024
For consistency with the other nolints files. Suggested in #19275.
mathlib-bors bot pushed a commit that referenced this pull request Nov 22, 2024
For consistency with the other nolints files. Suggested in #19275.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants