[READY] nix.checks: rework and clean up #803
Conversation
Adds facts source to the checks code that can be used for several checks. This is better than coping all of `self` which creates a new derivation every time anything changes in the repository.
Reworked pytest-facts to use `factsSrc` so it does not rebuild every time anything changes in the repository. Added the testPython environment to the build inputs so that `pylint` and `pytest` can be called directly.
Rework `duplicates-facts` to use `factsSrc` so it does not rebuild every time anything changes in the repository.
`fish` was raising warnings and errors because it is running in the sandbox. It tries to write files as if it had access to the $HOME directory but it does not. Adding the `--no-config` flag tells it not to read from a configuration file so it will not go hunting for one or try to write files where it cannot. The warnings/errors below are what `fish` was writing to the shell. These do not show up with this fix. > duplicates-facts> error: can not save history > duplicates-facts> warning-path: Unable to locate data directory derived from $HOME: '/homeless-shelter/.local/share/fish'. > duplicates-facts> warning-path: The error was 'Permission denied'. > duplicates-facts> warning-path: Please set $HOME to a directory where you have write access. > duplicates-facts> error: can not save universal variables or functions > duplicates-facts> warning-path: Unable to locate config directory derived from $HOME: '/homeless-shelter/.config/fish'. > duplicates-facts> warning-path: The error was 'Permission denied'. > duplicates-facts> warning-path: Please set $HOME to a directory where you have write access.
Adds `switchConfigurationSrc` source to checks that can be used for other checks. This will replace derivations that use `self` for the source which creates a new derivation every time anything in the repository changes.
Rework perl-switches test to use `switchConfigurationSrc` so it does not rebuild every time anything changes in the repository. Cannot use this with runCommand because it does not set the correct permissions for source files and directories.
djacu
changed the title
There was a problem hiding this comment.
@djacu looking much better, no rebuilding on every change anywhere in the repo! Had one question for my clarification.
openwrt-golden still has the old behavior because I kept getting permission denied errors when gomplate tried to create new directories.
Hmm let me look at why this is. iirc theres some odd behavior with gomplate that Ive run into before. Id be nice to get all of these tests fixed while its top of mind.
| # sources | ||
|
|
||
| # Used for derivations where facts is the primary directory. | ||
| factsSrc = toSource { |
There was a problem hiding this comment.
Is the thought here that eventually we might have more (or less) required for the factSrc? Since at the moment they are the same dirs between factsSrc and switchConfigurationSrc
There was a problem hiding this comment.
It was just a coincidence that they share the same source. When I started with factSrc, I didn't realize it needed more that the facts directory because the scripts reach up and over into the switch-configuration directory for files. And then the same happened when I made switchConfigurationSrc. I started with it only having the switch-configuration directory but then the scripts failed because they reach into the facts directory.
I moved forward with keeping them separate because I wasn't sure if or how files would be added in new directories or consolidated into fewer. I thought about writing a fileset source in situ for each derivation. I also though about consolidating factSrc and switchConfigurationSrc into a single source. Opted to leave it as is for us to discuss.
Ideally, some time in the future, it would be ideal that these scripts don't reach across parent directories like they do but that can be tackled another day.
There was a problem hiding this comment.
Hmm let me look at why this is. iirc theres some odd behavior with gomplate that Ive run into before. Id be nice to get all of these tests fixed while its top of mind.
Yeah I hacked at it for a couple hours and kept running into issues with gomplate. FYI if you want to hack on this, switch to using mkDerivation like I did for perl-switches. It sets write permissions on the source that you don't get with runCommand.
Including a openwrtSrc for any openwrt specific checks that we have. Additionally, reworking the openwrt-golden test to consume from this new src.
sarcasticadmin
force-pushed
the
djacu/cleanup-checks
branch
from
b19540e to
06e5bf5
Compare
All checks have been moved over to their respective fileset sources so we no longer need to use the cleanSource function against inputs.self.
|
@djacu This looks like its working as expected now for |
|
@sarcasticadmin now that I see your implementation, I know what I was doing wrong. In the build phase, you have and I had which does not It was trying to make new files/directories in the source input which is read only. |
|
Ooo yep that'll definitely cause problems 😅 Im glad we know what was going wrong |
Makes the test.sh script executable so that bash does not need to be invoked in the build environment which is already running bash.
|
Alright I pushed one more commit. Check that you are good with it and this should be good to go. Also I don't have merge authority |
Description of PR
Reworks the nix checks to reduce the use of
selfas a source. Switch to usingfilesetsto limit the scope of what is considered as part of the closure and gets copied into the nix store.Previous Behavior
self.duplicate-factsusesfishto run a script file.fishtries to find and create config files but cannot because it is running in the sandbox and throws warnings/errors. They do not cause the derivation to fail to build but they do pollute the output.New Behavior
pytest-facts,duplicate-facts, andperl-switcheshave restricted source inputs which will limit what causes them to rebuild.openwrt-goldenstill has the old behavior because I kept gettingpermission deniederrors whengomplatetried to create new directories.fishstops throwing warnings/errors. See c2c6d4e.Tests
How was this PR tested?
nix flake check