-
Notifications
You must be signed in to change notification settings - Fork 420
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(tofs
): allow TOFS for master+minion configuration
#398
Conversation
Wow, this will take some time.... |
@aboe76 Tell me now if this is a bad idea!! Is this something that the formula should handle or not? |
@aboe76 What it really boils down to is whether this should be supported by this formula or not. If not, we will have to reconsider removing the current Update: Retract this proposal due to the findings in my comment below. |
Would having a template for each major release vs one template with a bunch of if statements be easier to maintain long term? |
@getSurreal Thanks for the feedback. That's what I've mentioned in the comment just above yours. Personally, I'm not so sure the |
@aboe76 @getSurreal Actually, I've looked into this and I'm convinced that using multiple files would be far worse to maintain. Only 4 releases need to covered by the file at any one timehttps://s.saltstack.com/product-support-lifecycle/:
To assist with this process of maintenance, I propose using the same method as the main docs as a comment for each
That will make it much easier to clear blocks once a major version is no longer supported (without having to check each Git similarity between versions is always > 90%Tested using the similarity algorithm used by git for detecting file renames:
|
@myii You make a good case for a single file and I like matching it to the actual salt release cycle. |
@getSurreal Thanks again for your time and feedback. |
I've just pushed another commit to show these comments added in after each |
What about a radical change, instead of mimicking because the f_defaults.conf file is specially there so the package system of the OS can update the files in Downside to that aproach is we don't have any documentation on what features could be set.... |
@aboe76 above:
So if I understand you correctly:
That makes sense and saves unnecessary duplication of commented values from the
What documentation are you referring to here? The documentation that is kept in While we're here throwing curveballs, I've got another idea to add to the discussion. Make use of a new
|
@myii why do you want to keep the deafults per salt-version, they are already programmed inside salt. |
@aboe76 There are a number of reasons. Listing them in no particular order:
Reading the above is very dense because I'm summarising each point to avoid writing an essay. I'm happy to explain if further clarification is needed. |
@aboe76 Let me approach this from the other direction. As you said, the right idea is to only use the values from the pillar to populate |
@myii the salt understanding of its configuration can be default yaml or json structured, so if we can make the pillar yaml input and output the same then there is no need for jinja manipulation. Downside to this approach is:
|
Now this sounds very interesting if it can be achieved.
Is there any risk of the yaml coming out differently at the other end due to this process?
If the output matches the pillar, then that can be relied on instead. Ultimately, it's an output file controlled by Salt. The actual |
@aboe76 If we do follow through with this method, will have to carefully consider what to do about these available settings: Lines 7 to 11 in 5eee4c6
|
@aboe76 Just realised, we don't just have to worry about these pillar values... rather, there may be installations out there that have used these values and no longer have the default |
@myii for those setups that have removed the files there is also docs.saltstack.... |
@aboe76 After some testing, I'm definite that this idea is not achievable without serious consequences to existing setups. The following diff is based on using --- Current implementation
+++ YAML-only proposal
@@ -1,4 +1,4 @@
-auth.ldap.filter: "uid={{ username }}"
+auth.ldap.filter: uid={{ username }}
engines:
- slack:
aliases:
@@ -16,47 +16,6 @@
file_roots:
base:
- /srv/salt
- - [Numerous file_roots sourced from other parts of the pillar, e.g. formulas]
- - ...
- - ...
- - ...
- - ...
- - ...
- [Second env]:
- - [Numerous file_roots sourced from other parts of the pillar, e.g. formulas]
- - ...
- - ...
- - ...
- - ...
- - ...
- [Third env]:
- - [Numerous file_roots sourced from other parts of the pillar, e.g. formulas]
- - ...
- - ...
- - ...
- - ...
- - ...
fileserver_backend:
- git
- s3fs
@@ -83,16 +42,20 @@
pillar_roots:
base:
- /srv/pillar
-reactor:
- - 'master/deploy':
+reactors:
+ - master/deploy:
- /srv/salt/reactors/deploy.sls
rest_tornado:
- debug: False
- disable_ssl: False
+ debug: false
+ disable_ssl: false
port: 8000
ssl_crt: /etc/pki/api/certs/server.crt
ssl_key: /etc/pki/api/certs/server.key
-s3.buckets: ["bucket1", "bucket2", "bucket3", "bucket4"]
-s3.key: "askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs"
-s3.keyid: "GKTADJGHEIQSXMKKRBJ08H"
-winrepo_provider: "gitpython"
+s3.buckets:
+ - bucket1
+ - bucket2
+ - bucket3
+ - bucket4
+s3.key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
+s3.keyid: GKTADJGHEIQSXMKKRBJ08H
+winrepo_provider: gitpython
Who knows what will happen with all of the other possible settings? Basically, testing a massive change like this would take even more effort than the changes proposed in this PR (which are bad enough). I see two ways forward for working with the 4 supported versions of Salt at any given time:
Update: Reversed the diff direction for clarity. |
@myii 4 supported versions? I thought only three? Salt Open 2018.3 | Oct 31, 2018 | Apr 30, 2019 | Oct 31, 2019 | Oct 31, 2020 see https://s.saltstack.com/product-support-lifecycle/ But then again Is it necessary to support the same versions as saltstack enterprise support? |
@myii you are right about the other files, I haven't considered those. |
@aboe76 According to the current schedule, there will only be 4 active, supported versions at the same time. Discontinued versions can be stripped out. Guessing the dates for the upcoming version
By the time the next version comes out, In terms of complexity, not really. Quoting from above again:
The files change very little per version. The work I've done for this PR is mostly done. The only reason it was so complex was due to the new features around |
@myii if it is possible we could cut out the macros and put them in a |
@aboe76 OK, I will look further into using Thanks for all of your feedback. |
@myii yes I'm happy with this method and a good solution. |
0fc5e22
to
fbe814a
Compare
f_defaults.conf
based on major releases of Salttofs
): allow TOFS for master+minion configuration
@aboe76 @vutny OK, I've finalised this particular PR so that the minion configuration can be provided via. TOFS (opt-in, mirroring same method for the master configuration). If all is OK, I'd appreciate if this can be merged, so that the next steps can commence. I'll start a new issue with a checklist based upon the one below (note, transferred to #417, this remains quoted here for reference):
In terms of the unrelated Update: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work @myii and thanks to all the reviewers
Thanks for the review, @noelmcloughlin. |
@vutny Are you satisfied with this implementation? Can we proceed with the merge? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great, good to go @myii
@vutny Thanks, do you mind merging? |
@myii Wanna me to do that? |
@vutny I could always do it myself but I prefer to let others do it when it's a PR authored by me! If it was a minor change, I would have merged it myself. |
Okay, let's wait a day if there would be no objections, I'll merge it. |
@vutny I don't think there's anyone left to put up an objection, this PR has been around since the end of January! But I'm absolutely fine with that, it's not like I'm going to be doing the next stage very soon anyway. |
LGTM !
Op wo 12 jun. 2019 09:02 schreef Imran Iqbal <[email protected]>:
… @vutny <https://github.com/vutny> I don't think there's anyone left to
put up an objection, this PR has been around since the end of January! But
I'm absolutely fine with that, it's not like I'm going to be doing the next
stage very soon anyway.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#398?email_source=notifications&email_token=AANXTVGFSPYWV5EK3WFEE3DP2CNR3A5CNFSM4GTT5RFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXPONKQ#issuecomment-501147306>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AANXTVCXIHYCC5NAWN35POTP2CNR3ANCNFSM4GTT5RFA>
.
|
Thanks to all involved for your input along the way. |
🎉 This PR is included in version 0.58.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
2016.11
:2016.11
2017.7
2018.3
2019.2
The current master.d/f_defaults.conf is limited to settings from
2016.11
. There have been two more major releases since then and another on the way soon.This PR starts the process of ensuring that this file works for any master since
2016.11
. I've done most of the work but I'd appreciate this being reviewed, due to the importance of this configuration as well as some of the complexity in it.Of particular importance is the new
get_jinja_env_config_block
macro and how it populates the two Jinja environment sections. Problems here can cause breakages in templates and state files and I've spent a lot of time ensuring this all works. It has also resulted in uncovering a bug that I've reported in a new upstream issue. I've applied the workaround for that in this PR.If this all works well, the same can be done for the minion.d/f_defaults.conf file as well.
For ease of comparison, the four source (master config) files are: