Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop new kubeconfig workflow #666

Open
raliev12 opened this issue Nov 8, 2021 · 2 comments
Open

Develop new kubeconfig workflow #666

raliev12 opened this issue Nov 8, 2021 · 2 comments
Assignees
Labels
design needed New Design or Redesign required enhancement New feature or request
Milestone

Comments

@raliev12
Copy link
Contributor

raliev12 commented Nov 8, 2021

Problem description (if applicable)
Current kubeconfig management and workflow in airshipctl has outdone itself.
There are several common problems that we have with kubeconfigs right now:

  1. kubeconfigs are generated and dumped to filesystem per each phase, however it only slows deployment process and adds unnecessary complication since most of the time kubeconfig remains the same and we need to update or merge it only on demand;
  2. it's unclear which particular kubeconfig will be generated and used in a phase since kubeconfigs are retrieved iteratively using all the defined sources in cluster map.
    This approach makes delays and it's inconvenient for the end user to define a kubeconfig and its sources in a separate entity; also the information stored in clustermap has no practical benefit since it duplicates itself.
  3. since we generate kubeconfigs each time per phase we should carry kubeconfig entrypoint within phase config manifests, therefore we decrypt kubeconfig from manifests even if we don't need it. Thus all airshipctl subcommands doesn't work
    at all without defined in env SOPS decrypt keys and it makes it slow. There are already PS in downstream with workarounds solving that problem (by removing kubeconfig entrypoint from phase bundle) since it become really annoying for colleagues who use airshipctl every day;
  4. we don't have integration with all common kubernetes cli tools which support KUBECONFIG env var and --kubeconfig flag (it was removed from airshipctl in favor of cluster map);
  5. we still don't have handy and reliable approach to merge kubeconfigs - Kubeconfig interface in airshipctl doesn't provide functionality to do that properly and engineers make attempts to workaround this use-case using
    toolbox container and custom bash scripts with kubectl under the hood (for instance, capd phase plan).
  6. we work mostly with cluster names instead of contexts, assuming if they are not equal then the appropriate information should be stored in cluster map. However there are no verifications or enforcement of that. Kubernetes client which is beneficiary of kubeconfig information works with contexts instead of clusters, so we should either use context or enforce that defined context for cluster has a relation for that cluster.
  7. Kubeconfig and builder interfaces in airshipctl are bulky, non-efficient and most of their part aren't even used.

Proposed change
To solve described problems we should move forward and use completely different approach which must be:

  1. simple
  2. easy to understand
  3. fast
  4. reliable
  5. predictable

We have the following use cases to cover:
- be able to work with std kubeconfig (or other specified location) that already exist
- be able to modify the std kubeconfig (or other specified location) by adding/removing clusters/contexts
- added/removed parts of kubeconfigs are:
- newly created cluster (capd, capz...)
- generated kubeconfig (e.g. our current baremetal use-case)
- decrypted kubeconfig (if we want to keep kubecofnig parts repo)
- |
e.g. we may want to create kind cluster, add its kubeconfig to std kubconfig, deploy there capi/capz or capm3 deploy target cluster,
get its kubeconfig and add to std place, make move, kill kind cluster. In that case we'll get kubeconfig in std place.
we may want to encrypt its part and add to catalogue/encrypt that (can be done via phase)
we will need also a phase that decrypts and adds kubeconfig back. So the new user could work with that.
- we can see that kind follows the similar approach. it works with std location, when you add cluster it adds info to kubeconfig.
- |
lots of tools work with std location, nobody invents such complicated logic.
it's easier always to get kubeconfig from remote( special phase) and put it to std location, so other tools will also work.
This is more operator friendly thing.

The proposal of new kubeconfig workflow is the following:

  • airshipctl should work with one single kubeconfig file, by default located at ~/.kube/config.

  • there will be two options to override this location: standard CLI option (--kubeconfig path or KUBECONFIG env variable) or phase options

  • airshipctl and it's phases start to work with contexts instead of clusters. context name should be defined in phase options or user can override it using --context flag of phase run subcommand;

  • to perform operations with kubeconfig a new KRM function (kubeconfig-manager) will be introduced; it can perform following actions: saving kubeconfig from kustomize entrypoint (decryption); get kubeconfig from secret (and merge it into existing one); show parts or entire kubeconfig; remove contexts from kubeconfig;

  • the KRM function will be defined as a phase with appropriate options and will perform actions on demand;

Potential impacts
Improved performance and user experience.

@raliev12 raliev12 added enhancement New feature or request triage Needs evaluation by project members labels Nov 8, 2021
@raliev12
Copy link
Contributor Author

raliev12 commented Nov 8, 2021

The issue can be assigned to me.

@eak13 eak13 added design needed New Design or Redesign required and removed triage Needs evaluation by project members labels Nov 10, 2021
@eak13 eak13 added this to the Future milestone Nov 10, 2021
@eak13
Copy link

eak13 commented Nov 10, 2021

Review on next design call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design needed New Design or Redesign required enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants