Make dotfiles installation optional

[peakunshift]

Commitment

I confirm that I have read the following resources:

• How to ask questions The Smart Way

• Writing the perfect question

• Question checklist

• Could you please make my preference the defafault?

Current problem (if any)

Some files from creator's personal Dotfiles are required during some installation. As stated in the design guidlines, Qusal sould "provides a minimal modular isolated environment", and "should not focus on a specific Qubes OS user base". Currently we need to fork all Dotfiles which are personal settings.

Proposed solution

If some of these files are required for the installation to succeed, put them in this project. Otherwise, define a way to make dotfiles installation optionnal, by opting-in or opting-out.

The value to a user, and who that user might be

Make the installation process more generic without any personal preferences.

[ben-grande]

Did you know you can configure my dotfiles to have your own settings loaded after the defaults? It does not prevent my settings from being loaded, but you can override it anyway.

On one hand, I agree with you that they are my personal preference. On the other hand, I simply dislike using the small screen Xterm, the small font in Xterm, the non select-to-copy Xterm, not having handy aliases in the Shell. This is why on most projects I install at least:

include:
  - dotfiles.copy-sh
  - dotfiles.copy-x11

Currently we need to fork all Dotfiles which are personal settings

You only need to fork and delete files (not directories) in each dotfile configuration and no dotfiles will be applied (I haven't tested to check if this works). For example, delete all sh files:

cd dotfiles/files
find sh -type f -follow -delete

If some of these files are required for the installation to succeed, put them in this project. Otherwise, define a way to make dotfiles installation optionnal, by opting-in or opting-out.

And this... some projects require my dotfiles to succeed because they expect files in certain directories.

See the Electrum Xprofile for example, my dotfiles allow extra Xprofiles that are loaded when X starts, loading $XDG_CONFIG_HOME/x11/xprofile.d/*.sh. Or Mirage builder Opam initialization, also drop-in loaded in $XDG_CONFIG_HOME/sh/profile.d/*.sh.

No defauls shell files in Debian or Fedora or any distro is so versatile as my dotfiles to allow conditional inclusion of files instead of having to append to ~/.bashrc, which is a horrible idea as it creates a mess and doesn't load the files in a correct order.

In other words, my dotfiles, especially the Sh and X11 are usability wise, much better than the defaults and some other directories from the dotfiles such as vim, tmux, git can rely on them as I control how they work.

If some of these files are required for the installation to succeed, put them in this project. Otherwise, define a way to make dotfiles installation optionnal, by opting-in or opting-out.

Opting-out is possible, but this would require some work, not difficult, but time consuming, that I am not sure it is worth it.

find . -type f -name "*.sls" -exec grep -rn -m1 -- "- dotfiles\." {} \; | wc -l
58

There arre currently 58 files that have at least one inclusion of my dotfiles, this means that for each of these files, I would need to:

1. create a second state that installs the dotfiles
2. document in the readme how to install it

And this is a long, long work.

Make the installation process more generic without any personal preferences.

I am not closing this issue now because I tend to agree on a less personal preferences approach, but I also let the user deploy their own dotfiles, the only thing it really requires are the dotfiles.copy-NAME states to have the same name, it can be something that does not copy anything at all, such as

# copy-sh.sls
echo hello:
  cmd.run

But then some integration might break, and this integration can't be brought up here, to Qusal, because I either loose the power of my dotfiles or duplicate work that I make in my dotfiles to a more bare bones approach to Qusal, which means doubling code and this leads to eventually forgetting to update both codes and mess is created...

---

I agree that something needs to be done, but I don't know how to make it less time consuming for me over the time, especially not duplicating code.

Can you please point out which personal preference you find contradictory, just as an example?

[peakunshift]

Thanks for your detailed response!

I totally understand your points, but your explanation shows how closely the current state of Qusal is tight to your personal use cases, which is also understandable because you built it for yourself first and you specified that it's not production-ready. But then it makes it more difficult for new users to use it, because we not only install a project but also someone else's personal settings and preferences, and it partly deviates from QubesOS's philosophy.

Ideally it would be completely separated, but as you pointed out it would require a lot of work and maintenance.

My main concern is regarding changing dom0. One of the strong features of QubesOS is the ability to create VMs without polluting the other parts of the systems, because we can always delete the VM and start something new. For DE, sys-gui is coming and so it can also be in a dedicated VM. But dom0 is the most important part of the system, and the one we should protect the most. So I think it's preferable to make sure we don't install too many things in dom0, or at least, makes it optional. To me it applies to base templates and their configurations.

Can we at least think about updating some Essentials tasks? For example I didn't install Electrum one, and for this case I won't mind installing the Dotfiles, but for the base ones that everyone installs I think it can be a good thing to make it more general.

I already love Qusal and what it allows, I'd be happy to contribute!

[ben-grande]

So I think it's preferable to make sure we don't install too many things in dom0, or at least, makes it optional. To me it applies to base templates and their configurations.

My dotfiles can be skipped for dom0, you don't run the dom0.init, thus skipping dom0.dotfiles.copy-all, therefore, you only run the dom0.*.sls states that you want. This way, no dotfiles will be installed to dom0, they will still be present in the salt directory in dom0, but not applied to dom0 configuration.

Let's take for example KDE, this is my personal preference that I brought up to Qusal, still, my personal preference. I have no problems using Xfce in a normal Linux system, but Qubes menu is so huge that KDE just makes that experience much better.

For DE, sys-gui is coming and so it can also be in a dedicated VM.

I have plans for that, but upstream (Qubes OS) does not have a good integration with sys-gui, as I did sys-audio, I know the policy is permissive because of Qubes OS limitations.

Can we at least think about updating some Essentials tasks? For example I didn't install Electrum one, and for this case I won't mind installing the Dotfiles, but for the base ones that everyone installs I think it can be a good thing to make it more general.

Mapping this has to be done by manually because I don't remember every project that has integration with my dotfiles, but for the sake of answering, I don't believe there are many. For now, I can only find the aforementioned Electrum and Mirage-Builder, but any project that puts files in .local/bin for example, that are not in the default PATH variable of distribution shells, also depends on my dotfiles.

I already love Qusal and what it allows, I'd be happy to contribute!

Thank you!

I need to understand what you expect from a more compact version of my dotfiles, I need to see what you are imagining. A lot of dotfiles use external plugins to provide functionality, mine does not do that at all, so this impacts on the size of my dotfiles, but allows us to avoid making network connections and having a long loading time.

Although you are free to discuss this issue, this is not a starter task, it not difficult, but time consuming. In order for me to see you are really committed to this long task, hang around, do smaller tasks, so I can have some expectation that you will be around over the time and eventually we will be able to close this issue as completed.

For this issue to be fixed, I need to do changes changes in Qusal and my dotfiles. Qusal will hold a minimal configuration that can load extra files ~/.config/sh/sh.d, ~/.config/bash/bash.d, where my dotfiles will be deployed.

[peakunshift]

Of course I don't want to mess everything up, I'll see what I can do with the time I have, I'll send PRs for quick-wins and let's keep the discussion open for these bigger topics!

[ben-grande]

I started doing this change but I noticed the changes are going to take some days to complete because I have to recreate the qubes, as my dotfiles can't be in there, they need to be absent and the best method is to remove and create the qubes. And this is only for testing, not involving the writing of the changes.

See below my re-analysis of this matter.

Can we at least think about updating some Essentials tasks? For example I didn't install Electrum one, and for this case I won't mind installing the Dotfiles, but for the base ones that everyone installs I think it can be a good thing to make it more general.

How can I decide what is general and what is not?

Let's take tpope/vim-sensible and tmux-plugins/tmux-sensible for example. These projects do not aim to respect default settings, but to set sensible values.

vim-sensible:

sensible.vim - Defaults everyone can agree on

Think of sensible.vim as one step above 'nocompatible' mode: a universal set of defaults that (hopefully) everyone can agree on.

• If you're new to Vim, you can install this as a starting point, rather than copying some random vimrc you found.
• If you're pair programming and you can't agree on whose vimrc to use, this can be your neutral territory.
• If you're administrating a server with an account that's not exclusively yours, you can scp this up to make things a bit more tolerable.
• If you're troubleshooting a plugin and need to rule out interference from your vimrc, having this installed will ensure you still have some basic amenities.

tmux-sensible:

Principles

• tmux-sensible options should be acceptable to every tmux user! If any of the options bothers you, please open an issue and it will probably be updated (or removed).
  if you think a new option should be added, feel free to open a pull request.

• no overriding of user defined settings.
• Your existing .tmux.conf settings are respected and they won't be changed. That way you can use tmux-sensible if you have a few specific options.

The phrase set of defaults that (hopefully) everyone can agree on is a difficult goal because then the provided solution becomes so minimal that doesn't cover most use cases, just the basic.

So back to your request, on making my dotfiles more general, I think my dotfiles can be more general, it was built in a modular way. You may consider it also as a base anyone can built upon via the drop-in directories and the FILE.local configuration. If you've noticed something from my dotfiles is that I don't use values that would work only for me, I don't put my git user, mail and signing key in my dotfiles as they were made to be reusable, instead, I put them in my ~/.gitconfig.local:

% cat ~/.gitconfig.local
; vim: ft=gitconfig
[user]
        name = Ben Grande
        email = [email protected]
        signingKey = DF3834875B65758713D92E91A475969DE4E371E3

This way I can share my dotfiles without making the user change the defaults, only build upon it.

The dotfiles are not only providing a modular configuration but also a safe configuration

See these four git examples

The git configuration merge.VerifySignatures=true is used in the qubes-builder formula. Instead of letting the user forget to verify merge signatures that happens on merges and pulls, I enforce it.

Same thing with the gpg.conf, which unfortunately does not have a drop-in configuration file and hopefully will be substituted by Sequoia-PGP when possible. This GPG configuration is used for every GnuPG interaction with keys, I enforce a safe configuration, remove old ciphers, print keys in the long format etc.

Same thing with the ssh configuration. My SSH configuration, client side, was not vulnerable to terrapin attack that affected almost all OpenSSH servers and clients. Instead of waiting for upstream patches, I can enforce safe ciphers by default and when you update, you get them before distro releases. Even security fixes from upstream distributions takes time. Note that my dotfiles do not cover the SSH server configuration, but the client can reject to connect if the server provided algorythms and ciphers are not allowed by the client. Although there is an OpenSSH server in the ansible formula, it is not make over the network, but over Qrexec.

Xfce Thunar does not search for the x-terminal-emulator, it expects xfce4-terminal to exist, which is not present on a minimal system, therefore xfce helpers are provided.

Many other things are usability features, that I provide drop-in configuration for the user to modify without worrying that my formula will replace their configuration, no need to maintain a fork, just maintain your own configurations and apply them after the provided dotfiles.

Yes, the dotfiles have features that I find useful both security and usability wise, they are not however, unique to me, they serve anyone that are using the tools that I provide in Qusal, the SSH client hardening make sense if the client qube dev, ansible, any git client that uses SSH.

You can continue to discuss for a more general approach directly on the dotfiles repo. Treat it as a dotfiles targetted at Qubes users that use minimal templates and use packages provided by the Qusal formula.

---

Current problem (if any)

Some files from creator's personal Dotfiles are required during some installation. As stated in the design guidlines, Qusal sould "provides a minimal modular isolated environment", and "should not focus on a specific Qubes OS user base". Currently we need to fork all Dotfiles which are personal settings.

From

qusal/docs/DESIGN.md

Lines 18 to 23 in 5f10271

	## Goal
	Provide a minimal modular isolated environment for users to complete daily
	tasks in a secure manner. We should not focus on a specific Qubes OS user base
	as it would narrow our reach. We scope to have a diverse user base, with
	different needs and use case that could shape our project for the better.

When I wrote not focus on a specific user base, I was talking about a group of individuals that only have a use case, such a developers, only want developers formulas, journalists, do not care much about code but they need office suite and printing mechanisms, they also need audio and media usage.

You can consider the dotfiles projects as an extension of Qusal, a module, instead of being simply my personal preference. Like the Salt states, my dotfiles enforce a specific configuration, that can be changed per user discretion. If there is something that makes the dotfiles specific to me but not to a larger user base of Qusal users, please open an issue at the dotfiles repository. I want to make the dotfiles reusable.

I can also say that minimal modular isolated environment for users to complete daily tasks includes the dotfiles repository, they are usability wise, the best that I could make them. They are part of a minimal base system, they are modular because it allows drop-in configuration and they are isolated in a sense that they are portable and does not depend on networking capabilities.

Producing a minimal dotfile is of course possible but not something that I will be doing because after I start writing the first lines, I will think: why not add this, and this, and this and more thing are added. This is how my dotfiles grew, over time, it was very small in the beginning.

---

To answer the title: Making dotfiles installation optional

I do not aim to make it optional, it requires more work for me to do and I can't see clear benefits now, after my reanalysis. The dotfiles is a user configuration file that I created to be reusable and not only usable my me.

Also, opting-in and opting-out is not easy with Salt, there is no GUI pane for the user to select Salt options. Technically, with Jinja pillars, this would be possible, to opt-in or opt-out, but still doesn't cover the case where the states expect the dotfiles to be installed for certain functionalities.

[peakunshift]

Can we at least think about updating some Essentials tasks? For example I didn't install Electrum one, and for this case I won't mind installing the Dotfiles, but for the base ones that everyone installs I think it can be a good thing to make it more general.

By Essentials I mean Salt states as defined in the BOOTSTRAP doc. These will be the most installed and common ones, so it can the ones who are more general.

I see your distinctions between personal and defaults (that everyone can agree on). I do agree that we should provide the defaults one.

My proposal of making the dotfiles optional is also a matter of giving the user control and clarity over what is done. Currently Qusal is tigh to your usage and preferences (personal or default choices), we need to trust you on what is done when we run one single command. And usually opt-in is better that opt-out, because the user has better control. It's also easier to install than remove.

Your personal choices made on Qusal can also diverge from QubesOS philosophy. For example, some things are installed in dom0, and that's strongly discouraged. What you install here is not mandatory or necessary, it eases the experience in dom0, but if dom0 is that painful to interact with, it's also for a reason, because of the security choices. And in the future, dom0 won't even be accessible when used with sys-gui-gpu for example. I personally don't mind polluting a VM because it can be recreated easily, but dom0 is completely different and I don't want to install anything but the strict necessary in it. Here is a matter of personal preferences, but Qusal enforces a point of view.

I understand your points, it seems to be a lot of work, I hope in the future Qusal will be more generic, allows opt-ins and more clarity and control for the user.

Thanks for all your clarifications, as no actionables come from this issue I won't continue the debate. If I see any quick improvements I will open other issues!

[ben-grande]

By Essentials I mean Salt states as defined in the BOOTSTRAP doc. These will be the most installed and common ones, so it can the ones who are more general.

I defined hopefully all formulas in the bootstrap doc. The only thing special is that it is organized per function.

My proposal of making the dotfiles optional is also a matter of giving the user control and clarity over what is done. Currently Qusal is tigh to your usage and preferences (personal or default choices), we need to trust you on what is done when we run one single command. And usually opt-in is better that opt-out, because the user has better control. It's also easier to install than remove.

You already need to trust me on whatever is done if you are running something you don't fully understand. I understand that the dotfiles do increase the amount of code that you need to trust.

Your personal choices made on Qusal can also diverge from QubesOS philosophy. For example, some things are installed in dom0, and that's strongly discouraged. What you install here is not mandatory or necessary, it eases the experience in dom0, but if dom0 is that painful to interact with, it's also for a reason, because of the security choices. And in the future, dom0 won't even be accessible when used with sys-gui-gpu for example. I personally don't mind polluting a VM because it can be recreated easily, but dom0 is completely different and I don't want to install anything but the strict necessary in it. Here is a matter of personal preferences, but Qusal enforces a point of view.

Qubes documentation regarding installation of software in dom0 states: since software running in dom0 can potentially exercise full control over the system, it is important to install only trusted software in dom0

The problem with installing software in dom0 is not that one extra package is against QubesOS philosophy, but that we should prefer delegating the task that requires a package to a DomU so we don't increase our TCB (Trusted Computing Base).

Installation of software in Dom0 is not wrong in it is needed. The dom0.kde state is installing a lot of packages in Dom0, they are required for a KDE experience in Qubes.

And yes, Qusal enforces a point of view, install packages in Dom0 when necessary. I trust the dotfiles because I wrote them. You can consider qvm-terminal and qvm-port-forward as a package, you could say they are not necessary, but try to call them manually every time and see the burden. By installing Qusal, you are trusting me, the maintainer, to have written securely, you can review all you want, but in the end, you need to trust that I am taking the "right" decisions.

I understand your points, it seems to be a lot of work, I hope in the future Qusal will be more generic, allows opt-ins and more clarity and control for the user.

It can be in the future, just can't focus on this now. About control for the user and opt-ins, this is also a difficult choice, I want a good usability by default, asking for non-experts to write YAML for Salt Pillars to give users a choice during Salt states is also a complicated matter, one to be discussed in the future.

Thanks for all your clarifications, as no actionables come from this issue I won't continue the debate. If I see any quick improvements I will open other issues!

Please do.

[ben-grande]

@peakunshift check out #74