Example NixOps State Backends

[grahamc]

Store in S3:

{
  network = {
    storage.s3 = {
      profile = "rootlol";
      region = "us-east-1";
      bucket = "grahamc-nixops-state";
      key = "personal.nixops";
      kms_keyid = "166c5cbe-b827-4105-bdf4-a2db9b52efb4";
    };
  };

  flexo = {
    deployment = {
      targetHost = "147.75.105.137";
    };
    ...
  };
}

Store in Memory:

{
  network = {
    storage.memory = {};
  };

  flexo = {
    deployment = {
      targetHost = "147.75.105.137";
    };
    ...
  };
}

Legacy, exactly as master works now:

{
  network = {
    storage.legacy = {};
  };

  flexo = {
    deployment = {
      targetHost = "147.75.105.137";
    };
    ...
  };
}

[grahamc]

This at one point did not abspath, which was causing problems with legacy paths to state files.

[kyrias]

It seems to me like it would be a bit safer to make an actual copy rather than a symlink.

That way we know that if something goes wrong in the middle of writing the state file to disk, the "proper" statefile won't be corrupted.

[grahamc]

If we were to copy, we would need to implement our own implementation of locking on top. The legacy backend already supports its own locking. My thinking is it would be better to find a way to "escape" the legacy backend, or implement a "backup" feature in the Legacy backend, than implement copying. What do you think?

[domenkozar]

Would be cool to use contextmanager for locking as well to ensure a bit of safety

[domenkozar]

This way it can be moved into Storage abstraction and then backend can implement it or not

[adisbladis]

I'm not sure we should completely conflate looking & storage.
S3, Backblaze B2 & friends don't have any locking mechanisms themselves, so you may want to utilise a dynamodb table for locking.

It's fine if a storage backend implements a default locking mechanism though, as long as it can be overriden by the user.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/using-nixops-in-a-team/6459/4

[grahamc]

Right now if you don't define network, you get a bad error:

RuntimeError: error: attribute 'network' in selection path 'network' not found

[grahamc]

If you define network.storage = {} you get a bad error:

    key = list(storage.keys()).pop()
IndexError: pop from empty list

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update/6525/1

[ip1981]

A PR can be (and converted to) to a draft, so you wouldn't need a "do not merge" warning.

[grahamc]

todo:

• split the locking from the storage
• treat these as context managers decided not to, to keep the explicit API's actions very clear. they're used within context managers already.
• remove the S3 one and relocate it to nixops-aws
• error messages / data validation is terrible, reimplement on top of Move from xml intermediate Nix representation to JSON #1275
• validate the network parameters wit the module system (todo: new PR) not going to do this
• tests, somehow (see: tests: Add functional tests using NixOS in Podman (Docker) #1326)
• docs. lots, lol
• talk to users who use nixops with many networks in one state file
• accept a deployment ID for the "legacy" backend type. Should we support it for other types, too? Not sure. Migration might be tricky. not going to do this
• a way to force-unlock a network
• copy data from one backend to another (docs, command, otherwise)

this PR removes the idea of "global " networks and the "-d ..." argument, because of the dependence on ./network.nix. So, some thinking of the implications of this are needed.

[adisbladis]

We're never going to move forward with this without getting this change out to more people for review.

Therefore I'm merging this, with the caveat that it might be somewhat changed before a final release.

[rehno-lindeque]

There's a reference to databasefile here that doesn't seem to exist in the code

[rehno-lindeque]

• remove the S3 one and relocate it to nixops-aws

Looks like the relocation part of this one still needs doing if I'm not mistaken? I couldn't find any reference to StorageBackend in nixops-aws.

[roberth]

Whew, I finally got around to playing with this. Good call merging this though!

My observations so far:

The interface is quite coarse grained, dealing with the whole db only and having no knowledge of which deployment is used. This coarseness means that state file history becomes noisy and locking more problematic as the number of deployments increases. Users can choose distinct backend configurations to compensate for this, but that's effectively duplicating the -d feature that already exists. I'd be ok removing -d altogether, but that's perhaps too radical. Making the interface fine grained seems like the way to go.

State is always written, even for read-only operations like nixops info. I could compensate for this in a backend, but that's not the right solution. The same problem applies to locking, where it can't be worked around in the backend. So if NixOps needs to know what it's about to do anyway, it may as well use that knowledge in order not to write state redundantly.

Sqlite isn't really an interchange format. If something goes wrong, it's hard to read. I don't know about its format/version strategy (maybe it's good though?) and it's not easy to diff. Perhaps the backends could be based on nixops import/export instead?

Which brings us to: nixops import --include-keys-like functionality doesn't seem to be included. I haven't tested it and I may have read the code wrong, but it also seems like a good idea to have a single code path for this stuff anyway.

[Mic92]

Is the memory backend functional at all? We tried to use both legacy backend and memory backend in nix-community/infra#123 but they both seem not to work. nixops stable is also broken, which means there is no functional nixops left.

[talyz]

Yeah, the legacy backend was broken by this, since it now ignores the nixAttrs state attribute. I've solved this by using the require attribute to import the deployment files from the network file. Specifying the network file is however cumbersome, since the --network flag currently expects a directory, not a file. To make it easier, I've opened #1480.

With that in place, I think we could essentially drop deployments altogether without losing granularity: instead of running nixops deploy -d my_deployment you'd get the same result by running nixops deploy -n my_deployment.nix, as long as my_deployment.nix requires the appropriate files and specifies the backend to use. Note however that this strategy doesn't work with the current legacy backend, since it doesn't allow choosing where the state is stored. A simple json file backend based on import/export, like @roberth is suggesting, would solve this.