Add the option for a sentinel file for the origin

[haoming29]

Closes #995

[haoming29]

This PR added an additional field SentinelFile to the Origin.Exports parameter. @CannonLock you might need to update the UI as well.

[bbockelm]

Oh! Should this be SentinelFile or SentinelLocation? The latter may be more reasonable for a S3 backend.

[CannonLock]

@haoming29 I will patch on the ui change after the name is decided.

[haoming29]

Trying to understand the intension for S3 here. Will the user provide a path to the sentinel file for either S3 and POSIX? Or only a file name? The latter is what I assumed in this PR and I actually checked and require that the parameter does not contain a directory.

[jhiemstrawisc]

I think you'll have to think about a good way to incorporate the origin's configured storage type. In the case of an S3 sentinel object, we might require that for a given s3endpoint, bucket and object, we can fetch <s3endpoint>/<bucket>/<object> if the origin is using path-style requests or <bucket>.<s3endpoint>/<object> if we're using virtual hosting.

Of course, if you want to check that an authenticated S3 object exists before starting xrootd, you'll also need to handle creating the signed AWSv4 URLs with the credentials Pelican knows about, and that's not a small task. You could either write all of the functionality to do this in Pelican (and wind up duplicating half the magic of the S3 plugin) or you could use the AWS SDK and add a lot of bloat...

Maybe the approach is to start XRootD first and immediately try to pull the file/object. If the origin can't get something from itself, we assume the sentinel doesn't exist.

[haoming29]

@jhiemstrawisc That clarifies things a lot! I like your suggestion as to start xrootd first then pull a file and see if we can make it for S3 backend. For POSIX backend, I think it's easy enough to stat the file before starting the xrootd and that way we eliminate other possibilities that can cause the error getting the file through the origin (auth/token/etc). I will incorporate the comments above and update the code.

[CannonLock]

Is the parameter naming finalized here yet?

[haoming29]

SentinelLocation

Let's do SentinelLocation for now. I'll revise this PR today but I think SentinelLocation sounds good to me

[haoming29]

As @jhiemstrawisc and I discussed. The scope of this PR is to only add the sentinel file as the option for a POSIX backend, given the upcoming large changes in S3 configurations. We will come back and revisit this for S3 once that's done.

[CannonLock]

@haoming29 In testing the UI I hit the following error that killed the web app.

Fatal error occurred at the start of the program. Cleanup started: fail to open SentinelLocation test for StoragePrefix sdfsdf. Directory check failed: stat sdfsdf/test: no such file or directory

Notably I used garbage in the config values so not sure if this is really a "bug"

origin:
    exports:
        - capabilities:
            - PublicReads
            - Writes
            - Listings
            - DirectReads
            - Reads
          federationprefix: /testtimeusa
          storageprefix: /test0123/cannonlock-test/98y230498q2yt0qwu8gf89q3gt2
        - capabilities:
            - DirectReads
          federationprefix: sdfsdf
          sentinellocation: test
          storageprefix: sdfsdf

[haoming29]

@CannonLock Do you have a file test under /sdfsdf?

[CannonLock]

@haoming29 No, more so just bummed this took out the web app. Would be nice if we could keep the webapp running through this type of thing so they could continue to edit the config file/read the logs right in the webapp... Just a idea for the future.

[haoming29]

@haoming29 No, more so just bummed this took out the web app. Would be nice if we could keep the webapp running through this type of thing so they could continue to edit the config file/read the logs right in the webapp... Just a idea for the future.

Good point. This would be a corner case where we expect user to have a test file in place before running Pelican. If they decided to change the test file on the go, they should be prepared to have the new test file before making the change. It would be nice if we can have a tooltip/description to emphasize this

[bbockelm]

Two small nitpicks on coding style. After that, I think this is pre-approved and feel free to merge!

[bbockelm]

Suggested change

	return false, fmt.Errorf("invalid SentinelLocation path for StoragePrefix %s, file must not contain a directory. Got %s", export.StoragePrefix, export.SentinelLocation)
	return false, errors.Errorf("invalid SentinelLocation path for StoragePrefix %s, file must not contain a directory. Got %s", export.StoragePrefix, export.SentinelLocation)

[bbockelm]

Suggested change

	return false, errors.Wrap(err, fmt.Sprintf("fail to open SentinelLocation %s for StoragePrefix %s. Directory check failed", export.SentinelLocation, export.StoragePrefix))
	return false, errors.Wrapf(err, "fail to open SentinelLocation %s for StoragePrefix %s. Directory check failed", export.SentinelLocation, export.StoragePrefix)