forked from PelicanPlatform/pelican
-
Notifications
You must be signed in to change notification settings - Fork 0
/
parameters.yaml
2862 lines (2707 loc) · 105 KB
/
parameters.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
#
# Copyright (C) 2024, Pelican Project, Morgridge Institute for Research
#
# Licensed under the Apache License, Version 2.0 (the "License"); you
# may not use this file except in compliance with the License. You may
# obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This file contains structured documentation about the Pelican parameters.
# While it is somewhat human-readable, it is meant to help with the documentation
# generation.
############################
# Top-Level Configs #
############################
---
name: ConfigBase
description: |+
The directory containing the Pelican configurations and data when the utility
is run as non-root. Automatically configured; cannot be overridden.
default: "~/.config/pelican"
components: ["*"]
type: filename
---
name: ConfigLocations
description: |+
`ConfigLocations` provides administrators a way to define a list of directories containing Pelican configuration files. Within a given directory,
files are read in lexicographical order, and any keys that are defined in multiple files will take the value from the last file read. Directories are
read in the order provided by the list. For example, specifying:
`ConfigLocations: ["/configs1", "/configs2"]`
will read files first from `/configs1` and then from `/configs2`. If a key is defined in both `/configs1`
and `/configs2`, the value from `/configs2` will be used. If `/configs1` contains files `a.yaml` and `b.yaml` where both define the same key, the value
from `b.yaml` will be used.
Subdirectories of the provided directories are not read. Only the root config file's `ConfigLocations` key is used, and any redefinitions are ignored.
type: stringSlice
default: []
components: ["*"]
---
name: Debug
description: |+
A bool indicating whether Pelican should emit debug messages in its log.
NOTE: this will override whatever is set within your configuration file under Logging.Level!
type: bool
default: false
components: ["*"]
---
name: TLSSkipVerify
description: |+
When set to true, Pelican will skip TLS verification. This allows a "man in the middle" attack on the connection but can simplify testing. Intended for developers.
type: bool
default: false
components: ["origin", "registry", "director"]
---
name: IssuerKey
description: |+
A filepath to the file containing a PEM-encoded ecdsa private key which later will be parsed
into a JWK and serves as the private key to sign various JWTs issued by this server.
A public JWK will be derived from this private key and used as the key for token verification.
type: filename
root_default: /etc/pelican/issuer.jwk
default: $ConfigBase/issuer.jwk
components: ["client", "registry", "director"]
---
name: Transport.DialerTimeout
description: |+
Maximum time allowed for establishing a connection to target host.
type: duration
default: 10s
components: ["client", "registry", "origin"]
---
name: Transport.DialerKeepAlive
description: |+
Maximum time a TCP connection should be kept alive without any activity.
type: duration
default: 30s
components: ["client", "registry", "origin"]
---
name: Transport.MaxIdleConns
description: |+
Maximum number of idle connections that the HTTP client should maintain in its connection pool.
type: int
default: 30
components: ["client", "registry", "origin"]
---
name: Transport.IdleConnTimeout
description: |+
Maximum duration an idle connection should remain open in the connection pool.
type: duration
default: 90s
components: ["client", "registry", "origin"]
---
name: Transport.TLSHandshakeTimeout
description: |+
Maximum time allowed for the TLS handshake to complete when making an HTTPS connection.
type: duration
default: 15s
components: ["client", "registry", "origin"]
---
name: Transport.ExpectContinueTimeout
description: |+
Timeout to control how long the client should wait for the "Expect: 100-continue" response from the server before sending the request
body.
type: duration
default: 1s
components: ["client", "registry", "origin"]
---
name: Transport.ResponseHeaderTimeout
description: |+
Maximum time the client should wait for the response headers to be received from the server.
type: duration
default: 10s
components: ["client", "registry", "origin"]
---
name: GeoIPOverrides
description: |+
A list of IP addresses whose GeoIP resolution should be overridden with the supplied Lat/Long coordinates (in decimal form). This affects
both server ads (for determining the location of origins and caches) and incoming client requests (for determining where a client request is
coming from).
Configuration takes an IP address (both regular and CIDR) and a Coordinate made up of a lat/long pair in decimal format. For example:
```yaml
GeoIPOverrides:
- IP: "123.234.123.234"
Coordinate:
Lat: 43.073904
Long: -89.384859
- IP: "ABCD::1234/112"
Coordinate:
Lat: 39.8281
Long: -98.5795
```
Will result in the IP address "123.234.123.234" being mapped to Madison, WI, and IP addresses in the range ABCD::0000-FFFF will be mapped
to a field in Kansas.
type: object
default: none
components: ["director"]
---
############################
# Log-Level Configs #
############################
name: Logging.Level
description: |+
A string defining the log level of the client. Options include (going from most info to least): Trace, Debug, Info, Warn, Error, Fatal, Panic.
type: string
default: Error
components: ["*"]
---
name: Logging.LogLocation
description: |+
A filename defining a file to write log outputs to, if the user desires.
type: filename
default: none
components: ["*"]
---
name: Logging.DisableProgressBars
description: |+
A bool defining if progress bars should be enabled or not.
type: bool
default: false
components: ["Client"]
---
name: Logging.Origin.Cms
description: |+
Trace level of XRootD cluster management service, one of the main XRootD executables.
Cms basically is a file (or asset) discovery service. Each server has a cmsd daemon which
connect to a master one informing it if a server is available. XRootD asks cms where a file
could be found and cms works to report back the server for where the file is located.
Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["origin"]
---
name: Logging.Origin.Scitokens
description: |+
Trace level of scitokens debug output within XRootD configuration. This entails token management
and security credentials within XRootD. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["origin"]
---
name: Logging.Origin.Xrd
description: |+
Trace level of the eXtended Request Daemon within XRootD, another main XRootD executable. This reports information
the XRootD protocol and works with cms. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["origin"]
---
name: Logging.Origin.Xrootd
description: |+
Trace options for XRootD debug output within XRootD configuration. This prefix is reserved for the xroot protocol,
which is the component that sits on sockets and talks to clients as they query file-system info, open files, and read data.
This is the protocol for XRootD (like http) and handles connections and requests. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: info
components: ["origin"]
---
name: Logging.Origin.Http
description: |+
Logging level for the HTTP component of the origin. Increasing to debug
will cause the Xrootd daemon to log all headers and requests.
Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["origin"]
---
name: Logging.Origin.Ofs
description: |+
Logging level of Xrootd's "Open File System" (ofs) subsystem. The OFS manages the file descriptor table and redirection/
error handling. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["origin"]
---
name: Logging.Origin.Oss
description: |+
Logging level of Xrootd's "Open Storage System" (oss) subsystem. The OSS manages the interaction with the underlying
POSIX storage (open, read, write, close, etc). Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["origin"]
---
name: Logging.Cache.Http
description: |+
Logging level for the HTTP component of the cache. Increasing to debug will cause the Xrootd daemon to log
all headers and requests. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["cache"]
---
name: Logging.Cache.Ofs
description: |+
Trace level of XRootD's Open File System. This component cares about files and directories from the administrative perspective.
This component is build on top of the Open Storage System component, which deals with things like file creation and reads and
writes for files and directories. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["cache"]
---
name: Logging.Cache.Pfc
description: |+
Trace level of XRootD Proxy File Cache (XCache), the caching mechanism used by XRootD. This component
entails information for caches/caching within XRootD. This component instantiates its own Open Storage
System (OSS) to write local files to. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: info
components: ["cache"]
---
name: Logging.Cache.Pss
description: |+
Trace level of XRootD Proxy System Service. Variables this component reports include: number of remotes file opens,
number of opens that failed, number of remote file closes, and number of closes that failed. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["cache"]
---
name: Logging.Cache.Scitokens
description: |+
Trace level of scitokens debug output within XRootD configuration. This entails token management
and security credentials within XRootD. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["cache"]
---
name: Logging.Cache.Xrd
description: |+
Trace level of the eXtended Request Daemon within XRootD, another main XRootD executable. This reports information
the XRootD protocol and works with cms. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["cache"]
---
name: Logging.Cache.Xrootd
description: |+
Trace options for XRootD debug output within XRootD configuration. This prefix is reserved for the xroot protocol,
which is the component that sits on sockets and talks to clients as they query file-system info, open files, and read data.
This is the protocol for XRootD (like http) and handles connections and requests. Accepted values: `trace`, `debug`, `info`, `warn`, `error`, `fatal`, `panic`
type: string
default: error
components: ["cache"]
---
############################
# Federation-Level Configs #
############################
name: Federation.DiscoveryUrl
description: |+
A URL pointing to the federation's metadata discovery host.
NOTE: this does not work if the url contains a path!
type: url
default: none
components: ["*"]
---
name: Federation.DirectorUrl
description: |+
A URL indicating where a director service is hosted.
type: url
osdf_default: Default is determined dynamically through metadata at <Federation.DiscoveryUrl>/.well-known/pelican-configuration
default: none
direct_access: false
components: ["client", "origin", "cache"]
---
name: Federation.RegistryUrl
description: |+
A URL indicating where the namespace registry service is hosted.
type: url
osdf_default: Default is determined dynamically through metadata at <federation URL>/.well-known/pelican-configuration
default: none
direct_access: false
components: ["client", "director", "origin", "cache"]
---
name: Federation.JwkUrl
description: |+
A URL indicating where the JWKS for the Federation is hosted.
type: url
osdf_default: Default is determined dynamically through metadata at <Federation.DiscoveryUrl>/.well-known/pelican-configuration
default: none
direct_access: false
components: ["*"]
---
name: Federation.TopologyUrl
description: |+
A URL for the top level OSG Topology location (a legacy integration). This URL is needed to retrieve authorization file information.
type: url
osdf_default: "https://topology.opensciencegrid.org"
default: none
components: ["origin", "cache"]
---
name: Federation.TopologyNamespaceUrl
description: |+
A URL containing namespace information for origins and caches configured via the OSG Topology application (a legacy integration). The URL
should point to the hosted namespace.json.
type: url
osdf_default: https://topology.opensciencegrid.org/osdf/namespaces
default: none
components: ["director", "registry"]
---
name: Federation.TopologyDowntimeUrl
description: |+
A URL for determining OSG topology server downtime information. The result of querying this URL is an XML file containing downtime information.
type: url
osdf_default: https://topology.opensciencegrid.org/rgdowntime/xml
default: none
components: ["director"]
---
name: Federation.TopologyReloadInterval
description: |+
The frequency, in minutes, that topology should be reloaded.
type: duration
osdf_default: 4.5m
default: none
components: ["director", "registry"]
---
name: Federation.BrokerUrl
description: |+
The URL of the broker endpoint used by the origin.
If left unset, it will be populated by the federation metadata discovery.
type: url
default: none
direct_access: false
components: ["origin"]
---
############################
# Client-Level Configs #
############################
name: Client.StoppedTransferTimeout
description: |+
A timeout indicating when a "stopped transfer" event should be triggered.
type: duration
default: 100s
components: ["client"]
---
name: Client.SlowTransferRampupTime
description: |+
A duration indicating the ramp up period for a slow transfer.
type: duration
default: 30s
components: ["client"]
---
name: Client.SlowTransferWindow
description: |+
A duration indicating the sliding window over which to consider transfer speeds for slow transfers.
type: duration
default: 30s
components: ["client"]
---
name: Client.DisableHttpProxy
description: |+
A bool indicating whether the client's HTTP proxy should be disabled.
type: bool
default: false
components: ["client"]
---
name: Client.WorkerCount
description: |+
An integer indicating the number of file transfer tasks that should be
executed in parallel.
type: int
default: 5
components: ["client"]
---
name: DisableHttpProxy
description: |+
[Deprecated] A legacy configuration for disabling the client's HTTP proxy. See Client.DisableHttpProxy for new config.
type: bool
deprecated: true
replacedby: "Client.DisableHttpProxy"
default: false
components: ["client"]
---
name: Client.DisableProxyFallback
description: |+
A bool indicating whether the a proxy fallback should be used by the client.
type: bool
default: false
components: ["client"]
---
name: DisableProxyFallback
description: |+
[Deprecated] A legacy configuration for disabling the client's proxy fallback. See Client.DisableProxyFallback for new config.
type: bool
deprecated: true
replacedby: "Client.DisableProxyFallback"
default: false
components: ["client"]
---
name: Client.MinimumDownloadSpeed
description: |+
The minimum speed (in bytes per second) allowed for a client download before an error is thrown.
type: int
default: 102400
components: ["client"]
---
name: MinimumDownloadSpeed
description: |+
[Deprecated] A legacy configuration for setting the client's minimum download speed. See Client.MinimumDownloadSpeed for new config.
type: int
deprecated: true
replacedby: "Client.MinimumDownloadSpeed"
default: 102400
components: ["client"]
---
name: Client.MaximumDownloadSpeed
description: |+
The maximum speed allowed for a client to download a given file (enforced via rate limits).
This is not intended for use by production clients but rather for unit tests; 0 disables the rate limit.
type: int
default: 0
components: ["client"]
hidden: true
---
############################
# Origin-level Configs #
############################
name: Origin.DbLocation
description: |+
A filepath to the intended location of the origin's database.
type: filename
root_default: /var/lib/pelican/origin.sqlite
default: $ConfigBase/origin.sqlite
components: ["origin"]
---
name: Origin.Url
description: |+
The origin's configured URL, as reported to XRootD. This is the file transfer endpoint for the origin.
type: url
default: https://${Server.Hostname}:${Origin.Port}
components: ["origin"]
---
name: Origin.Port
description: |+
The TCP port to be used by the origin service for serving files. If set to 0,
then a random open port will be used.
default: 8443
type: int
components: ["origin"]
---
name: Origin.Exports
description: |+
A list describing the origin's exports. Each item in the list describes a single namespace the origin exports:
- StoragePrefix: The relevant path from the object store, e.g. for posix /my/dir
- FederationPrefix: The namespace prefix that data from StoragePrefix is made available under within the federation
- Capabilities: A list of the capabilities the origin is willing to support for the given export. Capabilities include:
["Reads", "PublicReads", "Writes", "Listings", "DirectReads"]
where each of these has the same effect as the corresponding "Origin.Enable*" configuration, except scoped to the
given export. If "PublicReads" is included, "Reads" is inferred.
- SentinelLocation: A filename under `StoragePrefix` path for Pelican to check the storage directory exists and is correctly mounted.
The value must be a file and contain no directory. Leave it empty to skip the check.
You should always choose a distinct name for `SentinelLocation`. It should not be reused for other servers.
If running in a containerized environment it should not be the name of the underlying physical host as that may change and lead to confusion.
You need to manually create a file under path to `StoragePrefix` with the same name as `SentinelLocation`.
Note that this parameter is only available for the POSIX backend.
Example:
```yaml
Origin.Exports
- StoragePrefix: /home/foo/bar
FederationPrefix: /demo/project
Capabilities: ["Reads", "PublicReads", "Writes", "Listings", "DirectReads"]
SentinelLocation: demoproject_origin_A
```
If Origin.StorageType == "s3", the following additional fields are available:
- S3Bucket: [OPTIONAL] See `Origin.S3Bucket` for details
- S3AccessKeyfile: [OPTIONAL] See `Origin.S3AccessKeyfile` for details
- S3SecretKeyfile: [OPTIONAL] See `Origin.S3SecretKeyfile` for details
If Origin.StorageType == "globus", the following additional fields are available:
- GlobusCollectionID: [REQUIRED] See `Origin.GlobusCollectionID` for details
- GlobusCollectionName: [OPTIONAL] See `Origin.GlobusCollectionName` for details
type: object
default: none
components: ["origin"]
---
name: Origin.StorageType
description: |+
The type of storage underpinning the origin. Currently supported types are "posix", "https", "s3", "globus", and "xroot".
type: string
default: "posix"
components: ["origin"]
---
name: Origin.FederationPrefix
description: |+
The namespace prefix of the origin's contents within the federation.
NOTE: This config option is incompatible with multiple exports defined via `Origin.Exports` and is ignored when the origin
exports multiple prefixes.
type: string
default: none
components: ["origin"]
---
name: Origin.StoragePrefix
description: |+
A string indicating the path to the volume exported by an origin's underlying storage. For example, if the origin has a StorageType
of "posix", this constitutes the path on disk exported by the origin for the federation. If the origin has a StorageType of "s3",
this value is not currently used.
NOTE: This config option is incompatible with multiple exports defined via `Origin.Exports` and is ignored when the origin
exports multiple prefixes.
type: string
default: none
components: ["origin"]
---
name: Origin.ExportVolumes
description: |+
A list of docker-style export volumes for the origin. Each item in the list describes a single volume the origin exports.
This configuration is meant mostly to be used by passing the -v flag from the command line. Paths exported with this
configuration will inherit the origin's abilities, so individual export configurations are not possible.
type: stringSlice
default: []
components: ["origin"]
---
name: Origin.EnablePublicReads
description: |+
A boolean indicating whether the origin permits reads without valid authorization. When false, reads from the origin will require a
properly-scoped authorization token signed by the origin's issuer.
NOTE: This config option is meant to configure an _origin's_ capabilities, but can be used to configure a namespace when the origin
exports only a single prefix or when every exported namespace should inherit the same configuration.
type: bool
default: false
components: ["origin"]
---
name: Origin.EnableReads
description: |+
A boolean indicating whether the origin permits any reads. When false, the origin may still allow writes.
NOTE: This config option is meant to configure an _origin's_ capabilities, but can be used to configure a namespace when the origin
exports only a single prefix or when every exported namespace should inherit the same configuration.
type: bool
default: true
components: ["origin"]
---
name: Origin.EnableWrites
description: |+
A boolean indicating whether the origin permits writes. All writes require authorization.
NOTE: This config option is meant to configure an _origin's_ capabilities, but can be used to configure a namespace when the origin
exports only a single prefix or when every exported namespace should inherit the same configuration.
type: bool
default: true
components: ["origin"]
---
name: Origin.EnableListings
description: |+
A boolean indicating whether the origin permits object listings. When true, clients can list the contents of the origin.
NOTE: This config option is meant to configure an _origin's_ capabilities, but can be used to configure a namespace when the origin
exports only a single prefix or when every exported namespace should inherit the same configuration.
type: bool
default: true
components: ["origin"]
---
name: Origin.EnableDirectReads
description: |+
A boolean indicating whether the origin permits direct reads. When true, the origin indicates that it is willing to interact directly with clients.
When false, the origin is indicating it is only willing to interact with clients via a cache service.
NOTE: This config option is meant to configure an _origin's_ capabilities, but can be used to configure a namespace when the origin
exports only a single prefix or when every exported namespace should inherit the same configuration.
type: bool
default: true
components: ["origin"]
---
name: Origin.ExportVolume
description: |+
[Deprecated] Origin.ExportVolume is being deprecated and will be removed in a future release. It is replaced by Origin.ExportVolumes.
A path to the volume exported by an origin.
type: string
default: none
deprecated: true
replacedby: "Origin.ExportVolumes"
components: ["origin"]
---
name: Origin.RunLocation
description: |+
A directory where temporary configurations will be stored for the XRootD daemon
started by the origin.
For non-root servers, if $XDG_RUNTIME_DIR is not set, a temporary directory will
be created (and removed on shutdown).
type: filename
root_default: /run/pelican/xrootd/origin
default: $XDG_RUNTIME_DIR/pelican/origin
components: ["origin"]
---
name: Origin.NamespacePrefix
description: |+
[Deprecated] Origin.NamespacePrefix is being deprecated and will be removed in a future release. It's configuration is being replaced by either
Origin.Exports.FederationPrefix or by Origin.FederationPrefix. Note that Origin.FederationPrefix is incompatible with multiple exports and requires
that the origin exports only a single path.
The filepath prefix at which an origin's contents are made globally available, eg /pelican/PUBLIC.
type: string
default: none
deprecated: true
replacedby: "Origin.FederationPrefix"
components: ["origin"]
---
name: Origin.EnableWrite
description: |+
[Deprecated] Origin.EnableWrite is being deprecated and will be removed in a future release. It is replaced by Origin.EnableWrites.
A boolean indicating if an origin allows write access.
type: bool
default: true
deprecated: true
replacedby: "Origin.EnableWrites"
components: ["origin"]
---
name: Origin.EnableFallbackRead
description: |+
[Deprecated] Origin.EnableFallbackRead is being deprecated and will be removed in a future release. It is replaced by Origin.EnableDirectReads.
Set to `true` if the origin permits clients to directly read from it
when no cache service is available.
type: bool
default: false
deprecated: true
replacedby: "Origin.EnableDirectReads"
components: ["origin"]
---
name: Origin.Multiuser
description: |+
A bool indicating whether an origin is "multiuser", ie whether the underlying XRootD instance must be configured in multi user mode.
type: bool
root_default: true
default: false
components: ["origin"]
---
name: Origin.EnableCmsd
description: |+
A bool indicating whether the origin should enable the `cmsd` daemon.
type: bool
default: true
components: ["origin"]
---
name: Origin.EnableMacaroons
description: |+
A bool indicating whether the origin allows clients to authenticate using macaroons.
type: bool
default: false
components: ["origin"]
---
name: Origin.DirectorTest
description: |+
A bool indicating whether the director should send file transfer tests to the origin.
If `Origin.StorageType` is set to values other than `POSIX`, this parameter is set to false.
type: bool
default: true
components: ["origin"]
---
name: Origin.SelfTest
description: |+
A bool indicating whether the origin should perform self health checks.
If `Origin.StorageType` is set to values other than `POSIX`, this parameter is set to false.
type: bool
default: true
components: ["origin"]
---
name: Origin.SelfTestInterval
description: |+
The interval of which the origin starts a new file transfer test to itself.
type: duration
default: 15s
components: ["origin"]
---
name: Origin.EnableUI
description: |+
Indicate whether the origin should enable its web UI.
type: bool
default: true
components: ["origin"]
---
name: Origin.EnableOIDC
description: |+
Indicate whether the origin should allow users to login to the admin website via OAuth2/OIDC with third-party
authentication providers such as CILogon.
If set to true, it is recommended that you also set `Server.UIAdminUsers` to a list of users
to give admin privilege. This is because origin admin website doesn't have a public, non-admin view,
and an empty AdminUsers list will lead to "permission denied" error
for all users logged into origin admin website via OAuth.
type: bool
default: false
components: ["origin"]
---
name: Origin.EnableBroker
description: |+
Indicate whether the origin should utilize the broker service to avoid
the need for incoming connections.
type: bool
default: false
components: ["origin"]
---
name: Origin.EnableIssuer
description: |+
Enable the built-in issuer daemon for the origin.
type: bool
default: false
components: ["origin"]
---
name: Origin.ScitokensRestrictedPaths
description: |+
This parameter is used to configure
[XRootD's SciTokens authorization plugin](https://github.com/xrootd/xrootd/tree/master/src/XrdSciTokens).
Any restrictions on the paths that the issuer can authorize inside their
namespace. This is meant to be a mechanism to help with transitions, where
the underlying storage is setup such that an issuer's namespace contains
directories that should not be managed by the issuer.
type: stringSlice
default: []
components: ["origin"]
---
name: Origin.ScitokensMapSubject
description: |+
This parameter is used to configure
[XRootD's SciTokens authorization plugin](https://github.com/xrootd/xrootd/tree/master/src/XrdSciTokens).
If set to `true`, the contents of the token's `sub` claim will be copied
into the XRootD username. When `Origin.Multiuser` is also set to `true`,
this will allow XRootD to read and write files using the Unix username
specified in the token.
type: bool
default: false
components: ["origin"]
---
name: Origin.ScitokensDefaultUser
description: |+
This parameter is used to configure
[XRootD's SciTokens authorization plugin](https://github.com/xrootd/xrootd/tree/master/src/XrdSciTokens).
If set, then all authorized operations will be performed under the
provided username when interacting with the file system. This is useful
when all files owned by an issuer should be mapped to a particular Unix
user account.
type: string
default: none
components: ["origin"]
---
name: Origin.ScitokensUsernameClaim
description: |+
This parameter is used to configure
[XRootD's SciTokens authorization plugin](https://github.com/xrootd/xrootd/tree/master/src/XrdSciTokens).
If set, then the provided claim will be used to determine the XRootD
username, and it will override the
`Origin.ScitokensMapSubject` and `Origin.ScitokensDefaultUser` parameters.
type: string
default: none
components: ["origin"]
---
name: Origin.ScitokensNameMapFile
description: |+
This parameter is used to configure
[XRootD's SciTokens authorization plugin](https://github.com/xrootd/xrootd/tree/master/src/XrdSciTokens).
If set, then the referenced file is parsed as a JSON object and the
specified mappings are applied to the username inside the XRootD
framework. See the
[XrdSciTokens documentation](https://github.com/xrootd/xrootd/tree/master/src/XrdSciTokens#mapfile-format)
for more information on the mapfile's format.
type: string
default: none
components: ["origin"]
---
name: Origin.XRootDPrefix
description: |+
The directory prefix for the XRootD origin configuration files.
type: string
default: origin
components: ["origin"]
---
name: Origin.EnableVoms
description: |+
Enable X.509 / VOMS-based authentication. This allows HTTP clients to
present X.509 client credentials in order to authenticate. The configuration
of the authorization for these clients must be done by the admin; Pelican
does not support automatic VOMS authorization configuration.
type: bool
default: true
components: ["origin"]
---
name: Origin.EnableDirListing
description: |+
[Deprecated] Origin.EnableDirListing is being deprecated and will be removed in a future release. It is replaced by Origin.EnableListings.
Allows the origin to enable collection listings. Needs to be enabled for recursive
downloads to work properly and for directories to be visible.
type: bool
default: false
deprecated: true
replacedby: "Origin.EnableListings"
components: ["origin"]
---
name: Origin.Mode
description: |+
[Deprecated] Origin.Mode is being deprecated and will be removed in a future release. It is replaced by Origin.StorageType.
The backend mode to be used by an origin. Current values that can be selected from
are either "posix" or "s3".
type: string
default: posix
deprecated: true
replacedby: "Origin.StorageType"
components: ["origin"]
---
name: Origin.S3ServiceName
description: |+
[Deprecated] Origin.S3ServiceName was previously used in part to determine an export's FederationPrefix, but
upstream changes no longer rely on this value. As of Pelican `7.7.0`, setting this value no longer has any effect.
AWSv4 signatures used by S3 servers to handle authentication now hardcode "s3" as their service name.
When constructing signed URLs for S3, this value is used as a part of the signature. It is almost always "s3". For more
information about S3 service names, see https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html
type: string
deprecated: true
replacedby: none
default: none
components: ["origin"]
---
name: Origin.S3Region
description: |+
Objects in S3 are associated with a "region", which is specifically a part of AWS's infrastructure. Often, S3 endpoints that are not
provided by Amazon use "us-east-1" as their region. This value is used when constructing signed URLs for getting authenticated objects
from a bucket.
For more information about how Amazon uses regions, see https://docs.aws.amazon.com/general/latest/gr/s3.html
This value is REQUIRED for S3 origins.
type: string
default: none
components: ["origin"]
---
name: Origin.S3Bucket
description: |+
**Note**: This value is only for setting up an origin that exports **one** storage prefix. For multiple exports, use `Origin.Exports`
Objects in S3 are stored in "buckets", which have unique names at each S3 service URL (ie the URL that provides access to your objects).
Setting a bucket restricts the origin to only serving objects from that bucket.
However, if the origin is meant to export all of the buckets associated with a given service URL, this value can be left unset *IF* all
of those buckets are public and the origin is using path-style URLS. When this is the case, objects can be fetched from the origin at the
path `/federation/prefix/bucket-name/object-name`.
type: string
default: none
components: ["origin"]
---
name: Origin.S3ServiceUrl
description: |+
The URL that provides API access your objects. When the S3 instance is hosted by Amazon, this is often "https://s3.us-east-1.amazonaws.com".
This value is REQUIRED for S3 origins.
type: string
default: none
components: ["origin"]
---
name: Origin.S3AccessKeyfile
description: |+
**Note**: This value is only for setting up an origin that exports **one** storage prefix. For multiple exports, use `Origin.Exports`
A path to a file containing an S3 access keyfile (also sometimes called an API key) for authenticated buckets when an origin
is run in S3 mode.
This value is OPTIONAL for S3 origins, and only applies when an exported bucket requires authentication. It should not be used
if the bucket is public or if the origin is meant to export all public buckets from the S3 service URL.
type: filename
default: none
components: ["origin"]
---
name: Origin.S3SecretKeyfile
description: |+
**Note**: This value is only for setting up an origin that exports **one** storage prefix. For multiple exports, use `Origin.Exports`
A path to a file containing an S3 secret keyfile for authenticated buckets when an origin is run in S3 mode.
This value is OPTIONAL for S3 origins, and only applies when an exported bucket requires authentication. It should not be used
if the bucket is public or if the origin is meant to export all public buckets from the S3 service URL.
type: filename
default: none
components: ["origin"]
---
name: Origin.S3UrlStyle
description: |+
The style of S3 urls used by the service URL host. This can be either "path" if objects are fetched at `<service-url>/<bucket>/<object>`
or "virtual" if objects are fetched at `<bucket>.<service-url>/<object>`.
This value is REQUIRED for S3 origins, but defaults to "path" if not set.
type: string
default: path
components: ["origin"]
---
name: Origin.HttpServiceUrl
description: |+
If Origin.StorageType is set to `https`, the service URL is used as the base for requests to the backend. To generate the
request, the Origin.FederationPrefix is removed from the object name, then the result is joined with the service URL and storage prefix.
For example, if one sets `Origin.HTTPServiceUrl=https://example.com`, `Origin.StoragePrefix=/testfiles` and `Origin.FederationPrefix=/foo`,
then a request for an object named `/foo/bar` will generate a request to https://example.com/testfiles/bar.
type: string
default: none
components: ["origin"]
---
name: Origin.HttpAuthTokenFile
description: |+
When set, all requests from the origin to the http backend will include the contents of the file as a bearer token in the
Authorization header.
If the origin backend is configured with the `globus` storage type, any value set here will be overridden with the filepath to
the first file ending in `.tok` found in the $(Origin.GlobusConfigLocation)/tokens directory
type: filename
default: none
components: ["origin"]
---
name: Origin.XRootServiceUrl
description: |+
When the origin is configured to export another XRootD storage backend by setting `Origin.StorageType = xroot`, the `XRootServiceUrl`
is used as the base for `root` protocol requests and should point at the upstream XRootD server.
type: string
default: none
components: ["origin"]
---
name: Origin.GlobusCollectionID
description: |+
**Note**: This value is only for setting up an origin that exports **one** Globus collection. For multiple exports, use `Origin.Exports`
Required if `Origin.StorageType == "globus"` AND `Origin.Exports` is not set.
Globus stores objects in [Collections](https://docs.globus.org/guides/overviews/collections-and-endpoints/#collection).
The unique identifier of a Collection is the Collection UUID.
You can find the Collection UUID at the bottom of your Collection's overview page.
type: string
default: none
components: ["origin"]
---
name: Origin.GlobusCollectionName
description: |+
**Note**: This value is only for setting up an origin that exports **one** storage prefix. For multiple exports, use `Origin.Exports`