The GRR configuration system is the main way users can control and customize the behavior of GRR components. Previous versions of GRR used command line flags to configure components, however that became unwieldy with a lot of confusing and often contradictory command line options embedded in various shell scripts. We decided to replace the configuration system with a new file based system.
When software is written with the GRR configuration system, it can define parameters that it is going to use. Parameters have the general form Section.parameter_name where Section corresponds to the component which defines related parameters. For example, the Logging subsystem can define several parameters: Logging.path, Logging.verbose etc. The use of section names ensures that different components can safely reuse the same parameter name without clashes.
In addition to the parameter name, it is possible to declare a helpful message, to assist the user in understanding the purpose of this parameter. Parameters are also strongly typed. For example we can find the following code:
config_lib.DEFINE_float("Client.rss_max", 500,
"Maximum memory footprint in MB.")
config_lib.DEFINE_option(type_info.PEMPrivateKey(
name="Client.private_key",
description="Client private key in pem format. If not provided this "
"will be generated by the enrollment process.",
))The first definition declares the parameter Client.rss_max to be a float, with a default value of 500.
The second definition is more complex. It defines the Client.private_key parameter as a PEMPrivateKey type info declaration. This declaration allows the configuration system itself to verify that the parameter is a valid private key in PEM format. There is now no need for the code to specifically test for sanity of the private key. If the user provides an invalid or corrupt PEM encoded key, it is caught immediately by the config system and the program aborts. For example the following happens after removing a random character from the CA.certificate PEM string:
Traceback (most recent call last):
File "grr/client/client.py", line 48, in <module>
conf.StartMain(main)
File "grr/client/conf.py", line 93, in StartMain
main([sys.argv[0]])
......
File "grr/lib/config_lib.py", line 467, in _VerifyParameters
descriptor.Validate(value)
File "grr/lib/rdfvalues/crypto.py", line 173, in Validate
return self.ParseFromString(value)
File "grr/lib/rdfvalues/crypto.py", line 185, in ParseFromString
raise type_info.TypeValueError("Certificate %s is invalid." % self.name)
grr.lib.type_info.TypeValueError: Certificate CA.certificate is invalid.
Since every parameter is defined in advance and has a default value, the configuration file does not need to specify every parameter - only those which should be changed from the default value. In order to see all declared parameters and their default values, we can issue the --config_help command line flag. This is analogous to the --help flag:
Client.control_urls = ["http://www.example.com/control"] List of URLs of the controlling server. * Value = ['http://localhost:8080/control'] Client.max_out_queue = 10240000 Maximum size of the output queue. * Value = 10240000
We can see that Client.control_urls has a default value of http://www.example.com/control, it is used to specify the URL of the controlling server, and currently (after reading the configuration files), the value is set to http://localhost:8080/control. We can also see that Client.max_out_queue is not changed from its default value.
When GRR is installed on a system, a system wide, distribution specific, configuration file is also installed (by default /etc/grr/grr_server.yaml). This specifies the basic configuration of the GRR service (i.e. various distribution specific locations). However, the configuration typically needs to be updated with site specific parameters (for example new crypto keys, the Client.control_urls setting to allow the client to connect to the server etc.)
In order to avoid having to trash the user customized configuration files when GRR is updated in future, we write site specific configuration files into the Config.writeback location (by default /etc/grr/grr.local.yaml). This local file only contains those parameters which are changed from the defaults, or the system configuration file.
|
Note
|
The system configuration file is never modified, all updated configurations are always written to the writeback location. Do not edit the system configuration file as changes in this file will be lost when GRR is upgraded. |
Using the --config_help option will print all the supported options. This allows users to figure out what parameters can be changed in the writeback configuration file to override the system configuration, or the defaults. Generally it is not required for users to edit many configuration parameters. The most important parameters will be automatically set upon installation.
The next important concept to understand is that of configuration contexts. GRR consists of many components, which share more of their configuration. Often, however, we want to specify slight differences between each component. If we were to write a separate configuration file say for each client version (Currently, 3 operating systems, and 2 architectures) as well as for each component (e.g. worker, enroller, frontend etc), we would end up with an unmaintainable mess of many configuration files. It would make more sense to put most of the common configuration parameters in the same file, and only specify the different ones, depending on the component which is running.
GRR introduces the concept of running context. The context is a list of attributes about the currently running component. When the code accesses the configuration system, the configuration system considers the currently running context and returns the appropriate value of the required parameter.
For example, consider the parameter Logging.path which specifies the location of the log file. In a full GRR installation, we wish the admin UI to log to /var/log/grr/adminui.log while the frontend should log to /var/log/grr/frontend.log. We can therefore specify in the YAML config file:
Logging.path: /var/log/grr/grr_server.log
AdminUI Context:
Logging.path: /var/log/grr/adminui.log
HTTPServer Context:
Logging.path: /var/log/grr/httpserver.log
Client Context:
Logging.path: /var/log/grr/grr_client.log
Platform:Windows:
Logging.path: c:\\windows\\system32\\grr.log
When the adminui.py program starts up, it populates its context with AdminUI Context. This forces the configuration system to return the value specific to this context. Note that other components, such as the worker will set a context of Worker Context, which will not match any of the above overrides. Therefore the worker will simply log to the default of /var/log/grr/grr_server.log.
Note that it is possible to have multiple contexts defined at the same time. So for example, the client running under windows will have the contexts, Client Context and Platform:Windows, while a client running under Linux will have the context Client Context and Platform:Linux. When several possibilities can apply, the option which matches the most context parameters will be selected. So in the above example, linux and osx clients will have a context like Client Context and will select /var/log/grr/grr_client.log. On the other hand, a windows client will also match the Platform:Windows context, and will therefore select c:\windows\system32\grr.log.
The following is a non-exhaustive list of available contexts:
-
Client Context, Worker Context, Console Context etc. Are defined when running one of the main programs.
-
Test Context is defined when running unit tests.
-
Arch:i386 and Arch:amd64 are set when running a 32 or 64 bit binary.
-
Installer Context is defined when the windows installer is active (i.e. during installation)
-
Platform:Windows, Platform:Linux, Platform:Darwin are set when running under these platforms.
The GRR configuration file format allows for expansion of configuration parameters inside other parameters. For example consider the following configuration file:
Client.name: GRR
Nanny.service_name: "%(Client.name)service.exe"
Nanny.service_key_hive: HKEY_LOCAL_MACHINE
Nanny.service_key: Software\\%(Client.name)
Nanny.child_command_line: |
%(child_binary) --config "%(Client.install_path)\\%(Client.binary_name).yaml"The expansion sequence %(parameter_name) substitutes or expands the parameter into the string. This allows us to build values automatically based on other values. For example above, the Nanny.service_name will be GRRservice.exe and the service_key will be set to Software\GRR
Expansion is very useful, but sometimes it gets in the way. For example, if we need to pass literal % escape sequences. Consider the Logging.format parameter which is actually a python format string:
Logging.format: \%(levelname\)s \%(module\)s:\%(lineno\)s] \%(message\)s # (1)
Logging.format: %{%(levelname)s %(module)s:%(lineno)s] %(message)s} # (2)-
This form escapes GRR’s special escaping sequence by preceding both opening and closing sequences with the backslash character.
-
This variation uses the literal expansion sequence %{} to declare the entire line as a literal string and prevent expansion.
The configuration system may be extended to provide additional functionality accessible from the configuration file. For example consider the following configuration file:
Logging.path: "%(HOME|env)/grr.log" # (1)-
This expansion sequence uses the env filter which expands a value from the environment. In this case the environment variable HOME will be expanded into the Logging.path parameter to place the log file in the user’s home directory.
There are a number of additional interesting filters. The file filter allows including other files from the filesystem into the configuration file. For example, some people prefer to keep their certificates in separate files rather than paste them into the config file:
CA.certificate: "%(/etc/certificates/ca.pem|file)"It is even possible to nest expansion sequences. For example this retrieves the client’s private key from a location which depends on the client name:
Client.private_key: "%(/etc/grr/client/%(Client.name)/private_key.pem|file)"The above describes the GRR configuration system as an yaml file and that is the most commonly used format. However, configuration files are parsed using a parser implementation which can be switched on demand. Currently there are a few configuration parser implementations but additional ones can be added.
The windows client implements a registry based configuration parser. This allows GRR to use the registry instead of a flat file when running on windows. The registry base parser is selected by using a URL of the form:
client.py --config reg://HKEY_LOCAL_MACHINE/Software/GRRAdditionally an ini based configuration parser can be selected using a .conf extension:
client.py --config /etc/grr.confSince ini files can not represent nested sections, it is impossible to specify different parameters for different contexts. It is therefore not recommended to use ini files for the full system config, but they are sufficient for use as a write back location.