Validate variables specified in system tests

[andrewkroh]

When a developer writes a system test config like _dev/test/system/test-*-config.yml, they can specify variables that are to be passed in the POST /api/fleet/package_policies fleet API request.

As was observed in elastic/integrations#11554, there is no validation that the variables specified are correct. The feature request is to make elastic-package validate that the variables used in the test config actually correspond to real variables defined in the package or data stream manifests, and that they match the "level" at which they are defined in the package.

The fact that incorrect variables are accepted probably means the POST /api/fleet/package_policies fleet API is not strict about validating the specified parameters exist.

Surprisingly, there are only two levels of variables listed in the code when there are four levels of variables in packages (integration, policy template, policy template input, data stream). We should probably expand the test config to match with 4 "levels".

Related

• 1password - fix system test preserve_original_event integrations#11554