Report usage using a richer configuration traversing protocol #3229
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR introduces code to traverse the configuration object (in a similar
manner to the RepresentationMixin style of logging the supplied configuration
object) with the intention of giving each object a chance to report its
own usage information.
This PR modifies the HighThroughputExecutor to use this API to report
richer usage information: a specific usage query is to ask about use of the
enable_mpi_mode parameter and this modification supports that.
Beware that this reports on configuration of components, and does not
report any further usage unless those components are so augmented using
the new API: for example, configurations by default will include three
staging providers, even though I believe it is extremely rare that either
the FTP or HTTP staging providers are actually used to stage data.
(It is hopefully a straightforward change to add a UsageInformation
implementation to report if those classes are actually used to stage
data in any run).
To support UsageInformation instances which report on usage during a run,
the component tree is traversed both for the start message and the end
message, and may result in different information in each message.
An example start message looks like this: (pretty-formatted)
{'correlator': 'f7595d08-7b94-49bc-b3d7-1ea7532b2f51',
'parsl_v': '1.3.0-dev',
'python_v': '3.12.2',
'platform.system': 'Linux',
'start': 1710150467,
'components': ['parsl.config.Config',
{'c': 'parsl.executors.high_throughput.executor.HighThroughputExecutor', 'mpi': False},
'parsl.providers.local.local.LocalProvider',
'parsl.channels.local.local.LocalChannel',
'parsl.launchers.launchers.SingleNodeLauncher',
'parsl.data_provider.ftp.FTPInTaskStaging',
'parsl.data_provider.http.HTTPInTaskStaging',
'parsl.data_provider.file_noop.NoOpFileStaging',
'parsl.monitoring.monitoring.MonitoringHub']}
…info of Config object
benclifford
changed the title
khk-globus
approved these changes
Mar 18, 2024
Comment on lines
+46
to
+49
| for arg in argspec.args[1:]: # skip first arg, self | ||
| arg_value = getattr(obj, arg) | ||
| d = get_parsl_usage(arg_value) | ||
| me += d |
There was a problem hiding this comment.
Stylistic alternative (but by no means a blocker):
me.extend(get_parsl_usage(getattr(obj, arg)) for arg in argspec.args[1:])
|
I converted this to draft status, because others are working on usage tracking now (which @kylechard and @yadudoc are driving I think?) - so this can be merged if they want it, otherwise we can close it. |
NishchayKarle
added a commit
to NishchayKarle/parsl
that referenced
this pull request
Apr 24, 2024
Merge changes from Ben related to usage tracking update Parsl#3229
benclifford
pushed a commit
that referenced
this pull request
Jun 6, 2024
This PR introduces a choice of 3 levels for users to select based on their preferred level of usage reporting. It introduces updates on top of #3229. Tracking Levels Level 1: python version, parsl version, operating system details. Level 2: configuration details + Level 1 Level 3: total apps run, total failed apps, execution time + Level 2 Usage tracking if currently enabled will be defaulted to level 1. Usage Data sent at launch (Levels 1 and 2) • Capture Parsl version, Python version, and environment details at startup. • Configuration Reporting: Log details about providers, launchers, executors, channels, and storage access methods used. Usage Data sent on closure (Level 3 only) • Number of apps ran • Number of failed apps • Total time elapsed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces code to traverse the configuration object (in a similar manner to the RepresentationMixin style of logging the supplied configuration object) with the intention of giving each object a chance to report its own usage information.
The protocol now reports configured objects either as a JSON string class name, or as a JSON object containing the class name and any additional information that class wishes to report for usage (via the
UsageInformationabstract class)This PR modifies the HighThroughputExecutor to use this API to report richer usage information: a specific usage tracking query is to ask about use of the enable_mpi_mode parameter, and so the HighThroughputExecutor will now report the boolean value of that parameter.
Beware that this reports on configuration, not use, of components: for example, configurations by default will include three staging providers, even though I believe it is extremely rare that either the FTP or HTTP staging providers are actually used to stage data. The
UsageInformationAPI is intended to support reporting whether these staging providers actually stage anything, but this PR does not implement that in those staging provider components.To support UsageInformation instances which report on usage during a run, the component tree is traversed both for the start message and the end message, and may result in different information in each message - for example, the DFK report occurs only end in the end message.
An example start message looks like this: (pretty-formatted)
Changed Behaviour
Different information will be reported via usage tracking - anyone processing that usage data will need to adapt their code.
Type of change