Skip to content

Commit

Permalink
Updated summary charts howto, added color palette howto. (#1818)
Browse files Browse the repository at this point in the history 
* Updated documentation for customizing Summary charts
* Added documentation for customizing the color palette
---------
Co-authored-by: Andrew Stoltman <[email protected]>
  • Loading branch information
@Macman1234
Macman1234 authored Jul 26, 2024
1 parent 088611c commit 3305981
Show file tree
Hide file tree
Showing 3 changed files with 158 additions and 50 deletions.
101 changes: 101 additions & 0 deletions docs/howto-colors.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
title: HOWTO Change Metric Explorer Colors
---

The color palette available to style chart data in Metric Explorer can be edited
in the JSON file `colors1.json` (`/etc/xdmod/colors1.json` if you installed the RPM or
`PREFIX/etc/colors1.json` if you did a manual install).

The default version of `colors1.json` (and what is used as a fallback palette if `colors1.json`
is missing or malformed) is shown below

```json
[
[
["FFFFFF"], ["1199FF"], ["DB4230"], ["4E665D"], ["F4A221"], ["66FF00"], ["33ABAB"], ["A88D95"],
["789ABC"], ["FF99CC"], ["00CCFF"], ["FFBC71"], ["A57E81"], ["8D4DFF"], ["FF6666"], ["CC99FF"],
["2F7ED8"], ["0D233A"], ["8BBC21"], ["910000"], ["1AADCE"], ["492970"], ["F28F43"], ["77A1E5"],
["3366FF"], ["FF6600"], ["808000"], ["CC99FF"], ["008080"], ["CC6600"], ["9999FF"], ["99FF99"],
["969696"], ["FF00FF"], ["FFCC00"], ["666699"], ["00FFFF"], ["00CCFF"], ["993366"], ["3AAAAA"],
["C0C0C0"], ["FF99CC"], ["FFCC99"], ["CCFFCC"], ["CCFFFF"], ["99CCFF"], ["339966"], ["FF9966"],
["69BBED"], ["33FF33"], ["6666FF"], ["FF66FF"], ["99ABAB"], ["AB8722"], ["AB6565"], ["990099"],
["999900"], ["CC3300"], ["669999"], ["993333"], ["339966"], ["C42525"], ["A6C96A"], ["111111"]
]
]
```

To change the colors available, simply edit the hexadecimal codes for any of the swatches in the palette.
To add a new color or set of colors, make sure to keep the JSON array rectangular with the same width.

As an example, to add the colors `123456` and `654321` you'd need to pad the table with other colors.
In this case, they're padded with white, which Open XDMoD skips when showing charts.

```json
[
[
["FFFFFF"], ["1199FF"], ["DB4230"], ["4E665D"], ["F4A221"], ["66FF00"], ["33ABAB"], ["A88D95"],
["789ABC"], ["FF99CC"], ["00CCFF"], ["FFBC71"], ["A57E81"], ["8D4DFF"], ["FF6666"], ["CC99FF"],
["2F7ED8"], ["0D233A"], ["8BBC21"], ["910000"], ["1AADCE"], ["492970"], ["F28F43"], ["77A1E5"],
["3366FF"], ["FF6600"], ["808000"], ["CC99FF"], ["008080"], ["CC6600"], ["9999FF"], ["99FF99"],
["969696"], ["FF00FF"], ["FFCC00"], ["666699"], ["00FFFF"], ["00CCFF"], ["993366"], ["3AAAAA"],
["C0C0C0"], ["FF99CC"], ["FFCC99"], ["CCFFCC"], ["CCFFFF"], ["99CCFF"], ["339966"], ["FF9966"],
["69BBED"], ["33FF33"], ["6666FF"], ["FF66FF"], ["99ABAB"], ["AB8722"], ["AB6565"], ["990099"],
["999900"], ["CC3300"], ["669999"], ["993333"], ["339966"], ["C42525"], ["A6C96A"], ["111111"],
["123456"], ["654321"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"]
]
]
```

Charts in Open XDMoD will go sequentially through this list when deciding what colors to color grouped sets within
one dataset on a chart. As such, you can create a gradient by defining it sequentially in this list. The
following JSON list adds a 8-element "snapshot" of [the Viridis color gradient](https://bids.github.io/colormap/).

```json
[
[
["FFFFFF"], ["1199FF"], ["DB4230"], ["4E665D"], ["F4A221"], ["66FF00"], ["33ABAB"], ["A88D95"],
["789ABC"], ["FF99CC"], ["00CCFF"], ["FFBC71"], ["A57E81"], ["8D4DFF"], ["FF6666"], ["CC99FF"],
["2F7ED8"], ["0D233A"], ["8BBC21"], ["910000"], ["1AADCE"], ["492970"], ["F28F43"], ["77A1E5"],
["3366FF"], ["FF6600"], ["808000"], ["CC99FF"], ["008080"], ["CC6600"], ["9999FF"], ["99FF99"],
["969696"], ["FF00FF"], ["FFCC00"], ["666699"], ["00FFFF"], ["00CCFF"], ["993366"], ["3AAAAA"],
["C0C0C0"], ["FF99CC"], ["FFCC99"], ["CCFFCC"], ["CCFFFF"], ["99CCFF"], ["339966"], ["FF9966"],
["69BBED"], ["33FF33"], ["6666FF"], ["FF66FF"], ["99ABAB"], ["AB8722"], ["AB6565"], ["990099"],
["FDE725"], ["A0DA39"], ["4AC16D"], ["1FA187"], ["277F8E"], ["365C8D"], ["46327E"], ["440154"]
]
]
```

Open XDMOD indexes the colors based on the first instance of the color found in that list,
so any duplicate colors may wreck havoc on group-by coloring.
As an example, selecting the first `123456` in the following
palette would cause the next two groups in that dataset to be colored `AABBCC` and `CCBBAA`.
Selecting the second `123456` in the palette would result in the next two groups *still* being colored `AABBCC` and `CCBBAA`.

```json
[
[
["123456"], ["AABBCC"], ["CCBBAA"], ["123456"], ["111111"], ["222222"], ["333333"], ["444444"]
]
]
```

The palette below shows a workaround for this problem in order to add both an 8-element and 10-element Viridis gradient.
This is done by nudging the starting point over by a value of 1 in any of the 3 RGB channels. In this example, the starting point of the
8-element gradient is `FDE725`, and the starting point of the 10-element gradient is `FDE726`.

```json
[
[
["FFFFFF"], ["1199FF"], ["DB4230"], ["4E665D"], ["F4A221"], ["66FF00"], ["33ABAB"], ["A88D95"],
["789ABC"], ["FF99CC"], ["00CCFF"], ["FFBC71"], ["A57E81"], ["8D4DFF"], ["FF6666"], ["CC99FF"],
["2F7ED8"], ["0D233A"], ["8BBC21"], ["910000"], ["1AADCE"], ["492970"], ["F28F43"], ["77A1E5"],
["3366FF"], ["FF6600"], ["808000"], ["CC99FF"], ["008080"], ["CC6600"], ["9999FF"], ["99FF99"],
["969696"], ["FF00FF"], ["FFCC00"], ["666699"], ["00FFFF"], ["00CCFF"], ["993366"], ["3AAAAA"],
["C0C0C0"], ["FF99CC"], ["FFCC99"], ["CCFFCC"], ["CCFFFF"], ["99CCFF"], ["339966"], ["FF9966"],
["69BBED"], ["33FF33"], ["6666FF"], ["FF66FF"], ["99ABAB"], ["AB8722"], ["AB6565"], ["990099"],
["FDE725"], ["A0DA39"], ["4AC16D"], ["1FA187"], ["277F8E"], ["365C8D"], ["46327E"], ["440154"],
["FDE726"], ["B5DE2B"], ["6ECE58"], ["35B779"], ["1F9E89"], ["26828e"], ["31688E"], ["3E4989"],
["482878"], ["440155"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"], ["FFFFFF"]
]
]
```
102 changes: 54 additions & 48 deletions docs/howto-summary-charts.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,72 +2,78 @@
title: HOWTO Change Summary Page Charts
---

To change the default charts for all users you need to update the
`roles.json` file (`/etc/xdmod/roles.json` if you installed the RPM or
`PREFIX/etc/roles.json` if you did a manual install) and change the JSON
for the charts. Since it is not obvious how to do this, I suggest
looking at the JSON created from the Metric Explorer for the changes you
need to make in `roles.json`.
If a user wants to add a chart to their personal Summary tab, they can use Metric Explorer to create a chart,
then open the "Chart Options" context menu by left clicking in the chart, then check the "Show in Summary
tab" checkbox.

First, log into the portal and create the desired charts using the
Metric Explorer. You can add this chart to the summary page, but only
for that user, by checking "Show in Summary Tab".
To change the charts that appear in users' Summary tabs by default, you'll need to create
a chart and add its JSON definition to the XDMoD configuration files.

After saving the chart, select from the table `UserProfiles` in the
database `moddb`.
The easiest way to create a JSON definition for a chart is interactively,
using Metric Explorer's JSON export function:

mysql> SELECT * FROM moddb.UserProfiles;
* Login to Open XDMoD with a user account that has the "Developer" role.
* Create a chart in the Metric Explorer.
* Open the "Chart Options" context menu by left clicking in the chart.
* Select the "View chart json" menu item.

This will show the user profile data for all users including the chart
JSON serialized by PHP. If you are unsure of your `user_id`, check the
`Users` table.
Default summary charts are defined as lists of charts within the configuration files for user roles.
These are located in two different sets of locations: in the top level `roles.json` file
(`/etc/xdmod/roles.json` if you installed the RPM or `PREFIX/etc/roles.json` if you did a manual install),
and in any `.json` files located in `roles.d` (`/etc/xdmod/roles.d/` or `PREFIX/xdmod/roles.d/` respectively).

mysql> SELECT id, username FROM moddb.Users;
The base summary charts (the ones included with a install of Open XDMoD) are included implicity
for any user role if the field `summary_charts` is absent for that role in both `roles.json` and
in `roles.d`. If `summary_charts` contains any charts, those override displaying any of the summary charts.

Next you'll need to deserialize the data then examine the JSON. Copy
this code into a file and run it with the data from your user profile:
By default, in an out-of-the-box XDMoD configuration:

```php
<?php
$data = 'a:1:{....'; // Copy your data into this variable.
$profile = unserialize($data);
$queries = json_decode($profile['queries'], true);
foreach ($queries as $name => $query) {
echo $name, ":\n";
echo $query['config'], "\n\n";
}
```
* The role "pub" - which is applied to any users who are not logged in - has no `summary_charts` field and so implictly uses the base summary charts.
* The role "default" - which all other user roles inherit - has an empty `summary_charts` field in `roles.json`, but
has explicitly defined summary charts appended in `roles.d/jobs.json` that are equivalent to the base summary charts.

This will output the JSON for each chart saved for the user the data was
copied from. Find the chart that you want to add to the default summary
page and copy it into `summary_charts` section of `roles.json`. Add the
JSON object list after the name of the chart.
The recommended method to define new summary charts is to create a new `.json` file in `roles.d` that specifically handles
any summary chart customization, rather than using the existing `.json` files. If you'd like to keep the default charts along with
any newly created charts, it's recommended to move the ones included with `roles.d/jobs.json` and any other files that contain `+summary_charts`,
to your newly created `.json` file, deleting them from their original files.

As an example, if you wanted to add a default chart for the "default" role,
you would edit your summary chart customization `.json` file in `roles.d`.
In the example below, you'd replace the `...`
in the `+summary_charts` list with the JSON you copied.

```json
"summary_charts": [
{
...
},
{
{
"+roles": {
"+default": {
...
"+summary_charts": [
...
]
}
...
},
...
]
}
}

```

You should also add a title to JSON for the chart that will be displayed
on the summary page.
This would only change the summary charts for logged-in users - if you wanted to make this same change for non-logged-in users,
you would have to do the same do the same for the role "pub", rather than just for "default".

There's a known issue displaying charts with no `global_filters` field
in the Summary page. If the chart you're adding has no global filters,
add an empty `global_filters` field as shown below
(see the premade ones in `roles.d/jobs.json` for more examples).

```json
"summary_charts": [
"+summary_charts": [
{
"title": "Chart Title",
"global_filters": {
"data": [],
"total": 0
},
...
},
...
]
```

Note that it is possible to have a different set of charts for different
roles, but the default configuration uses a single set of charts for all
roles.
5 changes: 3 additions & 2 deletions docs/howto.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ title: HOWTOs

The Open XDMoD HOWTOs are "how to" documents on specific subjects.

- [Change Default Summary Page Charts](howto-summary-charts.html)
- [Change Summary Page Charts](howto-summary-charts.html)
- [Change Metric Explorer Colors](howto-colors.html)
- [Enable Node Utilization Statistics](howto-node-utilization.html)
- [Reconstruct Slurm accounting logs](howto-reconstruct-slurm.html)
- [Reconstruct Slurm Accounting Logs](howto-reconstruct-slurm.html)

0 comments on commit 3305981

Please sign in to comment.