Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(swingset): add tools to classify mainnet runs #8680

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

warner
Copy link
Member

@warner warner commented Dec 20, 2023

Feed a slogfile into stdin of classify-runs.py, and it will emit one line per run, describing why the run was performed (price-feed, psm-sell, etc, known as the "type" of the run), along with the b${blockNum}-r${runNum} identifier, the number of computrons consumed, and the number of wallclock seconds consumed.

Feed all those output lines into the stdin of classify-types.py, and it will emit a table of statistics about those types, like the median wallclock time consumed.

@warner warner added the SwingSet package: SwingSet label Dec 20, 2023
@warner warner requested a review from toliaqat December 20, 2023 15:12
@warner warner self-assigned this Dec 20, 2023
@warner
Copy link
Member Author

warner commented Jan 9, 2024

@toliaqat hey, what should we do about this one? I'm inclined to land it, so at least the classification logic is available for folks to look at, even if it's in python instead of js.

@toliaqat
Copy link
Contributor

toliaqat commented Jan 9, 2024

@warner I think we should merge this in the misc-tools even if it is in Python. I like the idea given in the last meeting that we can have this Python code run in a job on the slog files from blob storage.

I want to create this pipeline so that we can start injecting such tools to export the metrics on datadog at regular intervals. decoupling the developer who wrote the tool from potential users of the outcome.

@warner warner added the automerge:rebase Automatically rebase updates, then merge label Jan 9, 2024
Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

6 similar comments
Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

@warner warner added automerge:rebase Automatically rebase updates, then merge and removed automerge:rebase Automatically rebase updates, then merge labels Jan 23, 2024
Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

13 similar comments
Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Feb 1, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Feb 3, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Feb 5, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Feb 7, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Feb 9, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Apr 3, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

19 similar comments
Copy link

github-actions bot commented Apr 5, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Apr 7, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented Apr 9, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented May 1, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

Copy link

github-actions bot commented May 3, 2024

@warner - This PR appears to be stuck. It's had a merge label for > 24 hours

@warner warner force-pushed the warner/add-run-classifiers branch from 25fd5e8 to d04f3c0 Compare September 24, 2024 14:52
Copy link

cloudflare-workers-and-pages bot commented Sep 24, 2024

Deploying agoric-sdk with  Cloudflare Pages  Cloudflare Pages

Latest commit: dba9cce
Status: ✅  Deploy successful!
Preview URL: https://06eefd22.agoric-sdk.pages.dev
Branch Preview URL: https://warner-add-run-classifiers.agoric-sdk.pages.dev

View logs

@warner warner removed the automerge:rebase Automatically rebase updates, then merge label Sep 24, 2024
Feed a slogfile into stdin of `classify-runs.py`, and it will write
one line per run to two output files (runs-separate.csv and
runs-continued.jsonlines), describing why the run was
performed (price-feed, psm-sell, etc, known as the "type" of the run),
along with the b${blockNum}-r${runNum} identifier, the number of
computrons consumed, and the number of wallclock seconds consumed. See
the header of classify-runs.py for details.

Feed the .csv to the stdin of `classify-types.py`, and it will emit a
table of statistics about those types, like the median wallclock time
consumed.
@warner warner force-pushed the warner/add-run-classifiers branch from d04f3c0 to dba9cce Compare September 24, 2024 19:17
@warner warner added the automerge:rebase Automatically rebase updates, then merge label Sep 24, 2024
@mergify mergify bot merged commit f46aec9 into master Sep 24, 2024
90 checks passed
@mergify mergify bot deleted the warner/add-run-classifiers branch September 24, 2024 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge:rebase Automatically rebase updates, then merge SwingSet package: SwingSet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants