Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README: Add lambda submission workflow #91

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,39 @@ of the repository.

## Submitting a benchmark

### Manual
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved

To submit a benchmark, a user should create an issue in this repository using
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved
the `Benchmark submission` issue template and attach the result(s) with the
extension `.json.gz` to the issue by dragging and dropping the file on to the
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved
issue body.

### Automated with AWS lambda workflow

The submission of benchmark has been automated using [AWS lambda workflow](https://github.com/facebookexperimental/resctl-demo/blob/main/resctl-bench/doc/lambda.md) which does the work of submitting issue with link to `.json.gz`.
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved
The user needs to upload benchmark result to the AWS using lambda function url as:
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved
`resctl-bench -r <RESULT_JSON> upload --upload-url <AWS lambda function URL>`
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved
e.g
`$resctl-bench --result resctl-bench-result_2022_07_01-00_26_40.json.gz upload --upload-url https://ygvr6jnjckwamfao5xztg6idiu0ukjeb.lambda-url.eu-west-1.on.aws`
Copy link
Collaborator

@obbardc obbardc Jul 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`$resctl-bench --result resctl-bench-result_2022_07_01-00_26_40.json.gz upload --upload-url https://ygvr6jnjckwamfao5xztg6idiu0ukjeb.lambda-url.eu-west-1.on.aws`
resctl-bench --result resctl-bench-result_2022_07_01-00_26_40.json.gz upload --upload-url https://ygvr6jnjckwamfao5xztg6idiu0ukjeb.lambda-url.eu-west-1.on.aws

Is that the right lambda URL for the new instance? cc @davide125

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be https://iocost-submit-us-east-1.s3.us-east-1.amazonaws.com ?

Copy link
Collaborator Author

@santoshmahto7 santoshmahto7 Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fully sure but It might be https://yxvf2x7zfkqga6sknkxgczwonq0yvkrm.lambda-url.us-east-1.on.aws (copied from iocost-benchmark/iocost-benchmarks-ci#16 (comment))
But yeah we should have shared public URL, so that it is known to external user



## CI workflow

### Importing result

Once issue is created, the github action `import-result` gets triggered, which downloads the benchmark result and create the Pull request for merging it to `main` repo along with metadata information.
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved

### Merging result
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved

The Pull request created by importing steps should be approved and merged manually to `main` branch. Once merged, github action `Merge results` merges the benchmarks data and generates the final hwdb file.
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved

The following artifcats are uploaded to the CI workflow for each trigger:
- Final hwdb file
- Merged results
- PDFs
- hwdb input Files
santoshmahto7 marked this conversation as resolved.
Show resolved Hide resolved


## Contributing

The code which handles user-submissions behind-the-scenes is available under the
Expand Down