Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP COMPLETE RECODE #63

Merged
merged 33 commits into from
Feb 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
18974ac
start from a blank slate
lquenti Jan 21, 2024
e56a81c
create workfspace structre
lquenti Jan 21, 2024
9480197
get the C interop working in general
lquenti Jan 21, 2024
8e3d1f6
add C ci
lquenti Jan 21, 2024
ebb7918
does the CI run now?
lquenti Jan 21, 2024
d442c61
specify OS
lquenti Jan 21, 2024
8f33828
fic CI path
lquenti Jan 21, 2024
554ef83
add werror
lquenti Jan 21, 2024
d0907de
gcc should be happy now
lquenti Jan 21, 2024
82e395a
disable `unusedFunction` in cppcheck
lquenti Jan 21, 2024
c075e09
define C-Rust interface for benchmarker
lquenti Feb 21, 2024
04b14a4
fix C CI
lquenti Feb 21, 2024
9b917e2
benchmark init logic
lquenti Feb 21, 2024
b571a2f
initial implementation benchmarker C code
lquenti Feb 22, 2024
6144348
logic to restrict memory
lquenti Feb 22, 2024
aec9286
asan and udsan for testing
lquenti Feb 22, 2024
ce60474
more test cases
lquenti Feb 22, 2024
8b5650c
heap-verflow found with asan
lquenti Feb 22, 2024
e9dca70
get the C code building
lquenti Feb 22, 2024
c132139
called C code from Rust!
lquenti Feb 22, 2024
cbc63dd
provide a high level Rust interface to the C-benchmarker
lquenti Feb 22, 2024
8a74bca
get CLI parser running + human_panic
lquenti Feb 22, 2024
04325c7
CLI validation
lquenti Feb 22, 2024
3aece39
set up logging
lquenti Feb 22, 2024
4efc9ad
progress tracking structure
lquenti Feb 23, 2024
2995c6e
load the previous progress
lquenti Feb 23, 2024
eebcd4b
add the benchmarking procedure in main program
lquenti Feb 23, 2024
dec7ca9
fmt
lquenti Feb 23, 2024
f2e9fd0
restructure files out of main.rs
lquenti Feb 23, 2024
6e4cad7
clippy + fmt benchmarker
lquenti Feb 23, 2024
8cb97f8
clippy + fmt blackheap
lquenti Feb 23, 2024
c347f05
finish the benchmarker; no analysis yet
lquenti Feb 23, 2024
d2e3d5d
readme notice
lquenti Feb 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions .github/ISSUE_TEMPLATE/bug_report.md

This file was deleted.

1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/config.yml

This file was deleted.

10 changes: 0 additions & 10 deletions .github/ISSUE_TEMPLATE/custom.md

This file was deleted.

20 changes: 0 additions & 20 deletions .github/ISSUE_TEMPLATE/feature_request.md

This file was deleted.

6 changes: 0 additions & 6 deletions .github/PULL_REQUEST_TEMPLATE.md

This file was deleted.

21 changes: 0 additions & 21 deletions .github/workflows/build_all.yml

This file was deleted.

19 changes: 19 additions & 0 deletions .github/workflows/c.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Everything C
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
static_analysis:
name: Run gcc with warnings
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Dependencies
run: sudo apt-get install gcc cppcheck
- name: Compile Library with Warnings
run: gcc -Wall -Wextra -Wpedantic -Werror -std=c11 -c -o /dev/null ./blackheap-benchmarker/src/c_code/benchmarker_internal.c
- name: Run cppcheck
# This is everything but `unusedFunction` enabled
run: cppcheck --enable=warning,style,performance,portability,information,missingInclude --error-exitcode=1 ./blackheap-benchmarker/src/c_code/benchmarker.c
78 changes: 63 additions & 15 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,69 @@ on:
env:
CARGO_TERM_COLOR: always
jobs:
build:
build_works:
name: Build Works
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: |
rustup update
rustup component add clippy
rustup component add rustfmt
- name: Check
working-directory: ./blackheap-modeller
run: cargo check
- name: Run clippy
working-directory: ./blackheap-modeller
run: cargo clippy -- -Dwarnings
- name: Run Fmt
working-directory: ./blackheap-modeller
run: cargo fmt --check
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: run cargo build --release
run: |
cargo build --release
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: run cargo test
run: |
cargo test
fmt:
name: fmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: run cargo fmt --check
run: |
cargo fmt --check
clippy:
name: clippy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: run cargo clippy
run: |
cargo clippy
doc:
name: Doc
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: run cargo doc
run: |
cargo doc

18 changes: 14 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
.vscode
blackheap
# Generated by Cargo
# will have compiled files and executables
debug/
target/

# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock

# These are backup files generated by rustfmt
**/*.rs.bk

# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb

# If you just use blackheap locally, please don't commit
/**/default-model
8 changes: 8 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[workspace]

members = [
"blackheap",
"blackheap-benchmarker"
]

resolver = "2"
20 changes: 4 additions & 16 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,19 +1,7 @@
Copyright (c) 2022 Lars Quentin
Copyright (c) 2022-2024 Lars Quentin

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
33 changes: 0 additions & 33 deletions Makefile

This file was deleted.

64 changes: 61 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,65 @@
# RECODE IN PROGRESS SEE BRANCH ETA MID 2024
# Blackheap

![Blackheap Logo](./icon.png)
# BIG RECODE, `old-v0` is the old branch

This tool creates a performance model of your I/O speed and further allows to predict future preformance.

# [Link to the Documentation](https://lquenti.github.io/blackheap/book/)
## Note

There is a **big recode** right now. But how does one eat an elephant...

## Progress
- [x] Start new repo
- [ ] Requirements Engineering
- [x] Find out how to formalize what the code should do
- fuck formalization, we use pseudocode
- [ ] Look what the code did before by looking through...
- [ ] <https://lquenti.github.io/blackheap/book/>
- [ ] <https://gwdg.github.io/iofs/book/>
- [ ] `blackheap-benchmark`
- [ ] `blackheap-modeller`
- [ ] `blackheap-frontend`
- [ ] `preloadee`
- [ ] All open issues
- [ ] All closed isses
- Unstructured ideas:
- Has to be resumeable
- be able to re-analyze raw data again
- Provide machine generated README in the data
- `blackheap-benchmarker` does not rely on bindgen compile time since this would create LLVM as dep
- [ ] Also provide a standalone binary for the benchmarker?
- [ ] Find a way to have C linting (all warnings, formatter, pedantic C standard)
- [ ] Design a high level architecture based on the requirements
- [x] Finish the benchmarker
- [ ] Finish the Rust Code

- [ ] Try to move stuff out of the `bin` into the `lib` crates

## Issues for when the recode is done
- Add `O_DIRECT` support

## Architecture
Cargo workspace with the following crates
```
- blackheap-core (lib): stuff all other libraries need (like Definitions)
- blackheap-benchmarker (lib): C code with Rust wrapper
- blackheap-analyzer (lib): Analysis of the benchmarks
- blackheap (bin): The user facing code
```

High level workflow:
```
- user starts blackheap with config parameters
- blackheap checks which benchmarks are already done
- If folder doesnt exist / was not provided, do all benchmarks
- based on those benchmarks, do the analysis
- Create (human-readable) plots
- Create (machine-readable) parsing data
```

Benchmarker:
```
- get input from Rust struct
- give output in Rust struct
- Rust part should support a to json function for persistence
```
Loading
Loading