Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QAS Demo - Flower Model #273

Merged
merged 61 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
e6675b2
Started playing with Fairness property and demo
sei-secheverria Jul 12, 2023
32c7626
Merge branch 'feature-simple-props' into feature-new-properties
sei-secheverria Jul 12, 2023
d5d25ed
Updating fairness and adding robustness
sei-secheverria Jul 12, 2023
b8cfcb6
Started adding robustness. Measurement and validation not defined yet.
sei-secheverria Jul 12, 2023
0a63e71
Started playing with Fairness property and demo
sei-secheverria Jul 12, 2023
75385ad
Updating fairness and adding robustness
sei-secheverria Jul 12, 2023
18522df
Started adding robustness. Measurement and validation not defined yet.
sei-secheverria Jul 12, 2023
cf7acac
Simplified check
sei-secheverria Jul 25, 2023
278efc6
Format fix
sei-secheverria Jul 25, 2023
01ce766
Bug fix, and new ExternalMeasurement signature fix.
sei-secheverria Aug 11, 2023
c0669e9
Added remaining robustness code, still need to tweak tests
sei-secheverria Aug 14, 2023
57fb615
Merge branch 'feature-new-properties' of github.com:sebastian-echever…
sebastian-echeverria Aug 15, 2023
211a9b6
MInor fixes, addeed tests for all robustness blur cases
sei-secheverria Aug 16, 2023
05f4d37
Some explanations, typo fixes
sei-secheverria Aug 16, 2023
5c222d8
Added tests for Robustness, with Array and Ranksum values, and proper…
sei-secheverria Aug 16, 2023
104c637
Merge branch 'bug/missing-values' into feature-new-properties
sebastian-echeverria Sep 8, 2023
1119ab4
Updating to new structure
sebastian-echeverria Sep 8, 2023
107d2a5
Improving imports, spacing, linting errors
sebastian-echeverria Sep 8, 2023
4623def
Making accuracy derive from array, and making array item id derive fr…
sebastian-echeverria Sep 8, 2023
34bb237
Adding force true to ensure demo can be run multiple times.
sebastian-echeverria Sep 8, 2023
0ac02fb
Changing default name and starting updating report notebook for demo2
sebastian-echeverria Sep 8, 2023
a21eb72
Simplifications, demo fixes, properties export consistency
sebastian-echeverria Sep 8, 2023
200d152
Adding method to print validated spec results
sebastian-echeverria Sep 8, 2023
128f2a7
More details printed
sebastian-echeverria Sep 8, 2023
e7a1bdc
Formatting
sebastian-echeverria Sep 8, 2023
22b7f31
Minor demo 2 improvements
sebastian-echeverria Sep 11, 2023
47740d9
Merge branch 'master' into feature-new-properties
sebastian-echeverria Sep 11, 2023
37fe4d2
More improvements for demo-2, now with second test.
sebastian-echeverria Sep 11, 2023
2bd7252
Finished passing Robustness notebook to using MLTE. Still need to ana…
sebastian-echeverria Sep 11, 2023
786e560
Minor formatting
sebastian-echeverria Sep 11, 2023
ccd64b2
Added initial definitions for Fairness and Robustness
sebastian-echeverria Sep 13, 2023
fed0d95
Minor formatting
sebastian-echeverria Sep 13, 2023
95c2070
Merge branch 'mlte-team:master' into feature-new-properties
sebastian-echeverria Sep 13, 2023
c080e0a
Initial performance measurements addition to demo2
sebastian-echeverria Sep 20, 2023
ce05d2c
Test data
sebastian-echeverria Sep 20, 2023
65dcb9d
Merge branch 'master' into feature-new-properties
sebastian-echeverria Sep 20, 2023
4c7286f
Fixed formatting and tensorflow dep version to avoid dependency confl…
sebastian-echeverria Sep 20, 2023
a5d93ad
Integrated interpretability into demo2; also added Interpretability a…
sebastian-echeverria Sep 25, 2023
514c44d
Updated demo2 report to validate/add new properties
sebastian-echeverria Sep 25, 2023
ca06979
Formatting
sebastian-echeverria Sep 25, 2023
c7b9c8c
Updated demo 2 with explicit QAS mentions
sebastian-echeverria Sep 25, 2023
d7c546e
Merge branch 'feature/extending-properties' into feature-new-properties
sebastian-echeverria Sep 25, 2023
1c67159
Updating to use extended properties, and to organize new properties a…
sebastian-echeverria Sep 26, 2023
c2e6ce2
Merge branch 'feature/extending-properties' into feature-new-properties
sebastian-echeverria Sep 26, 2023
3c10e30
Moved functions and TF dependency to separate file
sebastian-echeverria Sep 26, 2023
1394919
Fixed demo tensorflow requirements and added notes related to that
sebastian-echeverria Sep 27, 2023
106f18f
Started adding report code form previous demo, but it is not working yet
sebastian-echeverria Sep 27, 2023
043f5cc
Added script to download model
sebastian-echeverria Sep 27, 2023
38fd26d
Updated definitions for Fairness, Interpretability and Robustness.
sebastian-echeverria Sep 27, 2023
be25c7e
Merge branch 'master' into feature-new-properties
sebastian-echeverria Oct 2, 2023
ee8d5fb
Merge branch 'feature/simpler-properties' into feature-new-properties
sebastian-echeverria Oct 3, 2023
77de18c
Adapting to new Property constructor
sebastian-echeverria Oct 3, 2023
1c5c687
Internal reformatting
sebastian-echeverria Oct 4, 2023
828479a
Merge branch 'master' into feature-new-properties
sebastian-echeverria Oct 4, 2023
c3f2e30
Merge branch 'master' into feature-new-properties
sebastian-echeverria Oct 6, 2023
3e93466
Merge branch 'master' into feature-new-properties
sebastian-echeverria Oct 11, 2023
00af2c4
Minor upadates to demo-2 data, changed flower image to PD one.
sebastian-echeverria Oct 11, 2023
2335705
Added updated report generation
sebastian-echeverria Oct 11, 2023
8bf65d4
Notes to indicate that for Robustness and Fairness we are not running…
sebastian-echeverria Oct 11, 2023
d14d696
Restructured demo folders
sebastian-echeverria Oct 18, 2023
f265227
Fixed make for new demo structure
sebastian-echeverria Oct 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,5 @@ build/
# OSX artifacts
.DS_Store

# Demo artifacts
demo/data
demo/models
demo/store
demo/media
demo/reports

# VS Code local configs.
.vscode/
14 changes: 8 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ isort:
poetry run isort mlte/
poetry run isort test/
poetry run isort testbed/
poetry run isort demo/*.py
poetry run isort demo/
poetry run isort tools/

.PHONY: check-isort
check-isort:
poetry run isort --check mlte/
poetry run isort --check test/
poetry run isort --check testbed/
poetry run isort --check demo/*.py
poetry run isort --check demo/
poetry run isort --check tools/

# Format all source code
Expand All @@ -26,17 +26,19 @@ format:
poetry run black mlte/
poetry run black test/
poetry run black testbed/
poetry run black demo/*.py
poetry run black demo/*.ipynb
poetry run black demo/
poetry run black demo/simple/*.ipynb
poetry run black demo/scenarios/*.ipynb
poetry run black tools/

.PHONY: check-format
check-format:
poetry run black --check mlte/
poetry run black --check test/
poetry run black --check testbed/
poetry run black --check demo/*.py
poetry run black --check demo/*.ipynb
poetry run black --check demo/
poetry run black --check demo/simple/*.ipynb
poetry run black --check demo/scenarios/*.ipynb
poetry run black --check tools/

# Lint all source code
Expand Down
6 changes: 6 additions & 0 deletions demo/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Demo artifacts
simple/data
simple/models
**/store
**/media
**/reports
Loading
Loading