-
Notifications
You must be signed in to change notification settings - Fork 17
TEAM 1 WORK IN PROGRESS
Jakob Heinrich edited this page Jun 29, 2021
·
53 revisions
- "Daily" Scrum Meetings every Tuesday at 9:00 AM here and every Thursday at 12:00 PM here
- Sprint Review & Planning every even calendar week on Wednesday at 12:00 PM here
- adhoc meetings are often planned via our gitter channel
Collection of all relevant Links
- marisas pr
- restructuring of mia for pointbase attack
- all tests and the notebook had to be changed, so it looks like a lot
- attack() does not fit the attack modifier every time
- now an error is thrown when attack is not fitted
- franziska merged everything
- team1sprint3 now up to date
- everyone is on different python version 3.7 - 3.9. that's ok.
- marisa found a differential training privacy thingy (literature [8])
- art implementations
- papers inked in literature
- blackbox attack has no shadow models, question github (see literature)
- difficulties understanding implementations of the privacy risk score
TODOS:
- Mail an franziska -> @blauertee
- ART Privacy Score?
- Tf privacy risk score?
- no shadow models in art?
- think about we wanna be rated -> @everyone
- Marisa and Milos fixed their stuff in the PR
- setup.py import error
- still worked for marisa
- try with venv
- marisa will link some docs
- is there a priv risk score in art?
- privacy risk score
- only for blackbox attack?
- quantifies how good an attack model is?
- we have to read the paper again
- k datapoints
- visualization of the priv risk score
- how good is our attack model?
- claas will work on more ideas for visualisation
- infer returns only 1 or 0 for every datapoint
- readme abstracts
- is a lot of work
- since all of the attacks have to be fully understood
- we should start working on it now and then improve iteratively
- is a lot of work
TODOS:
- everybody sets up venv
- read privacy risk score
- link docs for venv in wiki -> @marisanest, see Collection of all relevant Links
- more ideas for visualization -> @erdnaf
-
PR of the last sprint
- Should we merge directly to the pr branch or make a PR?
- Commit directly to the branch
- Marisa already started fixing Franziska's requested changes
- Little changes requested from jury and Milos
- Everybody fixes their own work? YES
- Marisa and Claas will discuss the specific changes of Marisa's work
-
Installing via setup.py did that work for anyone? (import error)
- marisa did the imports as in ART
- importing modules works normally like that
- create an issue
-
Actions And branches?
- call the branch only team1?
- nobody can push on overlapping branch names
- so no.
- create different branches for sprints
- running tests for every branch?
- might be annoying for team2
- can't they then just ignore the tests?
- tests take a really long time (since attack models are trained)
- Milos will look into that
- call the branch only team1?
-
Assigning Issues/Work
- Little Tasks:
- Create team1sprint3 branch -> @marisanest
- Create Wiki Page for work documentation and literature
- fixing setup.py -> issue
- create separate Webex room -> @blauertee
- GitHub actions and branches -> @budmil
- Issues:
- See assignments in the issues
- privacy score calculation should happen in the metrics class
- visualisation as in here should be possible
- Two many issues about MIA on Single data points?
- Nah ... just close all of them with one PR
- writing all the texts for the README
- renaming
model_card_info- not that important
- will also require adjustments in the notebooks
- metrics output as JSON
- goes to the metrics class
- only calculate that once, because these values do not change
- Little Tasks:
- Presentation for tomorrow
- TODOs:
- prepare presentation -> everybody
- improve docs of privacy risk score -> @blauertee
- make privacy risk score tests work -> @blauertee
- open PRs to team1sprint3 -> everbody who made changes
- open PR to main -> @marisanest
- improve READMEs for MIAs -> @Friedrich-Müller
- We're bit tail heavy and did everything in the end, it got stressful, we should change that
- Things have to work on our systems
- Not everybody knows what the other's are/where doing
- Maybe improve github working structure?
- Franziska will share a model that overfits to make testing MIAs easier
- Zalando want's to test our stuff after this sprint
- Franziskas Blogpost about tf privacy is really helpfull
- Everybody should have the same environment
- We should write down how to install
- Use Anaconda
- Anaconda has a huge overhead maybe miniconda?
- we created an issue for it
- making sprints less tail heavy
- marisa wasn't happy with waiting for merges
- faster reviews
- what do reviews even mean?
- from now on we rotate reviews and do the thoroughly
- that means understanding the code & running the tests
- deadline 2 days before increment presentation
- PR deadline Sunday
- bug fixing Monday
- presentation Tuesday
- work on feature branches directly on privML/privacy-evaluator
- changing to relative imports
- will do
- think about the other team
- make use of team2needed label
- make biweekly meetings with team2
TODOs:
- look over relative import PR -> @marisanest
- create new sprint4 branch -> @blauertee
- ask team2 for regular meetings every 2 weeks -> @blauertee
- maybe merge User Output and the attack analyses class in a later sprint
- user output has been improved
- user output is now exportable as json
- redundant arguments
- @jrtorhoff has worked a lot on slicing
- Meeting with team 2
- only when needed
- put all parameters except target_model into MIA instead of attack interface
- For Rule based attack we don't need these parameters (only needed in the fit method) could be an argument there
- some PRs merged
- fixture
autouse = truenecessarily? - MIA BB from art shuffles output
- why??
- you don't know which data points have been successfully inferred.
- open an issue
- different outputs for different attacks in different notebooks, franziska might not be happy
- maybe out put should be in grammatically correct sentences (make it more human readable)
- Decision Boundary attack takes ages
- maybe try with franziskas models
- what do some parameters do
- empirically speaking: they have an influence on how long it takes the function to terminate
- a paper is linked
- READMEs
- get more technical
- maybe the docu is too much orientated on the paper then on our actual implementation
- we should be clear about where our implementation diverges from the papers
- rename MIAs to more expressive things
- test of team2 taking too long
- you have to load the whole dataset in our CIFAR10 pytorch implementation
- rethink tests and do them in a better way
- how sophisticated should they be -> ask franziska
- do global CPU vs. GPU option
- Presentation for sprint review
- assigning issues
- get in progress stuff ready till Saturday evening
- put that into sprint4 branch
TODO:
- ask team two about logging/verbose -> @blauertee
Please add newly imported packages to the requirements.txt file and to the setup.py file (into the install_requires list). I am not 100% sure if writing down all packages twice is the best solution, but I found some references to why this should be done that way: stackoverflow and packaging.python.. And I also asked Adrin about it: #114.
If you have files, that are not included by default when installing the `privacy-evaluator (e.g. model files, test directories, docs, etc.) you can add them into the MANIFEST.in file.