TEAM 1 WORK IN PROGRESS
- "Daily" Scrum Meetings every Tuesday at 9:00 AM here and every Thursday at 12:00 PM here
- Sprint Review & Planning every even calendar week on Wednesday at 12:00 PM here
- adhoc meetings are often planned via our gitter channel
Collection of all relevant Links
- marisas pr
- restructuring of mia for pointbase attack
- all tests and the notebook had to be changed, so it looks like a lot
- attack() does not fit the attack modifier every time
- now an error is thrown when attack is not fitted
- franziska merged everything
- team1sprint3 now up to date
- everyone is on different python version 3.7 - 3.9. that's ok.
- marisa found a differential training privacy thingy (literature [8])
- art implementations
- papers inked in literature
- blackbox attack has no shadow models, question github (see literature)
- difficulties understanding implementations of the privacy risk score
TODOS:
- Mail an franziska -> @blauertee
- ART Privacy Score?
- Tf privacy risk score?
- no shadow models in art?
- think about we wanna be rated -> @everyone
- Marisa and Milos fixed their stuff in the PR
- setup.py import error
- still worked for marisa
- try with venv
- marisa will link some docs
- is there a priv risk score in art?
- privacy risk score
- only for blackbox attack?
- quantifies how good an attack model is?
- we have to read the paper again
- k datapoints
- visualization of the priv risk score
- how good is our attack model?
- claas will work on more ideas for visualisation
- infer returns only 1 or 0 for every datapoint
- readme abstracts
- is a lot of work
- since all of the attacks have to be fully understood
- we should start working on it now and then improve iteratively
- is a lot of work
TODOS:
- everybody sets up venv
- read privacy risk score
- link docs for venv in wiki -> @marisanest, see Collection of all relevant Links
- more ideas for visualization -> @erdnaf
-
PR of the last sprint
- Should we merge directly to the pr branch or make a PR?
- Commit directly to the branch
- Marisa already started fixing Franziska's requested changes
- Little changes requested from jury and Milos
- Everybody fixes their own work? YES
- Marisa and Claas will discuss the specific changes of Marisa's work
-
Installing via setup.py did that work for anyone? (import error)
- marisa did the imports as in ART
- importing modules works normally like that
- create an issue
-
Actions And branches?
- call the branch only team1?
- nobody can push on overlapping branch names
- so no.
- create different branches for sprints
- running tests for every branch?
- might be annoying for team2
- can't they then just ignore the tests?
- tests take a really long time (since attack models are trained)
- Milos will look into that
- call the branch only team1?
-
Assigning Issues/Work
- Little Tasks:
- Create team1sprint3 branch -> @marisanest
- Create Wiki Page for work documentation and literature
- fixing setup.py -> issue
- create separate Webex room -> @blauertee
- GitHub actions and branches -> @budmil
- Issues:
- See assignments in the issues
- privacy score calculation should happen in the metrics class
- visualisation as in here should be possible
- Two many issues about MIA on Single data points?
- Nah ... just close all of them with one PR
- writing all the texts for the README
- renaming
model_card_info- not that important
- will also require adjustments in the notebooks
- metrics output as JSON
- goes to the metrics class
- only calculate that once, because these values do not change
- Little Tasks:
- Presentation for tomorrow
- TODOs:
- prepare presentation -> everybody
- improve docs of privacy risk score -> @blauertee
- make privacy risk score tests work -> @blauertee
- open PRs to team1sprint3 -> everbody who made changes
- open PR to main -> @marisanest
- improve READMEs for MIAs -> @Friedrich-Müller
- We're bit tail heavy and did everything in the end, it got stressful, we should change that
- Things have to work on our systems
- Not everybody knows what the other's are/where doing
- Maybe improve github working structure?
- Franziska will share a model that overfits to make testing MIAs easier
- Zalando want's to test our stuff after this sprint
- Franziskas Blogpost about tf privacy is really helpfull
- Everybody should have the same environment
- We should write down how to install
- Use Anaconda
- Anaconda has a huge overhead maybe miniconda?
- we created an issue for it
- making sprints less tail heavy
- marisa wasn't happy with waiting for merges
- faster reviews
- what do reviews even mean?
- from now on we rotate reviews and do the thoroughly
- that means understanding the code & running the tests
- deadline 2 days before increment presentation
- PR deadline Sunday
- bug fixing Monday
- presentation Tuesday
- work on feature branches directly on privML/privacy-evaluator
- changing to relative imports
- will do
- think about the other team
- make use of team2needed label
- make biweekly meetings with team2
TODOs:
- look over relative import PR -> @marisanest
- create new sprint4 branch -> @blauertee
- ask team2 for regular meetings every 2 weeks -> @blauertee
Trying to write up and easy to understand definition which is w&p as u can c :)
Definition:
In our MI black box attack, we supply train and test data to train shadow models. Which are not the same as the target classifiers train and test data to simulate an actual attackers behaviour. For the purpose of measuring how successful the attack was (or how vulnerable the target classifier is) these results should be compared at some point to the actual train and test data of the target classifier. But for our MIAs only one set of train and test data is ever supplied. This feels contradictory to me. What am I missing?
Notes with respect to this question (by @marisanest):
- The ART attack
MembershipInferenceBlackBoxdoes not use shadow models. It's a different approach, which is sadly not enough explained in the code and I could not find a referenced paper either. So no idea where this implementation is coming from, but to use this attack you need to know which data was used for the target model training and which was not. So you train the attack model with data that is split into training data and test data/non-training data of the target model. You use a part of this data for training and the other part to later test the performance of the attack model.- @blauertee: I just asked them in an issue
- The other two ART attacks also follow a different approach. For both a paper is referenced (see Collection of all relevant Links).
Mail to Franziska:
Hey Franziska,
we've been trying to implement the privacy risk score as described in the paper. And encountered some problems.
The implementation of the privacy risk score depends on the results of MIA based on shadow models. We thought the the MIA implemented by art utilizes shadow models to perform it's attack. But since a few things where confusing (e.g. that the documentation says the attack has to be supplied with the train and test data of the actual target model) we took a closer look and are now sure that only one attack model is trained and no shadow models (see line 221). We're not sure on which literature the attack implemented by ART is based, since none is linked, so we asked them in an issue. Because of the lack of shadow models in ART we don't think the privacy risk score is really applicable in our situation.
ART implements a different metric for membership privacy (also on point basis) called Differential Training Privacy. Would this metric also be sufficient for our purposes? Since Art has an implementation of it, it's compatible with the attacks we used.
If we have to use the privacy risk score, we also have to implement another MIA which actually utilizes shadow models. I don't think that we'd manage to do this in this sprint.
best wishes & thanks in advance,
Jakob
Franziskas answer:
Hi Jakob,
thank you for your email.
Actually, it's all right. Recent work has shown [1] that one shadow model is sufficient to perform membership inference. Therefore, the original target model can be used as the shadow model (since it perfectly mimics the target model, because it IS the target model). Therefore, the privacy risk score is applicable. Also in TensorFlow Privacy, they only have one shadow model, which is the target model itself and they are able to calculate the score. Therefore, you should be too and if you are unsure how to do it, check how they implement it in TensorFlow.
Good idea to open an issue and ask in ART.
Differential Privacy is no metric, even though it has the parameter epsilon. But eplison needs to be calculated during training and cannot be determined after training. It is hence not suitable for your task.
Therefore, please find a solution on how to integrate the attack score into your implementation, it should work exactly as in TensorFlow. You won't need to implement another MIA.
And one more thing: could you in the future please contact me over my AISEC address where I have more regular access to: [email protected]
Thank you and have a nice weekend Franziska
Please add newly imported packages to the requirements.txt file and to the setup.py file (into the install_requires list). I am not 100% sure if writing down all packages twice is the best solution, but I found some references to why this should be done that way: stackoverflow and packaging.python.. And I will also ask Adrin.