Opacus is a library that enables training PyTorch models with differential privacy. It supports training with minimal code changes required on the client, has little impact on training performance and allows the client to online track the privacy budget expended at any given moment.
This code release is aimed at two target audiences:
- ML practitioners will find this to be a gentle introduction to training a model with differential privacy as it requires minimal code changes.
- Differential Privacy scientists will find this easy to experiment and tinker with, allowing them to focus on what matters.
The latest release of Opacus can be installed via pip:
pip install opacus
⚠️ NOTE: This will bring in the latest version of our deps, which are on Cuda 10.2. This will not work if you environment is using an older Cuda version (for example, Google Colab is still on Cuda 10.1).
To install on Colab, run this cell first:
pip install torchcsprng==0.1.3+cu101 -f https://download.pytorch.org/whl/torch_stable.htmlThen you can just pip install opacus like before. See more context in this issue.
You can also install directly from the source for the latest features (along with its quirks and potentially ocassional bugs):
git clone https://github.com/pytorch/opacus.git
cd opacus
pip install -e .To train your model with differential privacy, all you need to do is to declare a PrivacyEngine and attach it to your optimizer before running, eg:
model = Net()
optimizer = SGD(model.parameters(), lr=0.05)
privacy_engine = PrivacyEngine(
model,
sample_rate=0.01,
alphas=[10, 100],
noise_multiplier=1.3,
max_grad_norm=1.0,
)
privacy_engine.attach(optimizer)
# Now it's business as usualThe MNIST example shows an end-to-end run using opacus. The examples folder contains more such examples.
Checkout the FAQ page for answers to some of the most frequently asked questions about Differential Privacy and Opacus.
See the CONTRIBUTING file for how to help out.
Do also check out our README files inside the repo to learn how the code is organized.
If you want to cite Opacus in your papers (much appreciated!), you can cite it as follows:
@article{opacus,
title={Opacus: User-Friendly Differential Privacy Library in PyTorch},
author={A. Yousefpour and I. Shilov and A. Sablayrolles and D. Testuggine and K. Prasad and M. Malek and J. Nguyen and S. Ghosh and A. Bharadwaj and J. Zhao and G. Cormode and I. Mironov},
journal={arXiv preprint arXiv:2109.12298},
year={2021}
}
- Mironov, Ilya. "Rényi differential privacy." 2017 IEEE 30th Computer Security Foundations Symposium (CSF). IEEE, 2017.
- Abadi, Martin, et al. "Deep learning with differential privacy." Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2016.
- Mironov, Ilya, Kunal Talwar, and Li Zhang. "R'enyi Differential Privacy of the Sampled Gaussian Mechanism." arXiv preprint arXiv:1908.10530 (2019).
- Goodfellow, Ian. "Efficient per-example gradient computations." arXiv preprint arXiv:1510.01799 (2015).
- McMahan, H. Brendan, and Galen Andrew. "A general approach to adding differential privacy to iterative training procedures." arXiv preprint arXiv:1812.06210 (2018).
This code is released under Apache 2.0, as found in the LICENSE file.