README.md

Sample Analysis Notebooks

Python Notebooks

The following notebooks use the SmartNoise Core Rust library and validator through Python bindings.

Please note: SmartNoise Core is being rewritten to build on top of OpenDP. For this reason, you should focus especially on notebooks that already make use of the OpenDP library. The "Basic PUMS Analysis with SmartNoise" notebook below has been updated to be "Basic PUMS Analysis with OpenDP" and other notebooks will follow. The new OpenDP-based notebooks can be found at https://docs.opendp.org/en/stable/examples.html

Library Use Overviews

• Basic PUMS Analysis with SmartNoise: Brief tutorial on doing data analysis within the SmartNoise system, with particular examples on how to understand returns from the analysis validator.
  • Notebook: basic_data_analysis.ipynb

Introductions to Statistical Releases

• Histograms: Demonstrate using the geometric and Laplace mechanisms to generate histograms on continuous, categorical and Boolean variables.
  • Notebook: histograms.ipynb
• Differentially Private Covariance: Demonstrate different approaches to releasing a covariance matrix.
  • Notebook: covariance.ipynb

Introductions to Core Concepts

• Demonstrate the Utility of a Differentially Private Release by Simulation: Demonstrate through simulation how the noise of the final answer from a differentially private query on a dataset mean varies by the size of the dataset and the choice of privacy-loss (epsilon) parameter.

  • Notebook: utility_laplace_mean.ipynb

• Utility of Statistical Releases using the Mental Health in Tech Survey The purpose of this demo is to showcase the utility of the OpenDP SmartNoise library using the Mental Health in Tech Survey. The notebook will focus on statistical releases in the trusted curator setting.

  • Notebook: tutorial_mental_health_in_tech_survey.ipynb

• Working with Unknown Dataset Sizes: Explain how SmartNoise preserves privacy of dataset size by resizing, demonstrate how to estimate dataset size when unknown, and compare utility of SmartNoise's resizing approach to a standard 'plug-in' approach.

  • Notebook: unknown_dataset_size

• Accuracy: Pitfalls and Edge Cases: This notebook describes SmartNoise's accuracy calculations, and ways in which an analyst might be tripped up by them.

  • Notebook: accuracy_pitfalls.ipynb