Skip to content

Commit

Permalink
Update the Readme for responsible-ai folder (#2775)
Browse files Browse the repository at this point in the history
  • Loading branch information
@rkamat25
rkamat25 authored Oct 31, 2023
1 parent 643c083 commit e584d49
Showing 1 changed file with 35 additions and 11 deletions.
46 changes: 35 additions & 11 deletions sdk/python/responsible-ai/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,49 @@ The Responsible AI components are supported for MLflow models with `scikit-learn
The components accept both models and SciKit-Learn pipelines as input as long as the model or pipeline implements `predict` and `predict_proba` functions that conforms to the `scikit-learn` convention.
If not compatible, you can wrap your model's prediction function into a wrapper class that transforms the output into the format that is supported (`predict` and `predict_proba` of `scikit-learn`), and pass that wrapper class to modules in this repo.

## Sample directory 📖
## Directory 📖




| Scenario | Dataset | Data type | RAI component included | Link to sample | Documentation |
| --- | --- | --- | --- | --- | --- |
| Regression | [sklearn Diabetes](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html) | Tabular | Explanation, Error Analysis, Counterfactuals | [responsibleaidashboard-diabetes-regression-model-debugging.ipynb](./tabular/responsibleaidashboard-diabetes-regression-model-debugging/responsibleaidashboard-diabetes-regression-model-debugging.ipynb) | [Tabular Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard?view=azureml-api-2) |
| Regression | [Programmers MLTable data](./tabular/responsibleaidashboard-programmer-regression-model-debugging/data-programmer-regression) | Tabular | Explanation, Error Analysis, Causal analysis, Counterfactuals | [responsibleaidashboard-programmer-regression-model-debugging.ipynb](./tabular/responsibleaidashboard-programmer-regression-model-debugging/responsibleaidashboard-programmer-regression-model-debugging.ipynb) | [Tabular Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard?view=azureml-api-2) |
| Binary Classification | [Finance Story](./tabular/responsibleaidashboard-finance-loan-classification/Fabricated_Loan_data.csv) | Tabular | Explanation, Error Analysis, Causal analysis, Counterfactuals | [Finance_Dashboard.ipynb](./tabular/responsibleaidashboard-finance-loan-classification/responsibleaidashboard-finance-loan-classification.ipynb) | [Documentation](./tabular/responsibleaidashboard-finance-loan-classification/readme.md) |
| Binary Classification | [Healthcare Story](./tabular/responsibleaidashboard-healthcare-covid-classification/data_covid_classification/) | Tabular | Explanation, Error Analysis, Causal analysis, Counterfactuals | [Covid_Healthcare_Dashboard.ipynb](./tabular/responsibleaidashboard-healthcare-covid-classification/responsibleaidashboard-healthcare-covid-classification.ipynb) | [Documentation](./tabular/responsibleaidashboard-healthcare-covid-classification/readme.md) |
| Binary Classification | [Education Story](./tabular/responsibleaidashboard-education-student-attrition-classificaton/Fabricated_Student_Attrition_Data.csv) | Tabular | Explanation, Error Analysis, Causal analysis, Counterfactuals | [Education_Dashboard.ipynb](./tabular/responsibleaidashboard-education-student-attrition-classificaton/responsibleaidashboard-education-student-attrition-classificaton.ipynb) | [Documentation](./tabular/responsibleaidashboard-education-student-attrition-classificaton/readme.md) |
| Classification | [Kaggle Housing](https://www.kaggle.com/alphaepsilon/housing-prices-dataset) | Tabular | Explanation, Error Analysis, Causal analysis, Counterfactuals | [responsibleaidashboard-housing-classification-model-debugging.ipynb](./tabular/responsibleaidashboard-housing-classification-model-debugging/responsibleaidashboard-housing-classification-model-debugging.ipynb) | [Tabular Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard?view=azureml-api-2) |
| Decision Making | [Kaggle Housing](https://www.kaggle.com/alphaepsilon/housing-prices-dataset) | Tabular | Causal analysis, Counterfactuals | [responsibleaidashboard-housing-decision-making.ipynb](./tabular/responsibleaidashboard-housing-decision-making/responsibleaidashboard-housing-decision-making.ipynb) | [Tabular Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard?view=azureml-api-2) |
| Decision Making | [sklearn Diabetes](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html) | Tabular | Causal analysis, Counterfactuals | [responsibleaidashboard-diabetes-decision-making.ipynb](./tabular/responsibleaidashboard-diabetes-decision-making/responsibleaidashboard-diabetes-decision-making.ipynb) | [Tabular Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard?view=azureml-api-2) |
| Text Classification | [DBPedia dataset](https://huggingface.co/datasets/DeveloperOats/DBPedia_Classes) | Text | Explanation, Error Analysis | [responsibleaidashboard-text-classification-DBPedia.ipynb](./text/responsibleaidashboard-text-classification-DBPedia.ipynb) | [Text Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-text-dashboard?view=azureml-api-2) |
| Multi-label Text Classification | [Covid19 Emergency Event Dataset](https://huggingface.co/datasets/joelito/covid19_emergency_event) | Text | Explanation, Error Analysis | [responsibleaidashboard-multilabel-text-classification-covid-events.ipynb](./text/responsibleaidashboard-multilabel-text-classification-covid-events.ipynb) | [Text Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-text-dashboard?view=azureml-api-2) |
| Binary Text Classification | [blbooksgenre dataset](https://huggingface.co/datasets/blbooksgenre) | Text | Explanation, Error Analysis | [responsibleaidashboard-text-classification-blbooksgenre.ipynb](./responsibleaidashboard-text-classification-blbooksgenre.ipynb) | [Text Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-text-dashboard?view=azureml-api-2) |
| Text Classification | [News Article Classification](./text/responsibleaidashboard-text-classification-financial-news/Text_classification_dataset.xlsx) | Text | Explanation, Error Analysis | [Financial_News_Text_classifier.ipynb](./text/responsibleaidashboard-text-classification-financial-news/responsibleaidashboard-text-classification-financial-news.ipynb) | [Documentation](./text/responsibleaidashboard-text-classification-financial-news/readme.md) |
| Text Question Answering | [Stanford Question Answering Dataset (SQuAD)](https://huggingface.co/datasets/squad) | Text | Explanation, Error Analysis | [responsibleaidashboard-text-question-answering-squad.ipynb](./text/responsibleaidashboard-text-question-answering-squad.ipynb) | [Text Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-text-dashboard?view=azureml-api-2) |
| AutoML Image Classification | [Fridge Images](https://github.com/microsoft/computervision-recipes/tree/master/scenarios/classification) | Image | Explanation, Error Analysis | [responsibleaidashboard-automl-image-classification-fridge.ipynb](./vision/responsibleaidashboard-automl-image-classification-fridge.ipynb) | [Vision Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-image-dashboard?view=azureml-api-2) |
| Object Detection | [MIT Computer Vision datasets](https://github.com/microsoft/computervision-recipes) | Image | Explanation, Error Analysis | [responsibleaidashboard-automl-object-detection-fridge-private-data.ipynb](./vision/responsibleaidashboard-automl-object-detection-fridge-private-data.ipynb) | [Vision Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-image-dashboard?view=azureml-api-2) |
| Image Classification | [Fridge Images](https://github.com/microsoft/computervision-recipes/tree/master/scenarios/classification) | Image | Explanation, Error Analysis | [responsibleaidashboard-image-classification-fridge.ipynb](./vision/responsibleaidashboard-image-classification-fridge.ipynb) | [Vision Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-image-dashboard?view=azureml-api-2) |
| Multilabel Image Classification | [Fridge Images](https://github.com/microsoft/computervision-recipes/tree/master/scenarios/classification) | Image | Explanation, Error Analysis | [responsibleaidashboard-image-multilabel-classification-fridge.ipynb](./vision/responsibleaidashboard-image-multilabel-classification-fridge.ipynb) | [Vision Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-image-dashboard?view=azureml-api-2) |
| Image Object Detection | [Object Detection Fridge Images](https://github.com/microsoft/computervision-recipes/tree/master/scenarios/detection) | Image | Explanation, Error Analysis | [responsibleaidashboard-object-detection-MSCOCO.ipynb](./vision/responsibleaidashboard-object-detection-MSCOCO.ipynb) | [Vision Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-image-dashboard?view=azureml-api-2) |


To learn more about the different types of Dashboard visit the below tutorials:
1) [Tabular Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard?view=azureml-api-2)
2) [Text Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-text-dashboard?view=azureml-api-2)
3) [Vision Dashboard Generation](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-image-dashboard?view=azureml-api-2)

| Scenario | Dataset | Data type | RAI component included | Link to sample |
| --- | --- | --- | --- | --- |
| Regression | [sklearn Diabetes](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html) | Tabular | Explanation, Error Analysis, Counterfactuals | [responsibleaidashboard-diabetes-regression-model-debugging.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/responsible-ai/responsibleaidashboard-diabetes-regression-model-debugging/responsibleaidashboard-diabetes-regression-model-debugging.ipynb) |
| Regression | [Programmers MLTable data](https://github.com/Azure/azureml-examples/tree/main/sdk/python/responsible-ai/responsibleaidashboard-programmer-regression-model-debugging/data-programmer-regression) | Tabular | Explanation, Error Analysis, Causal analysis, Counterfactuals | [responsibleaidashboard-programmer-regression-model-debugging.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/responsible-ai/responsibleaidashboard-programmer-regression-model-debugging/responsibleaidashboard-programmer-regression-model-debugging.ipynb) |
| Classification | [Kaggle Housing](https://www.kaggle.com/alphaepsilon/housing-prices-dataset) | Tabular | Explanation, Error Analysis, Causal analysis, Counterfactuals | [responsibleaidashboard-housing-classification-model-debugging.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/responsible-ai/responsibleaidashboard-housing-classification-model-debugging/responsibleaidashboard-housing-classification-model-debugging.ipynb) |
| Decision making | [Kaggle Housing](https://www.kaggle.com/alphaepsilon/housing-prices-dataset) | Tabular | Causal analysis, Counterfactuals | [responsibleaidashboard-housing-decision-making.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/responsible-ai/responsibleaidashboard-housing-decision-making/responsibleaidashboard-housing-decision-making.ipynb) |
| Decision making | [sklearn Diabetes](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html) | Tabular | Causal analysis, Counterfactuals | [responsibleaidashboard-diabetes-decision-making.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/responsible-ai/responsibleaidashboard-diabetes-decision-making/responsibleaidashboard-diabetes-decision-making.ipynb) |

## Supportability 🧰
Currently, we support datasets having numerical and categorical features. The following table provides the scenarios supported for each of the four responsible AI components:
> **Note**: Model overview (performance metrics and fairness disparity metrics) and Data explorer are generated for every Responsible AI dashboard by default and do not require a component to be configured.
| RAI component | Binary classification | Multi-class classification | Multilabel classification | Regression | Timeseries forecasting | Categorical features | Text features | Image Features | Recommender Systems | Reinforcement Learning |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -- |
| Explainability | Yes | Yes | No | Yes | No | Yes | No | No | No | No |
| Error Analysis | Yes | Yes | No | Yes | No | Yes | No | No | No | No |
| Explainability | Yes | Yes | No | Yes | No | Yes | Yes | Yes | No | No |
| Error Analysis | Yes | Yes | No | Yes | No | Yes | Yes | Yes | No | No |
| Causal Analysis | Yes | No | No | Yes | No | Yes (max 5 features due to computational cost) | No | No | No | No |
| Counterfactual | Yes | Yes | No | Yes | No | Yes | No | No | No | No |

Read more about how to use the Responsible AI dashboard [here](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard).
Read more about how to use the Responsible AI dashboards [here](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-responsible-ai-dashboard).

0 comments on commit e584d49

Please sign in to comment.