Docs for the text release (#497)

* Update README.md

* Update README.md

* Update tutorial.md

* Update tutorial.md

* Update all-presets.md

* Update examples.md

* Update README.md

* Update README.md

* Update README.md

* Update install-evidently.md

* Update install-evidently.md

* Update install-evidently.md

* Update notebook-environments.md

* Update notebook-environments.md

* Update notebook-environments.md

* Update notebook-environments.md

* Update notebook-environments.md

* Update notebook-environments.md

* Update notebook-environments.md

* Update column-mapping.md

* Update data-requirements.md

* Update hello-world.md

* Update README.md

* Update README.md

* Update README.md

* Update data-drift-algorithm.md

* Update data-drift-algorithm.md

* Update options-for-statistical-tests.md

* Update data-drift.md

* Update data-drift.md

* Update data-drift.md

* Update target-drift.md

* Update data-drift.md

* Update target-drift.md

* Update class-performance.md

* Update class-performance.md

* Update data-quality.md

* Update no-target-performance.md

* Update reg-performance.md

* Update data-quality.md

* Add files via upload

* Update data-quality.md

* Add files via upload

* Update data-quality.md

* Delete metric_column_summary_text.png

* Create text-overview.md

* Update text-overview.md

* Add files via upload

* Delete metric_text_descriptors_distribution_ovv-min.png

* Update text-overview.md

* Update text-overview.md

* Update all-presets.md

* Update all-presets.md

* Update all-presets.md

* Update all-metrics.md

* Update all-metrics.md

* Update data-drift.md

* Update data-quality.md

* Update data-quality.md

* Update data-drift.md

* Update target-drift.md

* Update reg-performance.md

* Update class-performance.md

* Update no-target-performance.md

* Update text-overview.md

* Update SUMMARY.md

* Update options-for-statistical-tests.md

* Update text-overview.md

* Update all-metrics.md

* Update data-drift-algorithm.md

* Update text-overview.md

* Update text-overview.md

* Update README.md

README.md

@@ -17,7 +17,7 @@
 
 # :bar_chart: What is Evidently?
 
-Evidently is an open-source Python library for data scientists and ML engineers. It helps evaluate, test, and monitor the performance of ML models from validation to production.
+Evidently is an open-source Python library for data scientists and ML engineers. It helps evaluate, test, and monitor the performance of ML models from validation to production. It works with tabular and text data.
 
 Evidently has a modular approach with 3 interfaces on top of the shared `metrics` functionality. 
 
@@ -34,7 +34,7 @@ Tests are best for automated batch model checks. You can integrate them as a pip
 ## 2. Reports: interactive dashboards
 
 > **Note**
-> We added a new Report object starting from v0.1.57.dev0. Reports unite the functionality of Dashboards and JSON profiles with a new, cleaner API. You can still use the old [Dashboards API](https://docs.evidentlyai.com/features/dashboards/generate_dashboards) but it will soon be depreciated.
+> We added a new Report object starting from v0.1.57.dev0. Reports unite the functionality of Dashboards and JSON profiles with a new, cleaner API. The old Dashboards API is deprecated and will be removed.
 
 ![Report example](docs/images/evidently_reports_main-min.png)
 
@@ -108,7 +108,7 @@ To run the **Data Stability** test suite and display the reports in the notebook
 data_stability= TestSuite(tests=[
     DataStabilityTestPreset(),
 ])
-data_stability.run(current_data=iris_frame.iloc[:90], reference_data=iris_frame.iloc[90:], column_mapping=None)
+data_stability.run(current_data=iris_frame.iloc[:60], reference_data=iris_frame.iloc[60:], column_mapping=None)
 data_stability 
 ```
 
@@ -146,7 +146,7 @@ data_drift_report = Report(metrics=[
     DataDriftPreset(),
 ])
 
-data_drift_report.run(current_data=iris_frame.iloc[:90], reference_data=iris_frame.iloc[90:], column_mapping=None)
+data_drift_report.run(current_data=iris_frame.iloc[:60], reference_data=iris_frame.iloc[60:], column_mapping=None)
 data_drift_report
 
 ```

docs/book/README.md

@@ -2,7 +2,7 @@ Evidently is an open-source Python library for data scientists and ML engineers.
 
 It helps evaluate, test, and monitor the performance of ML models from validation to production.
 
-# Quick Start 
+# Quick Start
 
 Quickly check it out (1 min):
 {% content-ref url="get-started/hello-world.md" %}
@@ -29,9 +29,9 @@ You need to provide the data, choose what to evaluate, and the output format. Ev
 
 You can integrate Evidently into various ML stacks as a monitoring or evaluation component.
 
-Evidently currently works with tabular data. 
+Evidently currently works with tabular and text data. 
 
-# 1. [Tests](tests/README.md): batch model checks 
+# 1. Tests suites: batch model checks 
 
 Tests perform structured data and ML model quality checks. You typically compare two datasets: **reference** and **current**. You can set test parameters manually or let Evidently learn the expectations from the reference. Tests verify a condition and return an explicit **pass** or **fail** result. 
 
@@ -52,7 +52,7 @@ Tests are best for automated batch checks.
 * [User guide: how to generate tests](tests-and-reports/run-tests.md) 
 * [Reference: available tests and presets](reference/all-tests.md) 
 
-# 2. [Reports](reports/README.md): interactive dashboards
+# 2. Reports: interactive dashboards
 
 {% hint style="info" %}
 We added a new Report object starting from **v0.1.57.dev0**. Reports unite the functionality of Dashboards and JSON profiles with a new, cleaner API. 
@@ -81,7 +81,7 @@ Reports are best for exploratory analysis, debugging, and documentation.
 * [User guide: how to run reports](tests-and-reports/get-reports.md) 
 * [Reference: available metrics and metric presets](reference/all-metrics.md) 
 
-# 3. [Real-time ML monitoring](integrations/evidently-and-grafana.md)
+# 3. Monitors: real-time ML monitoring
 
 *Note: this functionality is in early development and subject to an API change*. 
 

docs/book/SUMMARY.md

@@ -13,6 +13,7 @@
   * [Regression Performance](presets/reg-performance.md)
   * [Classification Performance](presets/class-performance.md)
   * [NoTargetPerformance](presets/no-target-performance.md)
+  * [Text Overview](presets/text-overview.md)
 * [Examples](examples/examples.md)
 * [Integrations](integrations/README.md)
   * [Notebook environments](integrations/notebook-environments.md)

docs/book/customization/options-for-statistical-tests.md

@@ -134,3 +134,10 @@ TestShareOfDriftedColumns(lt=0.5)
   - only for categorical features
   - returns `p-value`
   - drift detected when `p_value < threshold`
+- `text_content_drift` - Text content drift (domain classifier)
+  - default for text features
+  - only for text features
+  - returns `roc_auc` as drift_score
+  - drift detected when roc_auc > roc_auc of the random classifier at a set percentile (`threshold`) 
+  - default threshold: 0.05
+  - `roc_auc` values can be 0 to 1 (typically 0.5 to 1); higher value mean more confident drift detection

docs/book/examples/examples.md

@@ -6,7 +6,7 @@ description: Sample notebooks and tutorials
 
 ## Sample notebooks
 
-Simple examples on toy datasets to quickly explore what Evidently can do right out of the box. 
+Simple examples on toy datasets to show what Evidently can do out of the box. Colab examples contain pre-rendered reports.
 
 Title| Jupyter notebook | Colab notebook | Contents
 --- | --- | --- | --- 
@@ -24,7 +24,6 @@ Title | Jupyter notebook | Colab notebook | Blog post | Data source
 Monitor production model decay | [link](../../../examples/data_stories/bicycle_demand_monitoring.ipynb) | [link](https://colab.research.google.com/drive/1xjAGInfh_LDenTxxTflazsKJp_YKmUiD) | [How to break a model in 20 days. A tutorial on production model analytics.](https://evidentlyai.com/blog/tutorial-1-model-analytics-in-production) | Bike sharing UCI: [link](https://archive.ics.uci.edu/ml/datasets/bike+sharing+dataset)
 Compare two models before deployment | [link](../../../examples/data_stories/ibm_hr_attrition_model_validation.ipynb) | [link](https://colab.research.google.com/drive/12AyNh3RLSEchNx5_V-aFJ1_EnLIKkDfr) | [What Is Your Model Hiding? A Tutorial on Evaluating ML Models.](https://evidentlyai.com/blog/tutorial-2-model-evaluation-hr-attrition) | HR Employee Attrition: [link](https://www.kaggle.com/pavansubhasht/ibm-hr-analytics-attrition-dataset)
 Evaluate and visualize historical drift | [link](../../../examples/integrations/mlflow_logging/historical_drift_visualization.ipynb) | [link](https://colab.research.google.com/drive/12AyNh3RLSEchNx5_V-aFJ1_EnLIKkDfr) | [How to detect, evaluate and visualize historical drifts in the data.](https://evidentlyai.com/blog/tutorial-3-historical-data-drift) | Bike sharing UCI: [link](https://archive.ics.uci.edu/ml/datasets/bike+sharing+dataset)
-Create a custom report (tab) with PSI widget for drift detection | [link](../../../examples/data_stories/california_housing_custom_PSI_widget_and_tab.ipynb) | [link](https://colab.research.google.com/drive/1FuXId8p-lCP9Ho_gHeqxAdoxHRuvY9d0) | --- | California housing sklearn.datasets 
 
 
 ## Integrations

docs/book/get-started/hello-world.md

@@ -26,11 +26,7 @@ $ jupyter nbextension enable evidently --py --sys-prefix
 Install **Evidently**:
 
 ```python
-try:
-    import evidently
-except:
-    !npm install -g yarn
-    !pip install git+https://github.com/evidentlyai/evidently.git
+!pip install evidently
 ``` 
 
 # Imports 

docs/book/get-started/tutorial.md

@@ -50,24 +50,24 @@ $ jupyter nbextension enable evidently --py --sys-prefix
 
 That's it!
 
-### Google Colab, Kaggle Kernel, Deepnote
+### Hosted notebooks
 
-To install `evidently`, run the following command in the notebook cell:
+If you are using Google Colab, Kaggle Kernel, Deepnote or Databricks notebooks, run the following command in the notebook cell:
 
 ```
 !pip install evidently
 ```
 
 ### Windows
 
-Unfortunately, building visual HTML reports inside a **Jupyter notebook** is **not yet possible** for Windows. You can still install Evidently and get the output as JSON or a separate HTML file.
-
-To install Evidently, run:
+To install Evidently in Jupyter notebook on Windows, run:
 
 ```bash
 $ pip install evidently
 ```
 
+**Note**: Nbextension does not work on Windows. If you want to generate visual reports in Jupyter notebook on Windows, you will need to use a different visualization method (see instructions in step 4). This is a new functionality with limited testing. If you face issues, you can get the output as a separate HTML file and view it in a browser.
+
 ## 2. Import Evidently
 
 After installing the tool, import `evidently` and the required components. In this tutorial, you will use several **test suites** and **reports**. Each corresponds to a specific type of analysis. 
@@ -148,7 +148,7 @@ report
 It will display the HTML report directly in the notebook. 
 
 {% hint style="info" %}
-**Visualizations might work differently in other notebook environments**. If you use Databricks, Kaggle and Deepnote notebooks, you should add an argument to display the report inline: report.show(mode='inline'). Consult [this section](../integrations/notebook-environments.md) for help.
+**Note**: If you are using other notebook environments, e.g., Databricks, Kaggle and Deepnote notebooks, or Jypyter notebook on Windows, you should add an argument to display the report inline: report.show(mode='inline'). Consult [this section](../integrations/notebook-environments.md) for help.
 {% endhint %}
 
 First, you can see the Data Drift summary.
@@ -173,7 +173,7 @@ The data drift report compares the distributions of each feature in the two data
 
 Evidently Reports are very configurable. You can define which Metrics to include and how to calculate them. 
 
-To create a custom Report, you need to list individual **Metrics**. Evidently has dozens of Metrics that help evaluate anything from descriptive feature statistics to model quality. You can calculate Metrics on the column level (e.g., mean value of a specific column) or dataset-level (e.g., share of drifted features in the dataset).  
+To create a custom Report, you need to list individual **Metrics**. Evidently has dozens of Metrics that evaluate anything from descriptive feature statistics to model quality. You can calculate Metrics on the column level (e.g., mean value of a specific column) or dataset-level (e.g., share of drifted features in the dataset).  
 
 In this example, you can list several Metrics that evaluate individual statistics for the defined column. 
 
@@ -191,7 +191,7 @@ You will see a combined report that includes multiple Metrics:
 
 ![Part of the custom report, ColumnSummaryMetric.](../.gitbook/assets/tutorial/get-started-column-summary_metric-min.png)
 
-If you want to generate multiple column-level Metrics, there is a helper function. For example, in order to to calculate the same quantile value for all the columns in the list, you can use the generator: 
+If you want to generate multiple column-level Metrics, there is a helper function. For example, in order to calculate the same quantile value for all the columns in the list, you can use the generator: 
 
 ```
 report = Report(metrics=[
@@ -221,9 +221,9 @@ report
 
 ## 6. Define the report output format
 
-You can render the visualizations directly in the notebook as shown above. There are also alternative options. 
+You can render the visualizations in the notebook as shown above. There are also alternative options. 
 
-If you only want to log the calculated metrics and test results, you can export the results as a Python dictionary.
+If you only want to log the metrics and test results, you can get the output as a Python dictionary.
 
 ```python
 report.as_dict()
@@ -234,21 +234,21 @@ You can also get the output as JSON.
 report.json()
 ```
 
-You can also save HTML or JSON externally. 
+You can also save HTML or JSON externally and specify a path and file name: 
 
 ```python
 report.save_html("file.html")
 ```
 
 ## 7. Run data stability tests
 
-Reports are useful when you want to visually explore the data or model quality or share results with the team. However, it is less convenient if you want to run your checks automatically and only react to meaningful issues.
+Reports help visually explore the data or model quality or share results with the team. However, it is less convenient if you want to run your checks automatically and only react to meaningful issues.
 
 To integrate Evidently checks in the prediction pipeline, you can use the **Test Suites** functionality. They are also better suited to handle large datasets.
 
 Test Suites help compare the two datasets in a structured way. A **Test Suite** contains several individual tests. Each **Test** compares a specific value against a defined condition and returns an explicit pass/fail result. You can apply Tests to the whole dataset or individual columns. 
 
-Just like with Reports, you can create a custom Test Suite or use one of the **Presets** that work out of the box. 
+Just like with Reports, you can create a custom Test Suite or use one of the **Presets**. 
 
 Let's create a custom one! Imagine you received a new batch of data. Before generating the predictions, you want to check if the quality is good enough to run your model. You can combine several Tests to check missing values, duplicate columns, and so on. 
 
@@ -324,19 +324,19 @@ To integrate Evidently checks in the prediction pipeline, you can get the output
 suite.as_dict()
 ```
 
-You can extract necessary information from the JSON or Python dictionary output and design a conditional workflow around it. For example, if some tests fail, you can trigger an alert, retrain the model or generate the report. 
+You can extract necessary information from the JSON or Python dictionary output and design a conditional workflow around it. For example, if tests fail, you can trigger an alert, retrain the model or generate the report. 
 
 ## 8. What else is there?
 
 * **Go through the steps in more detail**
 
-If you want to walk through all the described steps in more detail, refer to the **User Guide** section of the docs. A good next step is to explore how to pass custom test parameters to define your own [test conditions](../tests-and-reports/custom-test-suite.md).  
+To understand the described flow in more detail, refer to the **User Guide** section of the docs. A good next step is to explore how to pass custom test parameters to define your own [test conditions](../tests-and-reports/custom-test-suite.md).  
 
 * **Explore available presets**
 
 Both **Tests** and **Reports** have multiple Presets available. Some, like Data Quality, require only input data. You can use them even without the reference dataset. When you have the true labels, you can run Presets like **Regression Performance** and **Classification Performance** to evaluate the model quality and errors. 
 
-To understand the contents of each Preset, head to the [Preset overview](../presets). If you want to see the pre-rendered examples of the reports, browse Colab notebooks in the [Examples](../examples/examples.md) section. 
+To understand the contents of each Preset, head to the [Preset overview](../presets/all-presets.md). If you want to see the pre-rendered examples of the reports, browse Colab notebooks in the [Examples](../examples/examples.md) section. 
 
 * **Explore available integrations**
 

docs/book/how-to-guides/README.md

@@ -7,8 +7,11 @@ These example notebooks and how-to guides show how to solve specific tasks.
 Topic | Question| Guide or example 
 | --- | --- | --- 
 Input data | How to load data from different sources to pandas.Dataframes? | <ul><li>[Guide](../input-data/load-data-to-pandas.md)</li></ul> 
-Test and reports | How to generate multiple tests or metrics quickly? | <ul><li>[Jupyter notebook](https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_apply_tests_and_metrics_generators.ipynb)</li></ul> 
+Test and reports | How to generate multiple Tests or Metrics quickly? | <ul><li>[Jupyter notebook](https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_apply_tests_and_metrics_generators.ipynb)</li></ul> 
+Test and reports | How to run evaluations on raw text data? | <ul><li>[Jupyter notebook](https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_run_calculations_over_text_data.ipynb)</li></ul> 
 Customization | How to assign a particular method for Data Drift detection?|<ul><li>[Guide](../customization/options-for-statistical-tests.md)</li><li> [Jupyter notebook](https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_specify_stattest_for_a_testsuite.ipynb)</li></ul> 
 Customization | How to define a custom list of Missing Values?|<ul><li>[Jupyter notebook](https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_run_tests_with_different_missing_values.ipynb)</li></ul> 
-Customization | How to specify a color scheme in Reports and Test Suites? (Needs an update for new API). |<ul><li>[Guide](../customization/options-for-color-schema.md)</li></ul>  
-Customization | How to add a custom metric or test? | <ul><li>[Colab](https://colab.research.google.com/drive/1kZ6RlbKfDiEoRkULnfPh_MbUPscB_iP_?usp=sharing)</li></ul> 
+Customization | How to specify a color scheme in Reports and Test Suites? (Needs an update). |<ul><li>[Guide](../customization/options-for-color-schema.md)</li></ul>  
+Customization | How to add a custom Metric or Test? | <ul><li>[Colab](https://colab.research.google.com/drive/1kZ6RlbKfDiEoRkULnfPh_MbUPscB_iP_?usp=sharing)</li></ul> 
+Outputs | How to get Report or Test Suite output in csv? | <ul><li>[Jupyter notebook](https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_get_report_results_in_csv.ipynb )</li></ul> 
+

docs/book/input-data/column-mapping.md

@@ -69,6 +69,17 @@ column_mapping.categorical_features = ['season', 'holiday'] #list of categorical
 **Why map them:** the column types affect some of the tests, metrics and visualizations. For example, the [drift algorithm](../reference/data-drift-algorithm.md) selects a statistical test based on the column type and ignores DateTime features. Some of the data quality visualizations are different for specific feature types. Some of the tests (e.g. on value ranges) only considers numeral columns, etc.
 {% endhint %}
 
+## Text data 
+
+To specify that columns contain raw text data: 
+
+```python
+column_mapping.text_features = ['email_subject', 'email_body']
+```
+
+**Why map them:** if you want to apply text-specific drift detection methods or call other metrics relevant to text data, you should specify them explicitly. Text columns are also excluded from certain tests and metrics similar to ID column.
+{% endhint %}
+
 # Additional mapping options
 
 There are additional mapping options that apply to specific test suites and reports.

docs/book/input-data/data-requirements.md

@@ -37,7 +37,7 @@ To use Evidently, you need a dataset that contains model prediction logs. It mig
 
 * Input feature columns
 * Prediction column
-* Target column
+* Target column (if known)
 * Additional columns such as DateTime and ID
 
 The exact schema requirements differ based on the contents of the report or test suite. For example, to evaluate Data Drift or Data Quality, you can pass only the feature columns. To evaluate Model Performance, you also need model prediction and target (true labels or actuals).
@@ -58,4 +58,4 @@ If the dataset is too large, you might need to downsample it before passing the
 
 ## Supported data types
 
-Right now, Evidently works only with tabular data. We are working to cover other data types.
+Right now, Evidently works with tabular and raw text data. You can also pass a dataset that contains different data types: for example, some columns may contain numerical or categorical data, while others contain text.