Skip to content

Commit

Permalink
Update version to 0.17.0
Browse files Browse the repository at this point in the history
  • Loading branch information
deliahu committed May 22, 2020
1 parent 03144e6 commit c6c6aea
Show file tree
Hide file tree
Showing 25 changed files with 68 additions and 76 deletions.
15 changes: 4 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,7 @@
<!-- Delete on release branches -->
<img src='https://s3-us-west-2.amazonaws.com/cortex-public/logo.png' height='88'>

# Machine learning model serving infrastructure

<br>

<!-- Delete on release branches -->
<!-- CORTEX_VERSION_README_MINOR -->
[install](https://cortex.dev/install)[docs](https://cortex.dev)[examples](https://github.com/cortexlabs/cortex/tree/0.16/examples)[we're hiring](https://angel.co/cortex-labs-inc/jobs)[chat with us](https://gitter.im/cortexlabs/cortex)<br><br>

<!-- Set header Cache-Control=no-cache on the S3 object metadata (see https://help.github.com/en/articles/about-anonymized-image-urls) -->
![Demo](https://d1zqebknpdh033.cloudfront.net/demo/gif/v0.13_2.gif)

Expand All @@ -32,7 +25,7 @@

<!-- CORTEX_VERSION_README_MINOR -->
```bash
$ bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.16/get-cli.sh)"
$ bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.17/get-cli.sh)"
```

### Implement your predictor
Expand Down Expand Up @@ -152,6 +145,6 @@ Cortex is an open source alternative to serving models with SageMaker or buildin
## Examples

<!-- CORTEX_VERSION_README_MINOR x3 -->
* [Image classification](https://github.com/cortexlabs/cortex/tree/0.16/examples/tensorflow/image-classifier): deploy an Inception model to classify images.
* [Search completion](https://github.com/cortexlabs/cortex/tree/0.16/examples/pytorch/search-completer): deploy Facebook's RoBERTa model to complete search terms.
* [Text generation](https://github.com/cortexlabs/cortex/tree/0.16/examples/pytorch/text-generator): deploy Hugging Face's DistilGPT2 model to generate text.
* [Image classification](https://github.com/cortexlabs/cortex/tree/0.17/examples/tensorflow/image-classifier): deploy an Inception model to classify images.
* [Search completion](https://github.com/cortexlabs/cortex/tree/0.17/examples/pytorch/search-completer): deploy Facebook's RoBERTa model to complete search terms.
* [Text generation](https://github.com/cortexlabs/cortex/tree/0.17/examples/pytorch/text-generator): deploy Hugging Face's DistilGPT2 model to generate text.
2 changes: 1 addition & 1 deletion build/build-image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ set -euo pipefail

ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. >/dev/null && pwd)"

CORTEX_VERSION=master
CORTEX_VERSION=0.17.0

slim="false"
while [[ $# -gt 0 ]]; do
Expand Down
2 changes: 1 addition & 1 deletion build/cli.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ set -euo pipefail

ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. >/dev/null && pwd)"

CORTEX_VERSION=master
CORTEX_VERSION=0.17.0

arg1=${1:-""}
upload="false"
Expand Down
2 changes: 1 addition & 1 deletion build/push-image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

set -euo pipefail

CORTEX_VERSION=master
CORTEX_VERSION=0.17.0

slim="false"
while [[ $# -gt 0 ]]; do
Expand Down
34 changes: 17 additions & 17 deletions docs/cluster-management/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The Cortex cluster may be configured by providing a configuration file to `cortex cluster up` or `cortex cluster configure` via the `--config` flag (e.g. `cortex cluster up --config cluster.yaml`). Below is the schema for the cluster configuration file, with default values shown (unless otherwise specified):

<!-- CORTEX_VERSION_MINOR -->
<!-- CORTEX_VERSION_MINOR x2 -->
```yaml
# cluster.yaml

Expand Down Expand Up @@ -68,31 +68,31 @@ log_group: cortex
tags: # <string>: <string> map of key/value pairs

# whether to use spot instances in the cluster (default: false)
# see https://cortex.dev/v/master/cluster-management/spot-instances for additional details on spot configuration
# see https://cortex.dev/v/0.17/cluster-management/spot-instances for additional details on spot configuration
spot: false

# see https://cortex.dev/v/master/guides/subdomain-https-setup for instructions on how to set up HTTPS for APIs
# see https://cortex.dev/v/0.17/guides/subdomain-https-setup for instructions on how to set up HTTPS for APIs
ssl_certificate_arn: # if empty, APIs will still be accessible via HTTPS (in addition to HTTP), but will not use a trusted certificate
```
The default docker images used for your Predictors are listed in the instructions for [system packages](../deployments/system-packages.md), and can be overridden in your [API configuration](../deployments/api-configuration.md).
The docker images used by the Cortex cluster can also be overriden, although this is not common. They can be configured by adding any of these keys to your cluster configuration file (default values are shown):
The docker images used by the Cortex cluster can also be overridden, although this is not common. They can be configured by adding any of these keys to your cluster configuration file (default values are shown):
<!-- CORTEX_VERSION_BRANCH_STABLE -->
```yaml
# docker image paths
image_operator: cortexlabs/operator:master
image_manager: cortexlabs/manager:master
image_downloader: cortexlabs/downloader:master
image_request_monitor: cortexlabs/request-monitor:master
image_cluster_autoscaler: cortexlabs/cluster-autoscaler:master
image_metrics_server: cortexlabs/metrics-server:master
image_nvidia: cortexlabs/nvidia:master
image_fluentd: cortexlabs/fluentd:master
image_statsd: cortexlabs/statsd:master
image_istio_proxy: cortexlabs/istio-proxy:master
image_istio_pilot: cortexlabs/istio-pilot:master
image_istio_citadel: cortexlabs/istio-citadel:master
image_istio_galley: cortexlabs/istio-galley:master
image_operator: cortexlabs/operator:0.17.0
image_manager: cortexlabs/manager:0.17.0
image_downloader: cortexlabs/downloader:0.17.0
image_request_monitor: cortexlabs/request-monitor:0.17.0
image_cluster_autoscaler: cortexlabs/cluster-autoscaler:0.17.0
image_metrics_server: cortexlabs/metrics-server:0.17.0
image_nvidia: cortexlabs/nvidia:0.17.0
image_fluentd: cortexlabs/fluentd:0.17.0
image_statsd: cortexlabs/statsd:0.17.0
image_istio_proxy: cortexlabs/istio-proxy:0.17.0
image_istio_pilot: cortexlabs/istio-pilot:0.17.0
image_istio_citadel: cortexlabs/istio-citadel:0.17.0
image_istio_galley: cortexlabs/istio-galley:0.17.0
```
6 changes: 3 additions & 3 deletions docs/cluster-management/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<!-- CORTEX_VERSION_MINOR -->
```bash
$ bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/master/get-cli.sh)"
$ bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.17/get-cli.sh)"
```

Continue to [deploy an example](#deploy-an-example) below.
Expand All @@ -26,7 +26,7 @@ To use GPU nodes, you may need to subscribe to the [EKS-optimized AMI with GPU S
<!-- CORTEX_VERSION_MINOR -->
```bash
# install the CLI on your machine
$ bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/master/get-cli.sh)"
$ bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.17/get-cli.sh)"

# provision infrastructure on AWS and spin up a cluster
$ cortex cluster up
Expand All @@ -37,7 +37,7 @@ $ cortex cluster up
<!-- CORTEX_VERSION_MINOR -->
```bash
# clone the Cortex repository
$ git clone -b master https://github.com/cortexlabs/cortex.git
$ git clone -b 0.17 https://github.com/cortexlabs/cortex.git

# navigate to the TensorFlow iris classification example
$ cd cortex/examples/tensorflow/iris-classifier
Expand Down
2 changes: 1 addition & 1 deletion docs/cluster-management/update.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ cortex cluster configure
cortex cluster down

# update your CLI
bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/master/get-cli.sh)"
bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.17/get-cli.sh)"

# confirm version
cortex version
Expand Down
2 changes: 1 addition & 1 deletion docs/deployments/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,4 @@ deleting my-api
<!-- CORTEX_VERSION_MINOR -->
* [Tutorial](../../examples/sklearn/iris-classifier/README.md) provides a step-by-step walkthough of deploying an iris classifier API
* [CLI documentation](../miscellaneous/cli.md) lists all CLI commands
* [Examples](https://github.com/cortexlabs/cortex/tree/master/examples) demonstrate how to deploy models from common ML libraries
* [Examples](https://github.com/cortexlabs/cortex/tree/0.17/examples) demonstrate how to deploy models from common ML libraries
14 changes: 7 additions & 7 deletions docs/deployments/exporting.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Here are examples for some common ML libraries:
The recommended approach is export your PyTorch model with [torch.save()](https://pytorch.org/docs/stable/torch.html?highlight=save#torch.save). Here is PyTorch's documentation on [saving and loading models](https://pytorch.org/tutorials/beginner/saving_loading_models.html).

<!-- CORTEX_VERSION_MINOR -->
[examples/pytorch/iris-classifier](https://github.com/cortexlabs/cortex/blob/master/examples/pytorch/iris-classifier) exports its trained model like this:
[examples/pytorch/iris-classifier](https://github.com/cortexlabs/cortex/blob/0.17/examples/pytorch/iris-classifier) exports its trained model like this:

```python
torch.save(model.state_dict(), "weights.pth")
Expand All @@ -22,7 +22,7 @@ torch.save(model.state_dict(), "weights.pth")
It may also be possible to export your PyTorch model into the ONNX format using [torch.onnx.export()](https://pytorch.org/docs/stable/onnx.html#torch.onnx.export).

<!-- CORTEX_VERSION_MINOR -->
For example, if [examples/pytorch/iris-classifier](https://github.com/cortexlabs/cortex/blob/master/examples/pytorch/iris-classifier) were to export the model to ONNX, it would look like this:
For example, if [examples/pytorch/iris-classifier](https://github.com/cortexlabs/cortex/blob/0.17/examples/pytorch/iris-classifier) were to export the model to ONNX, it would look like this:

```python
placeholder = torch.randn(1, 4)
Expand Down Expand Up @@ -50,7 +50,7 @@ A TensorFlow `SavedModel` directory should have this structure:
```

<!-- CORTEX_VERSION_MINOR -->
Most of the TensorFlow examples use this approach. Here is the relevant code from [examples/tensorflow/sentiment-analyzer](https://github.com/cortexlabs/cortex/blob/master/examples/tensorflow/sentiment-analyzer):
Most of the TensorFlow examples use this approach. Here is the relevant code from [examples/tensorflow/sentiment-analyzer](https://github.com/cortexlabs/cortex/blob/0.17/examples/tensorflow/sentiment-analyzer):

```python
import tensorflow as tf
Expand Down Expand Up @@ -88,14 +88,14 @@ aws s3 cp bert.zip s3://my-bucket/bert.zip
```

<!-- CORTEX_VERSION_MINOR -->
[examples/tensorflow/iris-classifier](https://github.com/cortexlabs/cortex/blob/master/examples/tensorflow/iris-classifier) also use the `SavedModel` approach, and includes a Python notebook demonstrating how it was exported.
[examples/tensorflow/iris-classifier](https://github.com/cortexlabs/cortex/blob/0.17/examples/tensorflow/iris-classifier) also use the `SavedModel` approach, and includes a Python notebook demonstrating how it was exported.

### Other model formats

There are other ways to export Keras or TensorFlow models, and as long as they can be loaded and used to make predictions in Python, they will be supported by Cortex.

<!-- CORTEX_VERSION_MINOR -->
For example, the `crnn` API in [examples/tensorflow/license-plate-reader](https://github.com/cortexlabs/cortex/blob/master/examples/tensorflow/license-plate-reader) uses this approach.
For example, the `crnn` API in [examples/tensorflow/license-plate-reader](https://github.com/cortexlabs/cortex/blob/0.17/examples/tensorflow/license-plate-reader) uses this approach.

## Scikit-learn

Expand All @@ -104,7 +104,7 @@ For example, the `crnn` API in [examples/tensorflow/license-plate-reader](https:
Scikit-learn models are typically exported using `pickle`. Here is [Scikit-learn's documentation](https://scikit-learn.org/stable/modules/model_persistence.html).

<!-- CORTEX_VERSION_MINOR -->
[examples/sklearn/iris-classifier](https://github.com/cortexlabs/cortex/blob/master/examples/sklearn/iris-classifier) uses this approach. Here is the relevant code:
[examples/sklearn/iris-classifier](https://github.com/cortexlabs/cortex/blob/0.17/examples/sklearn/iris-classifier) uses this approach. Here is the relevant code:

```python
pickle.dump(model, open("model.pkl", "wb"))
Expand Down Expand Up @@ -157,7 +157,7 @@ model.save_model("model.bin")
It is also possible to export an XGBoost model to the ONNX format using [onnxmltools](https://github.com/onnx/onnxmltools).

<!-- CORTEX_VERSION_MINOR -->
[examples/xgboost/iris-classifier](https://github.com/cortexlabs/cortex/blob/master/examples/xgboost/iris-classifier) uses this approach. Here is the relevant code:
[examples/xgboost/iris-classifier](https://github.com/cortexlabs/cortex/blob/0.17/examples/xgboost/iris-classifier) uses this approach. Here is the relevant code:

```python
from onnxmltools.convert import convert_xgboost
Expand Down
20 changes: 10 additions & 10 deletions docs/deployments/predictors.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,10 @@ The `payload` parameter is parsed according to the `Content-Type` header in the
### Examples

<!-- CORTEX_VERSION_MINOR -->
Many of the [examples](https://github.com/cortexlabs/cortex/tree/master/examples) use the Python Predictor, including all of the PyTorch examples.
Many of the [examples](https://github.com/cortexlabs/cortex/tree/0.17/examples) use the Python Predictor, including all of the PyTorch examples.

<!-- CORTEX_VERSION_MINOR -->
Here is the Predictor for [examples/pytorch/iris-classifier](https://github.com/cortexlabs/cortex/tree/master/examples/pytorch/iris-classifier):
Here is the Predictor for [examples/pytorch/iris-classifier](https://github.com/cortexlabs/cortex/tree/0.17/examples/pytorch/iris-classifier):

```python
import re
Expand Down Expand Up @@ -155,7 +155,7 @@ xgboost==1.0.2
```

<!-- CORTEX_VERSION_MINOR x2 -->
The pre-installed system packages are listed in [images/python-predictor-cpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/master/images/python-predictor-cpu/Dockerfile) (for CPU) or [images/python-predictor-gpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/master/images/python-predictor-gpu/Dockerfile) (for GPU).
The pre-installed system packages are listed in [images/python-predictor-cpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/0.17/images/python-predictor-cpu/Dockerfile) (for CPU) or [images/python-predictor-gpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/0.17/images/python-predictor-gpu/Dockerfile) (for GPU).

If your application requires additional dependencies, you can install additional [Python packages](python-packages.md) and [system packages](system-packages.md).

Expand Down Expand Up @@ -190,7 +190,7 @@ class TensorFlowPredictor:
```

<!-- CORTEX_VERSION_MINOR -->
Cortex provides a `tensorflow_client` to your Predictor's constructor. `tensorflow_client` is an instance of [TensorFlowClient](https://github.com/cortexlabs/cortex/tree/master/pkg/workloads/cortex/lib/client/tensorflow.py) that manages a connection to a TensorFlow Serving container to make predictions using your model. It should be saved as an instance variable in your Predictor, and your `predict()` function should call `tensorflow_client.predict()` to make an inference with your exported TensorFlow model. Preprocessing of the JSON payload and postprocessing of predictions can be implemented in your `predict()` function as well.
Cortex provides a `tensorflow_client` to your Predictor's constructor. `tensorflow_client` is an instance of [TensorFlowClient](https://github.com/cortexlabs/cortex/tree/0.17/pkg/workloads/cortex/lib/client/tensorflow.py) that manages a connection to a TensorFlow Serving container to make predictions using your model. It should be saved as an instance variable in your Predictor, and your `predict()` function should call `tensorflow_client.predict()` to make an inference with your exported TensorFlow model. Preprocessing of the JSON payload and postprocessing of predictions can be implemented in your `predict()` function as well.

For proper separation of concerns, it is recommended to use the constructor's `config` paramater for information such as configurable model parameters or download links for initialization files. You define `config` in your [API configuration](api-configuration.md), and it is passed through to your Predictor's constructor.

Expand All @@ -199,10 +199,10 @@ The `payload` parameter is parsed according to the `Content-Type` header in the
### Examples

<!-- CORTEX_VERSION_MINOR -->
Most of the examples in [examples/tensorflow](https://github.com/cortexlabs/cortex/tree/master/examples/tensorflow) use the TensorFlow Predictor.
Most of the examples in [examples/tensorflow](https://github.com/cortexlabs/cortex/tree/0.17/examples/tensorflow) use the TensorFlow Predictor.

<!-- CORTEX_VERSION_MINOR -->
Here is the Predictor for [examples/tensorflow/iris-classifier](https://github.com/cortexlabs/cortex/tree/master/examples/tensorflow/iris-classifier):
Here is the Predictor for [examples/tensorflow/iris-classifier](https://github.com/cortexlabs/cortex/tree/0.17/examples/tensorflow/iris-classifier):

```python
labels = ["setosa", "versicolor", "virginica"]
Expand Down Expand Up @@ -235,7 +235,7 @@ tensorflow==2.1.0
```

<!-- CORTEX_VERSION_MINOR -->
The pre-installed system packages are listed in [images/tensorflow-predictor/Dockerfile](https://github.com/cortexlabs/cortex/tree/master/images/tensorflow-predictor/Dockerfile).
The pre-installed system packages are listed in [images/tensorflow-predictor/Dockerfile](https://github.com/cortexlabs/cortex/tree/0.17/images/tensorflow-predictor/Dockerfile).

If your application requires additional dependencies, you can install additional [Python packages](python-packages.md) and [system packages](system-packages.md).

Expand Down Expand Up @@ -270,7 +270,7 @@ class ONNXPredictor:
```

<!-- CORTEX_VERSION_MINOR -->
Cortex provides an `onnx_client` to your Predictor's constructor. `onnx_client` is an instance of [ONNXClient](https://github.com/cortexlabs/cortex/tree/master/pkg/workloads/cortex/lib/client/onnx.py) that manages an ONNX Runtime session to make predictions using your model. It should be saved as an instance variable in your Predictor, and your `predict()` function should call `onnx_client.predict()` to make an inference with your exported ONNX model. Preprocessing of the JSON payload and postprocessing of predictions can be implemented in your `predict()` function as well.
Cortex provides an `onnx_client` to your Predictor's constructor. `onnx_client` is an instance of [ONNXClient](https://github.com/cortexlabs/cortex/tree/0.17/pkg/workloads/cortex/lib/client/onnx.py) that manages an ONNX Runtime session to make predictions using your model. It should be saved as an instance variable in your Predictor, and your `predict()` function should call `onnx_client.predict()` to make an inference with your exported ONNX model. Preprocessing of the JSON payload and postprocessing of predictions can be implemented in your `predict()` function as well.

For proper separation of concerns, it is recommended to use the constructor's `config` paramater for information such as configurable model parameters or download links for initialization files. You define `config` in your [API configuration](api-configuration.md), and it is passed through to your Predictor's constructor.

Expand All @@ -279,7 +279,7 @@ The `payload` parameter is parsed according to the `Content-Type` header in the
### Examples

<!-- CORTEX_VERSION_MINOR -->
[examples/xgboost/iris-classifier](https://github.com/cortexlabs/cortex/tree/master/examples/xgboost/iris-classifier) uses the ONNX Predictor:
[examples/xgboost/iris-classifier](https://github.com/cortexlabs/cortex/tree/0.17/examples/xgboost/iris-classifier) uses the ONNX Predictor:

```python
labels = ["setosa", "versicolor", "virginica"]
Expand Down Expand Up @@ -316,7 +316,7 @@ requests==2.23.0
```

<!-- CORTEX_VERSION_MINOR x2 -->
The pre-installed system packages are listed in [images/onnx-predictor-cpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/master/images/onnx-predictor-cpu/Dockerfile) (for CPU) or [images/onnx-predictor-gpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/master/images/onnx-predictor-gpu/Dockerfile) (for GPU).
The pre-installed system packages are listed in [images/onnx-predictor-cpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/0.17/images/onnx-predictor-cpu/Dockerfile) (for CPU) or [images/onnx-predictor-gpu/Dockerfile](https://github.com/cortexlabs/cortex/tree/0.17/images/onnx-predictor-gpu/Dockerfile) (for GPU).

If your application requires additional dependencies, you can install additional [Python packages](python-packages.md) and [system packages](system-packages.md).

Expand Down
Loading

0 comments on commit c6c6aea

Please sign in to comment.