Skip to content

Commit

Permalink
Updating versions for v24.10.00
Browse files Browse the repository at this point in the history
  • Loading branch information
dagardner-nv committed Jul 3, 2024
1 parent 26cec63 commit 11f9417
Show file tree
Hide file tree
Showing 15 changed files with 43 additions and 43 deletions.
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
[submodule "external/morpheus-visualizations"]
path = external/morpheus-visualizations
url = https://github.com/nv-morpheus/morpheus-visualizations.git
branch = branch-24.06
branch = branch-24.10
[submodule "external/utilities"]
path = external/utilities
url = https://github.com/nv-morpheus/utilities.git
branch = branch-24.06
branch = branch-24.10
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ morpheus_utils_initialize_cuda_arch(morpheus)
# Note intentionally excluding CUDA from the LANGUAGES list allowing us to set some clang specific settings later when
# we call morpheus_utils_enable_cuda()
project(morpheus
VERSION 24.06.00
VERSION 24.10.00
LANGUAGES C CXX
)

Expand Down
4 changes: 2 additions & 2 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ dependencies:
- glog=0.6
- libcudf=24.02
- librdkafka>=1.9.2,<1.10.0a0
- mrc=24.06
- mrc=24.10
- nlohmann_json=3.11
- pybind11-stubgen=0.10.5
- rapidjson=1.1.0
Expand Down Expand Up @@ -331,7 +331,7 @@ dependencies:
- grpcio-status=1.59
# - libwebp=1.3.2 # Required for CVE mitigation: https://nvd.nist.gov/vuln/detail/CVE-2023-4863 ##
- mlflow>=2.10.0,<3
- mrc=24.06
- mrc=24.10
- networkx=2.8.8
- numpydoc=1.5
- nvtabular=23.08.00
Expand Down
12 changes: 6 additions & 6 deletions docs/source/cloud_deployment_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ The Helm chart (`morpheus-ai-engine`) that offers the auxiliary components requi
Follow the below steps to install Morpheus AI Engine:

```bash
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-ai-engine-24.06.tgz --username='$oauthtoken' --password=$API_KEY --untar
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-ai-engine-24.10.tgz --username='$oauthtoken' --password=$API_KEY --untar
```
```bash
helm install --set ngc.apiKey="$API_KEY" \
Expand Down Expand Up @@ -146,7 +146,7 @@ replicaset.apps/zookeeper-87f9f4dd 1 1 1 54s
Run the following command to pull the Morpheus SDK Client (referred to as Helm chart `morpheus-sdk-client`) on to your instance:

```bash
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-sdk-client-24.06.tgz --username='$oauthtoken' --password=$API_KEY --untar
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-sdk-client-24.10.tgz --username='$oauthtoken' --password=$API_KEY --untar
```

#### Morpheus SDK Client in Sleep Mode
Expand Down Expand Up @@ -184,7 +184,7 @@ kubectl -n $NAMESPACE exec sdk-cli-helper -- cp -RL /workspace/models /common
The Morpheus MLflow Helm chart offers MLflow server with Triton plugin to deploy, update, and remove models from the Morpheus AI Engine. The MLflow server UI can be accessed using NodePort `30500`. Follow the below steps to install the Morpheus MLflow:

```bash
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-mlflow-24.06.tgz --username='$oauthtoken' --password=$API_KEY --untar
helm fetch https://helm.ngc.nvidia.com/nvidia/morpheus/charts/morpheus-mlflow-24.10.tgz --username='$oauthtoken' --password=$API_KEY --untar
```
```bash
helm install --set ngc.apiKey="$API_KEY" \
Expand Down Expand Up @@ -402,7 +402,7 @@ To publish messages to a Kafka topic, we need to copy datasets to locations wher
kubectl -n $NAMESPACE exec sdk-cli-helper -- cp -R /workspace/examples/data /common
```

Refer to the [Morpheus CLI Overview](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/basics/overview.rst) and [Building a Pipeline](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/basics/building_a_pipeline.md) documentation for more information regarding the commands.
Refer to the [Morpheus CLI Overview](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/basics/overview.rst) and [Building a Pipeline](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/basics/building_a_pipeline.md) documentation for more information regarding the commands.

> **Note**: Before running the example pipelines, ensure the criteria below are met:
- Ensure models specific to the pipeline are deployed.
Expand Down Expand Up @@ -782,8 +782,8 @@ kubectl -n $NAMESPACE exec deploy/broker -c broker -- kafka-topics.sh \

## Additional Documentation
For more information on how to use the Morpheus Python API to customize and run your own optimized AI pipelines, Refer to below documentation.
- [Morpheus Developer Guides](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/developer_guide/guides.md)
- [Morpheus Pipeline Examples](https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/examples)
- [Morpheus Developer Guides](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/developer_guide/guides.md)
- [Morpheus Pipeline Examples](https://github.com/nv-morpheus/Morpheus/tree/branch-24.10/examples)


## Troubleshooting
Expand Down
4 changes: 2 additions & 2 deletions docs/source/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,10 @@ To run the built "release" container, use the following:
./docker/run_container_release.sh
```

The `./docker/run_container_release.sh` script accepts the same `DOCKER_IMAGE_NAME`, and `DOCKER_IMAGE_TAG` environment variables that the `./docker/build_container_release.sh` script does. For example, to run version `v24.06.00` use the following:
The `./docker/run_container_release.sh` script accepts the same `DOCKER_IMAGE_NAME`, and `DOCKER_IMAGE_TAG` environment variables that the `./docker/build_container_release.sh` script does. For example, to run version `v24.10.00` use the following:

```bash
DOCKER_IMAGE_TAG="v24.06.00-runtime" ./docker/run_container_release.sh
DOCKER_IMAGE_TAG="v24.10.00-runtime" ./docker/run_container_release.sh
```

## Launching Triton Server
Expand Down
2 changes: 1 addition & 1 deletion examples/digital_fingerprinting/production/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# limitations under the License.

ARG MORPHEUS_CONTAINER=nvcr.io/nvidia/morpheus/morpheus
ARG MORPHEUS_CONTAINER_VERSION=v24.06.00-runtime
ARG MORPHEUS_CONTAINER_VERSION=v24.10.00-runtime

FROM ${MORPHEUS_CONTAINER}:${MORPHEUS_CONTAINER_VERSION} as base

Expand Down
4 changes: 2 additions & 2 deletions examples/digital_fingerprinting/production/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ services:
target: jupyter
args:
- MORPHEUS_CONTAINER=${MORPHEUS_CONTAINER:-nvcr.io/nvidia/morpheus/morpheus}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.06.00-runtime}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.10.00-runtime}
deploy:
resources:
reservations:
Expand Down Expand Up @@ -109,7 +109,7 @@ services:
target: runtime
args:
- MORPHEUS_CONTAINER=${MORPHEUS_CONTAINER:-nvcr.io/nvidia/morpheus/morpheus}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.06.00-runtime}
- MORPHEUS_CONTAINER_VERSION=${MORPHEUS_CONTAINER_VERSION:-v24.10.00-runtime}
image: dfp_morpheus
container_name: morpheus_pipeline
deploy:
Expand Down
2 changes: 1 addition & 1 deletion examples/digital_fingerprinting/starter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ We show here how to set up and run the DFP pipeline for three log types: CloudTr

## Environment Setup

Follow the instructions [here](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/docs/source/developer_guide/contributing.md) to set up your development environment in either a Docker container or conda environment.
Follow the instructions [here](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/developer_guide/contributing.md) to set up your development environment in either a Docker container or conda environment.

## Morpheus CLI

Expand Down
2 changes: 1 addition & 1 deletion external/morpheus-visualizations
2 changes: 1 addition & 1 deletion external/utilities
6 changes: 3 additions & 3 deletions models/model-cards/abp-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,15 +73,15 @@ limitations under the License.
## Training Dataset:

**Link:**
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/abp-sample-nvsmi-training-data.json <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/abp-sample-nvsmi-training-data.json <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**
* Sample dataset consists of over 1000 nvidia-smi outputs <br>

## Evaluation Dataset:

**Link:**
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/abp-validation-data.jsonlines <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/validation-data/abp-validation-data.jsonlines <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**
* Sample dataset consists of over 1000 nvidia-smi outputs <br>
Expand Down Expand Up @@ -143,7 +143,7 @@ NVIDIA believes Trustworthy AI is a shared responsibility and we have establishe
## Model Card ++ Saftey & Security Subcard

### Link the location of the training dataset's repository.
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/abp-sample-nvsmi-training-data.json
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/abp-sample-nvsmi-training-data.json

### Describe the life critical impact (if present).
* None
Expand Down
32 changes: 16 additions & 16 deletions models/model-cards/dfp-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,36 +62,36 @@ The model architecture consists of an Autoencoder, where the reconstruction loss
* Linux<br>

## Model Version(s):
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/dfp-models/hammah-role-g-20211017-dill.pkl
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/dfp-models/hammah-user123-20211017-dill.pkl
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/dfp-models/hammah-role-g-20211017-dill.pkl
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/dfp-models/hammah-user123-20211017-dill.pkl

# Training & Evaluation:

## Training Dataset:

**Link:**
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/training-data/cloudtrail
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.10/models/datasets/training-data/cloudtrail

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**

The training dataset consists of AWS CloudTrail logs. It contains logs from two entities, providing information about their activities within the AWS environment.
* [hammah-role-g-training-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-role-g-training-part1.json): 700 records <br>
* [hammah-role-g-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-role-g-training-part2.json): 1187 records <br>
* [hammah-user123-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-user123-training-part2.json): 1000 records <br>
* [hammah-user123-training-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-user123-training-part3.json): 1000 records <br>
* [hammah-user123-training-part4.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/cloudtrail/hammah-user123-training-part4.json): 387 records <br>
* [hammah-role-g-training-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/cloudtrail/hammah-role-g-training-part1.json): 700 records <br>
* [hammah-role-g-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/cloudtrail/hammah-role-g-training-part2.json): 1187 records <br>
* [hammah-user123-training-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/cloudtrail/hammah-user123-training-part2.json): 1000 records <br>
* [hammah-user123-training-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/cloudtrail/hammah-user123-training-part3.json): 1000 records <br>
* [hammah-user123-training-part4.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/cloudtrail/hammah-user123-training-part4.json): 387 records <br>

## Evaluation Dataset:
**Link:**
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/validation-data/cloudtrail <br>
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.10/models/datasets/validation-data/cloudtrail <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**

The evaluation dataset consists of AWS CloudTrail logs. It contains logs from two entities, providing information about their activities within the AWS environment.
* [hammah-role-g-validation.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-role-g-validation.json): 314 records
* [hammah-user123-validation-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part1.json): 300 records
* [hammah-user123-validation-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part2.json): 300 records
* [hammah-user123-validation-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part3.json): 247 records
* [hammah-role-g-validation.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/validation-data/cloudtrail/hammah-role-g-validation.json): 314 records
* [hammah-user123-validation-part1.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part1.json): 300 records
* [hammah-user123-validation-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part2.json): 300 records
* [hammah-user123-validation-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part3.json): 247 records

## Inference:
**Engine:**
Expand Down Expand Up @@ -130,7 +130,7 @@ NVIDIA believes Trustworthy AI is a shared responsibility and we have establishe

### Describe how this model works.
* The model works by training on baseline behaviors and subsequently detecting deviations from the established baseline, triggering alerts accordingly.
* [Training notebook](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/training-tuning-scripts/dfp-models/hammah-20211017.ipynb)
* [Training notebook](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/training-tuning-scripts/dfp-models/hammah-20211017.ipynb)

### List the technical limitations of the model.
* The model expects cloudtrail logs with specific features that match the training dataset. Data lacking the required features or requiring a different feature set may not be compatible with the model.
Expand All @@ -150,7 +150,7 @@ NVIDIA believes Trustworthy AI is a shared responsibility and we have establishe
## Model Card ++ Saftey & Security Subcard

### Link the location of the training dataset's repository (if able to share).
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/training-data/cloudtrail
* https://github.com/nv-morpheus/Morpheus/tree/branch-24.10/models/datasets/training-data/cloudtrail

### Describe the life critical impact (if present).
* None
Expand Down Expand Up @@ -191,7 +191,7 @@ NVIDIA believes Trustworthy AI is a shared responsibility and we have establishe
* No

### Are we able to identify and trace source of dataset?
* Yes ([fully synthetic dataset](https://github.com/nv-morpheus/Morpheus/tree/branch-24.06/models/datasets/training-data/cloudtrail))
* Yes ([fully synthetic dataset](https://github.com/nv-morpheus/Morpheus/tree/branch-24.10/models/datasets/training-data/cloudtrail))

### Does data labeling (annotation, metadata) comply with privacy laws?
* Not applicable (dataset is fully synthetic)
Expand Down
2 changes: 1 addition & 1 deletion models/model-cards/gnn-fsi-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ NVIDIA believes Trustworthy AI is a shared responsibility and we have establishe
* None

### Link the relevant end user license agreement
* [Apache 2.0](https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/LICENSE)
* [Apache 2.0](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/LICENSE)

## Model Card ++ Saftey & Security Subcard

Expand Down
2 changes: 1 addition & 1 deletion models/model-cards/phishing-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ limitations under the License.
## Evaluation Dataset:

**Link:**
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/phishing-email-validation-data.jsonlines <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/validation-data/phishing-email-validation-data.jsonlines <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**
* Dataset consists of SMSs <br>
Expand Down
6 changes: 3 additions & 3 deletions models/model-cards/root-cause-analysis-model-card.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,15 +74,15 @@ limitations under the License.
## Training Dataset:

**Link:**
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/root-cause-training-data.csv <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/root-cause-training-data.csv <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**
* kern.log files from DGX machines <br>

## Evaluation Dataset:

**Link:**
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/validation-data/root-cause-validation-data-input.jsonlines <br>
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/validation-data/root-cause-validation-data-input.jsonlines <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**
* kern.log files from DGX machines <br>
Expand Down Expand Up @@ -141,7 +141,7 @@ limitations under the License.
## Model Card ++ Saftey & Security Subcard

### Link the location of the training dataset's repository.
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.06/models/datasets/training-data/root-cause-training-data.csv
* https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/models/datasets/training-data/root-cause-training-data.csv

### Describe the life critical impact (if present).
* None
Expand Down

0 comments on commit 11f9417

Please sign in to comment.