Skip to content

Commit

Permalink
DOCA: fix optional deps + remove PreallocatorMixin from source stage (#…
Browse files Browse the repository at this point in the history
…1729)

Address minor changes (README and optional_deps for DOCA) and bug #1728

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - eago (https://github.com/e-ago)
  - https://github.com/eagonv
  - David Gardner (https://github.com/dagardner-nv)
  - Michael Demoret (https://github.com/mdemoret-nv)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: #1729
  • Loading branch information
e-ago authored Jun 24, 2024
1 parent 5bc4836 commit 7133977
Show file tree
Hide file tree
Showing 6 changed files with 68 additions and 57 deletions.
2 changes: 1 addition & 1 deletion ci/runner/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ RUN rapids-dependency-file-generator \

ENV MORPHEUS_SUPPORT_DOCA=ON

COPY ./docker/optional_deps/doca.sh /tmp/doca/
COPY ./.devcontainer/docker/optional_deps/doca.sh /tmp/doca/

RUN apt update && \
DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC \
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ ARG MORPHEUS_SUPPORT_DOCA="FALSE"
ENV MORPHEUS_SUPPORT_DOCA=${MORPHEUS_SUPPORT_DOCA}

# Copy all of the optional dependency scripts
COPY ${MORPHEUS_ROOT_HOST}/docker/optional_deps docker/optional_deps
COPY ${MORPHEUS_ROOT_HOST}/.devcontainer/docker/optional_deps docker/optional_deps

# Install DOCA (If requested)
RUN --mount=type=cache,id=doca,target=/tmp/doca,sharing=locked \
Expand Down
1 change: 0 additions & 1 deletion docker/optional_deps/doca.sh

This file was deleted.

111 changes: 63 additions & 48 deletions examples/doca/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ See the License for the specific language governing permissions and
limitations under the License.
-->

# DOCA Sensitive Information Detection Example
# DOCA GPU Real-Time traffic analysis

Examples in this directory use the DOCA Source Stage to receive and pre-process network packets in real-time before passing packets info to the next Morphues stages.

## Obtaining the Morpheus DOCA Container
DOCA Support is in early access and may only be used via the Morpheus DOCA Container found in NGC. Please speak to your NVIDIA Morpheus contact for more information.

Expand All @@ -25,7 +28,17 @@ The container must be run in privileged mode and mount in hugepages as configure
docker run -v /dev/hugepages:/dev/hugepages --privileged --rm -ti --runtime=nvidia --net=host --gpus=all --cap-add=sys_nice ${MORPHEUS_DOCA_IMAGE} bash
```

# Preparing the environment

Prior to running the example, the `rdma-core` conda package needs to be _removed by force_ from the conda environment, otherwise the environment is incompatible with the DOCA-provided packages.
```
conda remove --force rdma-core
```

For more info about how to configure the machine, please refer to the [DOCA GPUNetIO programming guide](https://docs.nvidia.com/doca/sdk/doca+gpunetio/index.html).

## Finding the GPU and NIC PCIe Addresses

The DOCA example requires specifying the PCIe Address of both the GPU and NIC explicitly. Determining the correct GPU and NIC PCIe Addresses is non-trivial and requires coordinating with those who have configured the physical hardware and firmware according to the DOCA GPUNetIO documentation, but the following commands can help find a NIC and GPU situation on the same NUMA node.
```
$ lspci -tv | grep -E "NVIDIA|ella|(^\+)|(^\-)"
Expand Down Expand Up @@ -59,17 +72,61 @@ cf:00.0 3D controller: NVIDIA Corporation Device 20b9 (rev a1)
```
We can see the GPU's PCIe address is `cf:00.0`, and we can infer from the above commands that the nearest ConnectX-6 NIC's PCIe address is `cc:00.*`. In this case, we have port `1` physically connected to the network, so we use PCIe Address `cc:00.1`.

## Running the Example
The DOCA example is similar to the Sensitive Information Detection (SID) example in that it uses the `sid-minibert` model in conjunction with the `TritonInferenceStage` to detect sensitive information. The difference is that the sensitive information we will be detecting is obtained from a live TCP packet stream provided by a `DocaSourceStage`.

Prior to running the example, the `rdma-core` conda package needs to be _removed by force_ from the conda environment, otherwise the environment is incompatible with the DOCA-provided packages.
## Running the example for UDP traffic analysis

In case of UDP traffic, the sample will launch a simple pipeline with the DOCA Source Stage followed by a Monitor Stage to report number of received packets.

```
conda remove --force rdma-core
python3 ./examples/doca/run_udp_raw.py --nic_addr 17:00.1 --gpu_addr ca:00.0 --traffic_type udp
```
UDP traffic can be easily sent with nping to the interface where Morpheus is listening:
```
nping --udp -c 100000 -p 4100 192.168.2.27 --data-length 1024 --delay 0.1ms
```

Morpheus output would be:
```
====Pipeline Pre-build====
====Pre-Building Segment: linear_segment_0====
====Pre-Building Segment Complete!====
====Pipeline Pre-build Complete!====
====Registering Pipeline====
====Building Pipeline====
EAL: Detected CPU lcores: 64
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
EAL: Probe PCI driver: mlx5_pci (15b3:a2dc) device: 0000:ca:00.0 (socket 1)
EAL: Probe PCI driver: gpu_cuda (10de:2331) device: 0000:17:00.0 (socket 0)
====Building Pipeline Complete!====
DOCA GPUNetIO rate: 0 pkts [00:00, ? pkts/s]====Registering Pipeline Complete!====
====Starting Pipeline====
====Pipeline Started====
====Building Segment: linear_segment_0====
Added source: <from-doca-0; DocaSourceStage(nic_pci_address=ca:00.0, gpu_pci_address=17:00.0, traffic_type=udp)>
└─> morpheus.MessageMeta
Added stage: <monitor-1; MonitorStage(description=DOCA GPUNetIO rate, smoothing=0.05, unit=pkts, delayed_start=False, determine_count_fn=None, log_level=LogLevels.INFO)>
└─ morpheus.MessageMeta -> morpheus.MessageMeta
====Building Segment Complete!====
DOCA GPUNetIO rate: 100000 pkts [00:12, 10963.39 pkts/s]
```

As the DOCA Source stage output packets in the new RawMessage format that not all the Morpheus stages may support, there is an additional stage named DOCA Convert Stage which transform the data RawMessage to the Messagemeta format.

```
python3 ./examples/doca/run_udp_convert.py --nic_addr 17:00.1 --gpu_addr ca:00.0 --traffic_type udp
```

## Doca Sensitive Information Detection example for TCP traffic

The DOCA example is similar to the Sensitive Information Detection (SID) example in that it uses the `sid-minibert` model in conjunction with the `TritonInferenceStage` to detect sensitive information. The difference is that the sensitive information we will be detecting is obtained from a live TCP packet stream provided by a `DocaSourceStage`.
To run the example from the Morpheus root directory and capture all TCP network traffic from the given NIC, use the following command and replace the `nic_addr` and `gpu_addr` arguments with your NIC and GPU PCIe addresses.
```
# python examples/doca/run.py --nic_addr cc:00.1 --gpu_addr cf:00.0 --traffic_type tcp
# python examples/doca/run_tcp.py --nic_addr cc:00.1 --gpu_addr cf:00.0 --traffic_type tcp
```
```
====Registering Pipeline====
Expand Down Expand Up @@ -119,45 +176,3 @@ AddClass rate: 0 pkts [00:09, ? pkts/s]
```
The output can be found in `doca_output.csv`

## Running the Example for UDP traffic

In case of UDP traffic, the sample will launch a simple pipeline with the DOCA Source Stage followed by a Monitor Stage to report number of received packets.
Command line is similar to the TCP example.

```
python3 ./examples/doca/run.py --nic_addr 17:00.1 --gpu_addr ca:00.0 --traffic_type udp
```
UDP traffic can be easily sent with nping to the interface where Morpheus is listening:
```
nping --udp -c 100000 -p 4100 192.168.2.27 --data-length 1024 --delay 0.1ms
```

Morpheus output would be:
```
====Pipeline Pre-build====
====Pre-Building Segment: linear_segment_0====
====Pre-Building Segment Complete!====
====Pipeline Pre-build Complete!====
====Registering Pipeline====
====Building Pipeline====
EAL: Detected CPU lcores: 64
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
EAL: Probe PCI driver: mlx5_pci (15b3:a2dc) device: 0000:ca:00.0 (socket 1)
EAL: Probe PCI driver: gpu_cuda (10de:2331) device: 0000:17:00.0 (socket 0)
====Building Pipeline Complete!====
DOCA GPUNetIO rate: 0 pkts [00:00, ? pkts/s]====Registering Pipeline Complete!====
====Starting Pipeline====
====Pipeline Started====
====Building Segment: linear_segment_0====
Added source: <from-doca-0; DocaSourceStage(nic_pci_address=ca:00.0, gpu_pci_address=17:00.0, traffic_type=udp)>
└─> morpheus.MessageMeta
Added stage: <monitor-1; MonitorStage(description=DOCA GPUNetIO rate, smoothing=0.05, unit=pkts, delayed_start=False, determine_count_fn=None, log_level=LogLevels.INFO)>
└─ morpheus.MessageMeta -> morpheus.MessageMeta
====Building Segment Complete!====
DOCA GPUNetIO rate: 100000 pkts [00:12, 10963.39 pkts/s]
```
6 changes: 2 additions & 4 deletions examples/doca/run_tcp.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,11 @@ def run_pipeline(pipeline_batch_size, model_max_batch_size, model_fea_length, ou
config.mode = PipelineModes.NLP

# Below properties are specified by the command line
config.num_threads = 1
config.num_threads = 5
config.edge_buffer_size = 1024
config.pipeline_batch_size = pipeline_batch_size
config.model_max_batch_size = model_max_batch_size
config.feature_length = model_fea_length
config.mode = PipelineModes.NLP

config.class_labels = [
'address',
Expand All @@ -96,8 +96,6 @@ def run_pipeline(pipeline_batch_size, model_max_batch_size, model_fea_length, ou
'user'
]

config.edge_buffer_size = 128

pipeline = LinearPipeline(config)

# add doca source stage
Expand Down
3 changes: 1 addition & 2 deletions morpheus/stages/doca/doca_source_stage.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,14 @@
from morpheus.config import Config
from morpheus.config import PipelineModes
from morpheus.messages import RawPacketMessage
from morpheus.pipeline.preallocator_mixin import PreallocatorMixin
from morpheus.pipeline.single_output_source import SingleOutputSource
from morpheus.pipeline.stage_schema import StageSchema

logger = logging.getLogger(__name__)


@register_stage("from-doca-source", modes=[PipelineModes.NLP])
class DocaSourceStage(PreallocatorMixin, SingleOutputSource):
class DocaSourceStage(SingleOutputSource):
"""
A source stage used to receive raw packet data from a ConnectX-6 Dx NIC.
Expand Down

0 comments on commit 7133977

Please sign in to comment.