Forward-merge branch-24.03 into branch-24.06 (#1605)

## Description * Manually resolves merge conflicts encountered by the `rapids-bot` * Original forward-merger PR #1593 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes.
nv-morpheus · Apr 9, 2024 · c9346db · c9346db
2 parents 752e85f + cb9f5e6
commit c9346db
Show file tree

Hide file tree

Showing 182 changed files with 3,452 additions and 1,126 deletions.
diff --git a/.github/workflows/pr.yaml b/.github/workflows/pr.yaml
@@ -72,10 +72,11 @@ jobs:
       pr_info: ${{ steps.get-pr-info.outcome == 'success' && steps.get-pr-info.outputs.pr-info || '' }}
 
   checks:
+    needs: [prepare]
+    # Only run the CI pipeline if the PR does not have the skip-ci label and we are on a PR branch
+    if: ${{ !fromJSON(needs.prepare.outputs.has_skip_ci_label) && fromJSON(needs.prepare.outputs.is_pr )}}
     secrets: inherit
     uses: rapidsai/shared-workflows/.github/workflows/[email protected]
-    # Only run the CI pipeline if the PR does not have the skip-ci label
-    if: ${{ ! fromJSON(needs.prepare.outputs.has_skip_ci_label) }}
     with:
       enable_check_generated_files: false
 

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -15,6 +15,93 @@ See the License for the specific language governing permissions and
 limitations under the License.
 -->
 
+# Morpheus 24.03.00 (7 Apr 2024)
+
+## 🚨 Breaking Changes
+
+- Updating `nlohman_json` to 3.11 to match MRC ([#1596](https://github.com/nv-morpheus/Morpheus/pull/1596)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- Add retry logic and proxy support to the NeMo LLM Service ([#1544](https://github.com/nv-morpheus/Morpheus/pull/1544)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- Upgrade `openai` version to 1.13 and `langchain` to version 0.1.9 ([#1529](https://github.com/nv-morpheus/Morpheus/pull/1529)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- Make `start_async()` available to source stages ([#1523](https://github.com/nv-morpheus/Morpheus/pull/1523)) [@efajardo-nv](https://github.com/efajardo-nv)
+- RAPIDS 24.02 Upgrade ([#1468](https://github.com/nv-morpheus/Morpheus/pull/1468)) [@cwharris](https://github.com/cwharris)
+- Decouple TritonInferenceStage from pipeline mode ([#1402](https://github.com/nv-morpheus/Morpheus/pull/1402)) [@dagardner-nv](https://github.com/dagardner-nv)
+
+## 🐛 Bug Fixes
+
+- Serialize datetime objects into the module config ([#1592](https://github.com/nv-morpheus/Morpheus/pull/1592)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Remove the defaults channel from `dependencies.yml` ([#1584](https://github.com/nv-morpheus/Morpheus/pull/1584)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- Fix `iso_date_regex_pattern` config in `file_batcher` module and allow override ([#1580](https://github.com/nv-morpheus/Morpheus/pull/1580)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Update DFP MLflow ModelManager to handle model retrieval using file URI ([#1578](https://github.com/nv-morpheus/Morpheus/pull/1578)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Fix `configure_logging` in DFP benchmarks ([#1553](https://github.com/nv-morpheus/Morpheus/pull/1553)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Catch langchain agent errors ([#1539](https://github.com/nv-morpheus/Morpheus/pull/1539)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Adding missing dependency on `pydantic` ([#1535](https://github.com/nv-morpheus/Morpheus/pull/1535)) [@yuchenz427](https://github.com/yuchenz427)
+- Fix memory leak in the mutable dataframe checkout/checkin code ([#1534](https://github.com/nv-morpheus/Morpheus/pull/1534)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Fix pathlib.Path support for FileSourceStage ([#1531](https://github.com/nv-morpheus/Morpheus/pull/1531)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Make `start_async()` available to source stages ([#1523](https://github.com/nv-morpheus/Morpheus/pull/1523)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Update CI Containers ([#1521](https://github.com/nv-morpheus/Morpheus/pull/1521)) [@cwharris](https://github.com/cwharris)
+- Fix intermittent segfault on interpreter shutdown ([#1513](https://github.com/nv-morpheus/Morpheus/pull/1513)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Adopt updated builds of CI runners ([#1503](https://github.com/nv-morpheus/Morpheus/pull/1503)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Update mlflow plugin version for deployments fix ([#1499](https://github.com/nv-morpheus/Morpheus/pull/1499)) [@pdmack](https://github.com/pdmack)
+- Add runtime environment output to fix building the release container ([#1496](https://github.com/nv-morpheus/Morpheus/pull/1496)) [@cwharris](https://github.com/cwharris)
+- Fix logging of sleep time ([#1493](https://github.com/nv-morpheus/Morpheus/pull/1493)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Pin pytest to &lt;8 ([#1485](https://github.com/nv-morpheus/Morpheus/pull/1485)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Improve pipeline stop logic to ensure join is called exactly once for all stages ([#1479](https://github.com/nv-morpheus/Morpheus/pull/1479)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Fix expected JSON config file extension in logger ([#1471](https://github.com/nv-morpheus/Morpheus/pull/1471)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Fix Loss Function to Improve Model Convergence for `AutoEncoder` ([#1460](https://github.com/nv-morpheus/Morpheus/pull/1460)) [@hsin-c](https://github.com/hsin-c)
+- GNN fraud detection notebook fix ([#1450](https://github.com/nv-morpheus/Morpheus/pull/1450)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Eliminate Redundant Fetches in RSS Controller ([#1442](https://github.com/nv-morpheus/Morpheus/pull/1442)) [@bsuryadevara](https://github.com/bsuryadevara)
+- Updating the workspace settings to remove deprecated python options ([#1440](https://github.com/nv-morpheus/Morpheus/pull/1440)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- Improve camouflage startup issues ([#1436](https://github.com/nv-morpheus/Morpheus/pull/1436)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Fixes to modular DFP examples and benchmarks ([#1429](https://github.com/nv-morpheus/Morpheus/pull/1429)) [@efajardo-nv](https://github.com/efajardo-nv)
+
+## 📖 Documentation
+
+- Update minimum compute requirements to Volta ([#1594](https://github.com/nv-morpheus/Morpheus/pull/1594)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Fix broken link in getting started with Morpheus doc ([#1494](https://github.com/nv-morpheus/Morpheus/pull/1494)) [@edknv](https://github.com/edknv)
+- Update abp-model-card.md ([#1439](https://github.com/nv-morpheus/Morpheus/pull/1439)) [@drobison00](https://github.com/drobison00)
+- Update gnn-fsi-model-card.md ([#1438](https://github.com/nv-morpheus/Morpheus/pull/1438)) [@drobison00](https://github.com/drobison00)
+- Update phishing-model-card.md ([#1437](https://github.com/nv-morpheus/Morpheus/pull/1437)) [@drobison00](https://github.com/drobison00)
+- Document incompatible mlflow models issue ([#1434](https://github.com/nv-morpheus/Morpheus/pull/1434)) [@dagardner-nv](https://github.com/dagardner-nv)
+
+## 🚀 New Features
+
+- Adding retry logic to the `TritonInferenceStage` to allow recovering from errors ([#1548](https://github.com/nv-morpheus/Morpheus/pull/1548)) [@cwharris](https://github.com/cwharris)
+- Create a base mixin class for ingress &amp; egress stages ([#1473](https://github.com/nv-morpheus/Morpheus/pull/1473)) [@dagardner-nv](https://github.com/dagardner-nv)
+- RAPIDS 24.02 Upgrade ([#1468](https://github.com/nv-morpheus/Morpheus/pull/1468)) [@cwharris](https://github.com/cwharris)
+- Install headers &amp; morpheus-config.cmake ([#1448](https://github.com/nv-morpheus/Morpheus/pull/1448)) [@dagardner-nv](https://github.com/dagardner-nv)
+
+## 🛠️ Improvements
+
+- Updating `nlohman_json` to 3.11 to match MRC ([#1596](https://github.com/nv-morpheus/Morpheus/pull/1596)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- DOCA 2.6 from public repo ([#1588](https://github.com/nv-morpheus/Morpheus/pull/1588)) [@e-ago](https://github.com/e-ago)
+- Support `ControlMessage` for `PreProcessNLPStage` `PreProcessFILStage` `AddScoreStageBase` ([#1573](https://github.com/nv-morpheus/Morpheus/pull/1573)) [@yuchenz427](https://github.com/yuchenz427)
+- Update MLflow in Production DFP example to use Python 3.10 ([#1572](https://github.com/nv-morpheus/Morpheus/pull/1572)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Fix environment yaml paths ([#1551](https://github.com/nv-morpheus/Morpheus/pull/1551)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Add retry logic and proxy support to the NeMo LLM Service ([#1544](https://github.com/nv-morpheus/Morpheus/pull/1544)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- Update to match new MRC function sig for AsyncioRunnable::on_data ([#1541](https://github.com/nv-morpheus/Morpheus/pull/1541)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Expose max_retries parameter to OpenAIChatService &amp; OpenAIChatClient ([#1536](https://github.com/nv-morpheus/Morpheus/pull/1536)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Upgrade `openai` version to 1.13 and `langchain` to version 0.1.9 ([#1529](https://github.com/nv-morpheus/Morpheus/pull/1529)) [@mdemoret-nv](https://github.com/mdemoret-nv)
+- Update ops-bot.yaml ([#1528](https://github.com/nv-morpheus/Morpheus/pull/1528)) [@AyodeAwe](https://github.com/AyodeAwe)
+- Add the ability to attach Tensor objects and timestamps to `ControlMessage` ([#1511](https://github.com/nv-morpheus/Morpheus/pull/1511)) [@drobison00](https://github.com/drobison00)
+- Fix or silence warnings emitted during tests ([#1501](https://github.com/nv-morpheus/Morpheus/pull/1501)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Support ControlMessage output in the C++ impl of DeserializeStage ([#1478](https://github.com/nv-morpheus/Morpheus/pull/1478)) [@dagardner-nv](https://github.com/dagardner-nv)
+- DOCA Source Stage improvements ([#1475](https://github.com/nv-morpheus/Morpheus/pull/1475)) [@e-ago](https://github.com/e-ago)
+- Update copyright headers for 2024 ([#1474](https://github.com/nv-morpheus/Morpheus/pull/1474)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Add conda builds to CI ([#1466](https://github.com/nv-morpheus/Morpheus/pull/1466)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Grafana log monitoring and error alerting example ([#1463](https://github.com/nv-morpheus/Morpheus/pull/1463)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Misc Conda Improvements ([#1462](https://github.com/nv-morpheus/Morpheus/pull/1462)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Simplification of the streaming RAG ingest example to improve usability ([#1454](https://github.com/nv-morpheus/Morpheus/pull/1454)) [@drobison00](https://github.com/drobison00)
+- Replace GPUtil with pynvml for benchmark reports ([#1451](https://github.com/nv-morpheus/Morpheus/pull/1451)) [@efajardo-nv](https://github.com/efajardo-nv)
+- Misc test improvements ([#1447](https://github.com/nv-morpheus/Morpheus/pull/1447)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Add a --manual_seed flag to the CLI ([#1445](https://github.com/nv-morpheus/Morpheus/pull/1445)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Optionally skip ci based on a label in the pr ([#1444](https://github.com/nv-morpheus/Morpheus/pull/1444)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Refactor verification of optional dependencies ([#1443](https://github.com/nv-morpheus/Morpheus/pull/1443)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Use dependencies.yaml as source-of-truth for environment files. ([#1441](https://github.com/nv-morpheus/Morpheus/pull/1441)) [@cwharris](https://github.com/cwharris)
+- Add mocked test &amp; benchmark for LLM agents pipeline ([#1424](https://github.com/nv-morpheus/Morpheus/pull/1424)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Add benchmarks for stand-alone RAG &amp; vdb upload pipelines ([#1421](https://github.com/nv-morpheus/Morpheus/pull/1421)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Add benchmark for completion pipeline ([#1414](https://github.com/nv-morpheus/Morpheus/pull/1414)) [@dagardner-nv](https://github.com/dagardner-nv)
+- Decouple TritonInferenceStage from pipeline mode ([#1402](https://github.com/nv-morpheus/Morpheus/pull/1402)) [@dagardner-nv](https://github.com/dagardner-nv)
+
 # Morpheus 23.11.01 (7 Dec 2023)
 
 ## 🐛 Bug Fixes

diff --git a/ci/iwyu/mappings.imp b/ci/iwyu/mappings.imp
@@ -52,41 +52,10 @@
 
 # rxcpp
 # Hide includes that are exported by <rxcpp/rx.hpp>
-{ "include": [ "\"rx-includes.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-util.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-util.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-predef.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-predef.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-subscription.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-subscription.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rx-observable.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rx-observer.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-observer.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-observer.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rx-operators.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rx-predef.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-scheduler.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-scheduler.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-subscriber.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-subscriber.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rx-subscriber.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-notification.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-notification.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-coordination.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-coordination.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-sources.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-sources.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-subjects.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-subjects.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-operators.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-operators.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-observable.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-observable.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-connectable_observable.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-connectable_observable.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<rxcpp/rx-grouped_observable.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "\"rxcpp/rx-grouped_observable.hpp\"", private, "<rxcpp/rx.hpp>", "public" ] },
-{ "include": [ "<sources/rx-iterate.hpp>", private, "<rxcpp/rx.hpp>", "public" ] },
+{ "include": ["@<rxcpp/.*>", "private", "<rxcpp/rx.hpp>", "public" ] },
+{ "include": ["@\"rxcpp/.*\"", "private", "<rxcpp/rx.hpp>", "public" ] },
+{ "include": ["@<rxcpp/rx-.*>", "private", "<rxcpp/rx.hpp>", "public" ] },
+{ "include": ["@\"rxcpp/rx-.*\"", "private", "<rxcpp/rx.hpp>", "public" ] },
 
 #Triton Client
 { "include": ["\"common.h\"", "private", "<http_client.h>", "public"] },

diff --git a/cmake/package_config/bsd/Configure_bsd.cmake b/cmake/package_config/bsd/Configure_bsd.cmake
@@ -32,7 +32,7 @@ function(morpheus_configure_libbsd)
   if (bsd_ADDED)
     message(STATUS "libbsd was not installed and will be built from source")
 
-    find_package(bsd REQUIRED)
+    find_package(md REQUIRED)
 
     set(bsd_INSTALL_DIR ${bsd_BINARY_DIR}/install)
     file(MAKE_DIRECTORY ${bsd_INSTALL_DIR}/include)
@@ -46,8 +46,6 @@ function(morpheus_configure_libbsd)
 
     cmake_path(GET MD_LIBRARY PARENT_PATH MD_LINK_DIRECTORY)
 
-    message(STATUS "MD_LIBRARY: ${MD_LINK_DIRECTORY}")
-
     # Get the Compiler settings to forward onto autoconf
     set(COMPILER_SETTINGS
       "CXX=${CMAKE_CXX_COMPILER_LAUNCHER} ${CMAKE_CXX_COMPILER}"
@@ -87,7 +85,7 @@ function(morpheus_configure_libbsd)
 
     # Install only the headers
     install(
-      DIRECTORY ${md_INSTALL_DIR}/include
+      DIRECTORY ${bsd_INSTALL_DIR}/include
       TYPE INCLUDE
     )
 
@@ -108,8 +106,6 @@ function(morpheus_configure_libbsd)
 
     add_dependencies(bsd::bsd bsd)
 
-    message(STATUS "bsd_INSTALL_DIR: ${bsd_INSTALL_DIR}")
-
   endif()
 
   LIST(POP_BACK CMAKE_MESSAGE_CONTEXT)

diff --git a/conda/environments/all_cuda-121_arch-x86_64.yaml b/conda/environments/all_cuda-121_arch-x86_64.yaml
@@ -117,6 +117,7 @@ dependencies:
   - --find-links https://data.dgl.ai/wheels-test/repo.html
   - --find-links https://data.dgl.ai/wheels/cu121/repo.html
   - PyMuPDF==1.23.21
+  - databricks-cli < 0.100
   - databricks-connect
   - dgl==2.0.0
   - dglgo

diff --git a/conda/environments/dev_cuda-121_arch-x86_64.yaml b/conda/environments/dev_cuda-121_arch-x86_64.yaml
@@ -10,6 +10,7 @@ channels:
 - pytorch
 dependencies:
 - appdirs
+- automake
 - benchmark=1.8.3
 - boost-cpp=1.84
 - breathe=4.35.0
@@ -43,6 +44,7 @@ dependencies:
 - ipython
 - isort
 - librdkafka>=1.9.2,<1.10.0a0
+- libtool
 - mlflow=2.9.2
 - mrc=24.06
 - myst-parser=0.18.1
@@ -72,6 +74,7 @@ dependencies:
 - pytorch-cuda
 - pytorch=*=*cuda*
 - rapidjson=1.1.0
+- rdma-core>=48
 - requests
 - requests-cache=1.1
 - requests-toolbelt
@@ -92,6 +95,7 @@ dependencies:
 - zlib=1.2.13
 - pip:
   - PyMuPDF==1.23.21
+  - databricks-cli < 0.100
   - databricks-connect
   - milvus==2.3.5
   - pymilvus==2.3.6

diff --git a/conda/environments/examples_cuda-121_arch-x86_64.yaml b/conda/environments/examples_cuda-121_arch-x86_64.yaml
@@ -61,6 +61,7 @@ dependencies:
   - --find-links https://data.dgl.ai/wheels-test/repo.html
   - --find-links https://data.dgl.ai/wheels/cu121/repo.html
   - PyMuPDF==1.23.21
+  - databricks-cli < 0.100
   - databricks-connect
   - dgl==2.0.0
   - dglgo

diff --git a/conda/environments/runtime_cuda-121_arch-x86_64.yaml b/conda/environments/runtime_cuda-121_arch-x86_64.yaml
@@ -37,6 +37,7 @@ dependencies:
 - watchdog=3.0
 - websockets
 - pip:
+  - databricks-cli < 0.100
   - databricks-connect
   - milvus==2.3.5
   - pymilvus==2.3.6

diff --git a/dependencies.yaml b/dependencies.yaml
@@ -52,6 +52,7 @@ files:
       - cudatoolkit
       - data_retrieval
       - development
+      - doca
       - docs
       - python
       - runtime
@@ -270,6 +271,7 @@ dependencies:
           - websockets
           - pip
           - pip:
+            - databricks-cli < 0.100
             - databricks-connect
             - milvus==2.3.5 # update to match pymilvus when available
             - pymilvus==2.3.6

diff --git a/docker/Dockerfile b/docker/Dockerfile
@@ -149,9 +149,6 @@ FROM conda_env as base_extended
 # Add one or more optional dependencies to the base environment
 ARG MORPHEUS_ROOT_HOST
 ARG MORPHEUS_SUPPORT_DOCA="FALSE"
-ARG DOCA_REPO_HOST
-ARG DOCA_VERSION=2.6.0-0.0.1
-
 # Set this environment variable so it auto builds DOCA
 ENV MORPHEUS_SUPPORT_DOCA=${MORPHEUS_SUPPORT_DOCA}
 

diff --git a/docker/build_container.sh b/docker/build_container.sh
@@ -31,7 +31,6 @@ DOCKER_EXTRA_ARGS=${DOCKER_EXTRA_ARGS:-""}
 CUDA_MAJOR_VER=${CUDA_MAJOR_VER:-12}
 CUDA_MINOR_VER=${CUDA_MINOR_VER:-1}
 CUDA_REV_VER=${CUDA_REV_VER:-1}
-DOCA_REPO_HOST=${DOCA_REPO_HOST:-""}
 FROM_IMAGE=${FROM_IMAGE:-"nvidia/cuda"}
 LINUX_DISTRO=${LINUX_DISTRO:-ubuntu}
 LINUX_VER=${LINUX_VER:-22.04}
@@ -47,7 +46,6 @@ DOCKER_ARGS="${DOCKER_ARGS} --target ${DOCKER_TARGET}"
 DOCKER_ARGS="${DOCKER_ARGS} --build-arg CUDA_MAJOR_VER=${CUDA_MAJOR_VER}"
 DOCKER_ARGS="${DOCKER_ARGS} --build-arg CUDA_MINOR_VER=${CUDA_MINOR_VER}"
 DOCKER_ARGS="${DOCKER_ARGS} --build-arg CUDA_REV_VER=${CUDA_REV_VER}"
-DOCKER_ARGS="${DOCKER_ARGS} --build-arg DOCA_REPO_HOST=${DOCA_REPO_HOST}"
 DOCKER_ARGS="${DOCKER_ARGS} --build-arg FROM_IMAGE=${FROM_IMAGE}"
 DOCKER_ARGS="${DOCKER_ARGS} --build-arg LINUX_DISTRO=${LINUX_DISTRO}"
 DOCKER_ARGS="${DOCKER_ARGS} --build-arg LINUX_VER=${LINUX_VER}"
@@ -66,7 +64,6 @@ echo "Building morpheus:${DOCKER_TAG} with args..."
 echo "   CUDA_MAJOR_VER       : ${CUDA_MAJOR_VER}"
 echo "   CUDA_MINOR_VER       : ${CUDA_MINOR_VER}"
 echo "   CUDA_REV_VER         : ${CUDA_REV_VER}"
-echo "   DOCA_REPO_HOST       : ${DOCA_REPO_HOST}"
 echo "   FROM_IMAGE           : ${FROM_IMAGE}"
 echo "   LINUX_DISTRO         : ${LINUX_DISTRO}"
 echo "   LINUX_VER            : ${LINUX_VER}"