From c1b30a171f011c897f81e389dcc302f85d61324f Mon Sep 17 00:00:00 2001 From: David Gardner Date: Fri, 26 Jul 2024 10:51:40 -0700 Subject: [PATCH] Apply PR suggestions from Mike McKiernan --- docs/source/developer_guide/architecture.md | 6 +++--- .../guides/10_modular_pipeline_digital_fingerprinting.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/developer_guide/architecture.md b/docs/source/developer_guide/architecture.md index 6b9be8d5b4..6070304b4f 100644 --- a/docs/source/developer_guide/architecture.md +++ b/docs/source/developer_guide/architecture.md @@ -63,11 +63,11 @@ The stage identifier is a unique string used in both logging and creating the st ### Type Inference -To perform work, each stage needs to know what type of data it will be operating on. Since Morpheus can pass any type of data from stage to stage, the pipeline must ensure compatible types at every edge connection between stages. This process is called stage type inference and is performed during the pipeline build phase. +To perform work, each stage needs to know what type of data it will be operating on. Morpheus can pass any type of data from stage to stage, the pipeline must ensure compatible types at every edge connection between stages. This process is called stage type inference and is performed during the pipeline build phase. -Stage type inference is necessary because the output type of some stages may depend on the output type of the previous stage. For example, consider a simple pass through stage that passes the input message to the next stage unmodified. If our pass through stage is preceded by a stage generating a string, its output type will be a string. Instead, if it's preceded by a stage generating an integer, its output type will be an integer. +Stage type inference is necessary because the output type of some stages can depend on the output type of the previous stage. For example, consider a simple pass through stage that passes the input message to the next stage unmodified. If our pass through stage is preceded by a stage generating a string, its output type will be a string. Instead, if it's preceded by a stage generating an integer, its output type will be an integer. -Due to the dynamic nature of the output type of a stage, stages must specify a type inference function that accepts an input type and returns the output type. Starting at the source stages, the pipeline will use this function to determine the output type of the source stages. This result will then be passed to the type inference function of the next stage, until the input and output types of every stage in the pipeline have been determined. +Due to the dynamic nature of the output type of a stage, stages must specify a type inference function that accepts an input type and returns the output type. Starting at the source stages, the pipeline will use this function to determine the output type of the source stages. This result is then be passed to the type inference function of the next stage, until the input and output types of every stage in the pipeline have been determined. After the build phase, the output types of stages cannot be changed. Returning a different type than specified during the build phase will result in undefined behavior. diff --git a/docs/source/developer_guide/guides/10_modular_pipeline_digital_fingerprinting.md b/docs/source/developer_guide/guides/10_modular_pipeline_digital_fingerprinting.md index d92b3120e4..968119b08f 100644 --- a/docs/source/developer_guide/guides/10_modular_pipeline_digital_fingerprinting.md +++ b/docs/source/developer_guide/guides/10_modular_pipeline_digital_fingerprinting.md @@ -164,7 +164,7 @@ There are a number of modules that are used in both the training and inference p Source: `examples/digital_fingerprinting/production/morpheus/dfp/modules/dfp_preproc.py` -The `dfp_preproc` module is a functional component within the Morpheus framework that combines multiple data filtering and processing pipeline modules related to inference and training. This module simplifies the pipeline by consolidating various modules into a single, cohesive unit. The `dfp_preproc` module allows for the configuration of parameters such as the cache directory, timestamp column name, pre-filter options, batching options, user splitting options, and supported data loaders for various file types. +The `dfp_preproc` module is a functional component within the Morpheus framework that combines multiple data filtering and processing pipeline modules related to inference and training. This module simplifies the pipeline by consolidating various modules into a single, cohesive unit. The `dfp_preproc` module supports configuration parameters such as the cache directory, timestamp column name, pre-filter options, batching options, user splitting options, and supported data loaders for various file types. The module itself consists of a series of chained sub-modules, which are connected in a logical sequence: