Ensuring consistent use of the export macro MORPHEUS_EXPORT (#1672)

This PR makes exporting symbols consistent across header files. The docs and examples for the developer_guide is also updated. Closes #1595 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Aser Garcia (https://github.com/aserGarcia) - David Gardner (https://github.com/dagardner-nv) Approvers: - Yuchen Zhang (https://github.com/yuchenz427) - Michael Demoret (https://github.com/mdemoret-nv) URL: #1672
nv-morpheus · May 9, 2024 · 9719d9f · 9719d9f
1 parent bf80d93
commit 9719d9f
Show file tree

Hide file tree

Showing 55 changed files with 257 additions and 297 deletions.
diff --git a/docs/source/developer_guide/guides/3_simple_cpp_stage.md b/docs/source/developer_guide/guides/3_simple_cpp_stage.md
@@ -54,7 +54,7 @@ def supports_cpp_node(self):
     return True
 ```
 
-C++ message object declarations can be found in the header files that are located in the `morpheus/_lib/include/morpheus/messages` directory. For example, the `MessageMeta` class declaration is located in `morpheus/_lib/include/morpheus/messages/meta.hpp`. In code this would be included as:
+C++ message object declarations can be found in the header files that are located in the `morpheus/_lib/include/morpheus/messages` directory. For example, the `MessageMeta` class declaration is located in `morpheus/_lib/include/morpheus/messages/meta.hpp`. Since this code is outside of the morpheus directory it would be included as:
 
 ```cpp
 #include <morpheus/messages/meta.hpp>
@@ -89,6 +89,7 @@ While our Python implementation accepts messages of any type (in the form of Pyt
 To start with, we have our Morpheus and MRC-specific includes:
 
 ```cpp
+#include <morpheus/export.h>
 #include <morpheus/messages/multi.hpp>  // for MultiMessage
 #include <mrc/segment/builder.hpp>      // for Segment Builder
 #include <mrc/segment/object.hpp>       // for Segment Object
@@ -100,12 +101,10 @@ We'll want to define our stage in its own namespace. In this case, we will name
 ```cpp
 namespace morpheus_example {
 
-// pybind11 sets visibility to hidden by default; we want to export our symbols
-#pragma GCC visibility push(default)
-
 using namespace morpheus;
 
-class PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>
+// pybind11 sets visibility to hidden by default; we want to export our symbols
+class MORPHEUS_EXPORT PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>
 {
   public:
     using base_t = mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>;
@@ -119,7 +118,13 @@ class PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage
 };
 ```
 
-We explicitly set the visibility for the stage object in the namespace to default. This is due to a pybind11 requirement for module implementations to default symbol visibility to hidden (`-fvisibility=hidden`). More details about this can be found in the [pybind11 documentation](https://pybind11.readthedocs.io/en/stable/faq.html#someclass-declared-with-greater-visibility-than-the-type-of-its-field-someclass-member-wattributes).
+We explicitly set the visibility for the stage object to default by importing:
+```cpp
+#include <morpheus/export.h>
+```
+Then adding `MORPHEUS_EXPORT`, which is defined in `/build/autogenerated/include/morpheus/export.h` and is compiler agnostic, to the definition of the stage object.  
+This is due to a pybind11 requirement for module implementations to default symbol visibility to hidden (`-fvisibility=hidden`). More details about this can be found in the [pybind11 documentation](https://pybind11.readthedocs.io/en/stable/faq.html#someclass-declared-with-greater-visibility-than-the-type-of-its-field-someclass-member-wattributes).
+Any object, struct, or function that is intended to be exported should have `MORPHEUS_EXPORT` included in the definition.
 
 For simplicity, we defined `base_t` as an alias for our base class type because the definition can be quite long. Our base class type also defines a few additional type aliases for us: `subscribe_fn_t`, `sink_type_t` and `source_type_t`. The `sink_type_t` and `source_type_t` aliases are shortcuts for the sink and source types that this stage will be reading and writing. In this case both the `sink_type_t` and `source_type_t` resolve to `std::shared_ptr<MultiMessage>`. `subscribe_fn_t` (read as "subscribe function type") is an alias for:
 
@@ -134,7 +139,7 @@ All Morpheus C++ stages receive an instance of an MRC Segment Builder and a name
 We will also define an interface proxy object to keep the class definition separated from the Python interface. This isn't strictly required, but it is a convention used internally by Morpheus. Our proxy object will define a static method named `init` which is responsible for constructing a `PassThruStage` instance and returning it wrapped in a `shared_ptr`. There are many common Python types that pybind11 [automatically converts](https://pybind11.readthedocs.io/en/latest/advanced/cast/overview.html#conversion-table) to their associated C++ types. The MRC `Builder` is a C++ object with Python bindings. However there are other instances such as checking for values of `None` where the casting from Python to C++ types is not automatic. The proxy interface object fulfills this need and is used to help insulate Python bindings from internal implementation details.
 
 ```cpp
-struct PassThruStageInterfaceProxy
+struct MORPHEUS_EXPORT PassThruStageInterfaceProxy
 {
     static std::shared_ptr<mrc::segment::Object<PassThruStage>> init(mrc::segment::Builder &builder,
                                                                      const std::string &name);
@@ -146,6 +151,7 @@ struct PassThruStageInterfaceProxy
 ```cpp
 #pragma once
 
+#include <morpheus/export.h>            // for exporting symbols
 #include <morpheus/messages/multi.hpp>  // for MultiMessage
 #include <mrc/segment/builder.hpp>      // for Segment Builder
 #include <mrc/segment/object.hpp>       // for Segment Object
@@ -156,12 +162,10 @@ struct PassThruStageInterfaceProxy
 
 namespace morpheus_example {
 
-// pybind11 sets visibility to hidden by default; we want to export our symbols
-#pragma GCC visibility push(default)
-
 using namespace morpheus;
 
-class PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>
+// pybind11 sets visibility to hidden by default; we want to export our symbols
+class MORPHEUS_EXPORT PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>
 {
   public:
     using base_t = mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>;
@@ -174,13 +178,12 @@ class PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage
     subscribe_fn_t build_operator();
 };
 
-struct PassThruStageInterfaceProxy
+struct MORPHEUS_EXPORT PassThruStageInterfaceProxy
 {
     static std::shared_ptr<mrc::segment::Object<PassThruStage>> init(mrc::segment::Builder& builder,
                                                                      const std::string& name);
 };
 
-#pragma GCC visibility pop
 }  // namespace morpheus_example
 ```
 

diff --git a/docs/source/developer_guide/guides/4_source_cpp_stage.md b/docs/source/developer_guide/guides/4_source_cpp_stage.md
@@ -43,6 +43,7 @@ Our includes:
 ```cpp
 #include <SimpleAmqpClient/SimpleAmqpClient.h>  // for AmqpClient::Channel::ptr_t
 #include <cudf/io/types.hpp>                    // for cudf::io::table_with_metadata
+#include <morpheus/export.h>                    // for exporting symbols
 #include <morpheus/messages/meta.hpp>           // for MessageMeta
 #include <mrc/segment/builder.hpp>              // for Segment Builder
 #include <mrc/segment/object.hpp>               // for Segment Object
@@ -60,13 +61,11 @@ Our namespace and class definition is:
 ```cpp
 namespace morpheus_rabbit {
 
-// pybind11 sets visibility to hidden by default; we want to export our symbols
-#pragma GCC visibility push(default)
-
 using namespace std::literals;
 using namespace morpheus;
 
-class RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
+//pybind11 sets visibility to hidden by default; we want to export our symbols
+class MORPHEUS_EXPORT RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
 {
   public:
     using base_t = mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>;
@@ -115,6 +114,7 @@ Wrapping it all together, our header file should be similar to:
 
 #include <SimpleAmqpClient/SimpleAmqpClient.h>  // for AmqpClient::Channel::ptr_t
 #include <cudf/io/types.hpp>                    // for cudf::io::table_with_metadata
+#include <morpheus/export.h>                    // for exporting symbols
 #include <morpheus/messages/meta.hpp>           // for MessageMeta
 #include <mrc/segment/builder.hpp>              // for Segment Builder
 #include <mrc/segment/object.hpp>               // for Segment Object
@@ -126,13 +126,11 @@ Wrapping it all together, our header file should be similar to:
 
 namespace morpheus_rabbit {
 
-// pybind11 sets visibility to hidden by default; we want to export our symbols
-#pragma GCC visibility push(default)
-
 using namespace std::literals;
 using namespace morpheus;
 
-class RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
+// pybind11 sets visibility to hidden by default; we want to export our symbols
+class MORPHEUS_EXPORT RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
 {
   public:
     using base_t = mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>;
@@ -162,7 +160,7 @@ class RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<Mess
 /**
  * @brief Interface proxy, used to insulate Python bindings.
  */
-struct RabbitMQSourceStageInterfaceProxy
+struct MORPHEUS_EXPORT RabbitMQSourceStageInterfaceProxy
 {
     /**
      * @brief Create and initialize a RabbitMQSourceStage, and return the result.
@@ -175,7 +173,7 @@ struct RabbitMQSourceStageInterfaceProxy
                                                                            const std::string& queue_name,
                                                                            std::chrono::milliseconds poll_interval);
 };
-#pragma GCC visibility pop
+
 }  // namespace morpheus_rabbit
 ```
 

diff --git a/docs/source/developer_guide/guides/8_cpp_modules.md b/docs/source/developer_guide/guides/8_cpp_modules.md
@@ -31,13 +31,14 @@ The following example will create a simple C++ module that passes through the in
 
 ```c++
 #pragma once
+#include <morpheus/export.h>
 #include <mrc/modules/properties/persistent.hpp>
 #include <mrc/modules/segment_modules.hpp>
 #include <nlohmann/json.hpp>
 
 namespace morpheus {
-#pragma GCC visibility push(default)
-class MyTestModule: public mrc::modules::SegmentModule, public mrc::modules::PersistentModule
+
+class MORPHEUS_EXPORT MyTestModule: public mrc::modules::SegmentModule, public mrc::modules::PersistentModule
 {
     using type_t = MyTestModule;
 
@@ -52,7 +53,7 @@ class MyTestModule: public mrc::modules::SegmentModule, public mrc::modules::Per
   private:
     int my_persistent_value{0};
 };
-#pragma GCC visibility pop
+
 }  // namespace morpheus
 ```
 

diff --git a/examples/developer_guide/3_simple_cpp_stage/src/simple_cpp_stage/_lib/pass_thru.hpp b/examples/developer_guide/3_simple_cpp_stage/src/simple_cpp_stage/_lib/pass_thru.hpp
@@ -17,6 +17,7 @@
 
 #pragma once
 
+#include <morpheus/export.h>            // for exporting symbols
 #include <morpheus/messages/multi.hpp>  // for MultiMessage
 #include <mrc/segment/builder.hpp>      // for Segment Builder
 #include <mrc/segment/object.hpp>       // for Segment Object
@@ -32,12 +33,11 @@
 
 namespace morpheus_example {
 
-// pybind11 sets visibility to hidden by default; we want to export our symbols
-#pragma GCC visibility push(default)
-
 using namespace morpheus;
 
-class PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>
+// pybind11 sets visibility to hidden by default; we want to export our symbols
+class MORPHEUS_EXPORT PassThruStage
+  : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>
 {
   public:
     using base_t = mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage>, std::shared_ptr<MultiMessage>>;
@@ -50,11 +50,10 @@ class PassThruStage : public mrc::pymrc::PythonNode<std::shared_ptr<MultiMessage
     subscribe_fn_t build_operator();
 };
 
-struct PassThruStageInterfaceProxy
+struct MORPHEUS_EXPORT PassThruStageInterfaceProxy
 {
     static std::shared_ptr<mrc::segment::Object<PassThruStage>> init(mrc::segment::Builder& builder,
                                                                      const std::string& name);
 };
 
-#pragma GCC visibility pop
 }  // namespace morpheus_example
diff --git a/...ples/developer_guide/4_rabbitmq_cpp_stage/src/rabbitmq_cpp_stage/_lib/rabbitmq_source.hpp b/...ples/developer_guide/4_rabbitmq_cpp_stage/src/rabbitmq_cpp_stage/_lib/rabbitmq_source.hpp
@@ -19,6 +19,7 @@
 
 #include <SimpleAmqpClient/Channel.h>
 #include <cudf/io/types.hpp>           // for cudf::io::table_with_metadata
+#include <morpheus/export.h>           // for exporting symbols
 #include <morpheus/messages/meta.hpp>  // for MessageMeta
 #include <mrc/segment/builder.hpp>     // for Segment Builder
 #include <mrc/segment/object.hpp>      // for Segment Object
@@ -36,13 +37,11 @@
 
 namespace morpheus_rabbit {
 
-// pybind11 sets visibility to hidden by default; we want to export our symbols
-#pragma GCC visibility push(default)
-
 using namespace std::literals;
 using namespace morpheus;
 
-class RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
+// pybind11 sets visibility to hidden by default; we want to export our symbols
+class MORPHEUS_EXPORT RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
 {
   public:
     using base_t = mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>;
@@ -72,7 +71,7 @@ class RabbitMQSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<Mess
 /**
  * @brief Interface proxy, used to insulate Python bindings.
  */
-struct RabbitMQSourceStageInterfaceProxy
+struct MORPHEUS_EXPORT RabbitMQSourceStageInterfaceProxy
 {
     /**
      * @brief Create and initialize a RabbitMQSourceStage, and return the result.
@@ -85,5 +84,5 @@ struct RabbitMQSourceStageInterfaceProxy
                                                                            const std::string& queue_name,
                                                                            std::chrono::milliseconds poll_interval);
 };
-#pragma GCC visibility pop
+
 }  // namespace morpheus_rabbit
diff --git a/morpheus/_lib/doca/include/morpheus/doca/doca_source.hpp b/morpheus/_lib/doca/include/morpheus/doca/doca_source.hpp
@@ -18,6 +18,7 @@
 #pragma once
 
 #include "morpheus/doca/common.hpp"
+#include "morpheus/export.h"
 #include "morpheus/messages/meta.hpp"
 
 #include <mrc/segment/builder.hpp>
@@ -35,14 +36,12 @@ struct DocaRxPipe;
 struct DocaSemaphore;
 }  // namespace doca
 
-#pragma GCC visibility push(default)
-
 /**
  * @brief Receives a firehose of raw packets from a GPUNetIO-enabled device.
  *
  * Tested only on ConnectX 6-Dx with a single GPU on the same NUMA node running firmware 24.35.2000
  */
-class DocaSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
+class MORPHEUS_EXPORT DocaSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>
 {
   public:
     using base_t = mrc::pymrc::PythonSource<std::shared_ptr<MessageMeta>>;
@@ -68,7 +67,7 @@ class DocaSourceStage : public mrc::pymrc::PythonSource<std::shared_ptr<MessageM
 /**
  * @brief Interface proxy, used to insulate python bindings.
  */
-struct DocaSourceStageInterfaceProxy
+struct MORPHEUS_EXPORT DocaSourceStageInterfaceProxy
 {
     /**
      * @brief Create and initialize a DocaSourceStage, and return the result.
@@ -80,6 +79,4 @@ struct DocaSourceStageInterfaceProxy
                                                                        std::string const& traffic_type);
 };
 
-#pragma GCC visibility pop
-
 }  // namespace morpheus
diff --git a/morpheus/_lib/include/morpheus/io/data_loader.hpp b/morpheus/_lib/include/morpheus/io/data_loader.hpp
@@ -17,6 +17,7 @@
 
 #pragma once
 
+#include "morpheus/export.h"
 #include "morpheus/messages/control.hpp"
 #include "morpheus/messages/meta.hpp"
 
@@ -28,16 +29,14 @@
 
 namespace morpheus {
 
-#pragma GCC visibility push(default)
-
 /**
  * @brief Abstract class for loading data from a source.
  *
  * This class defines two virtual methods for loading and returning control messages, as well as a
  * protected method for accessing the configuration information. It can be extended to implement
  * specific loading logic for different types of data sources.
  */
-class Loader
+class MORPHEUS_EXPORT Loader
 {
   public:
     /**
@@ -89,7 +88,7 @@ class Loader
  * loading data using these objects. It also defines a method for loading control messages from data
  * sources using the registered loaders.
  */
-class DataLoader
+class MORPHEUS_EXPORT DataLoader
 {
   public:
     /**
@@ -136,5 +135,4 @@ class DataLoader
     nlohmann::json m_config;
 };
 
-#pragma GCC visibility pop
 }  // namespace morpheus
diff --git a/morpheus/_lib/include/morpheus/io/data_loader_registry.hpp b/morpheus/_lib/include/morpheus/io/data_loader_registry.hpp
@@ -17,6 +17,7 @@
 
 #pragma once
 
+#include "morpheus/export.h"
 #include "morpheus/io/data_loader.hpp"
 #include "morpheus/messages/control.hpp"
 #include "morpheus/objects/factory_registry.hpp"
@@ -28,13 +29,12 @@
 #include <string>
 
 namespace morpheus {
-#pragma GCC visibility push(default)
 
-extern template class FactoryRegistry<Loader>;
+extern template class MORPHEUS_EXPORT FactoryRegistry<Loader>;
 
 using LoaderRegistry = FactoryRegistry<Loader>;  // NOLINT
 
-struct LoaderRegistryProxy
+struct MORPHEUS_EXPORT LoaderRegistryProxy
 {
     static void register_proxy_factory_fn(
         const std::string& name,
@@ -45,5 +45,4 @@ struct LoaderRegistryProxy
     static void register_factory_cleanup_fn(const std::string& name);
 };
 
-#pragma GCC visibility pop
 }  // namespace morpheus