Architecture discussion

[vmayoral]

@KeplerC and @jeffi, after having validated a first PoC (#5) I had a closer look at the architecture and spent some time evaluating it. I didn't wanted to spend time on undesired features or change things too much without consensus, so, for now, find below some comments and thoughts that we can consider in the next meeting for upcoming iterations. @KeplerC, if you feel like prototyping any of these ideas and need help, ping me:

Current architecture

(approximate representation)

``` sequenceDiagram par Workstation Note left of User: Step 1 User->>User: "ros2 run fogros2 fogros2" User->>FogROS2 server: Note left of FogROS2 server: Waits until launchfile using FogROSLaunchDescription Note left of User: Step 2 User->>Launch file: "ros2 launch fogros2_examples talker.launch.py" Launch file ->> Launch file: Launch listener Launch file ->> FogROS2 server: FogROSLaunchDescription Note right of FogROS2 server: Triggers cloud "launch" FogROS2 server->>FogROS2 server: Set up VPN FogROS2 server->>Cloud: Bring up instances FogROS2 server->>Cloud: Set up VPN FogROS2 server-->>Cloud: Configure, deploy, launch, ... end Cloud ->> Cloud: Launch talker Cloud ->> Launch file: Interoperate ```

Current proposal is sound. Leaving aside improvements that could be made on the synchronization approach followed and/or the implementation itself, I see the following aspects that could be refined for wider ROS 2 consumption (ordered by relevance):

• ⚠️⚠️⚠️ 1. integrated cloud launch and provisioning: As far as I understand, the real value of FogROS2 is on its cloud launch and provisioning extensions to ROS 2. Right now, this is implemented splitted across 1) a FogROS2 server and 2) a FogROSLaunchDescription, which need to be launched in two steps. This is unnecesary and all of it could be merged into a single step that's implemented purely as part of the Launch extensions (i.e. FogROS2 becomes then a ROS 2 extension for cloud launch and provisioning).
• ⚠️⚠️ 2. reduce code overlap, subclass: The current implementation forks launch and launch_ros (which is 👍 for a first implementation!), but not much is modified to justify a complete fork. I'd instead suggest to subclass Python abstractions and extend the subclasses as appropriate, with the FogROS2 logic. This way we'll avoid copying files unnecesarily.
• ⚠️ 3. expose capabilities at Launch-level: From a user's perspective, right now, the consumption/use of FogROS2 requires architectural-level knowledge if one wants flexibility on the cloud instances/locations/configuration,etc. Ideally, a user should be write her/his launch file and provide these details right there, without additional complexity.
• 👐 4. ros2 fog : I think there's merit in proposing fog tooling to manage and facilitate introspection of cloud infrastructure. Things like ros2 fog connect <instance-name> (to ssh into it), or ros2 fog list (to list current cloud instances) will be of much value.

Proposed architecture

sequenceDiagram
    par Workstation
        Note left of User: Step 1
        User->>Launch file: "ros2 launch fogros2_examples talker.launch.py"
        Launch file->>Launch file: Set up VPN
        Launch file ->> Launch file: Launch listener        
        Launch file ->> Cloud: FogROSLaunchDescription
        Note right of Launch file: Triggers cloud "launch"
        
        Launch file->>Cloud: Bring up instances
        Launch file->>Cloud: Set up VPN
        Launch file-->>Cloud: Configure, deploy, launch, ...
    end
    Cloud ->> Cloud: Launch talker
    Cloud ->> Launch file: Interoperate

Implementing this architecture could be done through the following abstractions:

• FogROSLaunchDescription: LaunchDescription subclass which implements setup (e.g. VPN), and synchronization logic for provisining of cloud instances (the actual provisining does not happen in here, but it does ensure synchronization across provisioning and launch phases).
• fogros2.AWS: Class that abstracts the provisioning of a machine in the Cloud for a specific provider (e.g. AWS). Should create the appropriate artifacts, dump them into a acceleration/cloud subfolder relative to the ROS 2 workspace (so that it can be later used by tooling), and launches the cloud instance. When finished, it should notify FogROSLaunchDescription main thread for synchronization alignment.
• fogros2.CloudNode: A launch_ros.actions.Node subclass that launches the node in the corresponding cloud machine (various options in here, e.g. it can also send the workspace and build it as part of the CloudNode logic, making the FogROSLaunchDescription and fogros2.AWS thinner).

Altogether, this will allow users to do something like this:

def generate_launch_description():
    ld = FogROSLaunchDescription()

    # Provisioning
    machine1 = fogros2.AWS(region="us-west-1", ec2_instance_type="t2.medium")    

    # Nodes
    talker_node = fogros2.CloudNode(
        package="fogros2_examples", executable="talker", output="screen", machine=machine1
    )
    listener_node = Node(
        package="fogros2_examples",
        executable="listener",
        output="screen",
    )

    # Add actions
    ld.add_action(machine1)  # provision machine1
    ld.add_action(talker_node)  # launch talker, cloud
    ld.add_action(listener_node)  # launch listener, workstation
    return ld

[KeplerC]

I like the proposed approach, as it solves a lot of problems in my mind, specifically,

1. how user configures FogROS. In the proposed approach, user only needs to interact with the launch file to configure FogROS.
2. bootstrapping wg0 interface as mentioned in Fixed to_cloud path issue  #5.

I will think about how to implement the proposed approach, as I need to understand ROS2 more

[vmayoral]

@KeplerC, see the following resources which might a useful inspiration for the launch extensions:

• (WIP) design document for remote and multi-machine launching
  • https://github.com/ros2/design/blob/acb998756be1bdae5ce816794335e3950e3a3ed2/articles/153_roslaunch_remote_launch.md
  • Added design document for remote and multi-machine launching ros2/design#297
  • https://github.com/ros2/design/pull/297/files
• Associated source code:
  • https://github.com/pjreed/ssh_machine (this one particularly)
  • https://github.com/pjreed/launch/tree/multi-machine-launching

[KeplerC]

closed this issue since we have already refactored to this new architecture