Skip to content

Commit

Permalink
first draft
Browse files Browse the repository at this point in the history
  • Loading branch information
@AnthonyCvn
AnthonyCvn committed Mar 13, 2024
1 parent 0cf8c10 commit 77b9eaf
Show file tree
Hide file tree
Showing 3 changed files with 274 additions and 0 deletions.
79 changes: 79 additions & 0 deletions blog/2024-03-15-ROS-integration/img/ros-reductstore-example.drawio
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
<mxfile host="app.diagrams.net" modified="2024-03-13T19:05:26.528Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" etag="oHQXwmuDIeNn-u-KG8G6" version="23.1.4" type="google">
<diagram name="Page-1" id="hGl6-yu-bIhh6hzXzWU5">
<mxGraphModel grid="1" page="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="jcethup4zuZOfwS8kyeW-5" value="" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="70" y="160.63" width="350" height="161.25" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-26" value="ReductStore Bucket" style="text;html=1;align=left;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontStyle=1" vertex="1" parent="1">
<mxGeometry x="87.5" y="133.63" width="140" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-28" value="" style="group;strokeColor=default;container=0;" connectable="0" vertex="1" parent="1">
<mxGeometry x="87.5" y="192" width="315" height="96.88" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-29" value="&lt;p data-pm-slice=&quot;1 1 []&quot; style=&quot;font-size: 12px;&quot;&gt;/image_raw&lt;/p&gt;" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontSize=12;" vertex="1" parent="1">
<mxGeometry x="560" y="206.25" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-30" value="&lt;p data-pm-slice=&quot;1 1 []&quot;&gt;Image Entry&lt;/p&gt;" style="text;whiteSpace=wrap;html=1;align=left;" vertex="1" parent="1">
<mxGeometry x="87.5" y="155" width="170" height="40" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-46" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="jcethup4zuZOfwS8kyeW-33" target="jcethup4zuZOfwS8kyeW-45">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-33" value="" style="sketch=0;pointerEvents=1;shadow=0;dashed=0;html=1;strokeColor=none;fillColor=#505050;labelPosition=center;verticalLabelPosition=bottom;verticalAlign=top;outlineConnect=0;align=center;shape=mxgraph.office.devices.video_camera;" vertex="1" parent="1">
<mxGeometry x="650" y="228.76" width="52" height="25" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-12" value="t1" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" vertex="1" parent="1">
<mxGeometry x="352.5" y="259.38" width="30" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-15" value="t2" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" vertex="1" parent="1">
<mxGeometry x="283" y="259.38" width="30" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-18" value="t3" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" vertex="1" parent="1">
<mxGeometry x="214" y="259.38" width="30" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-21" value="t4" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" vertex="1" parent="1">
<mxGeometry x="144.95" y="259.38" width="30" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-22" value="..." style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" vertex="1" parent="1">
<mxGeometry x="100" y="226.88" width="30" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-38" value="" style="image;aspect=fixed;html=1;points=[];align=center;fontSize=12;image=img/lib/azure2/other/Defender_Industrial_Robot.svg;" vertex="1" parent="1">
<mxGeometry x="740" y="206.25" width="71.36" height="95" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-39" value="" style="image;aspect=fixed;html=1;points=[];align=center;fontSize=12;image=img/lib/azure2/other/Defender_Industrial_Robot.svg;imageBorder=default;" vertex="1" parent="1">
<mxGeometry x="352.5" y="221.88" width="30.05" height="40" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-40" value="" style="image;aspect=fixed;html=1;points=[];align=center;fontSize=12;image=img/lib/azure2/other/Defender_Industrial_Robot.svg;imageBorder=default;" vertex="1" parent="1">
<mxGeometry x="283" y="221.88" width="30.05" height="40" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-41" value="" style="image;aspect=fixed;html=1;points=[];align=center;fontSize=12;image=img/lib/azure2/other/Defender_Industrial_Robot.svg;imageBorder=default;" vertex="1" parent="1">
<mxGeometry x="214" y="221.88" width="30.05" height="40" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-42" value="" style="image;aspect=fixed;html=1;points=[];align=center;fontSize=12;image=img/lib/azure2/other/Defender_Industrial_Robot.svg;imageBorder=default;" vertex="1" parent="1">
<mxGeometry x="144.89999999999998" y="221.88" width="30.05" height="40" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-47" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="jcethup4zuZOfwS8kyeW-45" target="jcethup4zuZOfwS8kyeW-5">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-45" value="ROS 2&lt;br&gt;Node" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
<mxGeometry x="470" y="211.26" width="70" height="60" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-53" value="Metadata" style="text;html=1;align=center;verticalAlign=top;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="124.95" y="199.88" width="70" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-54" value="Metadata" style="text;html=1;align=center;verticalAlign=top;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="194" y="199.88" width="70" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-55" value="Metadata" style="text;html=1;align=center;verticalAlign=top;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="263" y="199.88" width="70" height="30" as="geometry" />
</mxCell>
<mxCell id="jcethup4zuZOfwS8kyeW-56" value="Metadata" style="text;html=1;align=center;verticalAlign=top;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontSize=10;" vertex="1" parent="1">
<mxGeometry x="332.5" y="199.88" width="70" height="30" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>
Binary file added blog/2024-03-15-ROS-integration/img/ros-reductstore-example.webp
View file
Binary file not shown.
195 changes: 195 additions & 0 deletions blog/2024-03-15-ROS-integration/index.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
---
title: "Optimal Image Storage Solutions for ROS-Based Computer Vision"
description: "This blog post will guide you through setting up ROS 2 with ReductStore—a time-series database optimized for unstructured data. More specifically, we'll go through a practical example to show how to capture and store raw camera images from a ROS topic in ReductStore."
authors: anthony
tags: [tutorials, ros, computer-vision]
slug: tutorials/ros/optimal-image-storage-solutions-for-ros-based-computer-vision
date: 2024-03-15
image: ./img/ros-reductstore-example.webp
---

![ROS with ReductStore](./img/ros-reductstore-example.webp)

The Robot Operating System (ROS) stands as a versatile framework for developing sophisticated robotic applications with various sensors, including cameras. These cameras are relatively inexpensive and widely used as they can provide a wealth of information about the robot's environment.

Processing camera output with computer vision requires efficient solutions to handle massive amounts of data in real time. ROS 2 is designed with this in mind, but it is a communication middleware and does not provide a built-in solution for storing and managing large volumes of image data.

Addressing this challenge, this blog post will guide you through setting up ROS 2 with ReductStore—a time-series database for unstructured data optimized for edge computing, ensuring your robotic applications can process and store camera outputs effectively.

<!--truncate-->

## ROS 2 Distributions
To install ROS 2, you'll need to select a distribution that aligns with your project requirements and system compatibility. As of now, multiple distributions are available for ROS 2 [**here**](<https://docs.ros.org/en/foxy/Releases.html>).

Among them, two distributions are currently actively maintained :

- **Iron Irwini** (Release date: May 23rd, 2023; End of life: November 2024)

- **Humble Hawksbill** (Release date: May 23rd, 2022; End of life: May 2027)


<!-- -->

We recommend the "Humble Hawksbill" distribution for its long-term support until May 2027. This is the eighth release of ROS 2 and caters to various platforms including Ubuntu Linux, Windows, RHEL (Red Hat Enterprise Linux), or macOS.

To install the Humble Hawksbill distribution on your preferred operating system, follow the instructions provided in their [**installation guide**](<https://docs.ros.org/en/humble/Installation.html>). The installation process involves a series of commands specific to each platform and may require certain prerequisites like Python or C++ dependencies depending on your OS.

## Understanding the Advantages of Using ReductStore with ROS
Integrating ReductStore with ROS provides many benefits for robotic applications.

- **Best performance**: ReductStore's time-series design is tailored to the sequential nature of robotic applications and optimized for unstructured data, such as images, audio, and sensor readings.

- **Real-time data management**: ReductStore provides real-time First In First Out (FIFO) quota management, which is critical for maintaining a balance between storage space and continuous data flow on edge devices.

- **Metadata association**: It supports the association of labels or AI-generated analytics results directly with each stored image, enriching the dataset and facilitating subsequent processing or machine learning tasks.

- **Replication**: ReductStore offers the ability to replicate data across a distributed network based on user-defined filters, to copy important data to a central server or cloud storage for further analysis.


<!-- -->

## Example to Capture and Store Raw Camera Images
To capture and store raw camera images in a ROS-based system, you need to create a node that subscribes to the image topic, processes the image, and stores it in ReductStore.

You can use the [**reduct-ros-example**](<https://github.com/reductstore/reduct-ros-example>) as a guideline. This example demonstrates how to subscribe to a ROS topic, such as "/image\_raw", serialize the message in binary format, and store it in a bucket called "ros-bucket" under the entry "image-raw".

### Setting Up Your Node: Integrating ReductStore with a ROS Client Instance
To set up your node for integrating ReductStore with a ROS client instance, you will need to create a custom ROS 2 Node that listens to image messages and uses ReductStore client for data storage.

Below is an example demonstrating this integration within a Python class:

```python
import asyncio
from reduct import Client, Bucket
from sensor_msgs.msg import Image
from rclpy.node import Node

class ImageListener(Node):
"""Node for listening to image messages and storing them in ReductStore."""
def __init__(self, reduct_client: Client, loop: asyncio.AbstractEventLoop) -> None:
"""
Initialize the image listener node.
:param reduct_client: Client instance for interacting with ReductStore.
:param loop: The asyncio event loop.
"""
super().__init__("image_listener")
self.reduct_client: Client = reduct_client
self.loop: asyncio.AbstractEventLoop = loop
self.bucket: Bucket = None
self.subscription = self.create_subscription(
Image, "/image_raw", self.image_callback, 10
)
```

In this example `ImageListener` is a subclass of `Node`, which is part of ROS 2's client library (`rclpy`). It sets up a subscription to listen for incoming images from the `/image_raw` topic. When an image message is received by the node via `self.subscription`, it triggers `image_callback`.

In this callback function, each received frame is stored in the designated bucket in ReductStore using Python's built-in `asyncio` module, more on this later.

### Initialize a New ReductStore Bucket
This process involves setting up configuration parameters such as the bucket name and its storage quota settings.

In our example, we create a bucket named "ros-bucket" with a FIFO quota type, which is suitable for making sure that the disk doesn't run out of space.

The `exist_ok` parameter ensures that if the bucket already exists, it won't raise an exception but rather reuse the existing one.

Here's how you can define this initialization within our Python class:

```python
from reduct_py import BucketSettings, QuotaType

class ImageListener(Node):
# ... [other parts of ImageListener class] ...

async def init_bucket(self) -> None:
"""Asynchronously initialize the Reduct bucket for storing images."""
self.get_logger().info("Initializing Reduct bucket")
self.bucket = await self.reduct_client.create_bucket(
"ros-bucket",
BucketSettings(quota_type=QuotaType.FIFO, quota_size=1_000_000_000),
exist_ok=True,
)
```

This code snippet should be called within your existing `ImageListener` class during the node's initialization or before storing the first image. This ensures that the storage bucket is ready.

### Handling Images in Callbacks
When an image message from a ROS topic is received, it triggers the `image_callback` method. This method's role is to serialize the image data and organize its storage without blocking the main thread.

Serialization converts the ROS message format into a simpler binary representation that can be stored or transmitted more easily.

Here's an example code snippet demonstrating how to handle images in callbacks for storing them in ReductStore:

```python
from rclpy.serialization import serialize_message

class ImageListener(Node):
# ... [previous parts of ImageListener class] ...

def image_callback(self, msg: Image) -> None:
"""
Handle incoming image messages by scheduling storage.
This callback is triggered by ROS message processing. It schedules
the image storage coroutine to be executed in the asyncio event loop.
"""
self.get_logger().info("Received an image")
timestamp = self.get_timestamp(msg)
image_data = serialize_message(msg)
asyncio.run_coroutine_threadsafe(
self.store_data(timestamp, image_data), self.loop
)
```

In this context, `serialize_message` is used to convert the `Image` message object into a byte stream that can subsequently be passed along for storage.

Following serialization, an asynchronous coroutine (`store_data`) is scheduled on the event loop (`self.loop`) using `asyncio.run_coroutine_threadsafe`.

This function is particularly useful for integrating asynchronous operations within primarily synchronous ROS 2 callback handlers, ensuring that the processing doesn't block the executor.

### Store Images in a ReductStore Bucket Entry
The `store_data` method is designed to receive timestamped image data and write it into a specific Reduct bucket.

Here's an explanation of how this method works:

- **Timestamp Parameter**: The `timestamp` argument ensures that each piece of data can be associated with the exact time it was captured.

- **Data Serialization**: The `data` parameter expects a byte stream, which implies that image messages from ROS need to be serialized before being passed to this function.


<!-- -->

With these considerations in mind, here's how you can define the `store_data` method within the `ImageListener` class:

```python
class ImageListener(Node):
# ... [previous parts of ImageListener class] ...

async def store_data(self, timestamp: int, data: bytes) -> None:
"""
Store unstructured data in the Reduct bucket.
:param timestamp: The timestamp for the data.
:param data: The serialized data.
"""
if not self.bucket:
await self.init_bucket()
readable_timestamp = self.format_timestamp(timestamp)
self.get_logger().info(f"Storing data at {readable_timestamp}")
await self.bucket.write("image-raw", data, timestamp)
```

When the `store_data` method is called, it first checks if the bucket has been initialized. If not, it calls the `init_bucket` method to create the bucket. Then, it writes the serialized data to the bucket entry "image-raw" with the provided timestamp.

As you can see, the `store_data` method is designed to be non-blocking, ensuring that the main thread can continue processing other tasks without waiting for the data to be stored.

<!-- -->

## Conclusion
In conclusion, this blog post has demonstrated how to capture and store raw camera images from a ROS topic in ReductStore. The provided code snippets serve as a practical guide for setting up such a system, highlighting the importance of non-blocking operations and proper serialization to maintain system performance.

Using [**ReductStore**](<https://reduct.store/>) is straightforward to deploy and provides a robust solution for managing large volumes of image data in real time. The FIFO quota management, metadata association, and replication features make it an ideal choice for managing unstructured data in ROS-based computer vision applications.

---

I hope this tutorial has been helpful. If you have any questions or feedback, don’t hesitate to reach out in [**Discord**](https://discord.gg/8wPtPGJYsn) or by opening a discussion on [**GitHub**](https://github.com/reductstore/reductstore/discussions).

0 comments on commit 77b9eaf

Please sign in to comment.