REP 2003: Sensor Data and Map QoS Settings

[SteveMacenski]

This is a proposal to formalize the QoS settings for sensor and map data with the goal of making generally compliant drivers and state estimation software

[gavanderhoorn]

Could this be reformatted to one-sentence-per-line?

[SteveMacenski]

Sure, one moment

Edit: done

[gavanderhoorn]

Apologies for the many comments, I should've submitted a review instead.

High-level question: why is "map data" singled out in the REP? Could it be made an example of data that is not sensor data and make the accompanying text more generic?

Or is this specifically about map and sensor data in the context of navigation?

[SteveMacenski]

Apologies for the many comments, I should've submitted a review instead.

Makes no difference to me. All good comments.

why is "map data" singled out in the REP?

So map data is what got me to write this in the first place and the sensor data was an extension of the same logic.

We have a problem right now with a bunch of different consumers or producers of maps. AMCL, map server, cartographer, karto, gmapping, slam toolbox, ... the list goes on. While they all use the right messages, the QoS for a map should be strictly enforced so its ROS2-settings latched (as defined in the REP, phrasing it that way for simplification) since non-ROS2-settings latched is not compatible to normal consumers, its become a hassle on the Navigation stack maintainers to enforce compliance against other map servers/clients. Otherwise we have a constant on-set of tickets being filed that "XYZ-SLAM doesnt work with nav stack, fix it for me".

It got me thinking "well ok, who else has this problem" and sensor data came to mind. Since just like all the SLAM implementations, any sensor manufacturer can come out of the woodwork and slap a ROS2 driver down with non-compliant QoS settings. Since our friends at OR were so kind as to include a SensorDataQoS profile, I think its high time to formalize that this is what should be used.

I intentionally included an "out" that this only covers map providers and consumers in the context of maps for globally consistent representations (GCR) which doesn't cover things like costmaps or partial map updates. I think it would be inappropriate to say that for this particular set of data, you must use this standard. It's highly dependent on application, but for GCR, you're always going to want the best most recent data available to a new client. The same is true with data providers from sensors. It wouldn't be appropriate to say across an entire complex pipeline of data processing that it makes sense to use the SensorDataQoS profile, since once you've started processing, dropping of messages might be really bad. However that profile for sensor data coming out of the sensor itself, this is of desirable generalized interest. A GCR map could be thought of as another form of data, however trying to create some analog between sensor drivers and map (drivers?) seems like more words and opportunities for confusion than just treating them are relatively unique cases (with also unique prescribed QoS settings).

Or is this specifically about map and sensor data in the context of navigation?

These are the 2 areas I see those problems in. I expect in drafting this REP that someone comments on this with 1 or 2 more sets of messages we want to include but these are the ones I know of. This isn't relevant for just navigation, but also anything in sensor processing (Autoware, Moveit, perception pipelines) and mapping (primarily navigation and Autoware, but conceptually any field with an interest in some globally consistent representation of their environment, not restricted to occupancy grids). I thought about including things like geometry_msgs/Twist used for /cmd_vel in navigation, but that's too specific and the Twist message has other value to other users so prescribing a standard to them would be inappropriate. Though making a contextual standard for CustomMsg around when its used for sending control goals (Twist, Ackermann, control_msgs/*) could be a possible extension open for discussion. Though if included, someone would have to throw out their suggestion for the QoS to use, I don't necessarily have an opinion that I think generalizes well.

[gavanderhoorn]

We have a problem right now with a bunch of different consumers or producers of maps. AMCL, map server, cartographer, karto, gmapping, slam toolbox, ... the list goes on. While they all use the right messages, the QoS for a map should be strictly enforced so its ROS2-settings latched (as defined in the REP, phrasing it that way for simplification) since non-ROS2-settings latched is not compatible to normal consumers, its become a hassle on the Navigation stack maintainers to enforce compliance against other map servers/clients. Otherwise we have a constant on-set of tickets being filed that "XYZ-SLAM doesnt work with nav stack, fix it for me".

This reads to me as a justification for a REP on the Nav2 ROS API, which would then include the various QoS profiles associated with / configured for those topics.

I'm not saying there shouldn't be something that suggests best-practices when working with sensor data (on the topic of QoS settings), but the mix of sensor data and maps seems slightly odd to me.

[SteveMacenski]

I don't think this is navigation focused. This is message focused. We have a number of REPs for sensor data specifications and conventions (depth images, IMU, lidar, etc) for ROS1 but now that QoS is a new concept in ROS2 in the sense that without a specification and convention we limit re-use. I think of this as an extension of those REPs but grouped together because they all share the same QoS settings of SensorDataQos.

I'd be more than happy to separate this into 2 REPs, one around sensor data and the other around GCRs if that's what folks thinks makes most sense

[mikaelarguedas]

On the user / integrator side, one would expect that nodes (including drivers) provide the widest range of QoS allowing the end user / integrator to restrict them based on their application.
Some sensors have low bandwidth and may carry information one wouldn't want to miss (thinking a cliff sensor or a bumper for example). In this case one would expect the user to be able to request to subscribe to this sensor with a reliable QoS.
If the driver node offers reliable, it's up to the user to decide to subscribe using reliable or best effort. If the driver offers only best effort, the user cant use reliable and has to modify / configure or recompile the driver node to be able to subscribe to it.

Therefore I would suggest putting a recommendation for users to subscribe to sensor data using best effort (and all example code subscribing to sensors to do the same) but not add any constraint on the driver node's code as the default is the most permissive solution.

[SteveMacenski]

That’s an interesting suggestion. Are those QoS’s compatible with each other? A bit of the rationale for this was around accidentally creating QoS’s that were not compatible with each other so by creating a standard we stop accident mess-ups and “why isnt my [sensor] working?” questions.

I think to do something like that, we’d have to specify explicit settings for the publisher and an explicit range of subscribers of sensor. Can you create some table of these values?

On the other hand, I’m hesitant though that since we have a SensorDataQoS profile, that we should use it. Your suggestion makes sense, but for the vast majority of sensors that profile makes most sense. You bring up bump and cliff sensors and as far as I know those arent included as standard messages so theyre a bit of an edge case for which there isnt today a standard interface.

[mikaelarguedas]

Are those QoS’s compatible with each other?

Yes, the only combination of RELIABILITY that is not compatible is the subscriber requesting reliable and the publisher offering best_effort. Everything else is compatible. In the end the subscriber prescribes which one is used.

This is why the generic ROS 2 tools default to publish as reliable and subscribe as best effort to cover the widest range of compatible QoS.

e.g. ros2 topic pub and echo

echo uses sensor_data:

https://github.com/ros2/ros2cli/blob/6b0507e59a21eee4943f4468bea3cd474cc491c2/ros2topic/ros2topic/verb/echo.py#L62-L63

pub uses reliable (system_default):

https://github.com/ros2/ros2cli/blob/6b0507e59a21eee4943f4468bea3cd474cc491c2/ros2topic/ros2topic/verb/pub.py#L69-L70

---

I think to do something like that, we’d have to specify explicit settings for the publisher and an explicit range of subscribers of sensor. Can you create some table of these values?

I am not sure I got this bit, could you clarify what type of information such table would contain?
Something similar to the tables at https://index.ros.org/doc/ros2/Concepts/About-Quality-of-Service-Settings/#qos-compatibilities maybe?

---

we stop accident mess-ups and “why isnt my [sensor] working?”

This is totally understandable. Ironically I had the opposite issue with e.g turtlebot3 laser scans or AMCL particle clouds being published as sensor_data but the tools like RViz expecting them to be reliable (which is not compatible) making me say “why isnt my [sensor/localization] working?” ros-navigation/navigation2#1003

(which is not an argument against this proposal :) RViz should probably be updated to default to unreliable for most plugins anyway)

---

On the other hand, I’m hesitant though that since we have a SensorDataQoS profile, that we should use it. Your suggestion makes sense, but for the vast majority of sensors that profile makes most sense.

That makes sense, I just wanted to raise awareness about the fact that this is restricting users more than the default behavior (reliable publishers) as it seems to be an important thing to consider for such a REP.

I can easily picture the number of incoming ROS answers questions with "I created a subscriber with just the default settings and I can't subscribe to my image but if I do ros2 topic echo I receive it".

That being said I like the idea of a reference to point everyone at but wanted to point out the trade-offs being made in this proposal. THey appear to me to be the following:
👍 reference to tell everyone how to interact with sensor data on both publisher and subscriber side
👍 tools could all be updated to match the REP providing a consistent ecosystem
👎 users subscribing with default settings will not be able to receive messages
👎 users needing reliable sensors need to modify the sensor drivers themselves

[SteveMacenski]

users subscribing with default settings will not be able to receive messages

Is that true? I'm pretty sure that ros2 topic echo will work with sensor data QoS settings. That would be wild if that didnt work...

users needing reliable sensors need to modify the sensor drivers themselves

That would be true, but I think by in large for the sensor driver-class of data streams, the specifically curated SensorDataQoS is probably the right fit. Other than cliff / bump sensors (which I think of as safety mechanisms, not external sensing the way a camera or laser is, but perhaps that's semantics of my mind), I struggle to come up with any other examples of where SensorDataQoS isn't the right option. When looking over the list of sensor_msgs/... they all seem like this would be most appropriate and that's what I wrote this to target. Not to say that this can't or shouldn't change in light of that argument, but I suppose do you think making something more general and officially saying not to use SensorDataQoS for sensor drivers is the right way to go?

I just wanted to raise awareness about the fact that this is restricting users

Absolutely, this is a good discussion. I still think from my response in the paragraph above that SensorDataQoS still makes most sense, but frankly I don't care that much as long as we make some standard. If people want a more general and complex REP surrounding this to prohibit by lower level QoS settings inoperability, that's fine with me but I may then ask for help in drafting what more complex requirements would be. SensorDataQoS makes it easy and at least from what I can tell is purpose driven for what I was looking to standardize.

[SteveMacenski]

Hi Chris,

Glad that over time this was revived! I reviewed the open comments and reread the document so this is what I would propose:

• L33: explain that this is not exhaustive list, but a set of examples of sensors it applies to (but also try to be as exhaustive as possible)
• Add in an explicit mention that this REP fixes the "why isn't [my sensor] working?!" issues from QoS
• Change the description of the Sensor policy to have publishers of sensor data use the Reliable settings -- Do you think the new QoS policy is really required? I would think just having it explicitly set to the SystemDefaultsQoS would be sufficient.
• Change the description of the Sensor policy to have subscribers use SensorDataQoS

W.r.t. QoS overrides, I think that's a fair point, but also not commonly used still since it has to be explicitly enabled rather than being overwritable by default (unless that's been changed in the last few months). Most software that I've seen doesn't flip that flag to allow for those overwrites on QoS (yet), but I suppose for future proofing it's probably worth addressing. I'm just not sure what remark would be valuable to have made regarding it. Just a 1-liner saying 'QoS overrides exist, if a sensor data publisher node or sensor data subscription node enables it, it can be used to adjust settings'? I suppose that's true, but that doesn't - to me at least - really say anything new or set a requirement or suggestion. It seems more like just making users aware of existing features and I don't see this as the place for that (seems like QoS overrides rules / suggestions / best practices should be a REP of its own). Since we're setting in this document the standard for what you should do with sensor/map data, it seems odd to back-peddle about how to do something else - even if certainly that occasional need exists. That doesn't seem like the role of this document to me.

But if you feel strongly, I'd be fine adding in that 1-liner or something else if I'm not quite catching your point!

These are relatively minor changes, if what I've said above seems like the gameplan, I can execute on it early next week so we can have an updated discussion. I believe that should handle all pending items, but let me know if I missed something. Let me know and I'll get on it.

Pretty excited about this getting moved forward!

[SteveMacenski]

Pinging back on that -- does that sounds like a plan?

[clalancette]

I'm sorry again for the long delay here. I've left some minor notes inline, and here are some responses to your overall list:

L33: explain that this is not exhaustive list, but a set of examples of sensors it applies to (but also try to be as exhaustive as possible)

Sounds good.

Add in an explicit mention that this REP fixes the "why isn't [my sensor] working?!" issues from QoS

Sounds good.

Change the description of the Sensor policy to have publishers of sensor data use the Reliable settings -- Do you think the new QoS policy is really required? I would think just having it explicitly set to the SystemDefaultsQoS would be sufficient.

Yeah, I think having it use SystemDefaultsQoS for now would be best. We can consider whether we really want a new policy later on.

Change the description of the Sensor policy to have subscribers use SensorDataQoS

Yes, sounds good.

W.r.t. QoS overrides, I think that's a fair point, but also not commonly used still since it has to be explicitly enabled rather than being overwritable by default (unless that's been changed in the last few months). Most software that I've seen doesn't flip that flag to allow for those overwrites on QoS (yet), but I suppose for future proofing it's probably worth addressing. I'm just not sure what remark would be valuable to have made regarding it. Just a 1-liner saying 'QoS overrides exist, if a sensor data publisher node or sensor data subscription node enables it, it can be used to adjust settings'? I suppose that's true, but that doesn't - to me at least - really say anything new or set a requirement or suggestion. It seems more like just making users aware of existing features and I don't see this as the place for that (seems like QoS overrides rules / suggestions / best practices should be a REP of its own). Since we're setting in this document the standard for what you should do with sensor/map data, it seems odd to back-peddle about how to do something else - even if certainly that occasional need exists. That doesn't seem like the role of this document to me.
But if you feel strongly, I'd be fine adding in that 1-liner or something else if I'm not quite catching your point!

No, you've caught my point. While I agree with you that maybe we should have an REP that addresses QoS more broadly, I also think that this specific REP should address the best practice for what it is targeting. And in this case, I think best practice would be to default to SystemDefault QoS, and enable qos overrides so that users can change it if they really need it. This is the long way of saying that yes, I think we should add something about it.

Again, sorry for taking so long to get back to this.

[SteveMacenski]

This is the long way of saying that yes, I think we should add something about it.

I just want to be crystal clear so that I can do all these changes at once and knock them out. Would saying "QoS overrides exist, if a sensor data publisher node or sensor data subscription node enables it, it may be used to adjust settings to meet this standard" (with better grammar, more formal language) meet your request?

[clalancette]

I just want to be crystal clear so that I can do all these changes at once and knock them out. Would saying "QoS overrides exist, if a sensor data publisher node or sensor data subscription node enables it, it may be used to adjust settings to meet this standard" (with better grammar, more formal language) meet your request?

More-or-less. One thing I think should be clear in what we add is that the QoS overrides allow you to adjust things against this standard. That is, I would expect by default nodes that comply with this standard all use SystemDefaultQoS, so they interoperate out-of-the-box. But if that QoS doesn't work for you for some reason, you can use QoS overrides to set it to something that you want (you'll just have to do it on all publisher/subscribers to that topic). Does that make sense?

[SteveMacenski]

Sure thing - I'll add this to my queue to resolve over the next couple of days

[SteveMacenski]

Done - except for the Qos policy override comment. I tried for about 15 minutes to write something that would be sensible but couldn't find a way to communicate that subtly concisely. It ended up requiring so much context that the caveat we were addressing was longer than the actual specification itself for what you should use for each policy. It also involved so much "in case you don't want to actually follow this convention..." jardon it was backpeddling the entire value proposition of the standardization process.

As an analogy, of course, folks can create new LaserScan messages if the default one doesn't make sense for their specialized reason, but I don't think REP138 should be specifically stating to people that 'you can always make a new message for laser scanner if you like'. That's kind of how this comes out here.

In light of that and that I think this isn't really the place to be bringing up other features to advocate for them, I excluded that from these changes. I couldn't find a way to communicate it productively and I suspect its because the top may not be appropriate for this document as a standard. It feels more like a good note in a tutorial.

[SteveMacenski]

@clalancette I know you're busy as anyone could be - but any thoughts from these changes? Worth discussing at the next TSC meeting and/or triggering the next stages for this?

[ros-discourse]

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/vote-now-2023-ros-2-technical-steering-committee-community-representative-election/34019/1

[DLu]

I think you need to discuss map updates here, if nothing else than to draw a box around the GCR formats you discuss here. You say "all maps" but that comment seems to point to just some maps.

I understand that the representation is not part of this discussion, but there are separate message based approaches (namely map_msgs::msg::OccupancyGridUpdate) for handling differently aged subscriptions. The last time I looked at this, the state of the RCL/DDS cluster was such that the onNewSubscription functionality in the TODO here was not (yet?) implemented, although that may have changed.

The ROS 1 mechanism relied on being able to send a full map message to only new subscriptions, thus ensuring that you could subscribe ONLY to the full message topic and get the most up to date map. By saying that all maps should be transient local, are you also ensuring that all subscribers will always get a full update any time the data changes?

[clalancette]

The last time I looked at this, the state of the RCL/DDS cluster was such that the onNewSubscription functionality in the TODO here was not (yet?) implemented, although that may have changed.

As of Iron, the core supports this with the use of "Matched" callbacks. See http://docs.ros.org/en/iron/Releases/Release-Iron-Irwini.html#matched-events .

[SteveMacenski]

@DLu costmaps are not maps, so the costmap update topics are out of scope since those are collision avoidance/planning models and not environmental models that this REP is covering. The text is pretty clear in line about the scope that this is covering. But this is a good clarification to make:

• Explicitly restrict REP to environmental models, not to be confused with costmaps or other map-like structures for collision avoidance or C-space representation transportation

but also the OccGrid / Update topics have a stamped header, so it should not be challenging to disambiguate old updates to be disgarded. I still agree that Transient Local is the right answer even if we are using an update-structure. You wouldn't want (1) the main updated map to come in then start processing and wait on a polling basis for the update to come in, then (2) the algorithm could start planning with outdated data. Having it be event based on connection due to the QoS is still a preferable experience. There's certainly some technical decision making about how to handle the case, but this isn't a roadmap for map update schemes, this is a REP on the QoS to communicate them under.

Unless you disagree that updates should be also transient local, which we do today for for some time now, I think this REP does not require further updates on the topic. Though, some other documents about how to process the updates could be useful to provide a general roadmap for end-users, I don't disagree

If you think some clarify w.r.t. updates specifically as it applies to QoS, let me know what you're thinking is needed to clarify and I can make that happen.

[fmrico]

Should we extend this REP to cover the actuation? Maybe defining the ros2_control or cmd_vel topics could make this REP more complete. I know that everybody uses default settings in these topics (right?) but maybe it is good to have this written.

[SteveMacenski]

Should we extend this REP to cover the actuation? Maybe defining the ros2_control or cmd_vel topics could make this REP more complete.

@fmrico I'm on board with it in concept, but I would like to potentially have that either as a separate REP or as a v2 version of this REP. This has been some years in the making and I don't want to pile on the scope which gets it delayed further. Plus, there are some conversations happening right now on a migration to TwistStamped for control use with Nav2/Gazebo/Control, so that itself could be a REP which the QoS could be covered within. So I want to wait for that chat before we move in that direction to know what makes the most sense.

[ros-discourse]

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2023-11-16/34757/1

[mkhansenbot]

Am I understanding this correctly, that the publisher should use SystemDefaultsQoS and the subscriber SensorDataQoS? Why different QoS settings for those? Am I missing something?

[alsora]

IMO SystemDefaultsQoS is not safe to be used and it should be deprecated, rather than being encouraged in a REP.
Components should explictly declare their QoS and not rely on the underlying "system default" which are system specific and RMW specific

[SteveMacenski]

See the discussions above. This was a request from OSRF folks and I think its a sensible choice

[SteveMacenski]

We could call out what SystemDefaultsQoS means under the hood and I think that would be fine too (e.g. Volatile, Reliable, History depth of a user's choice).

[alsora]

some sensor drivers may produce data only "on change" rather than at a fixed, usually fast, period.
these topics should use transient local reliable publication

[SteveMacenski]

For example?

[SteveMacenski]

That doesn't seem like raw sensor feeds to me, but the output of a node that is doing some pre-processing (e.g. stage). The REP calls out that we only consider raw sensor data and aren't covering between stages which might have other operating contexts. I think that should cover it unless there are actual "on change" raw sensor data inputs - which I can't think of an example of

[SteveMacenski]

It was accepted! Merge baby merge! Is there anything I need to do to get it on REP 0?

[SteveMacenski]

This was accepted - can we get this merged?

[ros-discourse]

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/new-rep-2003-sensor-data-and-map-qos-settings/35758/7

[alanxuefei]

Since rmw/include/rmw/qos_profiles.h already provides profiles like rmw_qos_profile_sensor_data, would it be helpful to add a new QoS profile, rmw_qos_profile_map, specifically tailored for map data?

static const rmw_qos_profile_t rmw_qos_profile_map =
{
  RMW_QOS_POLICY_HISTORY_KEEP_LAST,
  1, // Depth of 1 is typical for map data to only keep the latest map
  RMW_QOS_POLICY_RELIABILITY_RELIABLE,
  RMW_QOS_POLICY_DURABILITY_TRANSIENT_LOCAL,
  RMW_QOS_DEADLINE_DEFAULT,
  RMW_QOS_LIFESPAN_DEFAULT,
  RMW_QOS_POLICY_LIVELINESS_SYSTEM_DEFAULT,
  RMW_QOS_LIVELINESS_LEASE_DURATION_DEFAULT,
  false
};