Performances

iRobot ROS2 performance evaluation framework

ROS2 Performance Framework

We developed a framework that allows to easily create arbitrary ROS2 systems and then measures their performance. With this framework you can define your system at runtime using json files or with command line options.

These are the main metrics that we are interested in:

• Latency
• Reliability
• CPU usage
• Memory usage

Due to our specific application scenario and to the new ROS2 multithread capabilities, we made most of the evaluations running all nodes in a single process. However, our frameworks also support having multiple processes running at the same time.

The goal is to develop a set of experiments which are easily reproducible and provide useful insights on the ROS2 performances. Moreover, it is possible to create proxy applications that mimic a real, complex, system and to measure what's the overhead of ROS2 communication in such a context.

We can easily repeat our tests with different ROS2 distributions or middlewares. For each ROS2 distribution we aim to:

Early results are suggesting us that this is the correct way to go, as we are spotting the strengths and weaknesses of ROS2 and helping the ROS2 community to grow, by signaling and fixing bugs to the ROS2 core implementation. Running our application also allowed to find regressions with respect to the previous releases.

Cross-Compilation

The iRobot cross-compilation framework has been moved to its own separate repository at: https://github.com/irobot-ros/ros2-cross-compilation

Proposals

Proposals

This directory contains the design proposals that we made to improve the ROS2 core and its performances.

External tools and resources

Apex AI ROS2 Evaluation tool

ApexAI provides an alternative valid performance evaluation framework, which allows to test different type of messages. Our implementation is inspired by their work.

• Github repo
• Apex AI tool ROSCon slides
• Apex AI tool ROSCon video

Other evaluation tools

• ROS2 communication performance evaluation
• ROS2 benchmarking framework
• ROS2 QOS throughput tests

DDS Vendors advertised performance

• FastRTPS Latency and Throughput
• FastRTPS vs ZeroMQ
• Connext Micro Heap usage

Performances discussions

• Evaluation of ROS2 Communication Layer
• ROS2 latency and throughput
• Evaluating QoS performances using ROS demos

Papers

• Exploring the performance of ROS2
• Time-Sensitive Networking for robotics
• Real-time Linux communications: an evaluation of the Linux communication stack for real-time robotic applications
• Towards a distributed and real-time framework for robots: Evaluation of ROS 2.0 communications for real-time robotic applications
• Time Synchronization in modular collaborative robots
• Reduce delay and jitter in wireless ROS1 networks