Narrow down by specifying conditions
現在1891件がヒットしています。check
![[Introduction to NVIDIA Isaac ROS GEMs] Episode 1 Overview of NVIDIA Isaac ROS GEMs](/business/semiconductor/articles/142899_head.png)
Introduction
The number of cases of using ROS (Robot Operating System) for the development of application software for robots has increased. The NVIDIA Isaac ROS GEMs we are going to introduce are a collection of acceleration packages for NVIDIA hardware provided for ROS developers.
NVIDIA Isaac ROS GEMs include a number of packages useful for adding AI capabilities to autonomous mobile robots.
In this series, we will provide 3 episodes of information for trying out this NVIDIA Isaac ROS GEMs on NVIDIA Jetson. If you want to see episode 2 and beyond, you can do so by registering in the simple form at the end of the article.
Episode 1 Overview of NVIDIA Isaac ROS GEMs
Episode 2 Try object detection
Episode 3 Try semantic image segmentation
*"Isaac" is read as "Isaac".
Configuring NVIDIA Isaac ROS GEMs
As you can see below, NVIDIA Isaac ROS GEMs are built on well-proven hardware acceleration libraries such as CUDA and TensorRT to take full advantage of the performance of NVIDIA hardware. Since it is ROS2 compliant, it can be combined with other ROS2 packages to develop ROS2 applications.
NVIDIA Isaac ROS GEMs package
As of February 2023, the packages offered are:
| AprilTag | AprilTag fiducial marker code reading and pose detection optimized for NVIDIA hardware |
| Argus Camera | Capture images from MIPI-CSI cameras connected to NVIDIA Jetson |
| Common | Utilities, packages, scripts, Dockerfiles and test environments commonly used by NVIDIA Isaac ROS GEMs |
| Compression | Video compression/decompression (encoding/decoding) optimized for NVIDIA hardware |
| DNN Inference | Deep neural network inference with NVIDIA Triton and NVIDIA TensorRT |
| DNN Stereo Disparity | Parallax calculation of stereo camera by deep neural network |
| Image Pipeline | Provides a mechanism for efficiently processing camera images using NVIDIA hardware |
| Image Segmentation | Semantic image segmentation optimized for NVIDIA hardware |
| Mission Client | A software package for assigning tasks to each robot from a fleet management system for autonomous mobile robots and monitoring them |
| NITROS | Provides a data transfer mechanism suitable for high-speed processing by NVIDIA hardware. |
| Nvblox | 3D scene reconstruction and Nav2 local cost map generation optimized for NVIDIA hardware |
| Object Detection | Object detection with DetectNet model optimized for NVIDIA hardware |
| Pose Estimation | 3D object pose estimation optimized for NVIDIA hardware |
| Proximity Segmentation | Stereo image depth estimation with Bi3D model optimized for NVIDIA hardware |
| Visual SLAM | Visual SLAM (self-localization and environment mapping) optimized for NVIDIA hardware |
Please see the latest information from the link below.
Isaac ROS Package Index and Examples
Platforms supported
Isaac ROS GEMs can be used on NVIDIA Jeton Orin/AGX Xavier/Xavier NX and x86_64 architecture PCs and workstations equipped with NVIDIA GPU cards.
* JetPack 5.0.2 or later is required, so it does not work with Jetson Nano or Jetson TX2.
|
platform |
hardware |
software |
| Jetson | NVIDIA Jetson (Orin series, AGX Xavier, Xavier NX) |
JetPack 5.0.2 or higher |
| x86_64 | x86_64 PC or workstation with NVIDIA GPU card | Ubuntu 20.04 or newer CUDA 11.6.1 or newer |
* ROS2 version is compatible with Humble or later. It will not work with ROS2 versions older than Humble, nor with ROS1.
About execution environment
A utility for building the execution environment of NVIDIA Isaac ROS GEMs is provided by NVIDIA in the Isaac ROS Common repository. When using this, the execution environment of NVIDIA Isaac ROS GEMs is built inside a Docker container. Ubuntu, ROS2, various libraries, NVIDIA TensorRT, NVIDIA VPI, NVIDIA Triton Inference Server, etc. are automatically installed in this container. Installing the necessary software one by one is not only time-consuming, but it may also be difficult to match each version, so this utility provided by NVIDIA is very useful.
The user selects the NVIDIA Isaac ROS GEMs packages and ROS2 packages needed to develop ROS2 applications and creates a ROS2 workspace, but on the host OS (not in a container). By mounting this ROS2 workspace directory from the container, you can read, write, and execute the ROS2 workspace from within the container. Even if you are not familiar with Docker, there is no need to worry, as this utility will handle the mount settings and so on.
Integration with NVIDIA Isaac Sim
NVIDIA offers NVIDIA Isaac ROS GEMs as well as a simulation toolkit for robotics called NVIDIA Isaac Sim. NVIDIA Isaac Sim is a scalable robotics simulation application and synthetic data generation tool powered by NVIDIA Omniverse. NVIDIA Isaac Sim has a bridge function with ROS2, so it is possible to work with Isaac ROS GEMs. Debug and test ROS2 applications running on Jetson inside a photorealistic virtual environment that faithfully reproduces the robot working environment provided by NVIDIA Isaac Sim.
* (NVIDIA Isaac ROS GEMs are compatible with ROS2 Humble or later, but) NVIDIA Isaac Sim has ROS1 bridge function in addition to ROS2 bridge. Therefore, it can also be linked with ROS1 applications.
Integration with NVIDIA Isaac Sim
Image source:
https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/overview.html
https://developer.nvidia.com/embedded/learn/get-started-jetson-agx-orin-devkit
Summary
NVIDIA Isaac ROS GEMs are a collection of ROS2-compliant packages that provide functions essential for realizing autonomous robots, such as object detection, segmentation, localization and environment mapping. Moreover, it is optimized for NVIDIA hardware, so it works very efficiently and at high speed.
Next time, I will explain how to try out the Isaac ROS object detection package from within the Isaac ROS GEMs and how the package is structured. Click the button below and fill out the form to view the next article.