Final Result:

https://www.youtube.com/watch?v=1ZyUGpBQ2yA&ab_channel=andrewgillies8

https://youtu.be/PtsvpXjY2wg

Goal:

Demonstrate how ROS2 can be used to process images from cameras in Isaac Sim, and take actions with those cameras.

Specifically, this demo will:

• Have a scene with a conveyor and 3 off-ramps with catch bins
• Side-kickers at each off-ramp with a camera above it
• Camera data piped to ROS
• A ros node that will perform instance segmentation on the image to determine object properties
• Another ros node that will actuate the kicker based on the ros cam node
• Boxes that auto spawn into the scene will get sorted into bins based on above

Steps:

1. Create the Scene

2. Create the Isaac Sim Components

3. Create the ROS2 package and associated nodes