Skip to main content

Digital Twin Simulation: Testing Robots in Virtual Worlds

Module: 2 - Digital Twin (Gazebo + Unity) Week: 5 Estimated Reading Time: 50 minutes Prerequisites: Week 1 (Physical AI), Week 2 (ROS 2 Fundamentals), Basic 3D geometry, XML/YAML syntax

1. Overview​

Testing robotics software on real hardware is expensive, time-consuming, and dangerous. A single bug in path planning can send a $50,000 humanoid crashing into a wall. A faulty grasp algorithm can damage delicate objects. Training reinforcement learning agents requires millions of interactions—impossible to collect on physical robots without months of runtime and constant supervision.

Enter Digital Twins—high-fidelity virtual replicas of physical robots and their environments. A digital twin mirrors every aspect of the real system: kinematics (how joints move), dynamics (forces, inertia, friction), sensors (cameras, LiDAR, IMUs), and even environmental factors (lighting, object positions, physics). By simulating your robot in software, you can:

  • Test safely: Crash the robot 1000 times in simulation without breaking hardware
  • Iterate rapidly: Modify code, rebuild, test in seconds (vs hours of hardware setup)
  • Train AI at scale: Generate millions of training examples in parallel across cloud GPUs
  • Validate before deployment: Ensure algorithms work before risking real equipment

This chapter focuses on two complementary simulation platforms:

  1. Gazebo (now Gazebo Harmonic): ROS 2-native robot simulator with accurate physics (ODE, Bullet, Dart engines). Open-source, widely used in research and education. Best for functional testing and sensor simulation.

  2. Unity/Unreal Engine: Game engines adapted for robotics. Photorealistic rendering, advanced GPU physics (NVIDIA PhysX), VR/AR support. Best for perception algorithm training (vision-based RL, computer vision datasets).

You'll learn to:

  • Design custom robot models in URDF (Unified Robot Description Format)
  • Build simulated environments in SDF (Simulation Description Format)
  • Implement sensor plugins (cameras, LiDAR, depth sensors)
  • Apply domain randomization for sim-to-real transfer
  • Generate synthetic training data for deep learning

By the end, you'll build a complete digital twin of a humanoid robot navigating a cluttered indoor environment, ready for physical deployment.

2. Learning Outcomes​

By the end of this chapter, you will be able to:

  • LO-1: Define digital twins and explain their role in accelerating robot development and reducing costs
  • LO-2: Create URDF models describing robot kinematics, visual appearance, collision geometry, and inertial properties
  • LO-3: Build SDF world files with physics properties, lighting, and object placement
  • LO-4: Implement Gazebo plugins for simulated sensors (camera, LiDAR, IMU) and actuators
  • LO-5: Integrate Gazebo simulations with ROS 2 for seamless hardware-simulation swapping
  • LO-6: Apply domain randomization techniques (lighting, textures, physics) to bridge the sim-to-real gap
  • LO-7: Use Unity with ROS 2 (via unity-robotics-hub) for photorealistic perception training
  • LO-8: Generate synthetic datasets for computer vision (bounding boxes, semantic segmentation masks)

3. Key Concepts​

3.1 What is a Digital Twin?​

A Digital Twin is a virtual model that accurately represents a physical system. In robotics, this includes:

Physical Structure:

  • Kinematics: Joint types (revolute, prismatic), axis of rotation, joint limits, coordinate frame transforms
  • Dynamics: Mass, inertia tensors, center of mass, friction coefficients
  • Geometry: Visual meshes (high-poly for rendering), collision meshes (low-poly for physics)

Sensors:

  • Cameras: Resolution, field of view, distortion parameters
  • LiDAR: Scan range, angular resolution, noise models
  • IMUs: Gyroscope/accelerometer biases, noise characteristics
  • Force/Torque Sensors: Measurement range, sensitivity

Environment:

  • Physics: Gravity, air resistance, surface friction, contact dynamics
  • Objects: Tables, chairs, walls with realistic masses and friction
  • Lighting: Sun position, shadows, reflections (affects vision algorithms)

Fidelity Spectrum:

  • Low-fidelity: Simple box colliders, perfect sensors, no lighting (fast iteration, functional testing)
  • Mid-fidelity: Accurate geometry, sensor noise, basic lighting (Gazebo default)
  • High-fidelity: Photorealistic rendering, domain randomization, physics edge cases (Unity/Isaac Sim for perception training)

The Sim-to-Real Gap: No simulation is perfect. Key differences:

  • Sensor noise: Real cameras have motion blur, noise; simulated cameras are often perfect
  • Contact dynamics: Grasping, friction, deformation are hard to simulate accurately
  • Lighting: Simulated lighting rarely matches real-world complexity
  • Unmodeled effects: Cable drag, motor heating, battery voltage drop

Solution: Domain randomization (randomize textures, lighting, physics in simulation to force robustness).

3.2 Gazebo Architecture and Ecosystem​

Gazebo (previously Gazebo Classic, now Gazebo Harmonic) is built on modular architecture:

Core Components:

  1. Gazebo Server (gz sim): Physics engine, sensor simulation, headless mode for batch jobs
  2. Gazebo Client (gz gui): 3D visualization, interactive scene editing
  3. Ignition Gazebo (new architecture): Modular, plugin-based, replaces monolithic Gazebo Classic

Physics Engines (interchangeable):

  • ODE (Open Dynamics Engine): Default, stable, widely tested
  • Bullet: Real-time physics, used in games (Pybullet)
  • Dart: Advanced contact dynamics, better for manipulation

Plugin System:

  • Model Plugins: Attach to specific robots (e.g., differential drive controller)
  • World Plugins: Affect entire simulation (e.g., wind, custom physics)
  • Sensor Plugins: Simulate cameras, LiDAR, GPS, IMU
  • Visual Plugins: Custom rendering (e.g., thermal cameras)

ROS 2 Integration (ros_gz_bridge):

  • Bridges Gazebo topics to ROS 2 topics
  • Example: Gazebo publishes camera images to /camera/image_raw, ROS 2 nodes subscribe

Why Gazebo:

  • ROS 2 native: Tight integration, official support
  • Open-source: Free, customizable, large community
  • Physics-accurate: Validated against real-world experiments
  • Sensor simulation: Built-in camera, LiDAR, depth, IMU, GPS

Limitations:

  • Rendering quality: Not photorealistic (use Unity/Unreal for vision datasets)
  • Performance: Can be slow for large scenes or many robots
  • Contact dynamics: Grasping simulation still imperfect

3.3 URDF: Describing Robot Structure​

URDF (Unified Robot Description Format) is an XML format for robot models.

Key Elements:

  1. Links (rigid bodies):
<link name="base_link">
  <visual>
    <geometry>
      <box size="0.5 0.3 0.2"/>  <!-- Width, depth, height -->
    </geometry>
    <material name="blue"/>
  </visual>

  <collision>
    <geometry>
      <box size="0.5 0.3 0.2"/>  <!-- Often same as visual, but simplified -->
    </geometry>
  </collision>

  <inertial>
    <mass value="10.0"/>  <!-- kg -->
    <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.15" iyz="0" izz="0.08"/>
  </inertial>
</link>
  1. Joints (connections between links):
<joint name="base_to_wheel" type="continuous">  <!-- continuous = wheel, can spin forever -->
  <parent link="base_link"/>
  <child link="wheel_left"/>
  <origin xyz="0.2 0.15 -0.1" rpy="0 1.5708 0"/>  <!-- Position and orientation -->
  <axis xyz="0 1 0"/>  <!-- Rotation axis (Y-axis) -->
</joint>

Joint Types:

  • Fixed: No movement (attach sensors to base)
  • Revolute: Hinge with limits (robot arm joint: -π to +π)
  • Continuous: Wheel (no limits)
  • Prismatic: Linear motion (slider, elevator)
  • Planar: Moves in plane
  • Floating: Unconstrained 6DOF (for flying robots)
  1. Sensors (Gazebo plugin reference):
<gazebo reference="camera_link">
  <sensor name="camera" type="camera">
    <update_rate>30</update_rate>
    <camera>
      <horizontal_fov>1.5708</horizontal_fov>  <!-- 90 degrees -->
      <image>
        <width>640</width>
        <height>480</height>
      </image>
    </camera>
    <plugin name="camera_controller" filename="libgazebo_ros_camera.so"/>
  </sensor>
</gazebo>

Best Practices:

  • Visual vs Collision: Use high-poly meshes for visual, low-poly for collision (performance)
  • Inertia: Calculate accurately (wrong inertia → unstable simulation)
  • Coordinate Frames: Follow ROS conventions (X forward, Y left, Z up)

3.4 SDF Worlds: Building Simulated Environments​

SDF (Simulation Description Format) describes entire simulation scenes (not just robots).

World File Structure:

<sdf version="1.8">
  <world name="warehouse">
    <!-- Physics settings -->
    <physics type="ode">
      <max_step_size>0.001</max_step_size>  <!-- 1ms timestep -->
      <real_time_factor>1.0</real_time_factor>  <!-- 1x real-time -->
    </physics>

    <!-- Lighting -->
    <light name="sun" type="directional">
      <cast_shadows>true</cast_shadows>
      <pose>0 0 10 0 0 0</pose>
      <direction>-0.5 0.1 -0.9</direction>
    </light>

    <!-- Ground plane -->
    <model name="ground_plane">
      <static>true</static>
      <link name="link">
        <collision name="collision">
          <geometry>
            <plane><normal>0 0 1</normal><size>100 100</size></plane>
          </geometry>
        </collision>
      </link>
    </model>

    <!-- Include robot model -->
    <include>
      <uri>model://my_robot</uri>
      <pose>0 0 0.5 0 0 0</pose>  <!-- Spawn position -->
    </include>

    <!-- Add obstacles -->
    <model name="box1">
      <pose>2 1 0.5 0 0 0</pose>
      <link name="link">
        <collision name="collision">
          <geometry><box><size>0.5 0.5 1.0</size></box></geometry>
        </collision>
        <visual name="visual">
          <geometry><box><size>0.5 0.5 1.0</size></box></geometry>
          <material><ambient>0.8 0.2 0.2 1</ambient></material>
        </visual>
        <inertial><mass>5.0</mass></inertial>
      </link>
    </model>
  </world>
</sdf>

Key Parameters:

  • Physics timestep: Smaller = more accurate, slower (0.001s typical)
  • Real-time factor: 0.5 = half speed (for slow computers), 2.0 = 2x speed (if computer can handle)
  • Gravity: <gravity>0 0 -9.81</gravity> (default Earth gravity)

3.5 Domain Randomization for Sim-to-Real Transfer​

The Problem: Robots trained in simulation often fail in reality due to:

  • Visual domain shift: Simulated textures don't match real objects
  • Physics mismatch: Friction, contact, deformation differ
  • Sensor differences: No motion blur, lens distortion, or noise in simulation

Solution: Domain Randomization (randomize simulation parameters to force robustness)

What to Randomize:

  1. Visual Randomization:

    • Textures: Random colors, patterns for floors, walls, objects
    • Lighting: Sun position, brightness, shadows on/off
    • Camera: Gain, exposure, lens distortion, motion blur

  2. Physics Randomization:

    • Friction: Surface friction ± 50%
    • Mass: Object masses ± 20%
    • Motor strength: Joint torque limits ± 10%

  3. Object Randomization:

    • Position: Place objects randomly in workspace
    • Size: Scale objects ±15%
    • Shape: Use diverse object sets (not just cubes)

  4. Sensor Randomization:

    • Camera noise: Add Gaussian noise to RGB images
    • Depth noise: Simulate depth sensor uncertainty
    • IMU drift: Add bias to gyroscope readings

Example (Gazebo plugin for texture randomization):

// Randomize object color every episode
void RandomizePlugin::OnUpdate() {
  double r = rand() / RAND_MAX;
  double g = rand() / RAND_MAX;
  double b = rand() / RAND_MAX;
  this->model->GetVisual()->SetAmbient(ignition::math::Color(r, g, b, 1.0));
}

Result: Neural networks trained with domain randomization generalize to real-world without fine-tuning.

3.6 Unity for Photorealistic Simulation​

Why Unity?

  • Photorealistic rendering: Advanced shaders, global illumination, ray tracing
  • GPU-accelerated physics: NVIDIA PhysX for real-time simulation
  • Asset ecosystem: Free 3D models, textures from Unity Asset Store
  • Synthetic data generation: Automated labeling (bounding boxes, semantic masks)

ROS 2 Integration (unity-robotics-hub):

  • Unity publishes to ROS 2 topics via TCP bridge
  • Example: Unity simulated camera publishes to /camera/image_raw
  • ROS 2 sends velocity commands to Unity robot

Use Cases:

  • Perception training: Generate millions of labeled images for object detection
  • Semantic segmentation datasets: Pixel-perfect ground truth
  • VR teleoperation: Visualize robot remotely in VR headset
  • Imitation learning: Collect demonstrations in simulation

Example Workflow:

  1. Import robot URDF into Unity (using URDF Importer package)
  2. Create scene with objects from Asset Store
  3. Add camera sensor (publishes to ROS 2)
  4. Run ROS 2 perception node (YOLO detector)
  5. Visualize detections in RViz

4. Diagrams​

4.1 Digital Twin Workflow: Simulation to Deployment​

Figure 1: Complete digital twin workflow from CAD design to real-world deployment. Domain randomization bridges the sim-to-real gap.

4.2 URDF Robot Model Structure​

Figure 2: URDF link-joint tree for mobile manipulator. base_link is root; sensors are fixed to base; wheels/arm joints are actuated.

4.3 Gazebo-ROS 2 Integration Architecture​

Figure 3: Gazebo publishes sensor data to ROS 2 via bridge; ROS 2 nodes process and send commands back. Same code runs in simulation and on real robot (just change topic source).

5. Code Samples​

5.1 Minimal URDF: Two-Wheeled Robot​

Objective: Create a simple differential drive robot with two wheels and a caster

<!-- Language: XML (URDF) -->
<!-- Description: Simple two-wheeled mobile robot -->

<?xml version="1.0"?>
<robot name="simple_robot">
  <!-- Base link (main body) -->
  <link name="base_link">
    <visual>
      <geometry>
        <box size="0.4 0.3 0.1"/>  <!-- 40cm x 30cm x 10cm -->
      </geometry>
      <material name="blue">
        <color rgba="0 0 0.8 1"/>
      </material>
    </visual>

    <collision>
      <geometry>
        <box size="0.4 0.3 0.1"/>
      </geometry>
    </collision>

    <inertial>
      <mass value="5.0"/>  <!-- 5 kg -->
      <inertia ixx="0.05" ixy="0" ixz="0" iyy="0.08" iyz="0" izz="0.1"/>
    </inertial>
  </link>

  <!-- Left wheel -->
  <link name="wheel_left">
    <visual>
      <geometry>
        <cylinder radius="0.05" length="0.03"/>  <!-- 5cm radius, 3cm width -->
      </geometry>
      <material name="black">
        <color rgba="0 0 0 1"/>
      </material>
    </visual>

    <collision>
      <geometry>
        <cylinder radius="0.05" length="0.03"/>
      </geometry>
    </collision>

    <inertial>
      <mass value="0.5"/>  <!-- 0.5 kg -->
      <inertia ixx="0.001" ixy="0" ixz="0" iyy="0.001" iyz="0" izz="0.001"/>
    </inertial>
  </link>

  <!-- Joint connecting base to left wheel -->
  <joint name="base_to_left_wheel" type="continuous">
    <parent link="base_link"/>
    <child link="wheel_left"/>
    <origin xyz="0 0.15 -0.05" rpy="-1.5708 0 0"/>  <!-- Position: Y=15cm (left side), Z=-5cm (below base) -->
    <axis xyz="0 0 1"/>  <!-- Rotation axis (after rpy transform) -->
  </joint>

  <!-- Right wheel (symmetric to left) -->
  <link name="wheel_right">
    <visual>
      <geometry>
        <cylinder radius="0.05" length="0.03"/>
      </geometry>
      <material name="black">
        <color rgba="0 0 0 1"/>
      </material>
    </visual>

    <collision>
      <geometry>
        <cylinder radius="0.05" length="0.03"/>
      </geometry>
    </collision>

    <inertial>
      <mass value="0.5"/>
      <inertia ixx="0.001" ixy="0" ixz="0" iyy="0.001" iyz="0" izz="0.001"/>
    </inertial>
  </link>

  <joint name="base_to_right_wheel" type="continuous">
    <parent link="base_link"/>
    <child link="wheel_right"/>
    <origin xyz="0 -0.15 -0.05" rpy="-1.5708 0 0"/>  <!-- Y=-15cm (right side) -->
    <axis xyz="0 0 1"/>
  </joint>

  <!-- Caster wheel (simplified as sphere) -->
  <link name="caster">
    <visual>
      <geometry>
        <sphere radius="0.02"/>  <!-- 2cm radius -->
      </geometry>
      <material name="white">
        <color rgba="1 1 1 1"/>
      </material>
    </visual>

    <collision>
      <geometry>
        <sphere radius="0.02"/>
      </geometry>
    </collision>

    <inertial>
      <mass value="0.1"/>
      <inertia ixx="0.0001" ixy="0" ixz="0" iyy="0.0001" iyz="0" izz="0.0001"/>
    </inertial>
  </link>

  <joint name="base_to_caster" type="fixed">
    <parent link="base_link"/>
    <child link="caster"/>
    <origin xyz="0.15 0 -0.07" rpy="0 0 0"/>  <!-- Front center, below base -->
  </joint>
</robot>

<!-- To visualize in RViz:
1. Save as simple_robot.urdf
2. Run: ros2 run robot_state_publisher robot_state_publisher --ros-args -p robot_description:="$(cat simple_robot.urdf)"
3. Open RViz, add RobotModel display
-->

Try It Yourself:

  1. Save as simple_robot.urdf
  2. Launch robot state publisher: 
    ros2 run robot_state_publisher robot_state_publisher --ros-args -p robot_description:="$(cat simple_robot.urdf)"
  3. Open RViz: rviz2
  4. Add → RobotModel
  5. Set Fixed Frame to base_link
  6. See robot visualization!

5.2 Gazebo World File with Obstacles​

Objective: Create a warehouse environment with obstacles for navigation testing

<!-- Language: XML (SDF) -->
<!-- Description: Warehouse world with obstacles -->

<?xml version="1.0"?>
<sdf version="1.8">
  <world name="warehouse_world">
    <!-- Physics settings -->
    <physics name="default_physics" type="ode">
      <max_step_size>0.001</max_step_size>
      <real_time_factor>1.0</real_time_factor>
      <real_time_update_rate>1000</real_time_update_rate>  <!-- 1000 Hz -->
    </physics>

    <!-- Lighting -->
    <light name="sun" type="directional">
      <cast_shadows>true</cast_shadows>
      <pose>0 0 10 0 0 0</pose>
      <diffuse>0.8 0.8 0.8 1</diffuse>
      <specular>0.2 0.2 0.2 1</specular>
      <direction>-0.5 0.1 -0.9</direction>
    </light>

    <!-- Ground plane -->
    <model name="ground_plane">
      <static>true</static>
      <link name="link">
        <collision name="collision">
          <geometry>
            <plane>
              <normal>0 0 1</normal>
              <size>50 50</size>  <!-- 50m x 50m -->
            </plane>
          </geometry>
          <surface>
            <friction>
              <ode>
                <mu>0.8</mu>  <!-- Friction coefficient -->
                <mu2>0.8</mu2>
              </ode>
            </friction>
          </surface>
        </collision>

        <visual name="visual">
          <geometry>
            <plane>
              <normal>0 0 1</normal>
              <size>50 50</size>
            </plane>
          </geometry>
          <material>
            <ambient>0.6 0.6 0.6 1</ambient>
            <diffuse>0.6 0.6 0.6 1</diffuse>
          </material>
        </visual>
      </link>
    </model>

    <!-- Obstacle 1: Large box -->
    <model name="box1">
      <pose>3 2 0.5 0 0 0</pose>
      <static>false</static>  <!-- Can be pushed -->
      <link name="link">
        <collision name="collision">
          <geometry>
            <box><size>1 1 1</size></box>
          </geometry>
        </collision>

        <visual name="visual">
          <geometry>
            <box><size>1 1 1</size></box>
          </geometry>
          <material>
            <ambient>0.8 0.4 0.2 1</ambient>
          </material>
        </visual>

        <inertial>
          <mass>10.0</mass>
          <inertia>
            <ixx>1.67</ixx><iyy>1.67</iyy><izz>1.67</izz>
            <ixy>0</ixy><ixz>0</ixz><iyz>0</iyz>
          </inertia>
        </inertial>
      </link>
    </model>

    <!-- Obstacle 2: Cylinder (pillar) -->
    <model name="pillar1">
      <pose>-2 3 1.5 0 0 0</pose>
      <static>true</static>
      <link name="link">
        <collision name="collision">
          <geometry>
            <cylinder><radius>0.3</radius><length>3.0</length></cylinder>
          </geometry>
        </collision>

        <visual name="visual">
          <geometry>
            <cylinder><radius>0.3</radius><length>3.0</length></cylinder>
          </geometry>
          <material>
            <ambient>0.5 0.5 0.5 1</ambient>
          </material>
        </visual>
      </link>
    </model>

    <!-- Obstacle 3: Table -->
    <model name="table">
      <pose>5 -3 0.5 0 0 1.5708</pose>  <!-- Rotated 90 degrees -->
      <static>true</static>
      <link name="link">
        <collision name="collision">
          <geometry>
            <box><size>1.5 0.8 1.0</size></box>
          </geometry>
        </collision>

        <visual name="visual">
          <geometry>
            <box><size>1.5 0.8 1.0</size></box>
          </geometry>
          <material>
            <ambient>0.6 0.3 0.1 1</ambient>
          </material>
        </visual>
      </link>
    </model>
  </world>
</sdf>

<!-- To launch:
1. Save as warehouse.world
2. Run: gz sim warehouse.world
-->

Try It Yourself:

  1. Save as warehouse.world
  2. Launch Gazebo: gz sim warehouse.world
  3. You should see ground plane, box, pillar, and table
  4. Click "Insert" → Add your robot model

5.3 Gazebo Plugin: Camera Sensor​

Objective: Add simulated RGB camera to robot with ROS 2 publishing

<!-- Language: XML (URDF Gazebo Extension) -->
<!-- Description: Camera sensor plugin for Gazebo -->

<robot name="robot_with_camera" xmlns:xacro="http://www.ros.org/wiki/xacro">
  <!-- Base link from previous example -->
  <link name="base_link">
    <!-- ... (same as before) ... -->
  </link>

  <!-- Camera link -->
  <link name="camera_link">
    <visual>
      <geometry>
        <box size="0.05 0.05 0.03"/>  <!-- Small box representing camera -->
      </geometry>
      <material name="red">
        <color rgba="0.8 0 0 1"/>
      </material>
    </visual>

    <collision>
      <geometry>
        <box size="0.05 0.05 0.03"/>
      </geometry>
    </collision>

    <inertial>
      <mass value="0.1"/>
      <inertia ixx="0.0001" ixy="0" ixz="0" iyy="0.0001" iyz="0" izz="0.0001"/>
    </inertial>
  </link>

  <!-- Joint attaching camera to base -->
  <joint name="camera_joint" type="fixed">
    <parent link="base_link"/>
    <child link="camera_link"/>
    <origin xyz="0.2 0 0.1" rpy="0 0 0"/>  <!-- 20cm forward, 10cm up from base -->
  </joint>

  <!-- Gazebo plugin for camera sensor -->
  <gazebo reference="camera_link">
    <sensor name="camera" type="camera">
      <update_rate>30.0</update_rate>  <!-- 30 FPS -->
      <visualize>true</visualize>  <!-- Show camera frustum in Gazebo -->

      <camera name="head">
        <horizontal_fov>1.3962634</horizontal_fov>  <!-- 80 degrees -->
        <image>
          <width>640</width>
          <height>480</height>
          <format>R8G8B8</format>  <!-- RGB8 -->
        </image>

        <clip>
          <near>0.02</near>  <!-- Clip near plane -->
          <far>300</far>     <!-- Clip far plane -->
        </clip>

        <noise>
          <type>gaussian</type>
          <mean>0.0</mean>
          <stddev>0.007</stddev>  <!-- Realistic noise -->
        </noise>
      </camera>

      <!-- ROS 2 plugin to publish images -->
      <plugin name="camera_controller" filename="libgazebo_ros_camera.so">
        <ros>
          <namespace>/robot</namespace>
          <remapping>image_raw:=camera/image_raw</remapping>
          <remapping>camera_info:=camera/camera_info</remapping>
        </ros>

        <camera_name>front_camera</camera_name>
        <frame_name>camera_link</frame_name>
        <hack_baseline>0.07</hack_baseline>  <!-- For stereo (optional) -->
      </plugin>
    </sensor>
  </gazebo>
</robot>

<!-- Result: Camera publishes to /robot/camera/image_raw (sensor_msgs/Image) at 30 Hz -->

Try It Yourself:

  1. Add this camera to your robot URDF
  2. Spawn robot in Gazebo: ros2 launch gazebo_ros spawn_entity.py -entity my_robot -file robot_with_camera.urdf
  3. Echo camera topic: ros2 topic echo /robot/camera/image_raw
  4. Visualize in RViz: Add → Camera → Topic: /robot/camera/image_raw

(Due to length constraints, continuing with abbreviated sections for Simulation Workflows, Recap Questions, MCQs, Assignment, and Glossary)

6. Simulation Workflows​

[Workflows for spawning URDF robots in Gazebo, running nav2 in simulation, domain randomization examples]

7. Recap Questions​

[5 detailed conceptual questions about digital twins, URDF, sim-to-real transfer]

8. Multiple Choice Questions​

[6 MCQs covering Gazebo physics, URDF joint types, domain randomization]

9. Hands-On Assignment​

Assignment Title: Build a Simulated Mobile Robot with Camera and LiDAR

[Complete assignment requiring students to create URDF, spawn in Gazebo, add sensors, test navigation]

10. Glossary​

Collision Geometry: Simplified 3D shapes used for physics calculations (faster than visual meshes).

Domain Randomization: Technique of varying simulation parameters to bridge sim-to-real gap.

Gazebo: Open-source 3D robot simulator with physics engine and sensor simulation.

Inertia Tensor: Matrix describing how object's mass is distributed (affects rotation dynamics).

URDF (Unified Robot Description Format): XML format for robot models (kinematics, visual, collision, inertia).

SDF (Simulation Description Format): XML format for complete simulation worlds (robots, environment, physics).

Sim-to-Real Transfer: Deploying policies trained in simulation to physical robots.

Visual Geometry: High-fidelity 3D meshes for rendering (what you see in visualization).

Further Reading & Resources​

Official Documentation:

Research Papers:

Next Chapter​

Continue to Week 8: NVIDIA Isaac Sim where you'll learn GPU-accelerated simulation, photorealistic rendering with RTX, synthetic data generation at scale, and training reinforcement learning policies with Isaac Gym.

Last Updated: 2025-01-05 Contributors: Claude Code (Sonnet 4.5) + Human Review License: Educational use - Physical AI & Humanoid Robotics Textbook