Texas A&M University    CCDC Army Research Laboratory

Peng Jiang1, Philip Osteen2, Maggie Wigness2 and Srikanth Saripalli1
1. Texas A&M University;  2. CCDC Army Research Laboratory
[Website] [Paper] [Github]


Semantic scene understanding is crucial for robust and safe autonomous navigation, particularly so in off-road environments. Recent deep learning advances for 3D semantic segmentation rely heavily on large sets of training data; however, existing autonomy datasets represent urban environments or lack multimodal off-road data. We fill this gap with RELLIS-3D, a multimodal dataset collected in an off-road environment containing annotations for 13,556 LiDAR scans and 6,235 images. The data was collected on the Rellis Campus of Texas A\&M University and presents challenges to existing algorithms related to class imbalance and environmental topography. Additionally, we evaluate the current state of the art deep learning semantic segmentation models on this dataset. Experimental results show that RELLIS-3D presents challenges for algorithms designed for segmentation in urban environments. Except for the annotated data, the dataset also provides full-stack sensor data in ROS bag format, including RGB camera images, LiDAR point clouds, a pair of stereo images, high-precision GPS measurement, and IMU data. This novel dataset provides the resources needed by researchers to develop more advanced algorithms and investigate new research directions to enhance autonomous navigation in off-road environments.

Recording Platform

Sensor Setup

Annotated Data:


With the goal of providing multi-modal data to enhance autonomous off-road navigation, we defined an ontology of object and terrain classes, which largely derives from the RUGD dataset but also includes unique terrain and object classes not present in RUGD. Specifically, sequences from this dataset includes classes such as mud, man-made barriers, and rubble piles. Additionally, this dataset provides a finer-grained class structure for water sources, i.e., puddle and deep water, as these two classes present different traversability scenarios for most robotic platforms. Overall, 20 classes (including void class) are present in the data.

Images Statics:

LiDAR Scans Statics:


Image Semantic Segmenation

LiDAR Semantic Segmenation

ROS Bag Raw Data

Data included in raw ROS bagfiles:

Topic Name Message Tpye Message Descriptison
/img_node/intensity_image sensor_msgs/Image Intensity image generated by ouster Lidar
/img_node/noise_image sensor_msgs/Image Noise image generated by ouster Lidar
/img_node/range_image sensor_msgs/Image Range image generated by ouster Lidar
/imu/data sensor_msgs/Imu Filtered imu data from embeded imu of Warthog
/imu/data_raw sensor_msgs/Imu Raw imu data from embeded imu of Warthog
/imu/mag sensor_msgs/MagneticField Raw magnetic field data from embeded imu of Warthog
/nerian_stereo/left_image sensor_msgs/Image Left image from Nerian Karmin2
/nerian_stereo/right_image sensor_msgs/Image Right image from Nerian Karmin2
/odometry/filtered nav_msgs/Odometry A filtered local-ization estimate based on wheel odometry (en-coders) and integrated IMU from Warthog
/os1_cloud_node/imu sensor_msgs/Imu Raw imu data from embeded imu of Ouster Lidar
/os1_cloud_node/points sensor_msgs/PointCloud2 Point cloud data from Ouster Lidar
/os1_node/imu_packets ouster_ros/PacketMsg Raw imu data from Ouster Lidar
/os1_node/lidar_packets ouster_ros/PacketMsg Raw lidar data from Ouster Lidar
/vectornav/GPS sensor_msgs/NavSatFix INS data from VectorNav-VN300
/vectornav/IMU sensor_msgs/Imu Imu data from VectorNav-VN300
/vectornav/Mag sensor_msgs/MagneticField Raw magnetic field data from VectorNav-VN300
/vectornav/Odom nav_msgs/Odometry Odometry from VectorNav-VN300
/vectornav/Pres sensor_msgs/FluidPressure  
/vectornav/Temp sensor_msgs/Temperature  
/velodyne_points sensor_msgs/PointCloud2 PointCloud produced by the Velodyne Lidar

Data Download:

Access Link


      title={RELLIS-3D Dataset: Data, Benchmarks and Analysis}, 
      author={Peng Jiang and Philip Osteen and Maggie Wigness and Srikanth Saripalli},


The DEVCOM Army Research Laboratory


All datasets and code on this page are copyright by us and published under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.

SemanticUSL: A Dataset for Semantic Segmentation Domain Adatpation

LiDARNet: A Boundary-Aware Domain Adaptation Model for Lidar Point Cloud Semantic Segmentation