The Michigan State University Four Seasons (MSU-4S) Dataset
We are at CVPR! Come see us on Friday, June 21st at 10:30 AM in the Arch Building, Hall 4E, Poster 302.
The Michigan State University Four Seasons (MSU-4S) Dataset is a unique dataset taken in and around MSU’s campus throughout all four seasons in various weather and environmental conditions.
MSU-4S was introduced at CVPR 2024 and is available to download under a non-commercial Creative Commons license.
Download Manuscript (Citation)
Download the Dataset
Current dataset version: v1.0. Release Notes.
Data labels will be available in a forthcoming release (est. Summer 2024).
To download the datset, we recommend using our downloader script. This script can be easily customized to only download the files you need.
All public dataset files are available to browse here.
Leaderboards
We plan to publish leaderboards for 3D and 2D object detection and domain adaptation tasks. We will announce more specific plans at a later date.
Selected Samples and Graphics
Technical Information
Vehicle and Sensor Hardware
Vehicle
The vehicle used for data collection is a modified 2017 Chevrolet Bolt EV. The vehicle was originally built as a part of the SAE AutoDrive competition, and as of publication remains the only one of these vehicles still operating in a research capacity.
Sensors
- Compute system: Crystal Rugged compact rugged server, 44-core Intel Xeon
- Network hardware: Netgear XXXX 10Gb link aggregation switch
- Cameras: FLIR Blackfly BFLY-PGE-31S4C-C, 2048x1536 resolution.
- Primary Lidar: 1x Ouster OS-1 (rev. 1) 64-line lidar
- Additional Lidar: 1x Velodyne VLP-32 32-line lidar* (see errata)
- Radar: 6x Continental ARS-430 radar
- IMU: Yost 3-Space Nano 9DOF; onboard 6DOF IMU
- Other hardware: custom-built CAN isolation system designed to allow one-way flow of vehicle CAN data to the vehicle compute system when system does not have autonomous driving capabilities enabled.
Data Format
There are currently 9 data collection segments in the dataset, separated by date of capture. Within each segment, there are subdirectories containing each data source, as follows:
top_left, top_mid, top_right
: Three front-facing cameras. Imagery is 2048x1536, in JPEG format. Images are not dedistorted; calibration parameters can be found in themisc
files. Optionally, pre-dedistorted imagery is available to download in both JPEG and PNG format.oust
: Ouster OS1-64 lidar, PCD format (main lidar data).velo
: Velodyne VLP-32 lidar, PCD format (additional lidar data; some data degradation present, see errata).misc
: Other sensor data including GPS, IMU, radar, transforms, camera calibration, and limited CAN data. YAML format.label3d
: 3D object data labels, YAML format.label2d
: 2D object data labels (center camera), YOLO format.
Each frame of the dataset is named with the time of collection. For example, one frame appears as such:
msu4s/
└── 2023_late_summer/
├── top_left/
│ └── 1692390643884248064_top_left.jpg
├── top_mid/
│ └── 1692390643884248064_top_mid.jpg
├── top_right/
│ └── 1692390643884248064_top_right.jpg
├── oust/
│ └── 1692390643884248064_oust.pcd
├── velo/
│ └── 1692390643884248064_velo.pcd
├── misc/
│ └── 1692390643884248064_misc.yaml
├── label2d/
│ └── 1692390643884248064_label2d.txt
└── label3d/
└── 1692390643884248064_label3d.yaml
All frames are time synchronized in postprocessing, using the timestamp from each frame of the Velodyne lidar as the synchronization target.
Not all frames have published 3D and 2D labels; these labels are being reserved for future applications.
Format of misc.yaml
The misc.yaml
file contains sensor readings from all non-camera and non-lidar sensors on the vehicle. Here is a selection of relevant keys:
gnss
: GNSS location data from the Piksi Multi RTK GNSS. acc
subkey is an estimated variance based on current fix type.
seasons
: Current season. Current labeled seasons are spring
, summer
, fall
, and winter
.
weather
: Labeled weather. Current labeled weather conditions are clear
, rain
, and snow
.
zone
: Sub-area of campus where frame was collected.
steering_angle
: Approximate angle of steering. Based on publicly available specifications, the 2017 Chevy Bolt has a steering ratio of 16.8:1, a wheelbase of 260.096 cm (102.4 in) and a track width of 150.1 cm (59 in).
/can/chassis/imu
- Data from the vehicle’s built-in IMU.
/sensors/imu/ouster_1/imu
- Data from the IMU on board the Ouster OS-1 lidar.
/sensors/imu/piksi/imu
- Data from the Piksi Multi GNSS’s onboard IMU. Orientation is not provided by the onboard IMU and has a reported covariance matrix of all -1.
/sensors/imu/yost/imu
- Data from a Yost 9DOF IMU. Orientation is not absolutely calibrated to true north.
/sensors/camera/{top_left,top_mid,top_right}/image_color/compressed
- Contains supplemental data for each of the cameras. Includes K and d matrices for dedistorting imagery, plus K_corr for a precomputed projection matrix for dedistorted imagery.
transforms
: The static transform tree for all sensors on the vehicle. See the Transforms subsection for more details.
Sensor Transforms / Frames of Reference
We strongly recommend code using this dataset uses per-frame transform data instead of static transform data.
Errata
Velodyne Lidar
We have noticed that in some segments, the Velodyne lidar data has artifacts in the output data. Based on manual review of data, we believe the sensor began to malfunction mid-collection, and thus should not be used when accuracy is desired. We are providing the Velodyne lidar data as-is in the hopes it can be useful for certain applications.
Radar
2023_late_summer
and 2023_early_fall
only have radar coverage from radars 2 and 5 (front and rear facing, respectively); all other radar channels in these segments have no data.