Research
We develop algorithms that enable machines to perceive, understand, and act in complex environments — spanning precision agriculture and autonomous vehicles.
Smart agriculture is the next big frontier for automation. New sensors combined with deep learning enable individual animal monitoring and health assessment, automated crop inspection, and precision phenotyping. We collaborate with agriculture experts to improve animal welfare and raise crop yields through computer vision.
A two-stage system that automates sow body condition scoring from ceiling-mounted RGB-D depth frames, jointly predicting backfat thickness, loin muscle depth, and total tissue thickness at the last rib. Reaches 3.87 mm mean absolute error, beating ResNet-18 and ViT-small baselines by 22% and 39%.
3,000+ training images with 23,000+ annotated instances for multi-view pig posture recognition. Baseline F1 scores are only 43–53%, so there is lots of room for improvement. The Kaggle challenge has concluded, but the dataset remains available for research. Paper Coming Soon!
Cameras and LED arrays automatically disrupt piling in cage-free poultry, reducing suffocation risk and improving welfare.
Multi-modal RGB and RGB-D dataset of gilt (female breeding pig) activities including eating, drinking, and moving. Presented at USPLF 2025 for precision livestock farming research and behavior modeling.
Affordable (under $3,000), open-source AgOpenGPS autosteer system integrated onto an electric tractor. Achieves centimeter-level row-following accuracy when combined with an RTK base station.
Survey of publicly available computer-vision datasets for precision livestock farming research, covering modalities, tasks, and coverage gaps.
Smeared pixels are a problem with many consumer depth sensors used in building 3D models. A self-annotated approach eliminates the need for manual annotation in training a smeared pixel removal algorithm.
Comprehensive review of learning with fewer annotations, addressing the challenge of limited labeled data across plant agriculture tasks.
Analysis of what is needed when building datasets to support Automated Behavior Analysis of animals, led by Janice Siegford with a focus on welfare-relevant behaviors.
MSU College of Agriculture and Natural Resources 2022 Innovation of the Year, awarded to Madonna Benjamin, Daniel Morris, Steven Yik, and Michael Lavagnino for sensor systems that monitor female swine body condition and lameness.
UAS-mounted LiDAR scans a maize diversity panel periodically over the growing season. Eleven phenotypes estimated with high correlation to manual measurements for six of them, using voxel-based growth models.
CNN-based segmentation of bean seed coats to analyze color genetics of dry beans, greatly speeding up the processing of large numbers of beans. Collaboration with Rie Sadohara and Karen Cichy.
Semi-supervised deep learning methods for accurate detection and posture estimation of pigs in depth images. Best-paper finalist for Agri-Robotics, with 2-sigma ellipses illustrating joint estimation precision.
Proposed Bean Split Ratio as a measure for quantifying bean quality related to splits after cooking and canning, with an automated algorithm for calculating it. If used to select new varieties, this can contribute towards improving bean quality.
Pyramid CNN approach for segmenting individual leaves in dense foliage with large internal variations and texture, automatically estimating boundaries for each leaf.
Object discrimination for off-road mobile robot navigation, finding hazards such as tree trunks, rocks, and cones that may be partially occluded by foliage.
Innovative sensor development to assess photosynthesis rates and their distribution on plant bodies, supporting crop breeding, selection, and modification.
We develop deep-learning methods to fuse data from multi-modal sensors — video, LiDAR, and radar — to achieve better accuracy in scene understanding. From depth completion to 3D object detection and multi-object tracking, our algorithms push the boundaries of what vehicles can perceive.
State-of-the-art fusion technique for radar and camera data. Probabilistically models radar returns on targets, enabling improved detection accuracy from an otherwise noisy data source.
Collaboration with Politecnico di Milano and University of Alabama developing sensing and control for an Indy autonomous racecar. Set a passing record and won the passing competition at Indianapolis Motor Speedway, September 2024. Races in Las Vegas, Monza, Goodwood, and Indianapolis.
Automotive radar + camera fusion for 3D object detection using transformer-based cross-attention between modalities.
Associates radar returns with image-based object centers to improve the performance of image-based 3D object detection.
Su Pang's latest LiDAR-camera fusion achieves a big speed-up and reduced memory over CLOCs, now running on a single-GPU computer for state-of-the-art 3D detection.
Yunfei Long's work calculates full-velocity radar returns using radar-camera fusion, overcoming the limitation of Doppler radar which only provides radial velocity measurements.
Saif Imran's improved depth completion addresses the tricky problem of super-resolving depth at occlusion boundaries by extrapolating both foreground and background surfaces across the boundary, then fusing them.
Yunfei Long's radar-camera depth completion method probabilistically associates radar returns to image pixels, then uses radar plus images to estimate a dense depth map.
Su Pang's tracking method using Random Finite Sets for more rigorous probabilistic scene modeling obtains state-of-the-art tracking performance.
Su Pang's Camera-LiDAR fusion combines 3D and 2D detection methods to achieve superior 3D performance. With very little overhead, the fusion network achieves state-of-the-art results. Our most-cited work.
Automated vehicles create dangers and opportunities for improved safety of pedalcyclists as they share the road. Surveys detection and localization methods across modalities and datasets.
Traditional ray models for LiDAR beams cause shape erosion or dilation, affecting width estimation of thin objects. A linearly diverging beam model with finite beam width yields more accurate object dimensions.
Saif Imran, Xiaoming Liu, and Daniel Morris developed color-image-guided upsampling of LiDAR that avoids smearing depth across object boundaries and depth discontinuities.
HD maps enable 10 cm-accuracy localization without GPS. FLAME (Feature Likelihood Acquisition Map Emulation) achieves similar localization accuracy with a map 1/1000th the size.
Accurate vehicle pose tracking and turning models enable rapid vehicle turning prediction, reducing collision risk especially at intersections.
The MSU Autodrive Challenge team integrated sensing and control hardware and software into a GM Bolt.
Robust ground measurement cost function that accounts for occlusions and clutter. Modeled with a Markov Random Field and optimized with Loopy Belief Propagation for high-quality ground segmentation of LiDAR data.
Within 3D LiDAR point clouds we seek objects such as people and vehicles, track them, and provide trajectory predictions — including clustering, clutter rejection, shape and motion models, and occlusion handling.
LiDAR range measurements are precise but sparse at long range, making object shape and category difficult to determine. 3D shape-based categorization methods classify object types from sparse point clouds.
Live Demos
Real-time video demonstrations of our computer vision systems running on livestock and poultry monitoring setups.
Real-time multi-pig keypoint detection and skeleton tracking with pose estimation overlays in a swine research pen.
Automated posture recognition — standing, sternal lying, lateral lying — with bounding box detection and confidence scores.
Camera-based monitoring system with real-time piling risk indicator for cage-free poultry welfare intervention.
Overhead camera view of cage-free poultry for automated behavior monitoring and welfare assessment.
Walkthrough of the LED intervention system in a poultry barn, showing green LED strips used to mitigate piling behavior.
