CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection
Our most-cited work, pioneering a late-fusion approach for combining 2D and 3D detectors to achieve state-of-the-art 3D object detection.
Faculty Director
Associate Professor
Biosystems & Agricultural Engineering
Electrical & Computer Engineering
Michigan State University
About
What is exciting right now is solving computer vision problems using machine learning and artificial intelligence techniques. Tasks that seemed way beyond reach a decade ago have been solved, but there are still many problems needing solutions.
Smart Agriculture, I believe, is the next big frontier for automation. We are exploring automated phenotyping and segmentation in cluttered scenes, as well as posture estimation of animals in depth images. I am actively involved in MSU NRT-IMPACTS, a program seeking to bridge the gap between computational and plant sciences through mentoring, teaching and research.
Automated Vehicles need vision to function safely and reliably. My students and I are finding ways to fuse video and lidar to obtain accurate, high-resolution depth maps that do not smear along the boundaries. We're also solving problems in radar-video fusion, tracking and object classification.
Expertise
Developing algorithms for scene understanding from heterogeneous sensor data, enabling machines to perceive complex 3D environments.
Fusing radar, LiDAR, and camera data to achieve superior accuracy for depth completion, object detection, and scene modeling.
Applying convolutional neural networks and novel architectures to solve challenging detection, segmentation, and tracking problems.
Pioneering late-fusion approaches combining 2D and 3D detectors for robust real-time object detection in autonomous driving scenarios.
Automated animal monitoring, posture estimation, crop phenotyping, and label-efficient learning for farm intelligence.
Contactless tremor measurement via pose and optical flow fusion (POFF) for Parkinson's Disease patient assessment.
Featured Work
Highlights spanning top venues including CVPR, ICCV, IROS, and WACV.
Our most-cited work, pioneering a late-fusion approach for combining 2D and 3D detectors to achieve state-of-the-art 3D object detection.
State-of-the-art probabilistic radar target modeling fused with camera data for next-generation autonomous driving perception.
Object-scene decoupling method enabling physics-based interaction with 3D Gaussian scenes.
Teaching
Circuits are fun and this course is a great way to learn the basics of analyzing linear circuits. Topics include Ohm's law, Kirchhoff's circuit laws, superposition, Thevenin and Norton equivalent circuits, Node Voltage Analysis, Mesh Current Analysis, as well as first and second order circuit responses.
We are on the cusp of a transportation revolution as our vehicles will soon take over the task of navigating us to where we want to go. This class covers sensor processing, object detection, planning, and control. Hands-on with ROS 2, Python, OpenCV, PyTorch, Turtlebots, Gazebo, and the CARLA driving simulator.
Created a new graduate-level course covering modern deep learning architectures, training techniques, and benchmark evaluation for computer vision tasks.
Beyond Academia
Co-Founder · 2021 – Present
Bringing computer vision technologies to precision livestock farming with automated monitoring and health assessment tools.
Collaborator · Indy Autonomous Challenge
Joint MSU – Politecnico di Milano team. First Place Passing Competition ($150K) and Second Place Timed Trials ($50K) at Indianapolis Motor Speedway, Sep 2024. Set the Autonomous Hill-Climb Time Record at Goodwood Festival of Speed, UK, Jul 2024.
MSU CANR Innovation of the Year
Novel sensor systems that monitor female swine, estimating body condition and lameness for proactive veterinary treatment. Team: M. Benjamin, D. Morris, S. Yik, M. Lavagnino.
Active Participant
A program seeking to bridge the gap between computational and plant sciences through mentoring, teaching and collaborative research.
Mentoring
Current and former PhD and Masters students advised as committee chair.
Service