Using DP Towards A Shortest Path Problem-Related Application

VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control

Real-time Semantic Segmentation in Autonomous Bus

The video shows the coarse segmentation in industrial zone, applying deep learning models.

Model Predictive Control on golfcar

Based on perception from LIDAR, we design a MPC on golfcar to control it’s steering wheel angle to follow the given trajectory.

Recent data collection of the USV platform on the HKUST shoreline

We record the data collection experiments of our USV platform with Lidar, omnidirectional camera, IMU and RTK DGPS.


Tranversibility extraction from 16-beam LIDAR

We extract in real-time the tranversible of an on-road vehicle under dynamic mobile situation.

A recent collection of 3D perception

Here is a collection of our recent results in 3D perception in the context of autonomous driving, including key object detection, path planning, local mapping, tranversibility extraction, etc.

A recent collection of research in autonomous boat

A recent collection of results on autonomous boat system including planning, tranversibility extraction, mapping, etc.

LIDAR Odometry Demo

LiDAR and Inertial Fusion

We demonstrate the state-of-the-art performance for real-time LiDAR-based odometry.

Socially-compliant Navigation through Generative Adversarial Imitation Learning

Real-time Nonholonomic Robot Navigation in Dynamic Environment

Neural SLAM

Deep Reinforcement Learning in Robotics Navigation

Multi-agent warehouse simulation and task allocation

Robotic engineering-aided neural engineering

We built and validated a high-speed high-precision tracking system for mouse test subjects, which aids neural engineering experiments on animals. The result is published recently on Biomaterials link

Mechanics for mouse tracking

Deep-learning-based human-like decision-making and exploration


Deep Visual Homing throug Omnidirectional Camera

Asynchronous Deep Reinforcement Learning

One of the first real Cloud Robotic Systems

A remote controlled mobile robot based on IntoRobot Cloud ( and IntoRobot Atom board. The control of the robot is through Internet. The robot can be shared to other via WeChat and WhatsApp easily.

Omnidirectional Camera Calibration Toolbox

Author:Jonas Eichenberger
This is a calibration toolbox for omnidirectional cameras which can be used with MATLAB. The most common omnidirectional camera models can be used for calibration and the toolbox has good corner extraction capabilities even for quite distorted omnidirectional images.

The toolbox is based on LIBOMNICAL.

This toolbox fixes some bugs, adds some additional features and was mainly used and tested with the Mei model. Therefore the other model implementations need probably some bug fixes too and everybody is welcome to contribute!

The idea is to have a general omnidirectional camera calibration toolbox for all the common models and a platform for future model implementations.

More Information see our GitHub

Tango-based application (on-going)


libcnn: A library for real-time robotic recognition applications based on CNN

libcnn is a modular deep learning libraray, useful for robotics and computer vision. More Information see our GitHub

Scene labeling: libcnn scene labeling

Object labeling: libcnn object labeling

Visible Light Communication-based Localization and Path-planning

Localization using a photonic sensor mounted on a tablet. It reaches to high localization precision as shown in the video. Please refer to:

Kejie Qiu, Fangyi Zhang, Ming Liu, Visible Light Communication-based Indoor Localization using Gaussian Process, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 2015

Ming Liu, Kejie Qiu, Shaohua Li, Fengyu Che, Liang Wu, C. Patrick Yue, Towards Indoor Localization using Visible Light Communication for Consumer Electronic Devices, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014

Multi-robot 3D SLAM based on LSD-SLAM (collaboration with ETH - ongoing)

The individual maps generated by LSD-SLAM over multiple robots are fused.

Multi-robot 2D SLAM without known initialization

Multiple robots will move across unknown environments, so that a complete map will be constructed once the co-localization can be achieved.

Internet-of-things control of a flowerpot

Online Extrinsic calibration between extroceptive Sensors

Guoyang Xie, Tao Xu, Carsten Isert, Michael Aeberhard, Shaohua Li, Ming Liu, Online Active Calibration for a Multi-LRF System, IEEE 18th International Conference on Intelligent Transportation Systems (ITS), 2015

Online Calibration

Visual Homing-based topological Navigation

The mobile robot navigates in an indoor environment with illuminant changes by using an Omnidirectional Camera. Please refer to:

Ming Liu, Cedric Pradalier, Roland Siegwart, Visual Homing from Scale with an Uncalibrated Omnidirectional Camera, IEEE Transactions on Robotics (TRO), Vol. 29, Issue 6, pp.1353 - 1365, Dec. 2013.

3D modeling and mapping in real-time

This a demo of 3D registration by ICP, based on the library proposed in:

Francois Pomerleau, Stephane Magnenat, Francis Colas, Ming Liu, Roland Siegwart, Tracking a Depth Camera: Parameter Exploration for Fast ICP, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011

Localization by 1-bit sensor for educational Robots

This is a joint work with EPFL, U Leuven, INRIA and CityU. We localize an educational robots using a inferred sensor on gray-scale images.

Shiling Wang, Francis Colas, Ming Liu, Francesco Mondada, Stéphane Magnenat, Localization of inexpensive robots with low-bandwidth sensors, DARS 2016, pdf

Point-cloud analysis for semantic labelling using Tensor Voting

Ming Liu, Francois Pomerleau, Francis Colas and Roland Siegwart, Normal Estimation for Pointcloud using GPU based Sparse Tensor Voting, IEEE International Conference on Robotics and Biomimetics (ROBIO), 2012,

Ming Liu, Efficient Segmentation and Plane Modeling of point-cloud for structured environment by Normal Clustering and Tensor Voting, in Proceedings of the IEEE International Conference on Robotics and Biomimetics, (ROBIO) 2014

Ming Liu, Roland Siegwart, Information Theory based Validation for Point-cloud Segmentation aided by Tensor Voting, Best Paper in Information IEEE International Conference on Information and Automation (ICIA), 2013

Incremental regional topological segmentation for environment surveillance

Ming Liu, Luc Oth, Francis Colas, Roland Siegwart, Incremental Topological Segmentation for Semi-structured Environments, Autonomous Robots (AURO), Vol. 37, Issue 3, Aug. 2014.

Color-based description of environment and topological scene recognition

Ming Liu, Roland Siegwart, Topological mapping and scene recognition with lightweight color descriptors for omnidirectional camera, IEEE Transactions on Robotics (TRO), Vol. 30, Issue 2, pp. 310- 324, April. 2014.

Resource allocation among networked robots using real-time wireless communication

Matthew Tan, Ming Liu and Roland Siegwart, An Experimental Evaluation of the RT-WMP Routing Protocol in an Indoor Environment, Best Paper Finalist IEEE International Conference on Information and Automation (ICIA), 2013