🛠️ Current Research - Digital Twin and Virtual Reality for Orchard Robotics
The primary objective of this research is to develop a highly integrated autonomous field robot system capable of precision pruning in smart orchards, including machine vision system development, automatic navigation system development, and end-effector control system development.
Ultimately, we aim to achieve Embodied AI systems based on end-to-end Vision-Language-Action models.
At the beginning, we focus on constructing simulation environments for the training and testing of robots through Digital Twin and Virtual Reality, designing a 3D World Foundation Model for orchards, and providing a systematic and comprehensive benchmark dataset for the field of agricultural engineering.
Supervisor:
- Tenured Associate Professor Wei GUO
- Assistant Professor James BURRIDGE
Related Support:
- Kubota KATR All-terrain Platform Vehicle
- FJD TRION™ P1 LiDAR Scanner
- XGRIDS LixelKity K1 Compact Handheld 3D Scanner
- Various farmlands, orchards and greenhouses located at the Institute for Sustainable Agro-ecosystem Services
✅ Completed Researches
-
Multi-agent collaboration and obstacle avoidance system for wheat production
This work focuses on object detection and tracking based on multi-sensor fusion in unstructured farmland environment. Especially, I’m dedicated to realizing the detection of unknown objects in open environment via few/zero-shot learning, open set recognition, etc.
In addition, I participated in the development of multi-agent collaborative system as a main member. In this project, I am responsible for path planning and task allocation algorithms in the harvesting-unloading-transportation process of wheat production. According to constraints of plot location, task number, operation time window, and path planning, intelligent scheduling model was established based on non-dominated sorting genetic and colony algorithms.
Supervisor:
Object detection and tracking based on multi-sensor fusion
Fusion
- AHRS: Xsens MTi series Attitude and Heading Reference Systems
- GNSS: CHCNAV series Global Navigation Satellite System
- 3D LiDAR: Velodyne Alpha Prime VLS-128 3D Light Detection and Ranging
- Panoramic Camera: FLIR Ladybug series panoramic camera
- 2D LiDAR: Hokuyo UTM-30LX 2D Light Detection and Ranging
Multi-agent collaborative system for agricultural production
-
Robot patrol based on Simultaneous Localization And Mapping (SLAM)
This robot is designed to perform random movement indoors and real-time object detection, which can check whether there are suspicious persons in the room. In this project, I am the team leader and mainly responsible for the development of 3D modeling, object detection, and Simultaneous Localization And Mapping (SLAM). In a ministerial-level innovation competition, our team won the second prize.
-
Robot grasping for fruit picking
This robot is designed to pick simulated tomatoes through narrow roads. In terms of the hardware, I am responsible for mechanical design, 3D modeling and printing of robot chassis and actuators. With regard to the software, I mainly research perception-motion coordinated control algorithm.
-
Grasping and quadruped robots
These robots are designed according to the rules of a national-level innovation competition. In this project, I am the team leader and mainly responsible for the arm control of the grasping robot and the inverse kinematics analysis of the quadruped robot. In this national-level innovation competition, our team won the third prize.
-
Corn morphology detection and distribution machine
This machine is designed for automatic distribution of corns according to their morphology as well as pest and disease conditions. In this project, I am the team leader and mainly responsible for corn detection, servo control, and their coordination. In a national-level innovation competition, our team won the second prize.
-
Remote sensing based on Unmanned Aerial Vehicle (UAV)
This project was supported by the National Natural Science Foundation of China, which is dedicated to the early discovery and continuous monitoring of Mikania Micrantha (an invasive plant).In this project, I am the core member of algorithm development and mainly responsible for feature analysis and semantic segmentation of remote sensing images.