Liyou Zhou

University of Cambridge in collaboration with Wayve Technologies Ltd

Research Interests

SLAM, NERF, VLA models.

Presentations

“Exploration of LLM-Enhanced State-Machine function-calls for Planning Robot Actions” (oral) – AgriFoRwArdS CDT Summer School: Robotic Phenotyping [July 2024] – Wageningen, The Netherlands.
“Splat Gym: A Neural Simulator Environment for Training Fruit Picking Robots” (poster) – AgriFoRwArdS CDT Summer School: Robotic Phenotyping [July 2024] – Wageningen, The Netherlands.

About me

I have a background in autonomous driving and IoT software development. I have been a long-time proponent and contributor of the ROS ecosystem. I am interested in exploring the emerging world of end-to-end machine learning in robotics through the CDT. I am eager to learn more about the agriculture domain area and the many opportunities for optimisation and disruption advanced robotics can bring

MSc Project

Robot Learning in Neural Simulator for fruit picking

In eye in hand fruit picking, the control policy is usually trained via Reinforcement Learning or Behaviour Cloning. It is desirable to have initial training in simulation to improve speed and minimize cost.

However, conventional 3d graphics environment is difficult to build and photo realism is difficult to achieve. While for agriculture deployment, fast adaptation to different working environments is crucial. The emergence of NeRF/Splatts provides a new possibility for building a 3D simulator for fruit picking.

NeRF/Splatts has the following advantages over traditional 3d graphics environment:

Quickly build a 3D training environment from few images.
Photo realistic rendering.
Infer missing information such as partially occluded fruits.

The proposed project will:

Curate an experimental dataset of robot pose and high resolution images.
Construct a 3d scene using neural representation.
Demonstrate the ability to train a control policy in simulation.
Transfer the learning to real-world by running scenarios in the real world robot.

PhD Project

AgriVision 2.0: Vision Language Action Robotics in Agriculture

With the advent of machine learning, purely modeless data driven robotic planning and control becomes a reality. Advanced capabilities are shown to be possible with self-supervised learning from data. Techniques such as Reinforcement Learning, world modelling, generative AI and language models have been successfully applied to robotics to solve a range of complex tasks. I intend to develop advanced robotic learning techniques in the specific context of agriculture applications. Innovation can be derived from the novel applications and push the boundaries of robotic capabilities and deployment in agriculture.

Liyou’s PhD project is being carried out in collaboration with Wayve Technologies Ltd, under the primary supervision of Dr Sebastian Pattinson.

EPSRC Centre for Doctoral Training in Agri-Food Robotics: AgriFoRwArdS site title logo image.