面向机器人整枝作业的温室番茄主茎识别方法与系统

Manual pruning is costly on time and labor, however, which is necessary for greenhouse tomato cultivation. In order to meet the need for robotic pruning, the system and method of tomato’s main-stem identification based on visional information was researched in this project. In view of the limitation on robot’s visual perception caused from the color similarity of stem and leaf and the complexity of background, a multi-spectral image acquiring system was designed based on the spectral characteristic difference of plant’s different organ. And the analysis method on the multi-spectral images was researched, so that the main-stem’s feature on grayscale and shape could be accurately described. The strategy for fusing the multi-spectral image information was implemented, before the classifying mode for the main-stem’s visible area was built. The multi-view image’s registration technique was adapted to measure the visible stem’s 3D parameter, including position and posture. Then, the main-stem curve-fitting method was researched to recover the occluded part of main-stem based on the discrete stem’s 3D information, to overcome the difficulty to identify and locate the whole main-stem from the complex plant background. The visual method and system for robotic pruning was also supposed to be important for the harvesting, spraying and pollinating robot.

整枝疏叶是番茄栽培管理的必要环节，人工作业耗时、费力。本项目面向温室番茄整枝机器人智能化作业需要，从多元图像信息获取和融合两个方面着手，研究番茄植株主茎形态视觉识别系统与方法。为了克服温室内植株茎叶色系相近、背景复杂多样的工况条件对视觉信息感知的客观限制，基于植物器官组织光谱反射特性差异，构建多波段光谱图像信息在线获取系统；研究番茄主茎光谱图像灰度与形态多元信息解析和特征提取方法，形成多波段光谱图像信息融合策略，建立针对主茎离散可见区域的图像分割模型；采用多视角图像动态采集配准技术，获取主茎可见区域的空间位姿参数；研究丛生番茄主茎整体形态曲线拟合方法，以实现对主茎受遮挡区域信息的恢复，解决茎叶交错重叠信息缺失的问题。项目最终形成番茄主茎识别定位系统和方法，可为番茄整枝、采摘、喷药以及授粉等环节的机器人作业提供技术支撑。

面向番茄植株机器人整枝的实际需要，以工厂化温室番茄植株为研究对象，结合温室环境光照多变、枝叶丛生、色系相近的工况特点，针对番茄整枝涉及的近色系目标识别、自然光照下图像色彩矫正、丛生目标识别和立体测量难题，围绕图像采集、色彩矫正、目标识别和立体测量等四方面共性关键技术开展研究突破。提出了茎秆、叶片和绿果等三类相近色目标的多波段图像融合方法，以凸显目标与背景亮度差异，提高目标视觉识别效率；研究了基于高动态范围成像技术的番茄植株图像色彩矫正方法，以克服农业复杂自然光照条件对作业对象色彩稳定呈现的客观限制；研究了基于Mask RCNN深度学习模型的目标区域分割与整枝操作点定位方法，为机器人整枝执行器提供定位依据；研究了植株主茎视觉动态跟踪与立体测量方法，以提高对叶、果和花等作业对象的搜索效率。形成了温室番茄智能化整枝视觉信息获取视觉系统，集成人工光源、滤光片切换转轮、灰度/彩色相机、双面视觉和旋转云台等关键部件；在此基础上，开发了温室番茄整枝打叶机器人控制软件，实现了相关算法的在线测试试验。视觉系统对番茄主茎动态识别准确率为85%，。距地面高度600mm-1500mm 作业区域内，对主茎图像拟合拼接平均偏差为9.86mm（姿态偏差3.77°），主茎长度、高度和生长倾角测量结果平均偏差分别为46.20mm、18.60mm 和4.33°。发表学术论文13篇、SCI/EI 检索12篇，申报发明专利4项、授权发明专利2项，获得软件著作权1项。项目成果作为整枝机器人的关键部件，是温室管理智能化作业的必要支撑，为后期作业机器人整机的开发形成必要技术支撑，对于降低设施番茄人力成本投入、提高种植产业效益具有重要意义。