非结构环境下串型水果防碰损采摘的视觉认知与夹剪机构耦合机制

Picking robot under the effect of non-structural factors including overlapping, adhesion and occlusion between bunched fruits may very easily damage the fruits due to inaccurate target positioning, incorrect clamping-cutting sequence and improper clamping-cutting gesture. The main cause of this damage is that the coupling mechanism between visual cognition and clamping-cutting actuators for undamaged picking is not settled well. To realize high efficient picking with low damage rate, this project taking grape and litchi as research objectives focused on the deep study as follows. Firstly, the innovative visual perception algorithms are proposed and multi-dimensional information of bunched fruits such as cutting point, stem gesture and the bounding volume to avoid damage are extracted. Secondly the rules to avoid collision fruit are built based on picking agronomic; the priority function of clamping-cutting is established by combining knowledge inference and intelligent decision theory; visual cognitive computing and programming on multi-objective undamaged picking sequence is conducted by use of mathematical programming methods. Thirdly, based on multi-dimensional information and picking sequence, the autonomous position coupling model between the actuator and anti-collision feeding gesture & clamping–cutting angle of bunches fruit is obtained by combining mechanism-vision collaborative modeling method and multivariate optimization theory.Forthly, the coupling model and actuator parameters are verified and corrected repeatedly through hardware-in-loop simulation and prototype experiments. The inner behavior mechanism between visual cognition and actuator coupling for undamaged picking are finally explored, and this research achievements will provide scientific base and theory support for the design of bunch fruits anti-collision picking actuator and its visual servo control system.

串型水果间相互重叠、贴靠或遮挡等非结构因素使得采摘机器人极易因目标定位不准、采摘顺序不当、夹剪位姿不合理等造成果实碰伤或刮落，造成该损伤的主要原因是防碰损采摘的视觉认知与夹剪机构耦合问题未得到解决，因此，本项目以葡萄和荔枝为对象，对如下内容进行深入研究：①创新视觉感知算法提取非结构环境下果串采摘点、果梗位姿、防碰包围体等多维信息；②依据采摘农艺构建果串防碰夹剪的规避准则，结合知识推理和智能决策理论建立果串夹剪优先度函数，再用数学规划方法对多目标防碰采摘顺序进行视觉认知规划；③以果串多维信息和采摘顺序为基础，运用机构-视觉协同建模与多元优化理论构建执行机构与果串间防碰进给方位和夹剪角度的自主位姿耦合模型；④通过硬件在环仿真和样机试验反复验证和修正耦合模型及夹剪机构参数。最终揭示防碰损采摘的视觉认知与夹剪机构耦合的内在行为机制，为串型水果防碰损采摘机构及其视觉系统设计提供科学依据与支撑理论。

针对非结构果园环境下串型水果防碰损采摘的视觉认知行为与采摘机构之间的协同规划问题，从视觉感知、认知规划、柔性采摘机构等方面入手开展了深入研究。①为准确提取果实目标的多维视觉信息，项目提出基于深度卷积神经网络的果园环境下葡萄图像识别方法，设计了叠贴葡萄果梗识别及最优采摘定位模型，研究基于局部点云数据的葡萄位姿估计方法，设计出葡萄采摘预备点的定位方法，在末端执行器到达最佳采摘点前进行姿态调整，使末端执行器处于最佳剪切姿态，降低采摘碰损风险。②针对现有技术采摘多串堆叠葡萄容易发生碰损的问题，本项目采用四象限法则与距离加权择优等方法对多个采摘目标进行顺序规划，根据果农的采摘经验建立紧急坐标系和重要坐标系，确定象限权重，计算目标优先函数。再利用能量优化方法计算路径规划中的采摘序列。基于人工势场和采样搜索法自动生成防碰撞路径点，并设计基于硬件在环仿真的采摘机器人视觉感知与作业规划试验系统，实现了采摘机器人无碰撞自主路径规划仿真。③依据葡萄生长形态及果串的物理参数，设计了夹持-托举-剪断式串型水果采摘机器人末端执行器；利用气驱软体控制原理，设计基于柔性抓持与夹剪一体的串果采摘执行器，试制采摘执行器物理样机，结合实验室已有的协作机器臂搭建了采摘机器人物理试验系统，通过采摘试验优化末端执行器和视觉规划算法，有效揭示了串型水果防碰损采摘中果实信息感知、视觉认知及智能规划与夹剪机构之间的关联机制，为串型水果防碰损采摘机构及其视觉伺服控制系统设计提供科学依据和支持理论。