船载多源感知系统及深度学习在其中的应用研究进展OA

Distant-water fisheries are vital to China's marine resource development and food security,with approximately 1 500 vessels operating on the high seas under extreme conditions including heavy fog,rain,snow,and dynamic sea states.Although electronic monitoring(EM)and vessel monitoring systems(VMS)have been widely deployed,they remain limited by single-modal data reliance,insufficient intelligence for automated analysis,and prohibitive satellite communication costs for raw video transmission.To address these challenges,shipborne multi-source perception systems integrating heterogeneous sensors with deep learning algorithms have emerged as a promising solution.This review systematically examines the hardware architecture,core algorithms,and future directions of such systems.The hardware system comprises visible and infrared cameras for all-weather video perception,conductivity-temperature-depth(CTD)sensors for in-situ seawater parameters measurement,and a multi-source data processing terminal serving as the central hub for data aggregation,intelligent fusion,and reliable communication.A complete engineering implementation of"shipborne multi-source perception and AI-integrated intelligent terminal equipment suite"is presented,which achieves edge-based real-time analysis through embedded AI computing platforms and supports multi-modal communication including GNSS and satellite links.In object detection,the technological evolution from two-stage detectors(Faster R-CNN,Cascade R-CNN)to single-stage detectors(YOLO series,SSD),anchor-free methods(CenterNet,FCOS),and end-to-end Transformer architectures(DETR,DyFish-DETR,FMRFT)is reviewed.Empirical comparisons show that YOLOv8 achieves 97.1%mAP@0.5 for vessel detection and 95.7%for fish detection at 62 FPS,significantly outperforming Faster R-CNN(88.4%and 81.2%at 18 FPS).A fundamental cause analysis reveals that two-stage detectors are more vulnerable to hull-induced geometric distortion because their region proposal networks rely on spatial feature consistency,while single-stage detectors perform dense regression directly on feature maps with higher tolerance.In action recognition,the progression from two-stream networks and 3D convolutional networks to temporal segment networks and skeleton-based models is analyzed.Skeleton-based models(ST-GCN,PoseConv3D)demonstrate unique advantages through three synergistic mechanisms:1)Physical-level robustness against illumination and background interference-PoseConv3D maintains 0.3%accuracy loss under 50%keypoint dropout versus 5.5%for GCN;2)Representation-level invariance to global motion from vessel rolling,which can be learned through data augmentation in graph convolutional networks;3)Computational-level efficiency with input dimensions of 50-75 compared to 150 000 for RGB frames,significantly reducing overfitting risk in data-scarce fishery scenarios.In multimodal fusion,joint representation methods(feature-level,model-level,decision-level fusion)and collaborative representation methods are compared,with applications in vessel detection and fish behavior analysis.Two novel paradigms for shipborne scenarios are proposed:physics-informed neural network-guided fusion that embeds the TEOS-10 equation of state as a physical constraint,and environment-adaptive dynamic fusion that adjusts visible,infrared,and CTD modality weights based on real-time sea state classification using lightweight MobileNetV3 classifiers.Key challenges are identified:1)Scarcity of specialized fishery datasets with extreme weather conditions and rare safety events;2)Systematic violation of fundamental computer vision assumptions——Brightness constancy is disrupted by sea surface specular reflection,temporal consistency is violated by six-degree-of-freedom vessel motion,and input clarity is degraded by sea fog and salt spray;3)Strict computational constraints on fishing vessels limiting model complexity;4)Lack of interpretability in safety-critical deep learning models.Future directions include digital twin-based synthetic data generation,domain adaptation for robust perception across varying sea conditions,and knowledge distillation combined with edge-cloud collaborative inference for model lightweighting on resource-constrained platforms.