基于VLM凸优化的网络直播视频场景图生成OA

Livestreaming video platforms have become an important medium for digital content dissemination,social interaction,and commercial activities.This is largely due to their large number of streamers,massive content supply,and ex⁃tremely high daily active user base.However,the real-time and unpredictable nature of livestreaming content poses serious challenges for online content supervision and regulation.Video scene graphs provide a structured representation for video understanding.They describe objects,attributes,and behavioral relationships within videos.By constructing a semantic net⁃work of"object-relation-action"in the spatiotemporal domain,video scene graphs enable structured modeling of video con⁃tent.In recent years,vision-language models(VLMs)have shown strong capabilities in cross-modal semantic understanding and complex scene reasoning.These advantages provide new technical support for livestreaming video scene graph genera⁃tion.Although VLMs can significantly improve semantic parsing accuracy in complex livestreaming scenarios,they still face an important challenge.Specifically,it is difficult to effectively capture the feature distribution patterns of livestream⁃ing videos.Convex optimization plays an important role in training VLMs.It helps guide the model to converge toward a global optimal solution.Based on this observation,this paper proposes a VLM-based convex optimization for scene graph generation(VCO-SGG).The method constructs a VLM-based approximately convex optimization framework that con⁃strains the geometric structure of the feature space for object semantics and their relationships,reducing feature distribution discrepancies and mitigating convergence oscillations during VLM training.A dynamic prototypical memory module is in⁃troduced,employing a parametric memory mechanism to strengthen the memory of key semantic elements'continuity and correlations across video frames.Furthermore,a feature association and relation filtering strategy is proposed to identify and filter redundant object indices online,which are generated in the scene graph due to dynamic changes,thereby enabling dy⁃namic generation and updating the scene graph.Experimental results demonstrate that our method achieves improvements of R@10 and mR@10 reaching 55.41％and 34.82％,on the self-built livestreaming video dataset BJUT-LGSD,respective⁃ly.In the publicly available datasets Mini Charades and Mini Action Genome datasets,R@10 and mR@10 are further im⁃proved to 48.19％/28.02％and 43.42％/26.02％,respectively,and the inference speed is 22.36 FPS.Overall,the results dem⁃onstrate greater competitiveness than other methods,indicating its capability to handle the task of generating scene graphs for livestreaming videos.