首页|期刊导航|Computers, Materials & Continua|Intra-Video Temporal-Aware RAG: A Self-Contained Framework for Video-Based Question Answering

Intra-Video Temporal-Aware RAG: A Self-Contained Framework for Video-Based Question AnsweringOA

中文摘要

Lecture videos are widely used in modern education,yet answering questions from them remains challenging.Relevant information is often distributed across time and expressed through multiple modalities,including speech,slides,and visual content.Existing VideoQA approaches,including recent retrieval-augmented generation(RAG)methods,typically rely on static text representations or global video features.Consequently,they may retrieve evidence that is semantically relevant but temporally misaligned,leading to inaccurate or weakly grounded responses.In addition,dependence on external knowledge sources can introduce hallucinations and reduce reliability in educational settings.To address these limitations,we propose a temporally aware,intra-video RAG framework tailored for lecture videos.The approach aligns automatic speech transcripts and visual captions into timestamped segments and performs retrieval constrained by temporal boundaries.Retrieved segments are further refined using a cross-encoder before answer generation,ensuring that responses are grounded in the correct portions of the video.We evaluate the proposed method on the LectQA-Vid dataset,consisting of 100 lecture videos and 3000 temporally annotated questions.Experimental results demonstrate improved factual alignment and robustness over non-temporal baselines,highlighting the importance of temporal grounding in lecture VideoQA.

Sumaira Shafiq;Naveed Ejaz;Munam Ali Shah;Rashid Kamal;Adnan Sohail;Sheraz Aslam

Department of Computing and Technology,Islamabad Campus,Iqra University,Islamabad,PakistanSchool of Computing,Ulster University,Belfast,UKDepartment of Computer Networks and Communication,College of Computer Science and Information Technology,King Faisal University,Al-Ahsa,Saudi ArabiaSchool of Computing,Ulster University,Belfast,UKDepartment of Computing and Technology,Islamabad Campus,Iqra University,Islamabad,PakistanDepartment of Computer Science,CTL Eurocollege,Limassol,Cyprus Department of Computer Science,American University of Cyprus,Larnaca,Cyprus International Digital Economy College,Minjiang University,Fuzhou,China

信息技术与安全科学

Video question answeringretrieval-augmented generationtemporal groundingmultimodal retrievaleducational videoswhisper ASRvisual captioninglarge language modelsexplainable AItimestamped evidence

《Computers, Materials & Continua》 2026 (8)

P.2276-2301,26

supported by the Deanship of Scientific Research,Vice Presidency for Graduate Studies and Scientific Research,King Faisal University,Saudi Arabia,under Grant KFU262069.

10.32604/cmc.2026.081534

评论