基于深度神经网络的煤矿数据中台元数据特征提取与清洗策略研究OA

To address the issues of significant quality differences in multi-source heterogeneous data,coarse metadata management,manual dependency and inability to dynamically evolve cleaning rules,and mismatch between cold and hot data storage and busi-ness characteristics in coal mine data middle platforms,a self-evolving metadata feature extraction and cleaning strategy based on deep neural networks is proposed:①embed an intelligent cleaning and quality feedback layer in the traditional data platform to build a six-in-one full-link governance architecture of"collection-cleaning-integration-storage-analysis-service";②design a deep learning model integrating convolutional neural networks,bidirectional long short-term memory networks,and attention mechan-isms(CNN-BiLSTM-Attention)to extract deep semantic features from field names and statistical features,and automatically gener-ate candidate sets of field-level cleaning rules;③establish a four-level quality label system and use Q-learning to dynamically adjust the weights of cleaning rules and achieve self-evolution of strategies;④build a cold and hot data stratified storage model based on data temperature perception,storing hot,warm,and cold data in Redis,SQL Server,and HBase respectively;⑤propose a five-di-mensional data quality index(DQI)and an automatic quality report generation framework to form a"cleaning-evaluation-feed-back-optimization"closed loop.Application in a coal mine with an annual output of over 10 million tons shows that the F1 value of the deep learning model reaches 91.7％,a 24.3 percentage points increase compared to regular matching;the rule iteration cycle is re-duced from 7 days to 4 hours,and the average repair time for rule failure is decreased from 4.2 hours to 0.5 hours;the data quality pass rate increases from 72.6％to 94.3％,and the DQI rises from 64.7 to 91.2;the average query delay is reduced from 3.8 seconds to 1.2 seconds,and storage costs are cut by 47％.This method realizes a fundamental transformation in coal mine data governance from manual experience-driven to deep neural network-driven and from static rules to self-evolving intelligent cleaning.