DMMP-Net: diffusion model-based missing part patching network for station air quality data generation completion

Zhenying Li

doi:10.1007/s13042-024-02468-x

Return

DMMP-Net: diffusion model-based missing part patching network for station air quality data generation completion

2024-11-27

PRE

DOI：10.1007/s13042-024-02468-x

Original

Original request for help

Save

Abstract

En 中文

Estimating the missing part of environmental monitoring ground station data is of great significance for environmental monitoring and prediction. However, it is difficult for existing methods to solve the problem of dealing with temporal correlation of station data, spatial correlation, and correlation between pollutant concentration values in missing data completions. Therefore, this paper proposes a diffusion model-based missing part patching network for station air quality data generation completion(DMMP-Net). First, the diffusion model is used to learn the data distribution pattern, and the data with missing values are used as conditional inputs to generate new data without missing values to fill in the data with missing values for the purpose of data enhancement, so that the data can be subsequently applied to the tasks of analyzing the sources of pollution, exploring the components of pollution, and predicting the air quality. Second, we use the attention mechanism to improve the noise estimation network to enhance the feature extraction capability of site air quality data in three dimensions: time, space, and between the concentrations of various pollutants, and to improve the ability of DMMP-Net to learn the features of the data distributions in order to generate accurate complementary data. Experiments are conducted on the data from Beijing regional air quality monitoring stations to prove the effectiveness of DMMP-Net. Compared with the forward substitution method, the mean padding method and the K-nearest neighbor padding algorithm, the evaluation indices of MAE and MRE have better results. In the three cases of random missing, time-continuous missing and space-continuous missing, the evaluation indices of MAE reach 5.532, 10.849 and 12.641, respectively, and the evaluation indices of MRE reach 0.129, 0.243 and 0.342, respectively, and the generation of the replacement effect is better than that of the traditional missing-value filling model.

Keywords:

Air quality

Missing value filling

Diffusion models

Attention mechanisms