.png){kind=icon}
.png){kind=icon}
Return
Evaluating Synthetic Data Generation from User Generated Text
.png){kind=icon}
OA
AI
DOI：10.1162/coli_a_00540.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
User-generated content provides a rich resource to study social and behavioral phenomena.Although its application potential is currently limited by the paucity of expert labels and theprivacy risks inherent in personal data, synthetic data can help mitigate this bottleneck. Inthis work, we introduce an evaluation framework to facilitate research on synthetic languagedata generation for user-generated text. We define a set of aspects for assessing data quality,namely, style preservation, meaning preservation, and divergence, as a proxy for privacy. Weintroduce metrics corresponding to each aspect. Moreover, through a set of generation strategiesand representative tasks and baselines across domains, we demonstrate the relation betweenthe quality aspects of synthetic user generated content, generation strategies, metrics, anddownstream performance. To our knowledge, our work is the first unified evaluation frameworkfor user-generated text in relation to the specified aspects, offering both intrinsic and extrinsicevaluation. We envisage it will facilitate developments towards shareable, high-quality syntheticlanguage data
Keywords:
SYNCHRONY
Journal
IF:
5.3
Papers:
837
Citations:
2.7K