Generating Topic-Agnostic Conversations With LLMs

Harshit Sandilya

doi:10.1109/ACCESS.2024.3473692

Generating Topic-Agnostic Conversations With LLMs

2024-01-01

0

OA

AI

DOI：10.1109/ACCESS.2024.3473692

原文链接

原文求助

摘要

En 中文

Conversational systems are important applications of Artificial Intelligence, encompassing a wide variety of implementations, from rule-based systems to complex systems using Natural Language Processing, Deep Neural Networks, and Transformer Architectures. With the growth of these implementations, the quality of conversational data has become a concern. Many attempts have been made to generate such data, focusing primarily on topical conversations. This article presents a generalized framework moving from generating topical conversation towards topic-agnostic conversational data consisting of three Large Language Model instances. Two of these models interact with each other to generate the conversation, while the third one plays the role of a judge to keep the conversation going. The synthetic data created by the proposed method exhibits higher quality and lower toxicity than four of the existing datasets (AmazonQA, Daily Dialog, Open Subtitles, and HUMOD) in terms of six performance measures, namely Toxicity, Severe Toxicity, Obscene, Threat, Insult, and Identity Attack. Compared to other datasets, the performance analysis of the generated data shows the lowest measures in terms of mean and maximum values. Specifically, the percentages for Toxicity, Obscene, Threat, Insult, and Identity Attack are 0.27%, 0.04%, 0.02%, 0.05%, and 0.02%, respectively, while the corresponding maximum values are 90.13%, 29.65%, 3.42%, 67.16% and 0.69%. The generated dataset also shows the maximum concentration, with 99.74% of the data in the range of 0-10% toxicity with just a few outliers.

Keyword:

Oral communication

Data models

Toxicology

Synthetic data

History

Computational modeling

Transformers

Training

Chatbots

Semantics

Large language models

synthetic data

conversational agents

Generating Topic-Agnostic Conversations With LLMs

摘要

期刊

机构

引用论文