Enhancing GPT-Based Input Method with Domain-Adaptive Pinyin Encoder(s)

doi:10.1007/978-981-95-3346-6_16

2026-01-01

PRE

W. Peng

Xiaodong Shi *

DOI：10.1007/978-981-95-3346-6_16

原文链接

原文求助

摘要

En 中文

近期研究已表明，在束搜索采样过程中应用字符级选择约束，可将中文GPT模型转变为性能优于传统输入法的拼音输入法引擎（IME）。然而，考虑到IME用户群体的多样性，满足不同领域的特定偏好至关重要。尽管商业IME提供领域特定词典，GPT基IME通常作为通用补全工具运行，且普遍缺乏高效的领域适应性。为解决此问题，我们提出一种参数高效微调方法，利用辅助Transformer编码器捕获拼音特征，仅通过调整编码器参数，即可使IME在少量数据下适应特定领域。该方法在PD基准测试（标准输入法评估数据集）上，相较于全微调的拼音GPT模型，将Top-1精确率提升了超过6%。此外，实验证明可通过数据混合和模型集成等策略，整合来自多个领域的数据与编码器，实现基于用户语境或偏好的灵活切换与组合。

Keyword:

Chinese input method

GPT

parameter efficient fine-tuning

期刊

NATURAL LANGUAGE PROCESSING AND CHINESE COMPUTING, NLPCC 2025, PT II

IF:

论文数:

被引数:

机构

xiamen university

学者数:

5.9W

论文数: 3.8W

被引数: 67

引用论文

WuDaoCorpora: A super large-scale Chinese corpora for pre-training language models WuDaoCorpora: 用于预训练语言模型的超大规模中文语料库

AI Open

IF0

2021-01-01

OAAI

Sha Yuan; Hanyu Zhao; Zhengxiao Du; Ming Ding; Xiao Liu; Yukuo Cen; Xu Zou; Zhilin Yang; Jie Tang

Fixed global memory for controllable long text generation

APPLIED INTELLIGENCE

IF3.5

2022-10-20

PREAI

Chen, Zheng; Liu, Zhejun

Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models

IF0

2022-06-22

OAAI

Ning Ding; Yujia Qin; Guang Yang; Fuchao Wei; Zonghan Yang; Yusheng Su; Shengding Hu; Yulin Chen; Chi-Min Chan; Weize Chen; Jing Yi; Weilin Zhao; Xiaozhi Wang; Zhiyuan Liu; Hai-Tao Zheng; Jianfei Chen; Yang Liu; Jie Tang; Juanzi Li; Maosong Sun

Moon IME: Neural-based Chinese Pinyin Aided Input Method with Customizable Association

Proceedings of ACL 2018, System Demonstrations

IF0

2018-01-01

OAAI

Yafang Huang; Zuchao Li; Zhuosheng Zhang; Hai Zhao

A Joint Graph Model for Pinyin-to-Chinese Conversion with Typo Correction

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

IF0

2014-01-01

OAAI

Zhongye Jia; Hai Zhao