Improving Multitask Retrieval by Promoting Task Specialization

Wenzheng Zhang

doi:10.1162/tacl_a_00597

Improving Multitask Retrieval by Promoting Task Specialization

2023-09-25

0

OA

AI

DOI：10.1162/tacl_a_00597

原文链接

原文求助

摘要

En 中文

In multitask retrieval, a single retriever is trained to retrieve relevant contexts for multiple tasks. Despite its practical appeal, naive multitask retrieval lags behind task-specific retrieval, in which a separate retriever is trained for each task. We show that it is possible to train a multitask retriever that outperforms task-specific retrievers by promoting task specialization. The main ingredients are: (1) a better choice of pretrained model-one that is explicitly optimized for multitasking-along with compatible prompting, and (2) a novel adaptive learning method that encourages each parameter to specialize in a particular task. The resulting multitask retriever is highly performant on the KILT benchmark. Upon analysis, we find that the model indeed learns parameters that are more task-specialized compared to naive multitasking without prompting or adaptive learning.1

Improving Multitask Retrieval by Promoting Task Specialization

摘要

期刊

机构