Multi-Task Distributed Learning Using Vision Transformer With Random Patch Permutation

doi:10.1109/TMI.2022.3218783

Multi-Task Distributed Learning Using Vision Transformer With Random Patch Permutation

2023-07-01

5

OA

AI

DOI：10.1109/TMI.2022.3218783

Original

Original request for help

Save

Abstract

En 中文

The widespread application of artificial intelligence in health research is currently hampered by limitations in data availability. Distributed learning methods such as federated learning (FL) and split learning (SL) are introduced to solve this problem as well as data management and ownership issues with their different strengths and weaknesses. The recent proposal of federated split task-agnostic (F eSTA) learning tries to reconcile the distinct merits of FL and SL by enabling the multi-task collaboration between participants through Vision Transformer (ViT) architecture, but they suffer from higher communication overhead. To address this, here we present a multi-task distributed learning using ViT with random patch permutation, dubbed ${p}$ -F eSTA. Instead of using a CNN-based head as in F eSTA, ${p}$ -F eSTA adopts a simple patch embedder with random permutation, improving the multi-task learning performance without sacrificing privacy. Experimental results confirm that the proposed method significantly enhances the benefit of multi-task collaboration, communication efficiency, and privacy preservation, shedding light on practical multi-task distributed learning in the field of medical imaging.

Keywords:

Federated learning

split learning

multi-task learning

vision transformer

privacy preservation

Multi-Task Distributed Learning Using Vision Transformer With Random Patch Permutation

Abstract

Journal

Organization