Multi-Task Distributed Learning Using Vision Transformer With Random Patch Permutation

The widespread application of artificial intelligence in health research is currently hampered by limitations in data availability. Distributed learning methods such as federated learning (FL) and split learning (SL) are introduced to solve this problem as well as data management and ownership issues with their different strengths and weaknesses. The recent proposal of federated split task-agnostic (F eSTA) learning tries to reconcile the distinct merits of FL and SL by enabling the multi-task collaboration between participants through Vision Transformer (ViT) architecture, but they suffer from higher communication overhead. To address this, here we present a multi-task distributed learning using ViT with random patch permutation, dubbed ${p}$ -F eSTA. Instead of using a CNN-based head as in F eSTA, ${p}$ -F eSTA adopts a simple patch embedder with random permutation, improving the multi-task learning performance without sacrificing privacy. Experimental results confirm that the proposed method significantly enhances the benefit of multi-task collaboration, communication efficiency, and privacy preservation, shedding light on practical multi-task distributed learning in the field of medical imaging.

Automated summary

The widespread application of artificial intelligence in health research is currently limited by data availability. To overcome this limitation, distributed learning methods such as federated learning (FL) and split learning (SL) are introduced, each with their own strengths and weaknesses in addressing data management and ownership issues. However, a recent proposal called federated split task-agnostic (F eSTA) learning suffers from high communication overhead. To address this, a new method called ${p}$ -F eSTA is presented, which uses ViT with random patch permutation and achieves improved multi-task learning performance without sacrificing privacy.