Abstract
Federated learning (FL) faces significant challenges in medical image analysis due to data heterogeneity among clients, where balancing personalization and generalization is challenging. Existing methods often struggle to achieve both objectives simultaneously, as excessive personalization reduces generalization, while over-generalization weakens adaptation to client-specific features. To address these challenges, we propose a perceptual hashing hypernetwork personalized FL (PHH-FL) to enhance both personalization and generalization. PHH-FL first uses a perceptual hashing (pHash) algorithm to construct a similarity matrix that captures data distribution differences among clients and employs a hypernetwork to generate personalized parameters for each client. Meanwhile, the shared hypernetwork is introduced to promote knowledge transfer between clients, thereby enhancing the generalization ability of the local model. By selectively generating parameters for the initial layers of the target network, PHH-FL reduces computational and communication costs while maintaining performance. Experiments on medical image classification and segmentation tasks show that PHH-FL outperforms state-of-the-art methods. Ablation studies further demonstrate that the proposed framework effectively balances personalization and generalization.