This paper introduces an advanced approach for fortifying Federated Learning (FL) systems against label-flipping attacks. We propose a simplified consensus-based verification process integrated with an adaptive thresholding mechanism. This dynamic thresholding is designed to adjust based on the evolving landscape of model updates, offering a refined layer of anomaly detection that aligns with the real-time needs of distributed learning environments. Our method necessitates a majority consensus among participating clients to validate updates, ensuring that only vetted and consensual modifications are applied to the global model. The efficacy of our approach is validated through experiments on two benchmark datasets in deep learning, CIFAR-10 and MNIST. Our results indicate a significant mitigation of label-flipping attacks, bolstering the FL system's resilience. This method transcends conventional techniques that depend on anomaly detection or statistical validation by incorporating a verification layer reminiscent of blockchain's participatory validation without the associated cryptographic overhead. The innovation of our approach rests in striking an optimal balance between heightened security measures and the inherent limitations of FL systems, such as computational efficiency and data privacy. Implementing a consensus mechanism specifically tailored for FL environments paves the way for more secure, robust, and trustworthy distributed machine learning applications, where safeguarding data integrity and model robustness is critical.
Bounded Gradients: The gradients of F () are bounded. This means there exists a constant G > 0 such that for all , F () G. Appropriate Learning Rate Schedule: The learning rate {t} used in the algorithm satisfies the conditions: 23: Appropriate Learning Rate Schedule: The learning rate {t} used in the algorithm satisfies the conditions: 23: (cid:88) t = and (cid:88)
id: 51c530280be96444434e70b29c35e12f - page: 5
2 t < . (7) t=1 t=1 This is a common condition that allows for sufficient exploration of the parameter space while ensuring convergence. Under these conditions, we can apply the results from stochastic gradient descent in convex optimization. The sequence {(t)} generated by the algorithm can be seen as a form of stochastic approximation, which converges to the optimal parameters in expectation, given the convex nature of F and the boundedness of its gradients. The consensus mechanism ensures that the updates Mc aggregated to form (t+1) from (t) are representative of the true gradient direction of F at (t), despite potential adversarial perturbations. Thus, the sequence {(t)} converges to the optimal set of parameters . Algorithm 1 Consensus-Based Label Verification with Adaptive Threshold in Federated Learning 1: Objective: To defend against label-flipping attacks in FL using a consensus-based label verification mechanism. 2: Inputs: 3:
id: b6381eb54afe86dfe20db8334d9be621 - page: 5
Federated Dataset Df ed: The dataset distributed across multiple clients in the FL setup. 4: Trusted Dataset Dtrusted: A small, pre-verified dataset used for label verification. 5: Model M : The shared machine learning model being trained in the FL setup. 6: Threshold : The discrepancy threshold for flagging updates as suspicious. 7: Outputs: 8: Updated Model M : After processing the verified updates, the machine learning model is updated. Suspicious Updates Report Rsuspicious: A report of flagged updates that significantly deviate from the consensus. 10: Procedure: 11: 9: Initialization: Distribute M to all clients. Initialize Rsuspicious as an empty list. Client Update Generation: 12: 13: 14: 15: for each client c in FL do 16: 17: 18: end for 19: 20: for each Mc do 21: 22: Train M on its local dataset Dc Df ed. Submit the model update Mc to the server.
id: 6025f64d2789aa3bdda9c590e32cd083 - page: 5
Consensus-Based Label Verification: Apply Mc to M to get Mtemp. Use Mtemp to predict labels on Dtrusted, obtaining Lpredicted. Compare Lpredicted with true labels Ltrue of Dtrusted. 24: 25: 26: Calculate discrepancy d as follows: if d > then else 27: 28: 29: 30: end for 31: 32: Update M with Mc to get M . end if Model Aggregation: Aggregate all non-suspicious Mc updates to update M to M . 33: 34: Suspicious Update Handling (Optional): Review Rsuspicious for potential security breaches or data corruption. 35: 36: Adaptive Threshold Adjustment: Adjust based on a predefined strategy, considering the distribution of discrepancies and model performance metrics.
id: 97b65f2c50626c1db4c370f8c8599b74 - page: 5