we visualize the sorted gradient similarity between each pair of clients in the final round。
we run 100 communication rounds involving 5 clients within two distinct non-IID scenarios.Subsequently, we observe from Fig. that gradient conflicts become more frequent as the training advances.This phenomenon is consistent with the understanding that the non-IID issue causes local minima to deviate from global minima [].Moreover。
and becomes notably more severe in non-IID scenarios.These experiments demonstrate that stronger heterogeneity leads to more severe local drifts,indicated by the reduction in minimum similarity from -0.12 to -0.22.Finally,consequently inducing more violent gradient conflicts. , First, even within the IID setting,。
as illustrated in Fig. . Our observation reveals the presence of gradient conflicts in heterogeneous settings,while a stronger heterogeneity results in a more severe conflict。
as evidenced by the minimum similarity decreasing from -0.06 to -0.12.Next, we increase the number of clients to 10; the results are presented in Fig. .We find that a larger number of clients further exacerbates the gradient conflict phenomenon, gradient conflict is widespread in FL。