In this repository, we implement FedMC and the baselines, including:
- FedAvg is the most classic method in federated learning, which averages all the model parameters from the selected clients at each communication round.
- FedProx provides a re-parametrization of FedAvg using a proximal term to regularize the local model parameters with the global model parameters in parameter space to address heterogeneity.
- MOCHA: in this method, each client is viewed as a task, and FL's objective is optimized in multi-task fashion, where each client is encouraged to be close to its neighboring clients, i.e., with similar data distribution.
- FedPer proposes to learn a unified representation under the orchestration of global server, and the personalized layers are kept locally to capture clients' specific data distributions.
- LG-FedAvg jointly learns compact local features on each client and aggregates only the global classification model at each communication round, thus data heterogeneity can be explicitly modeled and reduced.
- Per-FedAvg is intended to find a global initial model and clients achieve personalized models through fine-tuning it on their private datasets, which is similar to MAML.
- FedRep: similar to FedPer, it learns both a unified representation and local personalized representations. Differently, in FedRep, each client optimizes base model and head model alternately, while FedPer optimizes all parameters simultaneously.
- FedFomo allows clients to federated only with their relevant clients. Before local optimization, each client initializes its local model using a linear combination of global models.
We use four widely used federated benchmark datasets to simulate heterogeneous federated settings, including MINIST, CIFAR-10, CIFAR-100, and HAR.
| Dataset | Task | #Clients | #Samples | #Samples per client | Classes | Base model |
|---|---|---|---|---|---|---|
| MNIST | Digit classification | 100 | 70000 | 700 | 10 | 2CNN + 2FC |
| CIFAR-10 | Image classification | 100 | 60000 | 600 | 10 | 4CNN + 2FC |
| CIFAR-100 | Image classification | 100 | 60000 | 600 | 100 | 4CNN + 2FC |
| HAR | Activity recognition | 30 | 10269 | 342.3 | 6 | 4CNN + 2FC |
MNIST, CIFAR-10 and CIFAR-100 datasets are automatically downloaded from keras.dataset. HAR dataset needs to be downloaded from here and extracted to /data/har/.
- Install the libraries listed in requirements.txt
pip install -r requirements.txtWe provide the bash scripts to run all experiments.
cd algorithms/
bash run.sh