This is the implementation of "Personalized Edge Intelligence via Federated Self- Knowledge Distillation", also published in IEEE TPDS journal.
Please run the following commands below to install dependencies.
conda create -y -n fed_d python=3.7 conda activate fed_d conda install -y -c pytorch pytorch=1.7.1 torchvision=0.8.2 matplotlib python-lmdb cudatoolkit=11.3 cudnn pip install transformers datasets pytreebank opencv-python torchcontrib gpytorch Details for each argument are included in ./parameters.py.
The setup of the FedAvg/FedProx for resnet-8 on cifar10 with pathological distribution:
python run_gloo.py \ --arch resnet8 --complex_arch master=resnet8,worker=resnet8 --experiment demo \ --data cifar10 --pin_memory True --batch_size 64 --num_workers 2 --prepare_data combine \ --partition_data pathological --shard_per_user 2 \ --train_data_ratio 0.8 --val_data_ratio 0.0 --test_data_ratio 0.2 \ --n_clients 10 --participation_ratio 0.6 --n_comm_rounds 5 --local_n_epochs 5 \ --world_conf 0,0,1,1,100 --on_cuda True \ --fl_aggregate scheme=federated_average \ --optimizer sgd --lr 0.01 --local_prox_term 0 --lr_warmup False --lr_warmup_epochs 5 --lr_warmup_epochs_upper_bound 150 \ --lr_scheduler MultiStepLR --lr_decay 0.1 \ --weight_decay 1e-5 --use_nesterov False --momentum_factor 0.9 \ --track_time True --display_tracked_time True --python_path $HOME/anaconda3/envs/fed_d/bin/python \ --manual_seed 7 --pn_normalize True --same_seed_process True \ --algo fedavg \ --personal_test True \ --port 20001 --timestamp $(date "+%Y%m%d%H%M%S")The setup of the pFedSD for simple cnn on cifar10 with dirichlet distribution:
python run_gloo.py \ --arch simple_cnn --complex_arch master=simple_cnn,worker=simple_cnn --experiment demo \ --data cifar10 --pin_memory True --batch_size 64 --num_workers 2 --prepare_data combine \ --partition_data non_iid_dirichlet --non_iid_alpha 0.1 \ --train_data_ratio 0.8 --val_data_ratio 0.0 --test_data_ratio 0.2 \ --n_clients 10 --participation_ratio 0.6 --n_comm_rounds 5 --local_n_epochs 5 \ --world_conf 0,0,1,1,100 --on_cuda True \ --fl_aggregate scheme=federated_average \ --optimizer sgd --lr 0.01 --local_prox_term 0 --lr_warmup False --lr_warmup_epochs 5 --lr_warmup_epochs_upper_bound 150 \ --lr_scheduler MultiStepLR --lr_decay 0.1 \ --weight_decay 1e-5 --use_nesterov False --momentum_factor 0.9 \ --track_time True --display_tracked_time True --python_path $HOME/anaconda3/envs/fed_d/bin/python \ --manual_seed 7 --pn_normalize True --same_seed_process True \ --algo pFedSD \ --personal_test True \ --port 20002 --timestamp $(date "+%Y%m%d%H%M%S")TODO
The skeleton codebase in this repository was adapted from FedDF[1].
[1] T. Lin, L. Kong, S. U. Stich, and M. Jaggi, “Ensemble distillation for robust model fusion in federated learning,” in NeurIPS, 2020.