Implementing FedRep #127
Merged
Implementing FedRep #127
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… method. There is also a slight refactor of the base models for FedPer, MOON, and FedRep, as they all belong to the category of SequentiallySplit models, in some way. Also adding FedRep and Ditto to the smoke tests.
emersodb
requested review from
lotif,
fatemetkl,
jewelltaylor,
sanaAyrml and
yc7z
emersodb
commented
Apr 22, 2024
| from fl4health.utils.load_data import load_mnist_data | ||
| from fl4health.utils.metrics import Accuracy, Metric | ||
| from fl4health.utils.sampler import MinorityLabelBasedSampler | ||
|
|
||
|
|
||
| class MnistFedPerClient(MoonClient): | ||
| class MnistFedPerClient(FedPerClient): |
There was a problem hiding this comment.
Changing this client to no longer inherit from MoonClient. I liked the idea of mixing them, because they are so related, but it can be a little confusing. This also means that we can rip out the need to remember to specify the exchanger when using vanilla FedPer. We can still use the MoonClient to combine the idea with FedPer, just removing it from the examples to reduce confusion.
emersodb
commented
Apr 22, 2024
| @@ -56,10 +52,6 @@ def get_optimizer(self, config: Config) -> Optimizer: | |||
| def get_criterion(self, config: Config) -> _Loss: | |||
| return torch.nn.CrossEntropyLoss() | |||
|
|
|||
| def get_parameter_exchanger(self, config: Config) -> ParameterExchanger: | |||
There was a problem hiding this comment.
Now appears in FedPerClient
emersodb
commented
Apr 22, 2024
| @@ -1,31 +0,0 @@ | |||
| import torch | |||
There was a problem hiding this comment.
Moved this model into the same file as that used for FedRep since they are both just sequentially split models but for different datasets (MNIST vs. CIFAR)
emersodb
commented
Apr 22, 2024
|
|
||
| In this demo, APFL is applied to an augmented version of the MNIST dataset that is non--IID. The FL server expects three clients to be spun up (i.e. it will wait until three clients report in before starting training). Each client has a modified version of the MNIST dataset. This modification essentially subsamples a certain number from the original training and validation sets of MNIST in order to synthetically induce local variations in the statistical properties of the clients training/validation data. In theory, the models should be able to perform well on their local data while learning from other clients data that has different statistical properties. The proportion of labels at each client is determined by dirichlet distribution across the classes. The lower the beta parameter is for each class, the higher the degree of the label heterogeneity. | ||
| In this demo, APFL is applied to an augmented version of the MNIST dataset that is non--IID. The FL server expects three clients to be spun up (i.e. it will wait until three clients report in before starting training). Each client has a modified version of the MNIST dataset. This modification essentially subsamples a certain number from the original training and validation sets of MNIST in order to synthetically induce local variations in the statistical properties of the clients training/validation data. In theory, the models should be able to perform well on their local data while learning from other clients data that has different statistical properties. The proportion of labels at each client is determined by Dirichlet distribution across the classes. The lower the beta parameter is for each class, the higher the degree of the label heterogeneity. |
There was a problem hiding this comment.
Dirichlet is the name of a German mathematician, hence the capitalization.
emersodb
commented
Apr 22, 2024
emersodb
commented
Apr 22, 2024
| @@ -32,7 +32,7 @@ def __getitem__(self, index: int) -> Tuple[torch.Tensor, torch.Tensor]: | |||
| return self.data[index], self.targets[index] | |||
|
|
|||
|
|
|||
| class TestClient(InstanceLevelPrivacyClient): | |||
| class ClientForTest(InstanceLevelPrivacyClient): | |||
There was a problem hiding this comment.
These classes are just renamed to address a warning in pytest that it can't "discover" these tests. It just uses a rudimentary GREP for the prefix "Test" to find tests. So the rename just reduces the warnings.
sanaAyrml
approved these changes
May 7, 2024
…base classes for FedPer, FedRep, and MOON to bury the flatten features functionality in the base class for SequentiallySplitModels. With this change in functionality the FedPer class became essentially a pass through. So I eliminated it and made some modifications for the FedPerClient to make it clear that a SequentiallySplitExchangeBaseModel was the right type to be used for this client. Also added a test for the FedPerClient and modified the fedper base test.
sanaAyrml
approved these changes
May 8, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Type
Feature
Implementation of FedRep. This includes an example of the methods use, unit tests, and addition to the smoke tests.
Note: Most of the file changes are more very small typo fixes.
Clickup Ticket(s): Link(s) if applicable.
Tests Added
Added to the Smoke Tests
Tests have also been added in
tests/clients/test_fedrep_client.py