Fix _free_full_params()

I've been getting `AttributeError: 'FlatParameter' object has no attribute '_full_param_padded'` triggered by `p._full_param_padded.record_stream(current_stream)` Adding a check to not free full params if none have been added.
facebookresearch · Nov 21, 2023 · c5f8df9 · c5f8df9
1 parent 164cc0f
commit c5f8df9
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/fairscale/nn/data_parallel/fully_sharded_data_parallel.py b/fairscale/nn/data_parallel/fully_sharded_data_parallel.py
@@ -2096,6 +2096,8 @@ def _free_full_params(self, params: Optional[List[Parameter]] = None) -> None:
         """Free up storage for full parameters."""
         if params is None:
             params = self.params
+        if not self.has_full_params:
+            return
         self.has_full_params = False
         current_stream = torch.cuda.current_stream()
         for p in params: