Why the pp_rank equals to torch.distributed.get_rank(group=self.optimizer.dp_process_group)?

[CrossEntropyZenk]

    def _get_zero_ckpt_name(self, checkpoints_path, tag):
        mp_rank = 0 if self.mpu is None else self.mpu.get_model_parallel_rank()
        pp_rank = torch.distributed.get_rank(group=self.optimizer.dp_process_group)
        return self._get_rank_zero_ckpt_name(checkpoints_path, tag, mp_rank, pp_rank)

[SdAswin]

This pattern is commonly used in distributed training frameworks like PyTorch's Distributed Data Parallel (DDP) to determine the rank of a process within a specific group.the line of code you provided is a way to get the rank of the process within the optimizer's process group. This rank can then be used for various purposes in distributed training, such as deciding which part of the model to update or synchronizing operations within that group.