[not to be merged yet] added temp changes for fp32 main grad, might not work for TE#1151

Merged

ngoyal2707 merged 5 commits intongoyal_changes_for_pp_fp8from ngoyal_temp_fp32_main_grad

Dec 7, 2023

+31-9

Commits on Dec 4, 2023

added temp changes for fp32 main grad, might not work for TE
Naman Goyal
committed
post rebase
Naman Goyal
committed
changes to keep reduced grad in fp32 (#1152 )
vedanuj
authored

Commits on Dec 6, 2023

fix .grad=None issue when param is not sharded (#1153 )
jiecaoyu
authored

Commits on Dec 7, 2023

fixed broken clipping (#1154 )

ngoyal2707
and
Naman Goyal
authored