Skip to content

Latest commit

 

History

History
13 lines (8 loc) · 415 Bytes

README.md

File metadata and controls

13 lines (8 loc) · 415 Bytes

Tensor Parallelism

Code accompanying the deep-dive blog post on Tensor Parallelism.

  • The MLP and TP MLP layers are in layer.py
  • Matmul profiling code in matmul_profiling.py
  • MLP TP profiling code in tp_profiling.py
  • Tests of the rearranging tensor sums are in test_dot_product_{local,distributed}.py

Contributors