Code

Fix memory estimation#25

Code

Merged

cli99 merged 12 commits intomaincli99/llm-analysis:mainfrom fsdp-memcli99/llm-analysis:fsdp-memCopy head branch name to clipboardMay 28, 2024

+277-161

Commits

Commits on Mar 6, 2024

• wip

  cli99

  committed
  8135689

  Copy full SHA for 8135689

Commits on Mar 7, 2024

• wip

  cli99

  committed
  77ac4e8

  Copy full SHA for 77ac4e8

• wip

  cli99

  committed
  ff25f1f

  Copy full SHA for ff25f1f

Commits on May 21, 2024

• wip

  cli99

  committed
  fb6c84b

  Copy full SHA for fb6c84b

Commits on May 22, 2024

• add loss_bwd_memory

  cli99

  committed
  a2d7193

  Copy full SHA for a2d7193

• update log info

  cli99

  committed
  65cc34e

  Copy full SHA for 65cc34e

• format

  cli99

  committed
  2b92db3

  Copy full SHA for 2b92db3

• update bwd prefetch memory calc

  cli99

  committed
  fd6ad18

  Copy full SHA for fd6ad18

Commits on May 28, 2024

• fix test

  cli99

  committed
  4a02e9e

  Copy full SHA for 4a02e9e

• added hsdp through rdp_size

  cli99

  committed
  d32358c

  Copy full SHA for d32358c

• fix mlp flops calc

  cli99

  committed
  6ad5242

  Copy full SHA for 6ad5242

• Update coverage.yml

  cli99

  authored
  5ddbe75

  Copy full SHA for 5ddbe75