Code

Merge HF implementation into main#10

Code

Merged

onadegibert merged 6 commits intomainHelsinki-NLP/OpusDistillery:mainfrom hfHelsinki-NLP/OpusDistillery:hfCopy head branch name to clipboardOct 1, 2024

+350-71

Commits

Commits on Sep 16, 2024

• implementation of several HF models

  onadegibert

  committed
  221451c

  Copy full SHA for 221451c

Commits on Sep 20, 2024

• added hf configs

  onadegibert

  committed
  9f60d34

  Copy full SHA for 9f60d34

• translate_hf uses dataloader and accelerate

  onadegibert

  committed
  7d0ae7b

  Copy full SHA for 7d0ae7b

• fixed speed script

  onadegibert

  committed
  176c55b

  Copy full SHA for 176c55b

Commits on Sep 24, 2024

• fixed hf pipeline script

  onadegibert

  committed
  66ffb67

  Copy full SHA for 66ffb67

• added batchsize to config

  onadegibert

  committed
  6ac1531

  Copy full SHA for 6ac1531