MPS Device for SpaCy Transformers

[awindsor]

When I run HuggingFace transformer models independently of SpaCy, I am able to leverage the power of the M1 GPU to speed up inference about 30x. Is there a way to do this within SpaCy? The command I am using when running the models in torch is

mps_device = torch.device("mps")

with the associated model.to(mps_device) and device calls within the code.

This produces quite impressive speed ups in both training and inference and it would be useful to have those available within SpaCy.

[kadarakos]

Hey awindsor,

If you have thinc-apple-ops installed the PyTorch models should use MPS if you run your code with the --gpu-id flag or you add spacy.require_gpu() to your code. For more information we have written a blogpost about hardware acceleration on Apple https://explosion.ai/blog/metal-performance-shaders. You can skip to the last section for usage instructions. Hope this helps!

[awindsor]

Thank you. That is exactly what I am looking for. I will look to see what my resulting speed up looks like.

[luisjglz]

Hi, trying to set this up myself as well. Did you have any luck with it?
Thanks