Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support unpadded shapes in matmul1d w/ gather_in0 #16626

Open
avoraTT opened this issue Jan 10, 2025 · 0 comments
Open

Support unpadded shapes in matmul1d w/ gather_in0 #16626

avoraTT opened this issue Jan 10, 2025 · 0 comments
Assignees
Labels
metal tt-metal issue

Comments

@avoraTT
Copy link
Contributor

avoraTT commented Jan 10, 2025

Problem

Currently, Matmul1D with gather_in0=True does not handle shapes (K, N) that do not divide by the number of cores in the ring. In the current use case in the Llama models, the activations and weights need to be padded in order to use the matmul. However, this results in significant overhead caused by padding/slicing. Therefore, this matmul must support shapes that do not divide with the number of cores, and handle the padding implicitly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
metal tt-metal issue
Projects
None yet
Development

No branches or pull requests

1 participant