-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove dependency of flash_attn when use_flash_attn is set to false #20
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
19 times, most recently
from
January 25, 2024 08:06
9b67f33
to
aa7c10e
Compare
@@ -114,7 +118,47 @@ def forward(self, input): # pylint: disable=W0622 | |||
) | |||
|
|||
|
|||
class ColumnParallelLinearTorch(ColumnParallelLinear): | |||
class ColumnParallelLinearTorch(nn.Linear): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mark一下,如果这里变动的话,isp算法set_parallel_attr_for_param_groups函数需要相应更新
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里只是将继承类更换了,将init函数实现了一下,主题逻辑没有变动。
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
from
January 26, 2024 06:20
aa7c10e
to
6808bc8
Compare
sunpengsdu
reviewed
Jan 29, 2024
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
3 times, most recently
from
January 30, 2024 03:48
c04d9c7
to
4c786e9
Compare
9 tasks
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
3 times, most recently
from
February 2, 2024 11:10
c6b42f9
to
356d4be
Compare
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
3 times, most recently
from
February 19, 2024 09:04
407be0d
to
c22e806
Compare
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
7 times, most recently
from
February 26, 2024 03:58
e0ce40e
to
7ebc1c3
Compare
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
from
February 28, 2024 02:40
7ebc1c3
to
8c5e322
Compare
sallyjunjun
force-pushed
the
grj-no-flash-attn
branch
from
February 28, 2024 02:43
8c5e322
to
2518fb8
Compare
sunpengsdu
reviewed
Mar 8, 2024
should have some test cases? |
sunpengsdu
approved these changes
Mar 8, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
v100 is not supported in flash attention. In order to support v100 users using InternEvo, we remove dependency of flash attention module when use_flash_attn is set to false.
Modification
all flash attention related code.
BC-breaking (Optional)
Does the modification introduce changes that break the backward compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here and update the documentation.
Checklist
Before PR:
After PR: