Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: 开始实现 attention #89

Draft
wants to merge 17 commits into
base: dev
Choose a base branch
from
Draft

feat: 开始实现 attention #89

wants to merge 17 commits into from

Conversation

YdrMaster
Copy link
Collaborator

No description provided.

@YdrMaster YdrMaster added the kernel Add or modify kernel implementation label Jan 30, 2024
@YdrMaster YdrMaster self-assigned this Jan 30, 2024
@YdrMaster YdrMaster linked an issue Jan 30, 2024 that may be closed by this pull request
Attention #85
Open
7 tasks
@YdrMaster YdrMaster force-pushed the dev-attention branch 6 times, most recently from 60cebc8 to a9a56cb Compare February 2, 2024 08:41
Base automatically changed from dev to master February 19, 2024 10:18
@YdrMaster
feat: 开始实现 attention
cbe7c78 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
build(kernel): try to use local cccl instead of system
6d4465c 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
refactor(kernel): 实现一种不依赖模板参数的 BlockReduce 并用于 softmax
e54c7db 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
feat(kernel): 确认供 attention 调用的 softmax 接口
8e1564a 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
feat(kernel): 开始实现 softmax
ccf293f 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
feat(kernel): 完成不带 kv cache 的简单 attention
20a34ac 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
fix: 整理和改正 attention 构造问题
7286459 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
fix(kernel): 设置 sharedMemory
d7bbd3b 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
test(llm): 添加 attention 前端单测
9f736e9 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
feat(kernel): 封装 attLen 计算
4d47e5d 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
test(kernel): 添加 attention 单测
6211b92 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
fix(kernel): 解决 attention 访存错误的问题
1705126 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
fix(kernel): 仍然使用 cub::BlockReduce 并改正 Attention
f5d9756 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster
feat(kernel): 支持 kv cache attention
e16c679 
Signed-off-by: YdrMaster <[email protected]>
@YdrMaster YdrMaster changed the base branch from master to dev February 19, 2024 10:21
@YdrMaster YdrMaster marked this pull request as draft June 12, 2024 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@YdrMaster YdrMaster

Labels
kernel Add or modify kernel implementation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Attention
2 participants
@YdrMaster @PanZezhong1725