Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

需要34b-chat-16k 量化版本 #28

Open
weiminw opened this issue Jun 6, 2024 · 4 comments
Open

需要34b-chat-16k 量化版本 #28

weiminw opened this issue Jun 6, 2024 · 4 comments
Labels
question Further information is requested

Comments

@weiminw
Copy link

weiminw commented Jun 6, 2024 •
edited
Loading

感谢你们发布强悍的模型，是否可以发出 awq或者 gptq-int4
@Yimi81
Copy link
Contributor

Yimi81 commented Jun 8, 2024

好的 我会反馈需求。此外，社区已经有相应的版本，你可以测试使用~

@zhanghx0905
Copy link

目前还未找到34b-chat-16k的4bit量化模型，只见过4k上下文版本的量化模型。
如果大家发现了量化后的34b-chat-16k模型，能否分享一下？谢谢！

@Yimi81
Copy link
Contributor

Yimi81 commented Jun 11, 2024

@zhanghx0905 https://modelscope.cn/models/tclf90/Yi-1.5-34B-Chat-16K-GPTQ-Int4/summary

@masterwang22327
Copy link

原始的Yi-1.5-34B-Chat-16K太慢了 同样的Qwen1.5-32B 速度快它一倍 代码生成能力也烂到爆炸

@Haijian06 Haijian06 added the question Further information is requested label Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@weiminw @zhanghx0905 @Yimi81 @masterwang22327 @Haijian06