Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: support vllm in controller #635

Merged
merged 4 commits into from
Dec 3, 2024
Merged

feat: support vllm in controller #635

merged 4 commits into from
Dec 3, 2024

Conversation

zhuangqh
Copy link
Collaborator

@zhuangqh zhuangqh commented Oct 17, 2024 •
edited
Loading

Reason for Change:

  • support vllm runtime deployments
  • add a feature gate to enable vllm runtime
  • able to select runtime by annotation tag

Requirements

  • added unit tests and e2e tests (if applicable).

@zhuangqh zhuangqh force-pushed the controller-support-vllm branch from 936993c to 1fcd0b1 Compare October 28, 2024 11:09
@zhuangqh zhuangqh marked this pull request as ready for review October 28, 2024 11:13
zhuangqh
zhuangqh commented Oct 28, 2024
View reviewed changes
pkg/model/interface.go
ModelRunParams map[string]string // Parameters for running the model training/inference.
}

func (p *PresetParam) DeepCopy() *PresetParam {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may update some params according to the node counts. Thus, we must deepcopy it at first.

@zhuangqh zhuangqh force-pushed the controller-support-vllm branch from 3dbfdb6 to 3b52cc4 Compare November 6, 2024 11:07
@zhuangqh zhuangqh force-pushed the controller-support-vllm branch from 3b52cc4 to 8244ebc Compare November 14, 2024 00:05
@zhuangqh zhuangqh force-pushed the controller-support-vllm branch from 0545480 to 0a2e3a7 Compare November 22, 2024 12:16
@zhuangqh zhuangqh mentioned this pull request Nov 26, 2024
Closed
@zhuangqh
feat: support vllm in controller
933107e 
- set vllm as the default runtime by a featureflag

Signed-off-by: jerryzhuang <[email protected]>
@zhuangqh zhuangqh force-pushed the controller-support-vllm branch from 44ee56a to 933107e Compare December 2, 2024 04:25
@zhuangqh
set vllm as default
2c7eae5 
Signed-off-by: jerryzhuang <[email protected]>
@zhuangqh
fix
e9f5b03 
Signed-off-by: jerryzhuang <[email protected]>
@zhuangqh
Copy link
Collaborator Author

zhuangqh commented Dec 2, 2024

ready @Fei-Guo

@Fei-Guo Fei-Guo merged commit 0f9a11d into main Dec 3, 2024
3 of 6 checks passed
@Fei-Guo Fei-Guo deleted the controller-support-vllm branch December 3, 2024 15:53
@zhuangqh zhuangqh mentioned this pull request Dec 5, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Fei-Guo Fei-Guo Fei-Guo approved these changes

@helayoty helayoty Awaiting requested review from helayoty helayoty is a code owner

@ishaansehgal99 ishaansehgal99 Awaiting requested review from ishaansehgal99 ishaansehgal99 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zhuangqh @ishaansehgal99 @Fei-Guo