-
Notifications
You must be signed in to change notification settings - Fork 339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kwarg for additional ModelSettings parameters? #272
Comments
Hey @YanSte, Thanks for your question. Indeed, we've been having some discussions about this internally. One reason we're hesitant to introduce something like Perhaps a happy medium would be to have a from typing_extensions import TypedDict
class ModelSpecificSettings(TypedDict):
anthropic: AnthropicModelSettings
openai: OpenAIModelSettings
...
class AnthropicModelSettings(TypedDict):
x: int
y: str
.... This way, we retain type checking for those extra settings. I'll conclude with, I'm not actually opposed to having both |
Hi @sydney-runkle, Thanks for your detailed message and for elaborating. If I understand at the end you will have ?
Do you think it will not growing until new compagny will come with new model ? Or new params ? On my end I thought more about this, giving an extra_param: dict and use a validator compare to class init param. Here an example:
This approach ensures robust type checking while minimizing the need for manual updates. We will have the base settings (temperature etc..) and extra. What do you think? |
Hi @YanSte, Apologies for the delay. Thanks for your patience over the holidays! I think it might look more like: from typing_extensions import TypedDict
class ModelSpecificSettings(TypedDict):
anthropic: AnthropicSpecificSettings
openai: OpenAISpecificSettings
.... Indeed, I'm in favor of the extra param, though it does come with type checking drawbacks, as none of those extra kwargs can be type checked. Let's defer to @samuelcolvin re a decision here. |
I would also love being able to, e.g., pass a seed to the OpenAI models |
Hi there,
Would it make sense to introduce a kwarg in
ModelSettings
for this parameter?We could then include a validation step that checks whether the parameter is included in the model and raise an error.
This would make the model setup more generic and flexible, allowing for easier configuration across different models.
Best.
The text was updated successfully, but these errors were encountered: