Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[code_review] Explore different models #4587

Open
marco-c opened this issue Nov 1, 2024 · 3 comments
Open

[code_review] Explore different models #4587

marco-c opened this issue Nov 1, 2024 · 3 comments

Comments

@marco-c
Copy link
Collaborator

marco-c commented Nov 1, 2024

Similar to #4582, but across different models.

This depends on #4580 for the evaluation.

@suhaibmujahid
Copy link
Member

We have this as part of the experimental mode. We return the results generated by each of the models/configurations to the user for evaluation.

The following are the configurations in the experimental mode:

  • gpt-4o temp 0.2
  • gpt-4o temp 0.8
  • claude-3-5 temp 0.2
  • gemini-1.5-pro temp 0.2 (disabled due to a quota limitation error)

@suhaibmujahid
Copy link
Member

I deployed a new version of review helper, which enables back gemini-1.5-pro.

So currently, the following are the configurations in the experimental mode:

  • gpt-4o temp 0.2
  • gpt-4o temp 0.8
  • claude-3-5 temp 0.2
  • gemini-1.5-pro temp 0.2

@marco-c
Copy link
Collaborator Author

marco-c commented Jan 8, 2025

I deployed a new version of review helper, which enables back gemini-1.5-pro.

So currently, the following are the configurations in the experimental mode:

* gpt-4o temp 0.2

* gpt-4o temp 0.8

* claude-3-5 temp 0.2

* gemini-1.5-pro temp 0.2

And after #4731, it's going to be Gemini 2.0 Flash instead of Gemini 1.5 Pro.

@marco-c marco-c moved this from Backlog to In progress in Review Helper Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In progress
Development

No branches or pull requests

2 participants