Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guided decoding integration for autolabel #898

Merged
merged 22 commits into from
Sep 30, 2024
Merged

Conversation

DhruvaBansal00
Copy link
Contributor

Pull Review Summary

Description

A summary of the change. Please also include relevant motivation and context. This could include links to any docs/Slack threads/Github issues other artifacts.

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • This change requires a documentation update

Tests

Please describe the tests that you ran to verify your changes. This could include a test plan you executed locally, unit tests/integration tests that were run to verify the change works as expected.

Make sure to include screenshots, API response, log statements etc that point to the test being successful.

Put closes #XXXX in your comment to auto-close the issue that your PR addresses.

@DhruvaBansal00 DhruvaBansal00 marked this pull request as ready for review September 27, 2024 19:10
@DhruvaBansal00 DhruvaBansal00 requested review from rajasbansal, yadavsahil197 and nihit and removed request for yadavsahil197 September 27, 2024 21:58
@DhruvaBansal00
Copy link
Contributor Author

@nihit @tuxracer heads up here - we have to remove all instances of additionalProperties from the supplied JSON Schema for refuel models only due to a restriction from lm-format-enforcer (the backend we use for guided decoding). However, since OpenAI requires us sending this parameter (https://platform.openai.com/docs/guides/structured-outputs/supported-schemas) we can't just ignore it. Added additional methods for removing them post schema generation. Flagging this mostly since this small things breaks inference and we should push lm-format-enforcer to fix this so that client side code is cleaner.

Issue in lm-format-enforcer being tracked here: noamgat/lm-format-enforcer#129

@DhruvaBansal00 DhruvaBansal00 changed the title WIP guided decoding integration for autolabel Guided decoding integration for autolabel Sep 30, 2024
@DhruvaBansal00 DhruvaBansal00 merged commit d12ccc0 into main Sep 30, 2024
2 checks passed
@DhruvaBansal00 DhruvaBansal00 deleted the guided-generation branch September 30, 2024 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants