v0.2.6.dev3
Pre-release
Pre-release
What's Changed
- Set LitServer.stream using LitSpec.stream by @aniketmaurya in #398
- fix openai usage info for non-streaming response by @aniketmaurya in #399
- [pre-commit.ci] pre-commit suggestions by @pre-commit-ci in #400
- Async continuous batching loop by @aniketmaurya in #401
- add validation for
stream=False
withyield
usage by @aniketmaurya in #402 - fix callback runner to execute after predict by @aniketmaurya in #406
- integrate zmq by @aniketmaurya in #403
- warn users when predict/unbatch output length is not same as #requests by @aniketmaurya in #408
- move built in loops inside classes by @aniketmaurya in #409
- add justus and thomas as codeowners by @aniketmaurya in #410
- enable multiple workers for ZMQ by @aniketmaurya in #411
Full Changelog: v0.2.6.dev2...v0.2.6.dev3