You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When working with the Azure OpenAI service, we seem to be running into occasional "JSON::ParserError" errors. These don't happen frequently, but if you run a script that runs 100 prompts, it almost always triggers the error.
We tracked this down to the "helicone-stream-force-format" parameter suggested in the README. When that's present, the bug triggers. When it isn't, the bug doesn't trigger.
The error looks something like this:
unexpected token at '{"finish_reason":null,"index":0,"deltdata: {"id":"chatcmpl-8XckshxV3cEBasKbvvUD4MDSpUgYJ","object":"chat.completion.chunk","created":1703024490,"model":"gpt-35-turbo","choices":[{"finish_reason":null,"index":0,"delta":{"content":"."}}]}' (JSON::ParserError)
To Reproduce
Run this script:
client=OpenAI::Client.new(access_token: ENV.fetch("AZURE_OPENAI_API_KEY"),uri_base: "https://oai.hconeai.com/openai/deployments/gpt-35-turbo",request_timeout: 240,extra_headers: {"api-key": ENV.fetch("AZURE_OPENAI_API_KEY"),"Helicone-Auth": "Bearer #{ENV.fetch("HELICONE_API_KEY")}","Helicone-OpenAI-Api-Base": ENV.fetch("AZURE_OPENAI_ENDPOINT"),"Helicone-Property-Prompt": "TestUnexpectedTokenErrorsPrompt","helicone-stream-force-format": "true"},api_type: :azure,api_version: "2023-03-15-preview")100.times.peach(8)do |i|
puts""puts"Iteration #{i}"puts""client.chat(parameters: {model: "gpt-3.5-turbo",messages: [{role: "user",content: "Write me 3 paragraphs about the topic 'OpenAI parsing needs to be fixed'"}],response_format: {type: "text"},seed: i,# Default, can be overridden using parametersstream: procdo |chunk,_bytesize|
ifchunk.dig("choices",0,"finish_reason") == "stop"# last messageputs""puts"Iteration #{i} finished."puts""nextendcontent=chunk.dig("choices",0,"delta","content")unlesscontent.nil?
$stdout.write(".")endend})end
Expected behavior
JSON::ParserError shouldn't be triggered.
The text was updated successfully, but these errors were encountered:
Describe the bug
When working with the Azure OpenAI service, we seem to be running into occasional "JSON::ParserError" errors. These don't happen frequently, but if you run a script that runs 100 prompts, it almost always triggers the error.
We tracked this down to the "helicone-stream-force-format" parameter suggested in the README. When that's present, the bug triggers. When it isn't, the bug doesn't trigger.
The error looks something like this:
To Reproduce
Run this script:
Expected behavior
JSON::ParserError shouldn't be triggered.
The text was updated successfully, but these errors were encountered: