[Bug]: DefaultAzureCredential not working

[WGlenn43]

What happened?

I tried to follow this guide. My exact setup for the yaml is:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/gpt-4o
      api_base: https://[my_url].openai.azure.com/

litellm_settings:
    enable_azure_ad_token_refresh: true

Relevant log output

$ > litellm --config C:\litellm_config.yaml
INFO:     Started server process [19156]
INFO:     Waiting for application startup.

#------------------------------------------------------------#
#                                                            #
#               'A feature I really want is...'               #
#        https://github.com/BerriAI/litellm/issues/new        #
#                                                            #
#------------------------------------------------------------#

 Thank you for using LiteLLM! - Krrish & Ishaan



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new


LiteLLM: Proxy initialized with Config, Set models:
    gpt-4o
ERROR:    Traceback (most recent call last):
  File "C:\home\starlette\routing.py", line 732, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "C:\home\starlette\routing.py", line 608, in __aenter__
    await self._router.startup()
  File "C:\home\starlette\routing.py", line 709, in startup
    await handler()
  File "C:\home\litellm\proxy\proxy_server.py", line 3128, in startup_event
    await initialize(**worker_config)
  File "C:\home\litellm\proxy\proxy_server.py", line 2564, in initialize
    ) = await proxy_config.load_config(router=llm_router, config_file_path=config)
  File "C:\home\litellm\proxy\proxy_server.py", line 2029, in load_config
    router = litellm.Router(
  File "C:\home\litellm\router.py", line 389, in __init__
    self.set_model_list(model_list)
  File "C:\home\litellm\router.py", line 3968, in set_model_list
    self._create_deployment(
  File "C:\home\litellm\router.py", line 3887, in _create_deployment
    deployment = self._add_deployment(deployment=deployment)
  File "C:\home\litellm\router.py", line 4049, in _add_deployment
    InitalizeOpenAISDKClient.set_client(
  File "C:\home\litellm\router_utils\client_initalization_utils.py", line 373, in set_client
    _client = openai.AsyncAzureOpenAI(  # type: ignore
  File "C:\home\openai\lib\azure.py", line 409, in __init__
    raise OpenAIError(
openai.OpenAIError: Missing credentials. Please pass one of `api_key`, `azure_ad_token`, `azure_ad_token_provider`, or the `AZURE_OPENAI_API_KEY` or `AZURE_OPENAI_AD_TOKEN` environment variables.

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.53.9

Twitter / LinkedIn details

No response

[krrishdholakia]

Able to repro

[krrishdholakia]

Ah, the docs are missing details.

You need these keys

AZURE_CLIENT_ID=""
AZURE_CLIENT_SECRET=""
AZURE_TENANT_ID=""

in your environment

[krrishdholakia]

adding those env vars solves the issue

[WGlenn43]

Ah, the docs are missing details.

You need these keys

AZURE_CLIENT_ID=""
AZURE_CLIENT_SECRET=""
AZURE_TENANT_ID=""

in your environment

This still does not seem to fix the issue. I get the exact same log output after setting these environment variables

$ > $env:AZURE_CLIENT_ID = ""
$ > $env:AZURE_CLIENT_SECRET = ""
$ > $env:AZURE_TENANT_ID = ""

$ > litellm --config C:\litellm_config.yaml
INFO:     Started server process [10632]
INFO:     Waiting for application startup.

#------------------------------------------------------------#
#                                                            #
#              'I don't like how this works...'               #
#        https://github.com/BerriAI/litellm/issues/new        #
#                                                            #
#------------------------------------------------------------#

 Thank you for using LiteLLM! - Krrish & Ishaan



Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new


LiteLLM: Proxy initialized with Config, Set models:
    gpt-4o
ERROR:    Traceback (most recent call last):
  File "C:\home\starlette\routing.py", line 732, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "C:\home\starlette\routing.py", line 608, in __aenter__
    await self._router.startup()
  File "C:\home\starlette\routing.py", line 709, in startup
    await handler()
  File "C:\home\litellm\proxy\proxy_server.py", line 3128, in startup_event
    await initialize(**worker_config)
  File "C:\home\litellm\proxy\proxy_server.py", line 2564, in initialize
    ) = await proxy_config.load_config(router=llm_router, config_file_path=config)
  File "C:\home\litellm\proxy\proxy_server.py", line 2029, in load_config
    router = litellm.Router(
  File "C:\home\litellm\router.py", line 389, in __init__
    self.set_model_list(model_list)
  File "C:\home\litellm\router.py", line 3968, in set_model_list
    self._create_deployment(
  File "C:\home\litellm\router.py", line 3887, in _create_deployment
    deployment = self._add_deployment(deployment=deployment)
  File "C:\home\litellm\router.py", line 4049, in _add_deployment
    InitalizeOpenAISDKClient.set_client(
  File "C:\home\litellm\router_utils\client_initalization_utils.py", line 373, in set_client
    _client = openai.AsyncAzureOpenAI(  # type: ignore
  File "C:\home\openai\lib\azure.py", line 409, in __init__
    raise OpenAIError(
openai.OpenAIError: Missing credentials. Please pass one of `api_key`, `azure_ad_token`, `azure_ad_token_provider`, or the `AZURE_OPENAI_API_KEY` or `AZURE_OPENAI_AD_TOKEN` environment variables.

ERROR:    Application startup failed. Exiting.

[WGlenn43]

@krrishdholakia I'm still hitting this issue. I've tried setting all 3 items you suggested to a blank string. I'm running on Windows Powershell. I'm happy to get on a call to debug if it helps

[krrishdholakia]

Sure let's do that @WGlenn43

https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

[SeaDude]

Hm...this still doesn't create a passwordless experience. We exclusively use the 'DefaultAzureCredential()' method from the azure-identity library. When users create Azure OpenAI clients with this method, RBAC roles govern access.

We don't want to create, communicate or manage either AOAI API keys OR Service Principal client_secrets. And really, there is no need if the DefaultAzureCredential() can be used.

This is a huge deal for enterprise customers. Let's get aider in the hands of more enterprise GenAI devs.

[krrishdholakia]

When users create Azure OpenAI clients with this method

can you point me to how the azure openai client handles this @SeaDude