Engineering questions regarding the integration PR of LiteLLM

[TomExMachina]

Some proactive questions regarding #95:

• Why shouldn't autogen implement its own solution?
• What other options were considered before deciding LiteLLM was the appropriate solution?
• What are the foreseeable positive and negative implications of integrating LiteLLM?
• Why does LiteLLM need to be a required dependency rather than an [extra]?
• Does the PR make the usage of LiteLLM optional?
• What are the most urgent issues being tracked in the LiteLLM backlog, and for how long have they persisted?
• Since LiteLLM has a standalone "Proxy" mode which already works with Autogen, why does LiteLLM need to be installed as a dependency of Autogen? Why not just add a section to the README explaining how to use the proxy?
  
  I'm late to the PR-95 party, so figured I'd raise these before it's too late. I like LiteLLM. I'm just not sure if autogen developers will ultimately like being locked-into using it (If that's the plan).

Related Issue: #46
Related Discord Thread: https://discord.com/channels/1153072414184452236/1161283060277444648

[sonichi]

Thank you very much for raising these points. Many others and I have common concerns. The original proposed solution replaced the oai.Completion with LiteLLM, which is vetoed. The current solution is to only use it when api_type is specified as litellm. And it should be an optional dependency.

I'd like to hear alternative suggestions if you have any.

[TomExMachina]

Thanks for the reply. A few high-level suggestions come to mind:

• Require that any new dependency that isn't 'core', i.e., not required for autogen to function as expected in its most basic form be included as optional using the extras_require directive in setup.py.
• Require that a specific design pattern be used to implement optional dependencies. A single-level Strategy pattern sounds like good times.

I have additional ideas for helping maintain sanity without getting frozen in-place but I'll save those for PRs.