class_docstr: Example usage:
  ```python
  from litellm import Router
  model_list = [
  {
      "model_name": "azure-gpt-3.5-turbo", # model alias
      "litellm_params": { # params for litellm completion/embedding call
          "model": "azure/<your-deployment-name-1>",
          "api_key": <your-api-key>,
          "api_version": <your-api-version>,
          "api_base": <your-api-base>
      },
  },
  {
      "model_name": "azure-gpt-3.5-turbo", # model alias
      "litellm_params": { # params for litellm completion/embedding call
          "model": "azure/<your-deployment-name-2>",
          "api_key": <your-api-key>,
          "api_version": <your-api-version>,
          "api_base": <your-api-base>
      },
  },
  {
      "model_name": "openai-gpt-3.5-turbo", # model alias
      "litellm_params": { # params for litellm completion/embedding call
          "model": "gpt-3.5-turbo",
          "api_key": <your-api-key>,
      },
  ]
  
  router = Router(model_list=model_list, fallbacks=[{"azure-gpt-3.5-turbo": "openai-gpt-3.5-turbo"}])
  ```
docstr: Initializes Azure/OpenAI clients. Stores them in cache, b/c of this - https://github.com/BerriAI/litellm/issues/1278
code: def set_client(self, model: dict):
      """
      Initializes Azure/OpenAI clients. Stores them in cache, b/c of this - https://github.com/BerriAI/litellm/issues/1278
      """
      client_ttl = self.client_ttl
      litellm_params = model.get("litellm_params", {})
      model_name = litellm_params.get("model")
      model_id = model["model_info"]["id"]
      ####  for OpenAI / Azure we need to initalize the Client for High Traffic ########
      custom_llm_provider = litellm_params.get("custom_llm_provider")
      custom_llm_provider = (
          custom_llm_provider or model_name.split("/", 1)[0] or ""
      )
      default_api_base = None
      default_api_key = None
      if custom_llm_provider in litellm.openai_compatible_providers:
          (
              _,
              custom_llm_provider,
              api_key,
              api_base,
          ) = litellm.get_llm_provider(model=model_name)
          default_api_base = api_base
          default_api_key = api_key
      if (
          model_name in litellm.open_ai_chat_completion_models
          or custom_llm_provider in litellm.openai_compatible_providers
          or custom_llm_provider == "azure"
          or custom_llm_provider == "custom_openai"
          or custom_llm_provider == "openai"
          or "ft:gpt-3.5-turbo" in model_name
          or model_name in litellm.open_ai_embedding_models
      ):
          # glorified / complicated reading of configs
          # user can pass vars directly or they can pas os.environ/AZURE_API_KEY, in which case we will read the env
          # we do this here because we init clients for Azure, OpenAI and we need to set the right key
          api_key = litellm_params.get("api_key") or default_api_key
          if api_key and api_key.startswith("os.environ/"):
              api_key_env_name = api_key.replace("os.environ/", "")
              api_key = litellm.get_secret(api_key_env_name)
              litellm_params["api_key"] = api_key
  
          api_base = litellm_params.get("api_base")
          base_url = litellm_params.get("base_url")
          api_base = (
              api_base or base_url or default_api_base
          )  # allow users to pass in `api_base` or `base_url` for azure
          if api_base and api_base.startswith("os.environ/"):
              api_base_env_name = api_base.replace("os.environ/", "")
              api_base = litellm.get_secret(api_base_env_name)
              litellm_params["api_base"] = api_base
  
          api_version = litellm_params.get("api_version")
          if api_version and api_version.startswith("os.environ/"):
              api_version_env_name = api_version.replace(
                  "os.environ/", ""
              )
              api_version = litellm.get_secret(api_version_env_name)
              litellm_params["api_version"] = api_version
  
          timeout = litellm_params.pop("timeout", None)
          if isinstance(timeout, str) and timeout.startswith(
              "os.environ/"
          ):
              timeout_env_name = timeout.replace("os.environ/", "")
              timeout = litellm.get_secret(timeout_env_name)
              litellm_params["timeout"] = timeout
  
          stream_timeout = litellm_params.pop(
              "stream_timeout", timeout
          )  # if no stream_timeout is set, default to timeout
          if isinstance(
              stream_timeout, str
          ) and stream_timeout.startswith("os.environ/"):
              stream_timeout_env_name = stream_timeout.replace(
                  "os.environ/", ""
              )
              stream_timeout = litellm.get_secret(
                  stream_timeout_env_name
              )
              litellm_params["stream_timeout"] = stream_timeout
  
          max_retries = litellm_params.pop("max_retries", 2)
          if isinstance(max_retries, str) and max_retries.startswith(
              "os.environ/"
          ):
              max_retries_env_name = max_retries.replace(
                  "os.environ/", ""
              )
              max_retries = litellm.get_secret(max_retries_env_name)
              litellm_params["max_retries"] = max_retries
  
          if "azure" in model_name:
              if api_base is None:
                  raise ValueError(
                      f"api_base is required for Azure OpenAI. Set it on your config. Model - {model}"
                  )
              if api_version is None:
                  api_version = "2023-07-01-preview"
              if "gateway.ai.cloudflare.com" in api_base:
                  if not api_base.endswith("/"):
                      api_base += "/"
                  azure_model = model_name.replace("azure/", "")
                  api_base += f"{azure_model}"
                  cache_key = f"{model_id}_async_client"
                  _client = openai.AsyncAzureOpenAI(
                      api_key=api_key,
                      base_url=api_base,
                      api_version=api_version,
                      timeout=timeout,
                      max_retries=max_retries,
                      http_client=httpx.AsyncClient(
                          transport=AsyncCustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),  # type: ignore
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
  
                  cache_key = f"{model_id}_client"
                  _client = openai.AzureOpenAI(  # type: ignore
                      api_key=api_key,
                      base_url=api_base,
                      api_version=api_version,
                      timeout=timeout,
                      max_retries=max_retries,
                      http_client=httpx.Client(
                          transport=CustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),  # type: ignore
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
                  # streaming clients can have diff timeouts
                  cache_key = f"{model_id}_stream_async_client"
                  _client = openai.AsyncAzureOpenAI(  # type: ignore
                      api_key=api_key,
                      base_url=api_base,
                      api_version=api_version,
                      timeout=stream_timeout,
                      max_retries=max_retries,
                      http_client=httpx.AsyncClient(
                          transport=AsyncCustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),  # type: ignore
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
  
                  cache_key = f"{model_id}_stream_client"
                  _client = openai.AzureOpenAI(  # type: ignore
                      api_key=api_key,
                      base_url=api_base,
                      api_version=api_version,
                      timeout=stream_timeout,
                      max_retries=max_retries,
                      http_client=httpx.Client(
                          transport=CustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),  # type: ignore
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
              else:
                  verbose_router_logger.debug(
                      f"Initializing Azure OpenAI Client for {model_name}, Api Base: {str(api_base)}, Api Key:{api_key}"
                  )
  
                  cache_key = f"{model_id}_async_client"
                  _client = openai.AsyncAzureOpenAI(  # type: ignore
                      api_key=api_key,
                      azure_endpoint=api_base,
                      api_version=api_version,
                      timeout=timeout,
                      max_retries=max_retries,
                      http_client=httpx.AsyncClient(
                          transport=AsyncCustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),  # type: ignore
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
  
                  cache_key = f"{model_id}_client"
                  _client = openai.AzureOpenAI(  # type: ignore
                      api_key=api_key,
                      azure_endpoint=api_base,
                      api_version=api_version,
                      timeout=timeout,
                      max_retries=max_retries,
                      http_client=httpx.Client(
                          transport=CustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),  # type: ignore
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
  
                  # streaming clients should have diff timeouts
                  cache_key = f"{model_id}_stream_async_client"
                  _client = openai.AsyncAzureOpenAI(  # type: ignore
                      api_key=api_key,
                      azure_endpoint=api_base,
                      api_version=api_version,
                      timeout=stream_timeout,
                      max_retries=max_retries,
                      http_client=httpx.AsyncClient(
                          transport=AsyncCustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
  
                  cache_key = f"{model_id}_stream_client"
                  _client = openai.AzureOpenAI(  # type: ignore
                      api_key=api_key,
                      azure_endpoint=api_base,
                      api_version=api_version,
                      timeout=stream_timeout,
                      max_retries=max_retries,
                      http_client=httpx.Client(
                          transport=CustomHTTPTransport(),
                          limits=httpx.Limits(
                              max_connections=1000,
                              max_keepalive_connections=100,
                          ),
                      ),
                  )
                  self.cache.set_cache(
                      key=cache_key,
                      value=_client,
                      ttl=client_ttl,
                      local_only=True,
                  )  # cache for 1 hr
  
          else:
              verbose_router_logger.debug(
                  f"Initializing OpenAI Client for {model_name}, Api Base:{str(api_base)}, Api Key:{api_key}"
              )
              cache_key = f"{model_id}_async_client"
              _client = openai.AsyncOpenAI(  # type: ignore
                  api_key=api_key,
                  base_url=api_base,
                  timeout=timeout,
                  max_retries=max_retries,
                  http_client=httpx.AsyncClient(
                      transport=AsyncCustomHTTPTransport(),
                      limits=httpx.Limits(
                          max_connections=1000,
                          max_keepalive_connections=100,
                      ),
                  ),  # type: ignore
              )
              self.cache.set_cache(
                  key=cache_key,
                  value=_client,
                  ttl=client_ttl,
                  local_only=True,
              )  # cache for 1 hr
  
              cache_key = f"{model_id}_client"
              _client = openai.OpenAI(  # type: ignore
                  api_key=api_key,
                  base_url=api_base,
                  timeout=timeout,
                  max_retries=max_retries,
                  http_client=httpx.Client(
                      transport=CustomHTTPTransport(),
                      limits=httpx.Limits(
                          max_connections=1000,
                          max_keepalive_connections=100,
                      ),
                  ),  # type: ignore
              )
              self.cache.set_cache(
                  key=cache_key,
                  value=_client,
                  ttl=client_ttl,
                  local_only=True,
              )  # cache for 1 hr
  
              # streaming clients should have diff timeouts
              cache_key = f"{model_id}_stream_async_client"
              _client = openai.AsyncOpenAI(  # type: ignore
                  api_key=api_key,
                  base_url=api_base,
                  timeout=stream_timeout,
                  max_retries=max_retries,
                  http_client=httpx.AsyncClient(
                      transport=AsyncCustomHTTPTransport(),
                      limits=httpx.Limits(
                          max_connections=1000,
                          max_keepalive_connections=100,
                      ),
                  ),  # type: ignore
              )
              self.cache.set_cache(
                  key=cache_key,
                  value=_client,
                  ttl=client_ttl,
                  local_only=True,
              )  # cache for 1 hr
  
              # streaming clients should have diff timeouts
              cache_key = f"{model_id}_stream_client"
              _client = openai.OpenAI(  # type: ignore
                  api_key=api_key,
                  base_url=api_base,
                  timeout=stream_timeout,
                  max_retries=max_retries,
                  http_client=httpx.Client(
                      transport=CustomHTTPTransport(),
                      limits=httpx.Limits(
                          max_connections=1000,
                          max_keepalive_connections=100,
                      ),
                  ),  # type: ignore
              )
              self.cache.set_cache(
                  key=cache_key,
                  value=_client,
                  ttl=client_ttl,
                  local_only=True,
              )
  
imports: import copy, httpx
  from datetime import datetime
  from typing import Dict, List, Optional, Union, Literal, Any
  import random, threading, time, traceback, uuid
  import litellm, openai
  from litellm.caching import RedisCache, InMemoryCache, DualCache
  import logging, asyncio
  import inspect, concurrent
  from openai import AsyncOpenAI
  from collections import defaultdict
  from litellm.router_strategy.least_busy import LeastBusyLoggingHandler
  from litellm.router_strategy.lowest_tpm_rpm import LowestTPMLoggingHandler
  from litellm.router_strategy.lowest_latency import LowestLatencyLoggingHandler
  from litellm.llms.custom_httpx.azure_dall_e_2 import CustomHTTPTransport, AsyncCustomHTTPTransport
  from litellm.utils import ModelResponse, CustomStreamWrapper
  import copy
  from litellm._logging import verbose_router_logger
  import logging
label_desc: Use image node_id 14 for calling the Router obj's underlying member method code with example usage: obj.set_client(model) without return types