LiteLLM

LiteLLM simplifies model access, spend tracking, and fallbacks across 100+ LLMs, offering a unified OpenAI-compatible format. It allows platform teams to grant developers easy access to various LLMs while accurately tracking usage and costs. Features include automatic spend tracking across providers like OpenAI, Azure, Bedrock, and GCP, tag-based spend tracking, and logging spend to S3/GCS. LiteLLM supports prompt formatting for HF models, budgets, rate limits, and LLM observability. It facilitates efficient LLM management by standardizing logging, API, and authentication, significantly reducing operational complexities and enabling quick adaptation to new models.