inference｜支持GPT和LLM的推理APIxorbitsai/inference

通过修改一行代码即可替换GPT为任意LLM，支持云端、本地及笔记本运行，提供统一且生产级的推理API。 Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.