Text Generation Inference
★ 10.9kArchivedApache License 2.0Open Source — No Paywall2 Containers — 2 containers — light hosting footprintDocker · Binary · Source13 active committers (12mo)
About
Text Generation Inference (TGI) is a server toolkit for deploying and serving large language models. It is aimed at teams and operators who need a production-oriented inference backend for popular...
Community Ratings
No ratings yetFeatures
- LLM serving
- production-ready deployment
- distributed tracing
- Prometheus metrics
- tensor parallelism
- token streaming
- continuous batching
- OpenAI-compatible Messages API
- quantization
- safetensors weight loading
Details
- Last Updated
- Aug 5, 2026
- Created
- Oct 8, 2022
- Install Methods
- dockerbinarysource
- Requirements
- NVIDIA Container ToolkitNVIDIA drivers with CUDA 12.2 or higherAMD Instinct MI210 or MI250 GPUsNCCLPyTorch
- Authentication
- local
- API
- RESTGRPC
- Privacy & Independence
- Cloud: Optional
- Deployment
Deployment: Single container (docker run)2 Containers — light hosting footprintRuns as 2 containers
Docker Deployment
2 containers (docker run)
no resource limits declared
scanned Aug 7, 2026
Tags
Track your self-hosted stack
Bookmark software to try, rate tools you've used, and keep your collection in one place.
Metadata extracted from README on Aug 7, 2026
Related Software
hermes-agent
226.6k
Hermes Agent is a self-improving AI agent with a built-in learning loop, terminal UI, and messaging gateway for chatting and automating work across platforms.
Artificial IntelligenceAI AgentsAI Coding Assistant+2
2 Containers
n8n
199.6k
n8n is a fair-code platform for building and deploying AI agents and workflow automation with a visual canvas, custom code, and 1500+ integrations.
Automation ToolsPermissive LicenseLow Code Development+5
Single ContainerOpen CoreNot FOSS
opencode
194.4k
An open source AI coding agent.
Artificial IntelligenceAI Coding AssistantPermissive License+3
AutoGPT
186.1k
AutoGPT is a platform for creating, deploying, and managing continuous AI agents that automate complex workflows.
Artificial IntelligenceAutomation ToolsPermissive License+2
13 ContainersOpen Core
Ollama
177.9k
Ollama is a tool for running open models locally and exposing them through a REST API for apps, agents, and integrations.
Artificial IntelligenceLocal LLMsPermissive License+2
Langflow
152.9k
Langflow is a platform for building and deploying AI-powered agents and workflows with a visual builder, API deployment, and MCP server support.
Low Code DevelopmentAutomation ToolsModel Context Protocol+2
11 Containers
Dify.ai
151.6k
An open-source platform for building, testing, and operating LLM-powered applications with workflows, RAG, agents, and observability.
Low Code DevelopmentArtificial IntelligencePermissive License+6
36 ContainersOpen CoreFOSS with Issues
Open-WebUI
148.1k
Open WebUI is a self-hosted AI platform for chatting with LLMs, managing RAG, and building custom model workflows, designed to run entirely offline.
Artificial IntelligenceAI InterfacesLocal LLMs+6
2 ContainersOpen CoreFOSS with Issues
Claude Code
140.5k
Claude Code is an agentic coding tool that runs in the terminal and helps with coding tasks, code explanation, and git workflows using natural language commands.
Artificial IntelligenceAI Coding AssistantAI Harness+2
Cloud Required
ComfyUI
124.4k
ComfyUI is a modular node-based AI engine for creating images, video, 3D, audio, and other content with fine-grained control over models and parameters.
Artificial IntelligenceAI Media GenerationCopyleft License+2
Freemium
Codex
104.5k
Codex CLI is a local coding agent from OpenAI that runs on your computer.
Artificial IntelligenceAI Coding AssistantPermissive License+3
Uptime Kuma
89.9k
Uptime Kuma is an easy-to-use self-hosted monitoring tool for checking service and server uptime.
Status PagesPermissive LicenseNetwork Utilities+3
Single ContainerFully FOSS