vLLM
★ 96.3kCompare:Ollama vs vLLM
Apache License 2.0Open Source — No Paywall2 Containers — 2 containers — light hosting footprintPackage · SourceNo external DB57 documented maintainers
About
vLLM is an open-source library for serving and running large language models efficiently. It is aimed at developers and teams that need high-throughput inference for a wide range of model families,...
Community Ratings
No ratings yetNew this week
Read recap →Weekly Recap — Oct 23, 2026 – Oct 30, 2026Oct 30, 2026
Features
- LLM inference and serving
- PagedAttention memory management
- continuous batching
- chunked prefill
- prefix caching
- streaming outputs
- structured output generation
- tool calling
- OpenAI-compatible API server
- gRPC support
Details
- Last Updated
- Nov 5, 2026
- Created
- Feb 9, 2023
- Install Methods
- package-managersource
- Requirements
- uv or pip
- API
- RESTGRPC
- Privacy & Independence
- No Cloud Required
- Deployment
Deployment: Docker Compose ✓2 Containers — light hosting footprintRuns as 2 containers
Run vLLM yourself
Pick how much you want to manage.
netcupBest matchFull control
A small server, then Docker Compose.
About 20 minutes with Docker
HetznerFull control
A small server, then Docker Compose.
About 20 minutes with Docker
DigitalOceanFull control
A small server, then Docker Compose.
About 20 minutes with Docker
1 more way to host
Some links earn us a commission. netcup pays us one and is our preferred VPS host.
Is this your project? Sponsor a listing
Run vLLM with Docker Compose
Compose file from the vLLM project repository.
2 services · 2 unclassified (prometheus, grafana)
no resource limits declared
needs persistent storage (1 volume)
examples/observability/prometheus_grafana/docker-compose.yaml · @ 8d9b52f · scanned Nov 3, 2026
# docker-compose.yaml
version: "3"
services:
prometheus:
image: prom/prometheus:latest
extra_hosts:
- "host.docker.internal:host-gateway" # allow a direct connection from container to the local machine
ports:
- "9090:9090" # the default port used by Prometheus
volumes:
- ${PWD}/prometheus.yaml:/etc/prometheus/prometheus.yml # mount Prometheus config file
grafana:
image: grafana/grafana:latest
depends_on:
- prometheus
ports:
- "3000:3000" # the default port used by Grafana
This file may be out of date. Check the vLLM documentation for current setup steps.
Track your self-hosted stack
Bookmark software to try, rate tools you've used, and keep your collection in one place.
Metadata extracted from README on Aug 7, 2026
Alternatives to vLLM
Text Generation Inference
10.9k
A Rust, Python, and gRPC server for deploying and serving large language models with high-performance text generation.
Inference ServingArtificial IntelligenceLocal LLMs+1
2 Containers
Ollama
185.9k
Ollama is a tool for running open models locally and exposing them through a REST API for apps, agents, and integrations.
Artificial IntelligenceLocal LLMsGenerative AI+3
LocalAI
50.2k
LocalAI is an open-source AI engine that runs LLMs and other multimodal models locally on a wide range of hardware, with OpenAI-compatible APIs and on-demand backends.
Generative AIPermissive LicenseArtificial Intelligence+4
Single Container
KoboldCpp
12.2k
KoboldCpp is a self-contained AI text-generation runner for GGML and GGUF models, with bundled UI, multiple inference modes, and broad support for image, audio, video, and vision tasks.
Inference ServingLocal LLMsAI Interfaces+2
Single Container
GPT4All
77.3k
GPT4All runs large language models privately on everyday desktops and laptops, and provides a Python client around llama.cpp implementations.
Artificial IntelligenceLocal LLMsAI Interfaces+2
ollaya
1.4k
Ollaya runs open decision models locally and provides Ollama-like CLI, daemon, and API access for model serving.
Artificial IntelligenceLocal LLMsInference Serving+2
llamafile
26.6k
llamafile lets users distribute and run LLMs as a single-file executable with no installation. It also includes whisperfile for local speech-to-text transcription and translation.
Artificial IntelligenceLocal LLMsAI Tooling+2
Related Software
Dify.ai
160.4k
An open-source platform for building, testing, and operating LLM-powered applications with workflows, RAG, agents, and observability.
Low Code DevelopmentArtificial IntelligencePermissive License+6
36 ContainersOpen CoreFOSS with Issues
LiteLLM
62.7k
LiteLLM is an open source AI gateway and Python SDK for calling 100+ LLM providers through a unified OpenAI-compatible interface.
API ManagementProxy ServersArtificial Intelligence+2
3 ContainersOpen Core
LLMKube
252
Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API.
Artificial IntelligenceLocal LLMsInference Serving+2
LLM Harbor
3.3k
Harbor is a CLI and companion app for spinning up and managing a local LLM stack with pre-wired services like Open WebUI, Ollama, llama.cpp, SearXNG, Speaches, and ComfyUI.
Artificial IntelligenceLocal LLMsAI Tooling+5
private-gpt
57.6k
PrivateGPT is an open-source API layer that turns local models into production AI applications.
Artificial IntelligenceAI RAG SystemsPermissive License+2
Langfuse
37.0k
Langfuse is an open source LLM engineering platform for teams to develop, monitor, evaluate, and debug AI applications.
IDE ToolsArtificial IntelligencePermissive License+5
6 ContainersFOSS with Issues
HuggingFace Chat UI
11.0k
A SvelteKit chat interface for OpenAI-compatible LLM APIs, powering HuggingChat.
Artificial IntelligenceAI InterfacesSelf Hosting Solutions+1
LibreChat
46.8k
LibreChat is a self-hosted AI chat platform that unifies major AI providers in a ChatGPT-like interface.
Artificial IntelligenceAI InterfacesPermissive License+4
5 Containers
Open-WebUI
157.8k
Open WebUI is a self-hosted AI platform for chatting with LLMs, managing RAG, and building custom model workflows, designed to run entirely offline.
Artificial IntelligenceAI InterfacesLocal LLMs+7
2 ContainersOpen CoreFOSS with Issues
LibreChat
46.8k
LibreChat is a self-hosted AI chat platform that unifies major AI providers in a ChatGPT-like interface.
Artificial IntelligenceAI InterfacesLocal LLMs+2
6 Containers
AnythingLLM
68.0k
An all-in-one AI application for chatting with documents, building AI agents, and running local or cloud LLM workflows.
Artificial IntelligenceAutomation ToolsSelf Hosting Solutions+7
Single Container
text-generation-webui
47.7k
A desktop app for running local LLMs with chat, vision, tool-calling, training, image generation, and an API.
Artificial IntelligenceSelf Hosting SolutionsAI Interfaces+2
Single Container