Self-Hosted Local LLMs (30)
Best for your setup
Homelab Novice
Wants it running tonight, with one command.
AnythingLLM66.9k
An all-in-one AI application for chatting with documents, building AI agents, and running local or cloud LLM workflows.
- Single container
- Ships a Compose file
See all88/100 fitHow well this app matches homelab novices, 0 to 100
Homelab Expert
Runs a stack already and wants it to plug in.
Olares5.3k
Olares is an open-source personal cloud operating system for running and managing private services locally on your own hardware.
- SSO sign-in
- Multi-user
See allDefault rank #20
84/100 fitHow well this app matches homelab experts, 0 to 100
30 projects
AnythingLLM
66.9k
An all-in-one AI application for chatting with documents, building AI agents, and running local or cloud LLM workflows.
Artificial IntelligenceAutomation ToolsSelf Hosting Solutions+7
Single Container
SillyTavern
34.3k
SillyTavern is a locally installed interface for interacting with text generation LLMs, image generation engines, and TTS voice models.
Artificial IntelligenceAI InterfacesCopyleft License+3
Single Container
Ollama
182.7k
Ollama is a tool for running open models locally and exposing them through a REST API for apps, agents, and integrations.
Artificial IntelligenceLocal LLMsGenerative Artificial Intelligence Genai+3
vLLM
93.5k
A fast, easy-to-use library for LLM inference and serving.
Inference ServingLocal LLMsAI Tooling
2 Containers
LocalAI
49.5k
LocalAI is an open-source AI engine that runs LLMs and other multimodal models locally on a wide range of hardware, with OpenAI-compatible APIs and on-demand backends.
Generative Artificial Intelligence GenaiPermissive LicenseArtificial Intelligence+4
Single Container
private-gpt
57.6k
PrivateGPT is an open-source API layer that turns local models into production AI applications.
Artificial IntelligenceAI RAG SystemsPermissive License+2
LibreChat
45.5k
LibreChat is a self-hosted AI chat platform that unifies major AI providers in a ChatGPT-like interface.
Artificial IntelligenceAI InterfacesLocal LLMs+2
6 Containers
text-generation-webui
47.7k
A desktop app for running local LLMs with chat, vision, tool-calling, training, image generation, and an API.
Artificial IntelligenceSelf Hosting SolutionsAI Interfaces+2
Single Container
Jan
44.9k
Jan is an open-source desktop app for downloading and running local LLMs with privacy-focused, local control, while also offering optional connections to cloud model providers.
Artificial IntelligenceAI InterfacesPermissive License+1
llamafile
26.2k
llamafile lets users distribute and run LLMs as a single-file executable with no installation. It also includes whisperfile for local speech-to-text transcription and translation.
Artificial IntelligenceLocal LLMsAI Tooling+2
KoboldCpp
12.0k
KoboldCpp is a self-contained AI text-generation runner for GGML and GGUF models, with bundled UI, multiple inference modes, and broad support for image, audio, video, and vision tasks.
Inference ServingLocal LLMsAI Interfaces+2
Single Container
Open-WebUI
154.2k
Open WebUI is a self-hosted AI platform for chatting with LLMs, managing RAG, and building custom model workflows, designed to run entirely offline.
Artificial IntelligenceAI InterfacesLocal LLMs+7
2 ContainersOpen CoreFOSS with Issues