LLMKube
★ 267Apache License 2.02 documented maintainers
Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API.
Community Ratings
No ratings yetNew this week
Read recap →Weekly Recap — Nov 13, 2026 – Nov 20, 2026Nov 20, 2026
Details
- Last Updated
- Nov 21, 2026
- Created
- Nov 12, 2025
- Runtime / Stack
- GoDockerK8S
- Deployment
Deployment: no container deployment found
Track your self-hosted stack
Bookmark software to try, rate tools you've used, and keep your collection in one place.
Alternatives to LLMKube
LocalAI
50.5k
LocalAI is an open-source AI engine for running LLMs and other model types locally on a wide range of hardware, with drop-in API compatibility for several popular AI APIs.
Generative AIPermissive LicenseArtificial Intelligence+6
Single Container
KoboldCpp
12.4k
KoboldCpp is a self-contained AI text-generation runner for GGML and GGUF models, with bundled UI, multiple inference modes, and broad support for image, audio, video, and vision tasks.
Inference ServingLocal LLMsAI Interfaces+2
Single Container
Related Software
LLM
12.8k
A CLI tool and Python library for interacting with large language models from remote APIs or locally installed models.
Artificial IntelligenceLocal LLMsAI Tooling+1
Ollama
187.8k
Ollama is a tool for running open models locally and exposing them through a REST API for apps, agents, and integrations.
Artificial IntelligenceLocal LLMsGenerative AI+3
Text Generation Inference
10.9k
A Rust, Python, and gRPC server for deploying and serving large language models with high-performance text generation.
Inference ServingArtificial IntelligenceLocal LLMs+1
2 Containers
AnythingLLM
68.5k
An all-in-one AI application for chatting with documents, building AI agents, and running local or cloud LLM workflows.
Artificial IntelligenceAutomation ToolsGenerative AI+7
Single Container
GoModel
1.3k
GoModel is an AI gateway for routing, caching, tracking, and controlling access to LLM requests through an OpenAI- and Anthropic-compatible API.
Artificial IntelligenceGenerative AIAPI Management+4
6 ContainersOpen Core
vLLM
97.3k
vLLM is a fast, easy-to-use library and server for LLM inference and serving.
Inference ServingLocal LLMsAI Tooling
2 Containers
NextChatAI
89.0k
NextChat is a light, fast AI assistant web app and desktop client with support for multiple AI models and local browser storage.
Artificial IntelligenceAI InterfacesPermissive License+1
2 ContainersOpen Core
Dify.ai
161.6k
An open-source platform for building, testing, and operating LLM-powered applications with workflows, RAG, agents, and observability.
Low Code DevelopmentSelf Hosting SolutionsAutomation Tools+7
36 ContainersOpen CoreFOSS with Issues
Manifest
7.7k
Manifest is an OpenClaw plugin that routes requests to the most cost-effective LLM and provides a dashboard for tracking usage and costs.
Low Code DevelopmentArtificial IntelligenceAI Tooling+4
Telemetry: Opt-Out
openclaude
34.5k
OpenClaude is an open-source terminal-based coding agent CLI for cloud and local model providers.
AI Coding AssistantAI AgentsAI Tooling+1
GPT4All
77.3k
GPT4All runs large language models privately on everyday desktops and laptops, and provides a Python client around llama.cpp implementations.
Artificial IntelligenceLocal LLMsAI Interfaces+2
venator
35
Matrix Venator - versatile capital Matrix homeserver written from scratch in mautrix-go
Custom CommunicationsSelf Hosting Solutions