Try describing what you need:

LLMKube logo

LLMKube

★ 267
Apache License 2.02 documented maintainers

Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API.

Community Ratings

No ratings yet

Details

Last Updated
Nov 21, 2026
Created
Nov 12, 2025
Runtime / Stack
GoDockerK8S
Deployment

Deployment: no container deployment found

Track your self-hosted stack

Bookmark software to try, rate tools you've used, and keep your collection in one place.

Alternatives to LLMKube

LocalAI logo

LocalAI

50.5k
LocalAI is an open-source AI engine for running LLMs and other model types locally on a wide range of hardware, with drop-in API compatibility for several popular AI APIs.
Generative AIPermissive LicenseArtificial Intelligence+6
Single Container
Details
KoboldCpp logo

KoboldCpp

12.4k
KoboldCpp is a self-contained AI text-generation runner for GGML and GGUF models, with bundled UI, multiple inference modes, and broad support for image, audio, video, and vision tasks.
Inference ServingLocal LLMsAI Interfaces+2
Single Container
Details

Related Software

LLM logo

LLM

12.8k
A CLI tool and Python library for interacting with large language models from remote APIs or locally installed models.
Artificial IntelligenceLocal LLMsAI Tooling+1
Details
Ollama logo

Ollama

187.8k
Ollama is a tool for running open models locally and exposing them through a REST API for apps, agents, and integrations.
Artificial IntelligenceLocal LLMsGenerative AI+3
Details
Text Generation Inference logo

Text Generation Inference

10.9k
A Rust, Python, and gRPC server for deploying and serving large language models with high-performance text generation.
Inference ServingArtificial IntelligenceLocal LLMs+1
2 Containers
Details
AnythingLLM logo

AnythingLLM

68.5k
An all-in-one AI application for chatting with documents, building AI agents, and running local or cloud LLM workflows.
Artificial IntelligenceAutomation ToolsGenerative AI+7
Single Container
Details
GoModel logo

GoModel

1.3k
GoModel is an AI gateway for routing, caching, tracking, and controlling access to LLM requests through an OpenAI- and Anthropic-compatible API.
Artificial IntelligenceGenerative AIAPI Management+4
6 ContainersOpen Core
Details
vLLM logo

vLLM

97.3k
vLLM is a fast, easy-to-use library and server for LLM inference and serving.
Inference ServingLocal LLMsAI Tooling
2 Containers
Details
NextChatAI logo

NextChatAI

89.0k
NextChat is a light, fast AI assistant web app and desktop client with support for multiple AI models and local browser storage.
Artificial IntelligenceAI InterfacesPermissive License+1
2 ContainersOpen Core
Details
Dify.ai logo

Dify.ai

161.6k
An open-source platform for building, testing, and operating LLM-powered applications with workflows, RAG, agents, and observability.
Low Code DevelopmentSelf Hosting SolutionsAutomation Tools+7
36 ContainersOpen CoreFOSS with Issues
Details
Manifest logo

Manifest

7.7k
Manifest is an OpenClaw plugin that routes requests to the most cost-effective LLM and provides a dashboard for tracking usage and costs.
Low Code DevelopmentArtificial IntelligenceAI Tooling+4
Telemetry: Opt-Out
Details
openclaude logo

openclaude

34.5k
OpenClaude is an open-source terminal-based coding agent CLI for cloud and local model providers.
AI Coding AssistantAI AgentsAI Tooling+1
Details
GPT4All logo

GPT4All

77.3k
GPT4All runs large language models privately on everyday desktops and laptops, and provides a Python client around llama.cpp implementations.
Artificial IntelligenceLocal LLMsAI Interfaces+2
Details
venator logo

venator

35
Matrix Venator - versatile capital Matrix homeserver written from scratch in mautrix-go
Custom CommunicationsSelf Hosting Solutions
Details