ollaya

★ 1.5k

Apache License 2.0Open Source — No PaywallDocker · Binary16+ active committers (12mo)

About

Ollaya is a local model-serving system for open decision models, positioned as a counterpart to Ollama for non-generative decision workflows. It serves models from a local daemon, offers a...

Live Demo Source Code Docs Website

Community Ratings

No ratings yet

New this week

Read recap →

Weekly Recap — Oct 30, 2026 – Nov 6, 2026Nov 6, 2026

Features

Local model serving
CLI commands for run/pull/list/ps/show/rm/cp/stop/create
TypeSafe-compatible API
Native decision API
MCP client support
Router models for language detection
Modelfiles for custom question sets
CPU and GPU inference
Model browsing/search
Desktop app

Details

Last Updated: Nov 11, 2026
Created: Sep 23, 2026
Install Methods: dockerbinary
Requirements: Linux x86_64 or arm64 with glibc ≥ 2.38macOS on Apple siliconWindows x64NVIDIA GPU driver R580+ for CUDA runtime installationCPU, CUDA, or Metal depending on model/runtime
API: REST
Privacy & Independence: No Cloud RequiredOffline Capable
Deployment: Deployment: Container image available

Run ollaya yourself

Pick how much you want to manage.

netcupBest match
Full control
A small server, then Docker Compose.
About 20 minutes with Docker
Get a netcup serveraffiliate(opens in a new tab)
Hetzner
Full control
A small server, then Docker Compose.
About 20 minutes with Docker
Get a Hetzner server(opens in a new tab)
DigitalOcean
Full control
A small server, then Docker Compose.
About 20 minutes with Docker
Get a DigitalOcean server(opens in a new tab)

1 more way to host

OVHcloudA small server, then Docker Compose.(opens in a new tab)

Some links earn us a commission. netcup pays us one and is our preferred VPS host.

Is this your project? Sponsor a listing

Alternatives to ollaya

Ollama

186.6k

Ollama is a tool for running open models locally and exposing them through a REST API for apps, agents, and integrations.

Artificial IntelligenceLocal LLMsGenerative AI+3

GPT4All

77.4k

GPT4All runs large language models privately on everyday desktops and laptops, and provides a Python client around llama.cpp implementations.

Artificial IntelligenceLocal LLMsAI Interfaces+2

vLLM

96.7k

A fast, easy-to-use library for LLM inference and serving.

Inference ServingLocal LLMsAI Tooling

2 Containers

Text Generation Inference

10.9k

A Rust, Python, and gRPC server for deploying and serving large language models with high-performance text generation.

Inference ServingArtificial IntelligenceLocal LLMs+1

2 Containers

LocalAI

50.3k

LocalAI is an open-source AI engine for running LLMs and other model types locally on a wide range of hardware, with drop-in API compatibility for several popular AI APIs.

Generative AIPermissive LicenseArtificial Intelligence+4

Single Container

KoboldCpp

12.2k

KoboldCpp is a self-contained AI text-generation runner for GGML and GGUF models, with bundled UI, multiple inference modes, and broad support for image, audio, video, and vision tasks.

Inference ServingLocal LLMsAI Interfaces+2

Single Container

Related Software

kev

9.9k

Kev is a decision-model server built on a Qwen3 base model that reads a document once and returns calibrated probabilities for typed questions. It provides a TypeSafe-compatible API and a local playground.

Artificial IntelligenceLocal LLMsInference Serving

LLM Harbor

3.3k

Harbor is a CLI and companion app for spinning up and managing a local LLM stack with pre-wired services like Open WebUI, Ollama, llama.cpp, SearXNG, Speaches, and ComfyUI.

Artificial IntelligenceLocal LLMsAI Tooling+5

LLMKube

264

Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API.

Artificial IntelligenceLocal LLMsInference Serving+2

LibreChat

46.9k

LibreChat is a self-hosted AI chat platform that unifies major AI providers in a ChatGPT-like interface.

Artificial IntelligenceAI InterfacesPermissive License+4

5 Containers

Manifest

7.6k

Manifest is an OpenClaw plugin that routes requests to the most cost-effective LLM and provides a dashboard for tracking usage and costs.

Low Code DevelopmentArtificial IntelligencePermissive License+4

Telemetry: Opt-Out

Jan

45.7k

Jan is an open-source desktop app for downloading and running local LLMs with privacy-focused, local control, while also offering optional connections to cloud model providers.

Artificial IntelligenceAI InterfacesPermissive License+1

Langfuse

37.3k

Langfuse is an open source LLM engineering platform for teams to develop, monitor, evaluate, and debug AI applications.

IDE ToolsArtificial IntelligenceFOSS with Issues+5

6 ContainersFOSS with Issues

HuggingFace Chat UI

11.0k

A SvelteKit chat interface for OpenAI-compatible LLM APIs, powering HuggingChat.

Artificial IntelligenceAI InterfacesSelf Hosting Solutions+1

llamafile

26.6k

llamafile lets users distribute and run LLMs as a single-file executable with no installation. It also includes whisperfile for local speech-to-text transcription and translation.

Artificial IntelligenceLocal LLMsAI Tooling+2

GoModel

1.3k

GoModel is an AI gateway for routing, caching, tracking, and controlling access to LLM requests through an OpenAI- and Anthropic-compatible API.

Artificial IntelligenceProxy ServersAI Tooling+4

6 ContainersOpen Core