Hemera — A Lightweight Bridge for Local & Cloud AI Models

Running local LLMs often means dealing with moving targets. Runtimes like llama.cpp move at a breakneck speed—sometimes releasing multiple builds in a single day. While that pace brings support for new quantizations immediately, an engine update today can easily break or regress a model that ran perfectly yesterday.

On top of that, managing different GPU backends (CUDA, ROCm, Vulkan), missing dynamic libraries, and setting up models for different client tools usually means juggling multiple local setups.

If you want to run local GGUF models—or route to cloud providers—without worrying about an engine update breaking your setup, you quickly run into friction.

That's why I came with idea of Hemera—a flexible local AI engine manager, wrapper, and proxy router written in Java.

The Spirit of the Project

The philosophy behind Hemera is simple: unify your AI tooling into one quiet, self-contained service.

Instead of juggling different local wrappers or hardcoding provider-specific integrations into your apps, Hemera sits in the middle. It acts as both a local engine orchestrator and a multi-dialect API translator.

I wanted a tool that:

• Doesn't force a framework on you: Zero external dependencies—no Maven, no Gradle, no heavy web frameworks, and no npm build step. Just JDK 21+ and a single 112 KB JAR file.

• Never deletes what worked: Engine builds live side-by-side. Updating llama.cpp never removes older versions, so if a new build misbehaves, you can roll back or pin a model to a previous tag in one click.

• Handles local & remote natively: Reads local GGUF files directly, manages backends (CUDA, ROCm, Vulkan) per model, and makes remote models (DeepSeek, OpenAI, Gemini, Claude) look like local endpoints.

Key Features

• One Endpoint, Three Dialects: Supports Ollama, OpenAI, and Anthropic Claude endpoints on a single port (12434). Every tool in your stack—from the native ollama CLI to Claude Code—can drive your models.

• Engine Versioning & Rollbacks: Keep multiple engine releases and GPU backends (ROCm, CUDA, Vulkan, SYCL, CPU) installed simultaneously without interference.

• Smart GPU Discovery (ELF Header Inspection): If a GPU build is missing vendor dynamic libraries, it often quietly falls back to CPU execution at a crawl. Hemera reads ELF headers on startup, detects missing .sodependencies, searches system paths, and automatically sets LD_LIBRARY_PATH for that process.

• Virtual Model Aliases: A single GGUF file can be configured as multiple named models—each with its own context size, GPU offload settings, chat templates, and environment variables.

• Native GGUF Inspection & Safe Deletion: Built-in direct GGUF header reading (GgufReader.java) parses metadata without external CLI tools. Deleting a model in the UI only drops the configuration, keeping expensive GGUF files safe.

• Minimalist Web Dashboard: An embedded, zero-framework web UI (vanilla HTML/CSS/JS) that follows your OS light/dark theme for monitoring running models, engine PIDs, and proxy settings.

Under the Hood (Architecture)

Hemera’s codebase is structured around a few core modules inside ing.aether.hemera:

ing.aether.hemera
├── api         # API dialects (OpenAI, Claude, Ollama) & Proxy Dispatcher
├── engine      # Engine downloaders, ELF/Tar handling, Instance lifecycle
├── model       # Bare-metal GGUF reader & Model Registry
├── http & json # Custom HTTP router & lightweight JSON engine
└── web         # Admin REST API endpoints & static dashboard assets

1. The Proxy & API Layer (hemera.api): Intercepts incoming requests, normalizes payloads via ProxyDispatcher, translates between protocols on the fly, and routes requests to local processes or cloud APIs while preserving Server-Sent Events (SSE) streaming.

2. The Engine Orchestrator (hemera.engine): Manages executable runtimes. It resolves required native shared libraries (LibraryResolver.java), handles binary archives (Tar.java), and manages the lifecycle of local engine processes (InstanceManager.java).

3. The Model Registry (hemera.model): Instead of relying on external CLI tools to inspect files, ModelScanner searches local directories and uses GgufReader to parse tensor metadata directly from binary headers to feed the central registry.

4. Core Infrastructure (hemera.http & hemera.json): Powered by a custom hand-rolled JSON parser and HTTP router running on Java's virtual threads for maximum efficiency.

Check out the code on GitHub: github.com/peprasetya/hemera