Private LLM and Chat UI: Now Open-Source on Platform Mesh
2026-05-18
Today we are open-sourcing the Private LLM and Chat UI providers for Platform Mesh, together with a step-by-step tutorial that runs the entire demo on a single laptop in about 35 minutes.
Platform Mesh turns Kubernetes operators into managed services: a provider writes an operator once, registers it through Platform Mesh, and every customer reaches it through their own workspace, never touching the cluster where the operator runs. What we want to highlight with this example is composition. Private LLM and Chat UI are completely independent (different teams, different repos, different operators) but they compose into a working private chatbot through a single Secret contract. The customer wires them together; the provider teams never have to coordinate.
Private LLM runs a llama.cpp inference server. The customer creates an LLMInstance and the provider hands back a Secret with an OpenAI-compatible endpoint.
Chat UI deploys a per-tenant Open WebUI. The customer creates a ChatUIInstance pointing at any OpenAI-compatible Secret, including the one Private LLM just created.
Try it on your laptop. The full tutorial is published as part of the Apeiro Reference Architecture documentation. Everything runs inside a single Kind cluster, yet the logical layers match the production setup exactly. You play both roles: first the platform team, then the customer.