LLMxRayLocal LLM Observatory

See what your AI is actually doing — token by token, layer by layer.

Get Started

Developer Reference

GitHub

Chat Diagnostics

Stream tokens in real time with confidence coloring, inspect reasoning chains, and explore session metrics.

Model Comparison

Run the same prompt across up to 4 models or settings side by side. Grid view, diff view, and metrics bar.

Surgical Benchmark

Evaluate models with real logprobs on standardized test suites. Radar charts, per-category breakdowns.

Knowledge Base (RAG)

Upload documents, chunk and embed them locally, then search with natural language. All in IndexedDB.

Live

Point VS Code Copilot, Open WebUI or any other tool at LLMxRay instead of Ollama, and see every turn it sends, what the cache kept, and where the prompt broke it.

New in 0.7.2

Cache Lab

Measure how much of your prompt the KV cache reuses, find where reuse breaks, and measure what the rewrite saves every turn.

Decision Lab

Ask a decision model typed questions about a ticket or a message, and see the probability behind every answer, drawn as bars.

Tool Workshop

Build, edit, and test tool definitions on a visual node-based canvas with bidirectional code sync.

100% Local & Free

Everything runs on your hardware via Ollama. No cloud, no API keys, no cost. Your data stays yours.

Educators Kit

8-module progressive curriculum for AI/ML courses. From "What is a token?" to original research. Free, grounded in published science.

View Curriculum