Introduction
LLMxRay is a free, local-first dashboard that connects to Ollama running on your machine. It lets you chat with any model you've downloaded and then inspect everything that happened behind the scenes: how fast each token arrived, what the model might have been "thinking", how different settings change the output, and much more.
No cloud. No API keys. No cost. Everything runs on your hardware.
Who is this for?
| You are... | LLMxRay helps you... |
|---|---|
| Curious beginner | See AI responses form in real time and learn what "temperature" or "tokens" actually mean |
| Student / educator | Explore model behavior visually — great for AI/ML coursework and demos |
| Developer | Debug prompts, compare models, profile latency, inspect tool calls |
| Researcher | Run controlled experiments: same prompt, different settings, side-by-side results |
Navigating the App
LLMxRay has a sidebar with one item per major feature. This guide has one chapter per item, in the exact order they appear in the sidebar:
- Chat Diagnostics — The main chat interface with streaming and session analysis
- Compare — Side-by-side model and settings comparison
- Language Compare — Understanding the token tax across languages
- Embeddings — Text embedding visualization and similarity
- Knowledge Base — Document upload, chunking, and RAG search
- Tool Workshop — Visual tool builder with code sync
- Fill-in-the-Middle (FIM) — Code completion playground for FIM-capable models
- Live — Point your other tools at LLMxRay instead of Ollama and see every turn they send, and where their prompts break the cache
- Cache Lab — Measure why your prompt misses the KV cache, and what that costs every turn
- Decision Lab — Ask a decision model typed questions and see the probability behind every answer
- Protocol Observatory — Run the same prompt through native, OpenAI-compat, and Anthropic-compat against your local model
- AI Training — Training data curation and export
- Models — Model browser and capability detection
- Benchmark — Standardized model evaluation with logprobs
- My System — Hardware detection and Ollama status
- Settings — Connection, parameters, and preferences
Other Elements
- Header bar — Shows the current page title, a theme toggle (dark/light), and Ollama connection status (green = connected, red = disconnected).
- Feedback button — At the bottom of the sidebar. Opens an overlay to submit feedback directly.
Next Steps
If you haven't installed LLMxRay yet, head to the Installation page. Otherwise, pick any chapter from the sidebar to start exploring.