← Reference

Reference: Tested models

These results were recorded for the CUST/OS beta in April 2026. The tables distinguish combinations tested on hardware from those with adapter support but no recorded test. A result applies to the device, backend, and flow listed.

Status legend

Status Meaning
VERIFIED Full user flow run end-to-end on the reference device. A skill or live chat consumed this provider + model combo and returned correct results.
KNOWN_BROKEN Tried and failed in a specific, reproducible way. Workaround noted.
CODE_READY Adapter is included in the APK; no verification run has been recorded.
NOT_TESTED Pre-configured in example custos.yaml, no evidence it's been exercised.

Reference device: Samsung Galaxy S26 Ultra (Adreno 840 GPU, 16GB RAM). Other devices may behave differently — especially on GPU-backend LiteRT-LM, where precision and memory headroom vary.

On-device LLMs

Provider Model Backend Status Notes
litert gemma-4-E2B-it LiteRT GPU (FP32) VERIFIED Requires the custom litertlm-android-0.11.0-custos.aar — enables the FP32 activation patch so coordinate-precision survives GPU rounding. 16K context OK on 16GB devices.
litert gemma-4-E4B-it LiteRT GPU (FP32) KNOWN_BROKEN OOM at 16K context on the reference device. Drop context to 4–8K or use E2B.
litert gemma-4-E4B-it LiteRT CPU CODE_READY CPU avoids GPU precision concerns but is noticeably slower; tool calling not recently verified.
openai (file://) gemma-3-4b-it-q4_k_m.gguf llama.cpp (GGUF) CODE_READY Bundled in download-models.sh as an on-device fallback. Adapter supports tool-call + Hermes fallback; not recently re-verified on release APK.
openai (file://) Other GGUF LLMs (Qwen, Llama, Mistral, etc.) llama.cpp (GGUF) CODE_READY Requires support in the bundled llama-server and a matching tool-calling Jinja template set through chatTemplatePath. Tool calls need model-specific testing.

Known-broken tool-call mode: stock LiteRT-LM 0.10.0 with automaticToolCalling = true does not recover the numeric precision lost at GPU inference. The FP32 patch in the 0.11.0-custos AAR is the fix, and it's orthogonal to auto/manual tool calling. Our adapter keeps manual mode on so tool calls surface in the chat UI.

On-device embeddings / STT / Vision

Provider Model Runtime Status Notes
openai (file://) nomic-embed-text-v1.5.Q8_0.gguf llama.cpp NOT_TESTED Wired in example custos.yaml; no live-run evidence captured. Skill selector falls back to BM25 when embeddings are cold, so this is non-blocking.
openai (file://) all-MiniLM-L6-v2.Q8_0.gguf llama.cpp NOT_TESTED Alternative lightweight embedder.
openai (file://) ggml-tiny.bin (Whisper) whisper.cpp NOT_TESTED Transcription for PTT. Wired but no user flow captured.
vision yolo11m.onnx ONNX Runtime CODE_READY Tile-capture end-to-end verified by custos.detect_buildings; YOLO output correctness not independently checked.

Cloud LLMs

Provider Model Adapter Status Notes
openai grok-4-1-fast-reasoning (xAI) OpenAiCompatibleAdapter CODE_READY Configured in example custos.yaml; no user flow captured.
ollama user-configurable OllamaAdapter CODE_READY Tool-call support with JSON + Hermes fallback. Example config in custos.yaml commented-out.
vllm user-configurable VllmAdapter CODE_READY Dedicated adapter — handles vLLM's reasoning_content field and <think> tags so reasoning is stripped cleanly. Use protocol: vllm for vLLM servers; use protocol: openai for everything else OpenAI-compatible.
anthropic user-configurable AnthropicAdapter CODE_READY Messages API. Commented-out in example custos.yaml.

Adding your own results

When you verify a new combo end-to-end:

  1. Note the exact device, model path, adapter, backend settings.
  2. Move the row from CODE_READY / NOT_TESTED to VERIFIED with a one-line note on what flow it ran.
  3. Open a PR with the result and enough detail to repeat the test.

For a reproducible failure, use KNOWN_BROKEN and describe the failure and any tested workaround.

See also