Addis PulseStudio

Ollama v0.40.4: Decision Models in the Terminal and a Multimodal-Embeddings Path

A patch release that turns a shell command into a QC gate, and opens a local vector-embedding endpoint for the ComfyUI pipeline.

4 min read802 words

What happened

Ollama shipped v0.40.4 on October 14, 2026, adding CLI-level decision-model support β€” yes/no questions, text classification, rubric scoring β€” and a new multimodal-embeddings endpoint through llama-server. The patch also includes MLX-runner reliability fixes, a usage-credits billing mode for ollama launch, and a Codex-CLI integration that surfaces local Ollama models.

Context

Ollama has been the default local-inference wrapper for small studios: pull a model, run it, stream tokens back. v0.40.3 was the immediate predecessor. The decision-model primitives did not exist in Ollama's surface before this update. Classification and scoring tasks required either prompting a general LLM and parsing its prose, or standing up a separate classification service. The llama.cpp backend received a corresponding update with native decision-model support, so this is a new inference path in the C++ layer, not a prompt-template wrapper on top of the same model.

How it works

The decision-model path is invoked from the terminal: ollama run nimble --ask "Is this frame acceptable?" returns a structured yes/no. The same interface handles classification into user-defined labels and scoring against a rubric. Under the hood, the llama.cpp update adds native decision-model primitives, so the model emits a constrained answer rather than free text you parse.

On the MLX runner (Apple silicon), decision models are supported, and three reliability fixes landed: long Laya states are clipped instead of rejected, speculative-decoding rollback now handles recurrent state correctly, and Gemma 4 prefill was improved on NVIDIA GPUs β€” a pairing the release notes state without further clarification, since MLX is nominally an Apple-silicon backend.

Multimodal embeddings are now served through llama-server, with a fix for models that have small context windows. Cold-start overhead was reduced by reusing verified manifests and keeping already-loaded models resident in memory.

Our read

The headline reads as a feature addition, but the structural move is what matters. Before v0.40.4, gating a ComfyUI render β€” "does this frame pass QC?", "is the prompt adherent?" β€” meant prompting a general LLM and parsing its prose, or standing up a classification microservice. ollama run --ask collapses that into a shell command that returns a structured answer. For a studio whose pipeline is already mostly shell script and ComfyUI nodes, the difference between a Python client and a one-liner in a cron job is the difference between a tool you maintain and a tool you forget about.

The model name in the release example β€” "nimble" β€” signals something specific. It is not a 70B general model being coaxed into classification. The native llama.cpp primitives mean the output is constrained at the inference layer. A QC gate that runs in a few hundred milliseconds on a few gigabytes of VRAM is a different tool from one that needs 24 GB and four seconds.

What the release notes leave open: which model families support the decision path, what parameter counts are available, and whether classification labels are defined at inference time or baked in at training. Those three answers determine whether this is a production gate or a demo.

What this changes

If your build box is Apple silicon, the MLX fixes are direct: Laya-state clipping removes a class of long-context crashes, and the speculative-decoding rollback fix stabilises recurrent models. On NVIDIA, those specific fixes do not apply, but manifest-reuse and model-caching changes reduce cold-start time on any GPU.

The decision-model CLI is the actionable item. A ComfyUI pipeline can call ollama run nimble --ask from a shell wrapper to classify frames or score prompt adherence before queuing an export. No API client, no token-bucket management. The multimodal-embeddings llama-server endpoint opens a path to embed thumbnails into a local vector index for asset retrieval.

ollama launch now accepts usage credits without a paid-plan upgrade, and ollama launch codex surfaces local Ollama models in its picker. Neither changes a fully local ComfyUI workflow. The value is in the shell-script glue around the pipeline.

License

The sources for this release do not state a licence for Ollama v0.40.4 or the bundled llama.cpp update. Before building a commercial pipeline on the decision-model or multimodal-embeddings features, check the repository licence file and the relevant model card.

Key takeaways

  • Decision-model support (yes/no, classification, rubric scoring) is now a terminal command in Ollama, backed by native llama.cpp primitives rather than prompt engineering.
  • Multimodal embeddings are served through llama-server, with a context-window sizing fix for smaller models.
  • The MLX runner received Laya-state clipping, speculative-decoding rollback, and Gemma 4 prefill fixes, relevant to Apple-silicon build boxes.
  • ollama launch now accepts usage credits without a paid-plan upgrade, and ollama launch codex surfaces local Ollama models in its model picker.
  • The release notes do not specify which model families support the decision path, minimum VRAM requirements, or how classification labels are defined.

Sources

  1. v0.40.4 β€” tier 2
ollamacomfyuimultimodallocal aiautomation

How this post was made

Drafted from clustered primary sources by the models below, then read, edited and approved by a human before it was published. The sources are listed in full at the end of the article.

Drafted
Independent sources
1
cluster pair
clef:27b
cluster label
gemma4:12b
radar brief
gemma4:12b
research brief
qwen3.8:27b
draft article
qwen3.8:27b
short script
qwen3.8:27b
seo pack
gemma4:12b
Run
editorial-20261014T112927Z