A free public endpoint for google/diffusiongemma-26B-A4B-it running vLLM with structured reads (PR #57250): typed yes/no, choice and score questions answered in one denoise step, each with a probability. No text generated, nothing to parse. Plain OpenAI-compatible chat is on the same URL.
You send a state (a support ticket, a forum post, a photo) and a set of
questions. Each question has a type: noul (yes/no), choice
(pick one of up to 26 options) or score (an ordered scale). The endpoint
returns a probability for every answer, plus a confidence, in a few hundred milliseconds.
The wire format is the Jev "System One" API, so tools written for it work unchanged.
The trick is a discrete diffusion model: instead of generating tokens one by one, it denoises a whole canvas per forward pass. Seed that canvas with your answer template, leave one slot per question as noise, run a single step, and the logits at each slot are the answer distribution. The How a Read Works window has the details.
Shared and free, with limits sized for a decision API rather than a chatbot: about
120 requests in a burst, refilling to roughly 300 requests/minute, and
16 in flight, per IP. Firing decisions in parallel from an app is the point β
just don't run a load test against it. Over the line you get a 429 with
Retry-After; keep it up and the source is paused for 5 minutes.
No token, no signup. POST /v1/systemone with a state and your questions.
Option names and levels can be anything; the server maps them to single-token labels
for the model and maps them back in the answer.
curl https://6ab255d535c41fcea4a331db.endpoints.huggingface.cloud/v1/systemone \
-H 'Content-Type: application/json' \
-d '{
"model": "jev-latest",
"state": {"ticket": "Everything is down and we have a demo with our biggest client at noon."},
"questions": {
"urgent": {"type": "noul", "instructions": "Does the customer need a reply within the hour?"},
"team": {"type": "choice", "instructions": "Which team should handle it?",
"criteria": {"outage": "service down", "billing": "charges, refunds", "feature": "requests, how-to"}},
"tone": {"type": "score", "instructions": "How upset is the customer?",
"criteria": ["calm", "annoyed", "furious"]}
}
}'
import requests
ENDPOINT = "https://6ab255d535c41fcea4a331db.endpoints.huggingface.cloud"
def decide(state, questions, **extensions):
r = requests.post(f"{ENDPOINT}/v1/systemone", json={
"model": "jev-latest", "state": state, "questions": questions, **extensions})
r.raise_for_status()
return r.json()["answers"]
answers = decide(
"I was charged twice this month, please refund the duplicate.",
{
"urgent": {"type": "noul", "instructions": "Does the customer need a reply within the hour?"},
"team": {"type": "choice", "instructions": "Which team should handle it?",
"criteria": {"outage": "service down", "billing": "charges, refunds", "feature": "requests, how-to"}},
"tone": {"type": "score", "instructions": "How upset is the customer?",
"criteria": ["calm", "annoyed", "furious"]},
},
)
print(answers["urgent"]["noul"]) # P(yes), e.g. 0.02
print(answers["team"]["choice"], # "billing"
answers["team"]["confidence"]) # 1 - H(p)/ln K
print(answers["tone"]["score"], # expected level, 0-indexed
answers["tone"]["legend"]) # ["calm", "annoyed", "furious"]
Questions can be about pictures: pass images as data URLs (up to 8) and
the vision tower reads them ahead of the state.
import base64, requests
img = "data:image/jpeg;base64," + base64.b64encode(open("photo.jpg", "rb").read()).decode()
r = requests.post("https://6ab255d535c41fcea4a331db.endpoints.huggingface.cloud/v1/systemone", json={
"model": "jev-latest",
"state": "Look at the photo.",
"images": [img], # up to 8, ~280 input tokens each
"questions": {
"hotdog": {"type": "noul", "instructions": "The photo shows a hot dog."},
"setting": {"type": "choice", "instructions": "Where was it taken?",
"criteria": {"indoors": None, "outdoors": None, "studio": None}},
},
})
print(r.json()["answers"])
The same URL is an OpenAI-compatible server for the model. Any OpenAI SDK works with
base_url set to https://6ab255d535c41fcea4a331db.endpoints.huggingface.cloud/v1 and any API key. Thinking is off by
default; pass "chat_template_kwargs": {"enable_thinking": true} to turn it on.
curl https://6ab255d535c41fcea4a331db.endpoints.huggingface.cloud/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "google/diffusiongemma-26B-A4B-it",
"messages": [{"role": "user", "content": "Explain discrete diffusion for text in two sentences."}],
"chat_template_kwargs": {"enable_thinking": false},
"max_tokens": 256
}'
Generation on the shared box defaults to max_tokens 1024
and is capped at 4096; the context window served is 32,768 tokens. Streaming and tool calling
work; with thinking on, the thought comes back in message.reasoning. The diffusion
sampler has its own schedule, so temperature, seed, min_p,
logit_bias, min_tokens, bad_words and
allowed_token_ids are dropped by the proxy instead of returning the engine's 400. That
includes seed on raw reads: reproducible slot noise comes from your own seed canvas.
The endpoint serves its own playground: paste a request, attach an image or a webcam frame, read the answers. There is also Walk, a phone page that streams the back camera and reads one hazard label per frame.
Both pages are the example files shipped in vLLM PR #57250, served unmodified from the endpoint itself. Nothing is stored.
The prompt lists the questions and their allowed labels. The answer template
(urgent: @, team: @, tone: @, one line each)
is written into the diffusion canvas with the @ slots left as noise.
The engine runs one denoise step at temperature 1 and returns the logprobs at
every canvas position. The distribution over the label tokens at each slot, renormalized over that question's
labels, is the answer; confidence is the top label's averaged probability. By
default the server re-reads with fresh noise only when the first read is uncertain and
averages. These are the model's conditional label probabilities, not validated calibration.
| Type | Request | Answer |
|---|---|---|
noul | yes/no; optional criteria: {"true": β¦, "false": β¦} | {"noul": P(yes)} |
choice | criteria: {name: description}, 2β26 options | {"choice", "probabilities", "confidence"} |
score | criteria: [level0, level1, β¦], 2β26 ordered levels | {"score": Ξ£ iΒ·pα΅’, "legend", "probabilities", "confidence"} |
| Field | Values | What it does |
|---|---|---|
images | up to 8 data URLs | pictures the questions are about, placed ahead of the state |
samples | 1β8 or "auto" | read N times with different noise and average (auto: up to 4 re-reads only when uncertain); the shared endpoint caps N at 8 |
steps | 1β8 | denoise steps per read; more lets the answers settle against each other |
think | 0β2048 tokens (shared-endpoint cap) | the model writes a thought first, then reads with the thought in its prompt |
depends_on, ask_if | question ids | stage questions on earlier answers, or skip them |
The decision server is a thin layer over four new per-request fields in vLLM.
You can call them directly on /v1/chat/completions:
{
"model": "google/diffusiongemma-26B-A4B-it",
"messages": [{"role": "system", "content": "...question list, one label per question..."},
{"role": "user", "content": "{\"ticket\": \"...\"}"}],
"chat_template_kwargs": {"enable_thinking": false},
"max_tokens": 17, "logprobs": true, "top_logprobs": 20,
"logprob_token_ids": [11262, 951, 562, 603, 565],
"return_tokens_as_token_ids": true,
"vllm_xargs": {
"diffusion_seed_canvas": [100, 45518, 107, 101, 148436, 236787, 58369, 107, "... 32 ids"],
"diffusion_canvas_length": 32,
"diffusion_max_steps": 1,
"diffusion_read_only": true
}
}
diffusion_seed_canvas replaces the random canvas,
diffusion_read_only emits the argmax canvas on the converging step instead of
committing it, diffusion_max_steps caps the steps, and
diffusion_pinned holds template positions across steps. Every label must be a
single token so the canvas never shifts. Details and the example server: vllm#57250.
pi reads
custom model providers from ~/.pi/agent/models.json. Add a provider entry pointing
at this endpoint's OpenAI-compatible base URL, then select it from the CLI. Tool calls are
parsed server-side (Gemma 4 format), so pi's read/edit/bash tools work.
{
"providers": {
"hf-dgemma": {
"name": "DiffusionGemma 26B-A4B (HF public)",
"baseUrl": "https://6ab255d535c41fcea4a331db.endpoints.huggingface.cloud/v1",
"api": "openai-completions",
"apiKey": "not-needed",
"compat": {
"supportsReasoningEffort": false,
"maxTokensField": "max_tokens"
},
"models": [{
"id": "google/diffusiongemma-26B-A4B-it",
"name": "DiffusionGemma 26B-A4B",
"reasoning": false,
"input": ["text", "image"],
"contextWindow": 32768,
"maxTokens": 4096,
"cost": {"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0}
}]
}
}
}
Then run it β no restart needed, pi picks up models.json on launch:
# interactive pi --provider hf-dgemma --model google/diffusiongemma-26B-A4B-it --thinking off # one-shot, non-interactive pi -p --no-session --provider hf-dgemma \ --model google/diffusiongemma-26B-A4B-it --thinking off \ "Summarize this repo's README." # add it to your Ctrl+P model cycle alongside others pi --models "hf-dgemma/*,sonnet,haiku"
Verified with pi 0.85: a one-shot answer comes back in about a second.
Keep maxTokens under the 4096 generation cap; the model's own thinking mode is
off here and a diffusion sampler answers fast enough without it.
| Item | Value |
|---|---|
| Model | google/diffusiongemma-26B-A4B-it Β· Apache-2.0 Β· BF16 (unquantized) |
| Architecture | Discrete text diffusion on a Gemma 4 backbone Β· 26B MoE, 128 experts, ~4B active Β· causal encoder for the prompt, bidirectional decoder over a 256-token canvas Β· vision tower |
| Canvas served | 64 positions (a request may use fewer); up to 48 denoising steps for generation, 1 for a read Β· 32 concurrent sequences |
| Context window | 32,768 tokens served (model supports 262,144) |
| Modalities | Text + images in (max 8/request), text or decisions out |
| Hardware | 1Γ NVIDIA A100 (80 GB), single replica (launched on an H200 on 2026-09-20; moved to an A100 in us-east-1 on 2026-09-22 since usage is light) |
| Engine | vLLM nightly 3df4ae15 (main, 2026-09-21) + PR #57250 at 407326b7 + #57589 Β· Triton attention Β· prefix caching Β· async scheduling Β· gemma4 reasoning and tool parsers |
| Measured | A100, 2026-09-22: ~145 ms per decision (warm, single read, measured from Paris) Β· the 3-question ticket in ~0.23 s at the default policy Β· 64 concurrent: ~84 req/s, ~250 decisions/s, p50 0.6 s Β· generation ~215 tok/s single stream (0.45 s to first token), ~90 tok/s per stream at 8 concurrent Β· first read after a restart ~10 s (kernel compile). Launch-day H200 figures: ~230 ms per decision, ~130 decisions/s at 64 concurrent, ~530 tok/s single stream. |
| Rate limit | ~300 requests/min per IP (burst 120, 16 in flight) Β· 429 + Retry-After when exceeded Β· per request: β€24 questions, β€8 images, β€8 samples |
Measured numbers are from this deployment's own verification battery on launch day, not marketing. The PR is not merged yet: the container overlays its nine Python files onto a pinned nightly image at boot, so what runs here is exactly the PR as of its head commit.
This whole thing β H200, a patched vLLM, the decision server, autoscaling, rate limiting β is one deploy form on Inference Endpoints: dedicated, production-grade deployments of any model on the Hub.
Pick a model, pick your hardware (CPU to multi-GPU H200), bring any container, and get an OpenAI-compatible URL with autoscaling, scale-to-zero, and per-minute billing. No shared queues, no rate limits from strangers β your model, your GPUs.