Putting Qwen3.8-27B to Work on a DGX Spark

DGX Spark · GB10 · 128 GB unified memory (121 GiB OS-visible) · aarch64 · CUDA 13.0 · August 27, 2026

I wanted to know whether a 27-billion-parameter local coding model on a DGX Spark was actually useful in a real software-development workflow. Not whether it could produce an impressive benchmark number, but whether I could hand it work from a real backlog, verify the result mechanically, and decide what work should move off expensive frontier models and onto local compute.

I started by standing up Qwen3.8-27B with SGLang and DFlash2 speculative decoding, then measured it against Docker Model Runner's llama.cpp path on the same machine. After that I gave the model progressively harder work from my own codebase, with acceptance tests written before the model ran.

Some of the published claims held up very well. Some installation and configuration guidance did not survive contact with the current software stack. And the most useful result was not a tokens-per-second number at all: it was learning where this local model belongs in my development pipeline - and where it does not.

That distinction matters because high local inference throughput can create the impression that a model fitting comfortably inside a sub-100-GB working footprint is approaching frontier-model utility. My tests do not support that conclusion. The tokens per second are real. So is the useful work. But throughput and utility are different measurements: as the tasks became harder, limitations in reasoning, output volume, edit strategy, and harness behavior became visible long before raw decode speed became a problem.

1. DFlash2 works - and my coding workload is a particularly good case

The basic performance claim reproduced. I ran the same generation settings - max_tokens=3072, temperature zero, thinking disabled - on three prompt shapes chosen to span how predictable the next tokens are: a coding completion, a step-by-step math derivation, and a stretch of ordinary prose. Speculative decoding should help most on the structured end of that range.

On the coding prompt, Qwen3.8-27B reached 75.7 output tokens/second through SGLang/DFlash2 versus 12.5 through llama.cpp with a GGUF: a 6.1× gain.

The gain tracks predictability: code 6.1×, math 5.0×, prose 1.5×. That is exactly what I would expect from speculative decoding. A draft model does best when it can correctly predict blocks of upcoming tokens, and code is highly structured.

I also removed speculation while holding the SGLang path constant. Throughput fell to 12.6 tok/s - essentially llama.cpp's 12.5. In this configuration, essentially all of the measured throughput advantage came from DFlash2 speculative decoding, not from switching inference servers by itself.

llama.cpp GGUF Q4_K_M              12.5 tok/s   baseline
+ SGLang kernels + NVFP4 + graphs  12.6 tok/s     +1%
+ DFlash2 speculative decoding     75.7 tok/s   +501%

The recipe advertised a 3.43× gain. I measured 6.1× on code. I do not interpret that as disproving the published number; my prompts appear to sit near the favorable end of the speculative-decoding workload distribution. More importantly, coding is the workload I actually care about.

2. The current recipe contains stale assumptions

The runtime stack is a derivative of lmsysorg/sglang:qwen38-27b, with the 20 GB sakamakismile/Qwen3.8-27B-MTP-NVFP4 target and the 3.6 GB z-lab/Qwen3.8-27B-DFlash2 draft pinned at 50307d4c. The recipe overlays five DFlash2 Python modules.

The stated reason for that overlay is now stale. All five modules already exist in the August base image because DFLASH merged upstream earlier. But the overlay is still necessary: the base image knows the earlier DFlash architecture, not DFlash2DraftModel, and lacks the layer-ID mapping between the draft model's Hugging Face layer-output IDs and the target's layer-input capture. Without that mapping the failure is silent misalignment rather than a useful startup error.

A future image can be checked directly:

docker run --rm --entrypoint sh lmsysorg/sglang:qwen38-27b \
  -c 'grep -c DFlash2DraftModel \
      /sgl-workspace/sglang/python/sglang/srt/models/dflash.py'

3. On a unified-memory Spark, the crash threshold is not the operating point

The DGX Spark is specified with 128 GB of unified memory; Linux reports roughly 121 GiB usable to the operating system. Even that OS-visible number should not be read as memory available to a model in a real installation. This Spark also runs PostgreSQL, coordination services, containers, and normal system processes, all drawing from the same unified pool as the GPU workload.

The recipe discusses mem-fraction-static values around 0.90 and warns that 0.95 can hard-reboot the machine during CUDA-graph capture. On GB10 there is no separate VRAM pool: SGLang is competing for the same unified memory used by the operating system and everything else on the Spark.

For my machine, 0.75 is the sensible setting. I could not measure a throughput or quality penalty, and it returned roughly 12 GiB compared with 0.90. The remaining 1.1 million KV tokens are far beyond what my current coding workload can consume. The published crash boundary is useful information; it is not the same thing as a safe operating point for a Spark doing other work at the same time.

4. A fast model endpoint is not automatically a useful coding worker

Several integration details mattered more than I expected. Without --tool-call-parser qwen3_coder, the model emits function markup as ordinary text and an agentic client sees no tool call. The commonly suggested /no_think suffix did not reliably disable reasoning in my tests; passing {"chat_template_kwargs":{"enable_thinking":false}} did. That matters because reasoning consumed 57–66% of generated tokens in the tests where it remained enabled.

The Qwen Code CLI also exits with status zero when its model endpoint is unavailable. A bridge that trusts process exit status can therefore accept an API error string as completed work. I now preflight the endpoint and inspect output rather than treating zero as proof that a model answered.

Another startup failure came from the draft model's missing refs/main. The recipe downloads into a pinned snapshot directory, but SGLang's speculative-alias path later resolves configuration without the revision. In offline mode there is no fallback. Writing the pinned SHA into refs/main fixes it.

5. Then I stopped benchmarking tokens and gave it real work

The first backlog task was a small production-hardening change: refuse work from a watcher whose lock file is still on disk after the process has died. Fourteen acceptance tests were written and committed before any agent ran. The gate was verified failing for the expected reason. Each arm ran from the same base commit in an isolated worktree.

All three passed on the first attempt. The same Qwen model took 889 seconds through the agentic CLI and 22 seconds when given the specification plus the current file and asked to return the replacement file directly - about a 40× difference on this task.

That is not evidence that agent harnesses generally cost 40×. It is one task, deliberately favorable to a direct completion: a precise specification, one file, and a deterministic gate. What it did establish was that my harness could dominate the cost of this class of work. The agentic run made 43 model turns, opened with roughly 22,000 tokens of context, and grew beyond 45,000 while editing a file it ultimately got right on its first pass.

That changed the question. Instead of asking, “Can Qwen act as an autonomous coding agent?” I started asking, “What is the cheapest harness that reliably completes each shape of work?”

6. Multi-file work did not require an agent either

Round two was a genuinely multi-file task: implement a policy loader across three files and preserve a design contract that the specification stated but the code did not enforce. Twenty-one tests were preregistered.

The discovery penalty was only 14 seconds. Given the whole package with no indication of which files mattered, the model selected the correct three itself. Both versions also preserved the project's existing 116-test suite.

7. Two “model failures” were actually failures in my experiment

My first round-two result looked much worse: 10 tests passed, 11 failed, with exactly the same plateau across three attempts. Both causes were mine.

First, my file extractor used a non-greedy fenced-block regex. The task itself involved parsing fenced YAML, so a correct implementation naturally contained a fence inside its own source. My extractor truncated the returned file at that inner fence and produced a half-file that still happened to parse as Python.

Second, one preregistered test passed /dev/null as a findings path. The production code correctly checks is_file(); a character device is not a regular file. That made the intended revise path unreachable and corrupted the meaning of two test outcomes.

The lesson was more useful than the false failure: when repeated model attempts plateau on exactly the same failures, a broken gate should become the null hypothesis. An automated development system needs to distrust its evaluator as well as its worker.

8. The first real ceiling was output volume

Round three moved to a ~1,000-line orchestration module and a security-relevant path guard over git status output: quoted and escaped paths, renames checked on both sides, and path traversal. This required understanding and correcting subtle existing logic rather than adding a small isolated feature.

Whole-file output collapsed. Two rounds degraded into prose without usable edit blocks; two produced syntax errors. The same model had held its output protocol cleanly across roughly 250 lines and three files in round two. Surgical edit verbs plus AST splicing made the task tractable immediately.

That suggests the practical limit I hit was not “multi-file work” or even task complexity by itself. It was asking this model to emit too much existing source in one response.

9. Iteration did not rescue the hard case

The AST-spliced version scored 17/19 on its first attempt. Later attempts got worse: one dropped to 13/19 and two produced syntax errors. The two persistent failures were also the subtle cases - decoding consecutive octal escapes as the bytes of a single UTF-8 character, and normalizing .. before the allowlist check.

This matters because I had built a draft/review/revise loop specifically to trade free local tokens for wall-clock time. Across these three rounds it never improved the best result, and in round three it actively degraded it. For this model and these tasks, “one shot, run the gate, keep the best result” currently has more evidence behind it than repeated autonomous revision.

That is still a provisional conclusion. The experiment has not yet tested ambiguous requirements, broad exploratory work, or tasks without deterministic gates - exactly the places where an agentic loop may earn its overhead.

10. The routing policy I am actually using

Three rounds are not a model benchmark. They are enough, however, to establish a useful local operating policy for the work shapes actually tested.

The point is not that Qwen3.8-27B replaces Grok, Claude, Codex, or an agentic coding environment. The point is that I now have evidence that a meaningful subset of my backlog does not need them.

That is also the main qualification I would put beside the impressive throughput numbers. A local model can match the mechanics of some frontier-model work without matching the full utility envelope of a frontier model. In these experiments Qwen was excellent when the problem was bounded, the context could be controlled, and correctness was mechanically testable. Its limits appeared as the work demanded more indirection, larger outputs, or less constrained reasoning. Fast inference makes the model useful; it does not erase the capability difference.

Local inference is free at the margin once the hardware is sitting here. A 20–60 second bounded implementation pass, followed by a deterministic test gate, is valuable even if harder work still escalates. The economic win comes from routing, not from pretending one model or one harness should do everything.

11. The experiment changed how I think about autonomous coding

I began this work expecting the interesting question to be whether DFlash2 could make a dense 27B model fast enough to use interactively on the Spark. It can. On predictable coding output, the measured acceleration is substantial.

But once inference became cheap and fast, the bottleneck moved upward. Context construction, number of model turns, output protocol, edit strategy, gate correctness, and task routing started dominating the result.

That is also why I care about evaluating the entire autonomous loop rather than only the model. The worker proposes code; the build and test system measures it. When the measurement apparatus is wrong, more autonomous iteration can simply produce the wrong conclusion faster. When the harness is wasteful, a faster model mostly burns through an inefficient loop more quickly.

The useful unit is completed, validated work - not tokens generated and not agent turns.

What I would test next

The next round is a pure refactor: split that ~1,000-line module without changing behavior, using the existing 116-test suite as the acceptance criterion. That tests whether the model can choose sensible module seams and execute a higher-blast-radius many-file change without the crutch of new feature logic.

Two other boundaries remain open: an ambiguous specification, where the correct behavior is to ask rather than guess, and a task without a deterministic gate, where review quality and human verification cost become part of the result.

Todd Espy · August 27, 2026

Generated output stats available on request.