Nightshift job modes: what each mode is good for, and what it was not doing

The short version

Nightshift has many ways to run a job besides "race four attempts and pick one". This report measures every one of them on a DGX Spark. The controlled fleet was three non-thinking racers (Qwen2.5-Coder-14B, Qwen2.5-14B, Llama-3.1-8B) and a dedicated Muse-Glimmer-30B judge lane. The owner's own fleet, with its Qwen3.8-27B and no judge lane, was measured beside it. Ten fixtures with deterministic scorers were held constant, and every cell ran three times.

With a dedicated judge lane, the race modes work.

• The verifier panel scored 99 % of the candidates it shortlisted, and picked the best-scoring candidate 22 times in 26. On the four fixtures no model solved, it found the best partial answer 11 times in 11, against 1 in 10 for majority vote.
• Majority vote is three times faster, and right when most candidates are right.
• Bulk extracted 220 of 220 units correctly.
• Gate repair turned every seeded app with a missing click handler into a working app.

As shipped in 1.3.2, five modes did not do what they say on this box:

• Deliberate never gave a correct answer: it answered with its own progress notes, after up to nine chunks.
• The consistency judge never checked anything: every check timed out and was recorded as "consistent".
• A 45-second deadline ran up to five minutes.
• "Continue the night" was impossible on any fleet whose lanes hold 32k tokens.
• On the owner's own fleet the verifier panel judged nothing: 0 of 59 candidates scored, because it kept choosing a 3B "thinking" racer that never returns a verdict. This is the mechanism behind the published "8 % judge coverage".

All five are fixed, and re-measured on the installed fixes:

• deliberate answers the fact prompt correctly in 5 s;
• planted contradictions are flagged 3 of 3;
• the 45 s deadline finishes in 47–59 s;
• continue runs;
• the owner fleet's panel scored 17 of 20.

Each fix carries a regression test that fails on the build before it. In all, 35 defects were fixed: 15 that the Mac campaign had found and that also existed in this tree, and 20 new. 17 of them were confirmed live.

What is still weak:

• "Verified" means the generated app loads, not that it works: 12 of 12 apps verified, 6 of 12 working.
• Auto plans 256 attempts for a one-paragraph answer.
• The owner's thinking models spend most of a short job reasoning.
• This box's steady background chat traffic keeps it out of its night posture, and starved its watchers until 1.3.5.

Every number names its machine, build and source. Failures, refusals and wiped jobs stay in the denominators or are reported apart.

At a glance

1. The modes

A Nightshift job is durable work: it is queued, checkpointed, survives a restart, and ends with a record, signed receipts and an outcome. The first campaign measured one shape of job, best_of_n with its defaults. This report covers everything else the code offers. We re-derived the list below from the DGX source (js/engine/jobs.js and the engine modules, the 1.3.2 tree), not from product copy. It corrects the prompt's starting map in four places:

• best_of_n is itself a kind with many knobs.
• composition is minted by an import or the compose verb and cannot be submitted (the submit route says so).
• The selection-policy field is called selection, not selection_policy.
• The DGX edition has no tournament policy. The elimination tournament is a Mac-edition module (tournament.js); this build always spreads attempts across every lane. On 1.3.2 it accepted selection: "tournament" and silently ran full coverage (D-10, now refused with the reason).

The sync router modes (meterless/* chat through :8088) are not jobs and are out of scope.

What the DGX had already measured. Nothing, for most of this table. Across the 608 jobs in the store before this campaign (inventory-dgx-jobstore.md): no deliberate and no batch job had ever run; no job had used majority_vote or unit_test; there were 8 bulk jobs, 16 quality-mode jobs, 5 consistency-mode jobs, 43 creative-mode jobs, 14 deadline jobs and 14 Auto plans. None of them were scored against ground truth. The only prior measurement of these modes is the Mac campaign of 26–27 September (reports/MODES-BENCH-2026-10-01/mac-reference/), which this report cites beside the DGX numbers.

2. How we measured

The machine. One DGX Spark: NVIDIA GB10, 121 GB unified memory (the daemon admits lanes against 106 177 MB), Ubuntu 24.04.4 on aarch64. The installed build at the start was Nightshift 1.3.2 (all three debs). The fixes this campaign made were measured afterwards on 1.3.3, on the cells they touch (§2.4). Nothing on this box rotates: every fleet below stays fully resident, so memory waits were zero throughout (median 0 s, maximum 0 s, over 171 grid jobs). On the Mac, the memory plan decided more of a job's time than the mode did. Here, the time a job takes is the mode's time and the models' time.

Three fleets.

Why the controlled fleet is not the owner's. On the smoke run, a best-of-4 on one hard prompt took 716 s on the owner's racers. Qwen3.8-27B reasoned for 599 s, ran out of budget, then answered correctly in 48 s on its thinking-off retry. The 3B "VibeThinker" lane wrote 17 000 characters of reasoning twice and never answered. The product learns to turn thinking off for a lane, but only within one job, so every new job pays that burn again (T-1). Putting --reasoning-budget 0 on the lane does not stop it on this llama-server build (T-2). Mode comparisons on those racers would have measured reasoning budgets, not modes. So the controlled fleet uses three non-thinking racers from two model lines, and the owner's own fleet is measured separately, as it is.

The fixtures are the Mac campaign's eight, verbatim (code, fact and extract are themselves the best-of-n campaign's), plus two harder ones, because a 14B coder solves most of the Mac set:

• semver: Python compare_semver, the full SemVer 2.0.0 precedence rules, 19 cases (4 must raise). A naive integer-tuple compare scores 7/19.
• ledger: a bank statement to JSON with a computed closing balance, a debit count and a credit total; 6 exact fields. One arithmetic slip fails it.

Every scorer is deterministic: Python cases executed, JavaScript cases run in a VM, exact JSON fields, fixed fact keys, contradiction keys for the canon. Self-tests (harness/fixtures/selftest*.mjs) show a reference answer passing and a wrong one failing for each. Two scorer caveats:

• Python's fnmatch also passes wildcard 15/15.
• The v2 fact rule needs one of a fixed set of verbs ("combine", "integrate", "share", "a single pool"). Two winners this campaign scored 4/5 only because they said "access memory directly, without copying". By reading, they are correct, and §4 reports both counts.

Common settings. Every job used window: "now" (the default, quiet, waits for the night), auto_synthesize: "off" (end-of-night reports would bleed into the next job's timing; synthesis is measured on its own) and privacy: "local-only". For the grid only, jobQualityExecChecks was switched on, because the unit_test selector and [js] rubric checks need it. What the product does when it is off is a result of its own (§4.x). Repetitions were interleaved, three per cell: rep 1 of every cell, then rep 2, then rep 3.

The harness is reports/MODES-BENCH-2026-10-01/harness/, ported from the Mac campaign: mode-bench.mjs (the grid), extras.mjs, bulk.mjs, watch.mjs, resume.mjs, refusals.mjs and a shared lib.mjs. It submits through the public API (:8088), reads each job's record, events, attempts and makes back from /var/lib/meterless/jobs/<id>/, and writes one JSON record per job. Every 5 s it also samples the daemon's memory ledger, the host's MemAvailable and load, and GB10 power and utilisation. The protocols (PROTOCOLS.md) were written before any grid result was analysed.

Confounders we know about (all in RUNLOG.md, with times):

• An orphaned headless Chrome from an earlier session was rendering a generated WebGL app in software at about 4 CPU cores for the whole campaign. Stopping it was refused by this session's permissions, and the operator was told.
• The Nightshift cockpit stayed open (about one core).
• Two source-fix agents ran targeted test files, two full JS suites and four release builds overlapped parts of the grid.
• Posture: the jobs before 07:00 local ran in the Swarm posture (47 jobs); the rest ran in Depth (115), with fewer lanes at once.

Correctness columns are unaffected by any of these. Time columns carry medians with ranges, and we do not rank modes by speed within a few tens of seconds.

Infrastructure failures are kept apart, not hidden. On 1.3.2, nine grid jobs lost all their attempts in under a second, because the engine's model probe timed out under load and wiped its list of served models (D-11). They are excluded from the mode tables, re-run once each (runs/grid-1.3.2-rerun/), and reported as a defect with their job ids.

3. What was already known

Three sources existed before this campaign. None of them had measured these modes on the DGX against ground truth.

The Mac campaign's headline findings are a useful prior, and this report checks each one on the DGX:

• A dedicated judge lane makes the verifier panel work (Mac: 93 % of shortlisted candidates scored, a correct winner 13 of 13 times one existed). Checked in §4.1.
• Majority vote is fast and fails when most candidates share a mistake (Mac: 9 of 12). Checked in §4.1.
• More diversity hurts single-answer tasks (Mac: medium and high scored 0 of 6 winners). Checked in §4.5.
• The consistency judge over-flags (Mac: precision 0.75). On the DGX the opposite happened: it never flagged anything, because every check timed out (D-14; §4.4).
• Deliberate, batch and bulk dispatched to parked models. That was Mac-specific (rotation); on the DGX the same runners failed for different reasons (§4.6, §4.7).

4. Results

Every table here was computed from the raw records (reports/MODES-BENCH-2026-10-01/runs/, analysed by harness/analyze_grid.py, analyze_dgx.py and analyze_extras.py). The machine is the DGX Spark. The controlled fleet is the one in §2 unless a table says otherwise. Where a mode was measured on both 1.3.2 (as shipped) and a fixed build, both columns are shown.

4.1 Selectors: who picks the winner

Best-of-4 on the ten fixtures, three reps per cell, on 1.3.2. "Correct" means fully correct by the deterministic scorer. "Best available" means at least one candidate in the job was fully correct.

What the table says:

• The panel picks the best candidate far more often (22 of 26 against 16 of 27), and it is what separates the two when no candidate is perfect. On duration, jsduration, semver and ledger no candidate was fully correct. The panel picked the highest-scoring partial answer in 11 of 11 of those jobs; majority vote picked it in 1 of 10.
• On "is the winner fully correct" the two are level (11 and 12). Two of the panel's misses are a scorer artefact. Both fact winners it was "wrong" on fail only the v2 rule's verb keyword: they say the CPU and GPU "access memory directly, without copying" instead of "share a single pool". Read by a person, they are correct.
• Majority vote is 3× faster (33 s against 100 s median), because there is no judge seat. When most candidates share a mistake, the majority wins with it. On semver one job had a fully correct candidate and majority vote picked a 0.34 answer.
• unit_test never crowned a failing answer. It returned no winner 5 of 5 times, because no candidate passed every case. That is the honest answer. With jobQualityExecChecks off, 1.3.2 accepted the body, raced the whole night and only then refused (D-9, 147 s and 446 tokens for nothing). 1.3.3 refuses it at submit.
• Judge coverage, the published weak point, is not a property of the panel; it is a property of the fleet. With a dedicated judge lane, the panel scored 306 of 310 shortlisted candidates (99 %) across 89 panels on 1.3.2, with 15 abstentions and a median of 79 s per panel. On the owner's own fleet, which has no judge lane, it scored 0 of 59 (§4.11, D-17).

The 1.3.3 drift check (one rep of each fixture on the fixed build) matched 1.3.2 on 9 of 10 fixtures. On semver, the one job picked a 0.21 candidate over a 0.63 one.

4.2 Selection policy: no tournament in this edition

The Mac edition has an elimination tournament policy. This edition has none: js/engine/tournament.js does not exist, and attempts are always spread across every lane. On 1.3.2, selection: "tournament" was accepted and silently run as full coverage (rep 1 of each tour8- cell: same attempt spread, same result shape as full8-). On 1.3.3 it is refused with the reason (D-10).

Full coverage at n = 8 against the n = 4 baseline (1.3.2):

Doubling n bought one correct semver winner out of three, for 1.3× the wall time.

4.3 Quality judge mode: the rubric is the bar, for better and for worse

A frozen written rubric, deterministic checks, and an eligibility gate (Must band).

The two lessons are the Mac's, reproduced on another machine and another fleet:

• The quality mode refuses honestly when nothing meets the bar. It is as good as the rubric, and no better: a rubric that forgets a field lets every wrong answer through.
• stop_when_met saves half the attempts on an easy bar.

On 1.3.2 the quality default level stitched creative persona briefs into factual prompts (M-10). 1.3.3 keeps briefs for creative mode only. The quality cells above were unaffected in score (factual prompts were answered either way), but 1.3.3 no longer sends the briefs.

4.4 Consistency judge mode: from "never checks" to "catches what it is shown"

A five-passage canon about a fictional company. Memos are consistent, carry two planted contradictions, or carry one subtle changed term ("net 60 days" against the canon's "net 28 days"). Ground truth uses fixed contradiction keys on the winner.

On 1.3.2 the check never ran: every one of 9 consistency checks timed out and abstained. The check called the Muse judge on a flat 60 s clock with its reasoning on, while the verifier panel gives the same judge reasoning-off and 290 s. The abstention was recorded as verdict: "consistent" (D-14, D-15). The one subtle "flag" on 1.3.2 was not a real flag either. On 1.3.3 the judge runs and catches planted contradictions, but it was only ever shown the top 3 of 5 passages. A profile that restates all five facts pushed the payment-terms passage out of view (D-16).

Fail-closed behaviour is correct on every build. A missing canon_ref is refused at submit. On 1.3.2 an unindexed one ran the whole night and errored at the end, throwing away the selected winner (D-4); on 1.3.3 it is refused at submit with canon_ref_unretrievable.

4.5 Creative judge mode

Two open prompts (a four-line poem, a café name and pitch), n = 8, three reps on each build.

Creative mode does what it says: it keeps the variety, and it never collapses to one answer.

4.6 Diversity and attempt budget

On semver (the fixture this fleet finds hardest), n = 4. 1.3.2: three reps; 1.3.3: two.

• On 1.3.2, more diversity lowered the mean candidate score at every step (0.55 → 0.45 → 0.38), as on the Mac. The persona-briefs axis was the worst single axis on both builds.
• 1.3.3 removes persona briefs from non-creative nights (M-10), and the level effect weakened (0.70 → 0.63 → 0.59).
• The 2 / 2 correct winners for high and budget-max on 1.3.3 rest on 2 correct candidates out of 8 each, and do not repeat on 1.3.2. That is noise at this sample size, not a finding.
• The attempt budget showed no trend: no attempt hit the length limit at any level.

Leave diversity at its default for single-answer work.

4.7 Deliberate: from "answers with a progress note" to "the fastest correct answer here"

One Markovian chain per job, on five prompts, three reps per build.

On 1.3.2 deliberate never gave a correct answer, and for a reason unrelated to the models. After answering in chunk 1, a mid-size model does not write [TASK_COMPLETE]. It writes bare status blocks in the carryover format ("COMPLETED: … REMAINING: … DECISIONS: …") without the checkpoint marker, the chain keeps going, and the last chunk, a progress note, became the answer (D-13). A one-paragraph fact took 620 s and nine chunks to say nothing. On 1.3.3 a status-only chunk counts as no new work, a state that says nothing remains finishes the chain, and a status note is never the answer. The same prompts now finish in one chunk, correctly, in 5 s (fact) and 21 s (invoice): faster than any best-of-n here.

What stays a design limit, reported rather than changed:

• Deliberate has no selector. On semver (1.3.3 rep 1) the chain refined its code over nine chunks, scoring 8, 10, 13, 0, 15, 0, 17, 0 and 14 of 19 cases, and answered with chunk 9 (14/19) when chunk 7 had 17/19.
• The "long build task in bounded chunks" framing splits a short code answer across chunks.

Resume after a crash (M-2) and the stall stop (M-8) are fixed with engine tests. The live kill test could not run here: this session's permissions do not allow killing the engine host.

4.8 Batch

The ten fixtures as ten requests in one job, three reps. "Bad" inserts an over-long request at position 5.

Batch isolates a bad request correctly on both builds. On 1.3.2 the record hid the failure (M-4). On 1.3.3 a failing request is also retried once on another lane, and a lane that fails twice stops claiming (M-5, engine tests). Correctness per request is the fleet's, the same as a single attempt: about half the fixtures are beyond these 8–14B models.

4.9 Bulk

A corpus of 20 generated receipts with an answer key; one unit per receipt, judge off. 1.3.3.

Bulk is the most dependable mode on the DGX: 220 of 220 extraction units correct, a coverage ledger that adds up, a lossless pause. units_in_flight did not change throughput on units this small (each unit is about 3 s of model time). Model-free conversion needs no lane at all.

4.10 Windows, gates and stops

On 1.3.2 the deadline stopped generation but not the judge (M-11); the panel alone could take 209 s.

One property of this box shapes every window. An outside app (meterless-city-npc) chats through the local endpoint every few seconds: 3 937 turns during the campaign. The scheduler therefore saw "recent sync traffic" almost constantly. Quiet-window jobs waited, and most of the 1.3.3 runs ran in the Depth posture. That is how a real owner's box looks.

4.11 The owner's own fleet: better models, no judge

motion-q4 as the owner runs it: Qwen3.8-27B, Qwen2.5-14B, VibeThinker-3B and MiniCPM5-1B racers, a 37 GB video lane, no judge lane. Best-of-4 on six fixtures, three reps, on 1.3.2.

The owner's 27B solves what the controlled fleet cannot (ledger 3/3, a semver), at 2–5× the time, most of it reasoning. But the panel never judged on this fleet: 0 of 59 shortlisted candidates scored in 18 jobs. With no judge lane, the picker chose VibeThinker-3B, a thinking racer that sits in the lane named verifier and ignores the reasoning-off switch. All 18 panels came back unparseable, so every winner on the owner's fleet was the fallback pick (the largest cluster). This is the mechanism behind the public page's "8 % judge coverage" (D-17). 1.3.4 replaces an unusable judge inside the panel and strikes it for later panels. On 1.3.4 the same six fixtures, one rep each: the panel scored 17 of 20 shortlisted candidates (85 %). VibeThinker came back unparseable on the first panel and was struck; Qwen3.8-27B judged the other five (recorded as a racer-judge, median 21 s). Winners were correct on 5 of 6 fixtures, including semver and jsroman, which this fleet's 1.3.2 runs got wrong.

4.12 Makes: apps, gate repair, refine

App generation. An offline to-do app prompt, n = 4 × 3, 12 attempts (1.3.3):

The DGX has the runtime tier the Mac lacks: every attempt was loaded in the verification sandbox (sandbox_verify, tier runtime). But the tier checks that the page runs, not that it works: it clicks nothing. "Verified" here means "loads without an error", and half of the verified apps did not do the job.

Gate repair of seeded broken apps (the correct app with a defect planted, repair_rounds: 2):

On the Mac a missing handler passed the static gate and was never repaired (0 of 7). Here the repair loop fixed all 6. The two unrepaired syntax seeds were still marked verified.

Refine (artifact.refine.code on a seeded off-by-one bug): 6 of 6 fixed (5/7 → 7/7 cases), no regressions.

4.13 Synthesis and reports

On a creative night with 7 clusters (job_muq91i830d), every cluster report in the deterministic "cut" mode completed in under a second, citing the cluster's members. The night report completed in "cut" mode in 0.5 s, and in curator mode in 294 s on the judge seat. A curator synthesis could not be bound by hand on this fleet. Every local lane holds 32k per slot, the floor for a local curator is 100k, and the product refused with curator_window_too_small. That is correct, and named.

4.14 Rerun, continue and Methods

• Rerun: 9 reruns of three grid jobs. The winner's correctness was the same 9 / 9 times. The wording was never identical (median word overlap 0.91).
• Continue: on 1.3.3, continuing a night was impossible on this fleet. The plan picked the judge seat as curator (the documented fallback when no lane clears the floor), and the verb then refused that same lane curator_window_too_small (D-18). On 1.3.4: on 1.3.4 the verb was accepted (202) and bound the judge seat. The interlude ran the parent's 7 cluster synopses and its insight pack on that seat (34 minutes: the Muse judge is slow, and one at a time). The child night then raced 8 attempts in 4 minutes with the sealed insight pack folded in (tiers from 4 043 down to 159 tokens) and, creative like its parent, returned a wide spread and a shortlist of 6 (job_murw0n0i0h).
• Methods: a 24-attempt creative night was mined into a Method. A mined Method starts as a candidate and is never applied automatically; reuse is refused until it is promoted (409, by design). After promotion on 1.3.4: the Method was promoted to provisional (200), and reuse was then refused, 409 method_incompatible: "requires 3 racing lanes with distinct families; fleet modes-dgx provides 2". The Method was mined from nights that raced three model families, and this fleet has two (Qwen, Llama). That is a sound check, and it means reuse could not be measured on this fleet.

4.15 Auto

Auto, resolved exactly as the composer resolves it, then run as planned:

The best-of-4 baseline got the same three right in 49–114 s. Auto sizes the night by the clock, not the task: it plans "until morning" for a one-paragraph answer. That repeats the Mac's finding on a box 7× larger. The results are right, at 35–45× the time and 55–95× the tokens of a best-of-4.

4.16 Media autopilot

On the controlled fleet (no image model seated), a winner-mode image deliverable made two image makes. Both failed honestly ("no image model is available for a one-shot still — seat an image lane"). On 1.3.2 the families-only body was accepted and made nothing silently (M-13). On 1.3.3 it records outputs: ['image'], derived from media_autopilot.families. Image quality itself was not re-measured here (the job store holds 138 completed image makes from earlier DGX runs).

4.17 Watchers and schedules

A folder watcher (5-minute cadence, debounce 2) on an uploaded corpus, with controlled edits over 46 minutes, beside a 5-minute interval schedule:

The watcher defers observation while the scheduler sees recent sync traffic, and on this box that never stopped (§4.10). The deferral had no bound (D-19). The fix bounds it at one cadence interval, and ships in 1.3.5. On the Mac, without that traffic, the same test fired 3 of 3.

5. Across modes

The same tasks, different modes (controlled fleet; 1.3.2 unless marked):

How each mode compares with best-of-n (the first campaign's subject, ../NIGHTSHIFT_BENCHMARKS.md):

• On a short, checkable task, one deliberate chunk (fixed) or a majority vote are the cheapest correct answers here: 5–21 s and 9–18 s against 50–60 s.
• On a task where most candidates are wrong, only the panel finds the best partial answer (11 of 11 against 1 of 10), and only better models (the owner's 27B) produce a correct one.
• More attempts (n = 8), more diversity and Auto's n = 256 all cost more without moving the hard fixtures. The exception is one semver win in three at n = 8.

Where a comparison is not fair:

• The owner-fleet row uses different racers and has no judge.
• Auto's rows ran with heavy outside chat traffic.
• The deliberate rows on the two builds are different code.

The table names each.

6. Strengths

• A dedicated judge lane makes the verifier panel work on the DGX too. It scored 99 % of shortlisted candidates (306 of 310) and picked the best-scoring candidate 22 of 26 times. On the four fixtures nobody solved, it picked the best partial answer 11 of 11 times, against 1 of 10 for majority vote. This is the single most useful configuration choice on this box.
• Refusals are honest and early. On 1.3.3, 11 of 11 malformed or unavailable requests are refused at submit with a reason:
• a missing rubric or canon_ref;
• an unindexed canon;
• unit_test without exec checks;
• an unknown judge mode;
• diversity max;
• selection: "tournament";
• units_in_flight: 9;
• submitting app_stream or composition;
• a bad start_gate.
• Bulk is dependable: 220 of 220 units correct, 0 duplicates, a lossless pause, and model-free conversion at 6 700–8 300 files an hour.
• Repair works:
• Gate repair turned all 6 seeded missing-handler apps and 4 of 6 syntax-broken ones into working apps.
• Refine fixed 6 of 6 seeded code bugs without a regression.
• Stops save compute. stop_when_met: 2 and no_new_cluster_after: 2 both ended n = 8 jobs at 4–5 attempts, with correct winners.
• Batch isolates a bad request. The other requests landed in every run.
• Fixes land cleanly. 35 defects were fixed (15 of the Mac campaign's that were present in this tree, and 20 new), each with a regression test that fails on the unfixed build. The full JS suite went from 4 331 to 4 448 tests, all passing at each cut. 17 of the fixes were re-measured live (§7).

7. Weaknesses and open problems

Defects found in this campaign. "Fixed" means a regression test that fails on the previous build and passes on the fix, a green full suite, and a built and installed deb. "Live" means the fix was re-measured on the installed build. Full evidence, with file and line references: reports/MODES-BENCH-2026-10-01/DEFECTS.md.

Weaknesses that are not code defects:

• "Verified" means "runs", not "works". The DGX's sandbox tier loaded every generated app and marked all 12 verified; 6 of them did the job. It needs an interaction check.
• Thinking racers spend most of a short job reasoning (T-1). The owner's 27B reasoned 599 s on one prompt before its thinking-off retry answered in 48 s. The product learns to turn thinking off for a lane only within one job. --reasoning-budget 0 on the lane does not help on this llama-server build (T-2).
• The owner's fleet is at its memory ceiling. After the 1.3.3 restart its 1B router lane was refused admission by about 300 MB; on 1.3.4 the 37 GB video lane was. Its lanes' measured footprints have grown as they ran.
• Auto sizes by the clock. It plans n = 256 for a one-paragraph fact.
• Deliberate has no selector. It answers with its last real chunk, even when an earlier one was better.
• A rubric that omits a field passes every wrong answer.
• Steady outside chat traffic keeps this box out of its night posture, which delays quiet-window jobs and starved watchers (D-19).

8. What the numbers say

On a DGX Spark, as it ships in 1.3.2, the job modes split three ways:

1. Some work well: best-of-n with a dedicated judge lane, majority vote, bulk, batch isolation, quality mode with a careful rubric, creative mode, refine, and the stop conditions.
2. Some did not do what they say, and now do:
3. Deliberate answered with progress notes.
4. Consistency never checked.
5. Deadlines overran by minutes.
6. Continue was impossible on 32k fleets.
7. The owner's own panel judged nothing.
8. Some are honest but costly: Auto's 256-attempt plans, and the owner's thinking racers.

What to use, on this box:

What would change the picture:

• an interaction step in the sandbox's runtime tier;
• learned thinking-off across jobs;
• a fleet-level judge seat as the default on the owner's fleet (or D-17's re-pick, now shipped);
• a re-measure of the watcher on 1.3.5.

The gallery

The benchmark folder that ships with this report (Benchmarks/) holds a single-file 3D gallery (index.html) and the best jobs generated so far, each an unmodified Nightshift export (GET /v1/jobs/{id}/export), unzipped and renamed in JOBS/. Opened from disk, the gallery draws the jobs as a night sky: every star is a job, coloured by task; the moons circling it are the models that raced for it, and the gold one won. Selecting a star shows the ask, the race, every candidate's answer side by side, the scores, the residency plan, what the job made, and a link to the job's full offline replay.

Methodology notes and sources

Raw data is all under reports/MODES-BENCH-2026-10-01/ in the DGX source tree, copied to the USB drive. There is one JSON record per job, a bench.log per run, and a memory, load and GPU sample every 5 s (samples.jsonl).

Harness, fixtures and analysis (harness/):

• mode-bench.mjs, extras.mjs, bulk.mjs, watch.mjs, resume.mjs, refusals.mjs, d8-live.mjs, methods-reuse.mjs and lib.mjs, ported from the Mac campaign (~/nightshift-build/bench/modes/ on the Mac);
• fixtures/suite.mjs (prompts and scorers), with the DGX additions semver and ledger, plus fixtures/apps.mjs and fixtures/corpus20*;
• analyze_grid.py, analyze_dgx.py, analyze_extras.py, inventory.py;
• PROTOCOLS.md, written before any grid result was read;
• RUNLOG.md, every run, install, setpoint change and confounder, with times;
• DEFECTS.md, every defect with file and line evidence, its fix, its test and its live re-measure.

Builds:

Each release note is in src-tauri/target/releases/<v>/RELEASE.md (copies in reports/RELEASE_<v>.md). Each release:

• passed the full JS suite (4 331 → 4 448 tests; the only failures across all cuts were two timing tests under parallel load, each passing alone);
• passed check-jsx, cargo test for meterless-core and meterlessd, and the media packaging test;
• shipped engine files byte-identical to source, checked by cmp of the core deb and dist/.

1.3.3 and 1.3.4 were rebuilt in place before installation when the benchmark found more defects; none of those builds was ever installed in between.

DGX vs Mac. The Mac campaign (mac-reference/) measured the same modes on a 16 GB Apple M5 with small models that rotate through memory. Its findings that hold here:

• a dedicated judge lane is what makes the panel work;
• majority vote is fast and brittle;
• more diversity hurts single-answer tasks;
• a rubric passes what it forgets to check;
• Auto sizes by the clock.

Its findings that do not hold here:

• tournament selection (not in this edition);
• deliberate, batch and bulk dispatching to parked models (nothing rotates here);
• the consistency judge's false alarms (here it first never ran, then never cried wolf: precision 1.0).

None of the Mac's fixes were in this tree: each was re-derived here where the bug existed.

Confounders we know about (times in RUNLOG.md):

• An orphaned headless Chrome from an earlier session rendered a WebGL page in software at about 4 cores for the whole campaign. This session's permissions did not allow stopping it.
• An outside app (meterless-city-npc) sent 3 937 chat turns through the local endpoint during the campaign. Most arrived after 1 October 23:00 UTC, i.e. during the 1.3.3 and 1.3.4 runs.
• Two fix agents, five full test-suite runs and seven release builds overlapped parts of the runs.
• The posture changed with the quiet window: 47 grid jobs ran in Swarm, the rest in Depth.
• Two lanes were added to the controlled fleet by the operator at 09:11 UTC on 2 October. The tail of the watcher test ran with them.

Correctness columns are unaffected by any of these. Time columns carry medians with ranges, and modes are not ranked by speed within tens of seconds.

Limitations:

• Three repetitions per cell (two for the 1.3.3 diversity cells) show failures and large effects. They cannot rank close settings.
• The controlled fleet is three 8–14B models. Several fixtures have no fully correct candidate at that size, so those comparisons rest on partial credit.
• The app runtime check (add, toggle, persist) is the harness's own headless Chrome, not the product's.
• Not measured live:
• the deliberate resume after an engine kill (killing the engine host was not permitted to this session; engine tests cover M-2);
• Method reuse (the mined Method needs three model families, and the controlled fleet has two);
• the watcher fix (1.3.5 not installed);
• electricity.
• The sync chat router is out of scope.