/hub/runs/protectli-analyst-common-profile

Home SOC: Four Models with Shared Sampling

Each model answered the same 12 tasks once with the sampling settings made explicit. Read completion first, then check whether the alert explanations and full answer checklist passed.

This page keeps the shared controls, retained measurements, failures, and comparison limits together. The linked files are there when you need the underlying table or method.

What you can compare here

10 of the 12 tasks contain alerts. Completion uses all 12; explanation and checklist results use those 10. The 21 expected explanations count individual items across the same 10 tasks. This pass supports a choice for the scheduled report under these settings, not a general intelligence ranking.

Related pages

You can reuse Trevor-measured data from this page under the repository MIT terms. Retain the required notices when reusing the project documentation, and credit Trevor Unland when citing this run. Project media has separate terms in the repository.

Recorded results

Open the details under any row for the specific setup, outcome, artifact identity, and supporting file.

Qwen3 4B Instruct

Structured report; shared sampling

Partial result

Measured on Minisforum MS-S1 MAX · CPU

Finished reports
10 of 12
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
2 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
0 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
12 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)

2 answers stopped at the output cap. Of the 10 tasks with alerts, 2 showed every expected explanation and 0 met the full answer checklist. The 12/21 count tracks individual explanation items within those same tasks.

Setup and original evidence

Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile

Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.

Installed artifact digest: 0edcdef34593eac1aa2be9c7d06c432dcf81945adca5eca2f27662c18f168ba0

Qwen3.5 4B

Structured report; shared sampling

Partial result

Measured on Minisforum MS-S1 MAX · CPU

Finished reports
12 of 12
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
7 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
4 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
18 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)

All 12 finished. Of the 10 tasks with alerts, 7 showed every expected explanation and 4 met the full answer checklist. The 18/21 count tracks individual explanation items within those same tasks.

Setup and original evidence

Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile

Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.

Installed artifact digest: d8b0f5e9760cd1682034f292d7ef72ec46f432149be0df7574bf2d6e92e38c04

Qwen3.5 9B

Structured report; shared sampling

Partial result

Measured on Minisforum MS-S1 MAX · CPU

Finished reports
9 of 12
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
5 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
2 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
9 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)

3 requests timed out. Of the 10 tasks with alerts, 5 showed every expected explanation and 2 met the full answer checklist. The 9/21 count tracks individual explanation items within those same tasks.

Setup and original evidence

Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile

Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.

Installed artifact digest: 56671c2ab9385f9cfcb404638e32cd62d88e3501d44822208363c010179a3c90

Gemma 4 12B QAT

Structured report; shared sampling

Partial result

Measured on Minisforum MS-S1 MAX · CPU

Finished reports
4 of 12
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
2 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
2 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
2 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)

8 requests timed out. Of the 10 tasks with alerts, 2 showed every expected explanation and 2 met the full answer checklist. The 2/21 count tracks individual explanation items within those same tasks.

Setup and original evidence

Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile

Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.

Installed artifact digest: 38044be4f923e5a55264ed7df4eaac2676651a905f735197c504045140c02bd3

Supporting evidence

Results and method

These are the few files that support this page directly. The repository holds the wider project history.

Controls

  • Ollama 0.35.1 in the isolated SOC runtime; two CPU threads and no GPU layers
  • Frozen briefs, schemas and expected contexts; no weight training or per-model tuning
  • Context 4096; output cap 700; temperature 0; seed 17; thinking disabled; keepalive zero
  • One request at a time; resource guard; source-bound validation; original failures preserved
  • top_k 20; top_p 1; min_p 0; repeat_penalty 1; presence/frequency penalties 0
  • 12 tasks × 4 models = 48 requests; 120-second deadline; several settings changed together
  • Two CPU threads were a resource limit for the scheduled service. No thread-count sweep was run; larger-model timeouts do not establish that the models could not work with more compute.
  • The 4096-token context is the input-and-output window. The separate 700-token setting limits the answer. A timeout, output-cap stop and input truncation are different failures.

What this run established

  • $Qwen3.5 4B delivered 12/12 tasks, displayed complete context on 7/10 and met raw strict utility on 4/10.
  • $The original 4B capped twice; 9B timed out three times; Gemma timed out eight times.
  • $The selected newer 4B still missed context on three tasks. Code retains the facts and the operator reviews the source.
  • $No setting was isolated as the cause, and no model added proven detection or human time savings.

Get an email when I publish new hardware reviews, benchmarks, and security research. Unsubscribe anytime.

Email goes to Buttondown. Privacy details.

© 2026 Trevor Unland. All rights reserved.

PrivacyTermsRSS Feed

$ echo "Built with React + TypeScript"