Local AI Benchmarks, Inference Results, and Hardware Testing
The results outgrew the articles. At some point, another wall of bash output stopped being useful. The Hub puts the individual runs, hardware, model files, speeds, thermals, failures, and source material in one place. Now you can search by device or model and pull up the numbers, test settings, compatibility results, and evidence behind each run.
What am I looking at?
The Hub collects results from my different builds. A test group holds one question and its settings; a result row holds one model, configuration or stage of that test. Some rows are averages from repeated requests, while others count completed answers or show a temperature. The full test page explains which one you are reading.
I prepare these records from saved test outputs and reports, then include them in the website. This page does not collect live firewall traffic or combine the results into an overall model ranking.
Inference performance
Generation and prompt processing are measured in tokens per second. Check the model file, runtime and workload before comparing two rows.
Home SOC model tests
These count finished reports, expected explanations and evidence checks. The models ran on the MS-S1 MAX; the Protectli Vault supplied the firewall events. These are separate from the MS-S1 knowledge-assistant and fine-tuning project.
Hardware and setup checks
The Vault pages separate physical blocking, virtual packet checks, offline replays, path measurements, thermals and evidence recovery. The GPU report belongs to the Minisforum and checks Radeon offload. Each page names the system and the scope of its test.
For example, Qwen3.5 4B finished 12 of 12 reports with shared settings. Of the 10 tasks containing alerts, 7 showed every expected explanation and 4 passed the full answer checklist. Those numbers describe different checks on the same tasks.
122
result rows
88
published rows
9
rows labeled failed
21
test groups
122
matching now
These counts describe Hub rows. A row can summarize several requests or samples; the denominator on each measurement tells you what was counted. The Hub does not combine them into one accuracy score.
Find a result
Search model notes, devices, runtime versions, failures, or supporting filenames. Try “Qwen3.5 9B”, “inference speed”, or “matched-runtime-summary.csv”.
122 matching result rows.
Start with the test group: compare the same task and settings before drawing a conclusion. Tokens per second, finished reports, investigation checks and temperatures measure different things. The Vault pages hold firewall and supporting workflow checks; model experiments belong to the Minisforum.
Inference performance
Rates measure prompt processing and generation. Open a row to see its workload, repetitions and setup.
Qwen3 0.6B is the smallest Qwen3 model in these tests. The 0.6B label describes its parameter count; Q4_K_M identifies the quantized file I ran. Its small size made it a useful starting point for testing CPU inference and Intel GPU offload on the X1S.
Family
Qwen3
Recorded size
0.6B
It was small enough to show the CPU, mixed host-operation, and full-Vulkan differences without crossing into the larger-model i915 failures.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU in Matched server request mode on Youyeetoo X1S.
Three measured requests completed with a clean kernel window.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
Qwen3 0.6B is the smallest Qwen3 model in these tests. The 0.6B label describes its parameter count; Q4_K_M identifies the quantized file I ran. Its small size made it a useful starting point for testing CPU inference and Intel GPU offload on the X1S.
Family
Qwen3
Recorded size
0.6B
It was small enough to show the CPU, mixed host-operation, and full-Vulkan differences without crossing into the larger-model i915 failures.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Matched server request mode on Youyeetoo X1S.
Ollama reported 16.11% higher internal generation throughput than the paired llama.cpp row.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
Qwen3 1.7B is the next step up from the 0.6B model in the matched X1S tests. I used the Q4_K_M file in both CPU servers, then ran the same model through the three llama.cpp device modes. The two groups answer different questions, so their results are listed separately.
Family
Qwen3
Recorded size
1.7B
The original X1S work put it in the usable interactive tier, so it became the second clean model for checking whether the Vulkan pattern held beyond 0.6B.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU in Matched server request mode on Youyeetoo X1S.
Three measured requests completed with a clean kernel window.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
Qwen3 1.7B is the next step up from the 0.6B model in the matched X1S tests. I used the Q4_K_M file in both CPU servers, then ran the same model through the three llama.cpp device modes. The two groups answer different questions, so their results are listed separately.
Family
Qwen3
Recorded size
1.7B
The original X1S work put it in the usable interactive tier, so it became the second clean model for checking whether the Vulkan pattern held beyond 0.6B.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Matched server request mode on Youyeetoo X1S.
Ollama reported 16.46% higher internal generation throughput than the paired llama.cpp row.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
It marks the point where the clean small-model Vulkan result stopped scaling into a reliable larger-model run on this software stack.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU in Matched server request mode on Youyeetoo X1S.
Three measured requests completed with a clean kernel window.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
It marks the point where the clean small-model Vulkan result stopped scaling into a reliable larger-model run on this software stack.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Matched server request mode on Youyeetoo X1S.
Ollama reported 34.89% higher internal generation throughput than the paired llama.cpp row.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
Phi-4 Mini gave me a second model family for the matched CPU test. Both runtimes loaded the Q4_K_M file, so I could compare the same workload beyond the Qwen models. Its later Vulkan failure belongs to the GPU test, not the completed CPU comparison.
Family
Phi-4
Recorded size
Mini
It provides a second model family inside the matched runtime result and a retained failure outside the clean Vulkan pair.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU in Matched server request mode on Youyeetoo X1S.
Three measured requests completed with a clean kernel window.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
Phi-4 Mini gave me a second model family for the matched CPU test. Both runtimes loaded the Q4_K_M file, so I could compare the same workload beyond the Qwen models. Its later Vulkan failure belongs to the GPU test, not the completed CPU comparison.
Family
Phi-4
Recorded size
Mini
It provides a second model family inside the matched runtime result and a retained failure outside the clean Vulkan pair.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Matched server request mode on Youyeetoo X1S.
Ollama reported 32.00% higher internal generation throughput than the paired llama.cpp row.
Internal generation rates are comparable inside this eight-row test. Model load time was not an identical part of the timed path.
Qwen3 0.6B is the smallest Qwen3 model in these tests. The 0.6B label describes its parameter count; Q4_K_M identifies the quantized file I ran. Its small size made it a useful starting point for testing CPU inference and Intel GPU offload on the X1S.
Family
Qwen3
Recorded size
0.6B
It was small enough to show the CPU, mixed host-operation, and full-Vulkan differences without crossing into the larger-model i915 failures.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU in True CPU mode on Youyeetoo X1S.
Vulkan was hidden and host-operation offload was disabled. The five-repetition run had a clean kernel window.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
Qwen3 0.6B is the smallest Qwen3 model in these tests. The 0.6B label describes its parameter count; Q4_K_M identifies the quantized file I ran. Its small size made it a useful starting point for testing CPU inference and Intel GPU offload on the X1S.
Family
Qwen3
Recorded size
0.6B
It was small enough to show the CPU, mixed host-operation, and full-Vulkan differences without crossing into the larger-model i915 failures.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU + GPU in Mixed host operations mode on Youyeetoo X1S.
Zero model layers still allowed host operations on the Intel GPU. Prompt processing rose sharply while generation barely moved.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
Qwen3 0.6B is the smallest Qwen3 model in these tests. The 0.6B label describes its parameter count; Q4_K_M identifies the quantized file I ran. Its small size made it a useful starting point for testing CPU inference and Intel GPU offload on the X1S.
Family
Qwen3
Recorded size
0.6B
It was small enough to show the CPU, mixed host-operation, and full-Vulkan differences without crossing into the larger-model i915 failures.
What ran
Q4_K_M ran through llama.cpp 9a286ac using Vulkan in Full Vulkan mode on Youyeetoo X1S.
Full Vulkan improved generation 16.2% over true CPU and cut the recorded package peak by 24 °C.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
Qwen3 1.7B is the next step up from the 0.6B model in the matched X1S tests. I used the Q4_K_M file in both CPU servers, then ran the same model through the three llama.cpp device modes. The two groups answer different questions, so their results are listed separately.
Family
Qwen3
Recorded size
1.7B
The original X1S work put it in the usable interactive tier, so it became the second clean model for checking whether the Vulkan pattern held beyond 0.6B.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU in True CPU mode on Youyeetoo X1S.
Vulkan was hidden and host-operation offload was disabled. The five-repetition run had a clean kernel window.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
Qwen3 1.7B is the next step up from the 0.6B model in the matched X1S tests. I used the Q4_K_M file in both CPU servers, then ran the same model through the three llama.cpp device modes. The two groups answer different questions, so their results are listed separately.
Family
Qwen3
Recorded size
1.7B
The original X1S work put it in the usable interactive tier, so it became the second clean model for checking whether the Vulkan pattern held beyond 0.6B.
What ran
Q4_K_M ran through llama.cpp 9a286ac using CPU + GPU in Mixed host operations mode on Youyeetoo X1S.
Zero model layers still allowed host operations on the Intel GPU. Prompt processing rose sharply while generation barely moved.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
Qwen3 1.7B is the next step up from the 0.6B model in the matched X1S tests. I used the Q4_K_M file in both CPU servers, then ran the same model through the three llama.cpp device modes. The two groups answer different questions, so their results are listed separately.
Family
Qwen3
Recorded size
1.7B
The original X1S work put it in the usable interactive tier, so it became the second clean model for checking whether the Vulkan pattern held beyond 0.6B.
What ran
Q4_K_M ran through llama.cpp 9a286ac using Vulkan in Full Vulkan mode on Youyeetoo X1S.
Full Vulkan improved generation 15.9% over true CPU and cut the recorded package peak by 26 °C.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
It marks the point where the clean small-model Vulkan result stopped scaling into a reliable larger-model run on this software stack.
What ran
Q4_K_M ran through llama.cpp 9a286ac using Vulkan in Full Vulkan mode on Youyeetoo X1S.
The run reached an i915 reset timeout. No speed score is published for it.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
Phi-4 Mini gave me a second model family for the matched CPU test. Both runtimes loaded the Q4_K_M file, so I could compare the same workload beyond the Qwen models. Its later Vulkan failure belongs to the GPU test, not the completed CPU comparison.
Family
Phi-4
Recorded size
Mini
It provides a second model family inside the matched runtime result and a retained failure outside the clean Vulkan pair.
What ran
Q4_K_M ran through llama.cpp 9a286ac using Vulkan in Full Vulkan mode on Youyeetoo X1S.
The run window contained an i915 GPU hang even though llama-bench returned zero. The kernel event overrides the process exit code.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
The 8B Qwen3 entry was useful for testing the upper end of what I could run on the 16 GB X1S. Loading and completing a CPU request was one result; the larger full-Vulkan attempt was a separate test with its own failure record.
Family
Qwen3
Recorded size
8B
It establishes that fitting in 16 GB and completing on CPU did not guarantee a stable full-GPU path.
What ran
Q4_K_M ran through llama.cpp 9a286ac using Vulkan in Full Vulkan mode on Youyeetoo X1S.
The run window contained an i915 GPU hang even though llama-bench returned zero. No clean performance row is claimed.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
The Gemma 3 entry includes a separate text-only derivative made from the original artifact. That distinction matters when reproducing the run: the derivative completed true CPU, while the later Vulkan attempt still hit the device-loss path. Keep the file identity with the result.
Family
Gemma 3
Recorded size
4B
It records two different compatibility boundaries: converting the older multimodal artifact for current llama.cpp and the separate Vulkan failure that conversion did not fix.
What ran
Text-only derivative ran through llama.cpp 9a286ac using Vulkan in Full Vulkan mode on Youyeetoo X1S.
Fence and preemption timeouts led to an i915 reset and vk::DeviceLostError.
Compare the clean Qwen rows inside this group. The larger-model entries are failure outcomes, not performance rows.
Ling-mini-2.0 is listed as 16B total parameters with 3B active. For this test I pinned one 8.8 GB IQ4_XS file, then changed the CPU thread count. The model page keeps the three- and four-thread settings, repetitions and thermal limits beside their results.
Family
Ling-mini
Recorded size
16B total / 3B active
The thread test shows a real speed and temperature tradeoff on the N5095 using one verified 8.8 GB artifact rather than a mutable model tag.
What ran
IQ4_XS, pinned revision 8be84a0 ran through llama.cpp 9a286ac using CPU in 3 threads, 85 °C guard mode on Youyeetoo X1S.
The clean three-thread row stayed well below the normal thermal guard.
The four-thread row used a disclosed 90 °C abort after the first attempt reached the normal 85 °C guard.
Ling-mini-2.0 is listed as 16B total parameters with 3B active. For this test I pinned one 8.8 GB IQ4_XS file, then changed the CPU thread count. The model page keeps the three- and four-thread settings, repetitions and thermal limits beside their results.
Family
Ling-mini
Recorded size
16B total / 3B active
The thread test shows a real speed and temperature tradeoff on the N5095 using one verified 8.8 GB artifact rather than a mutable model tag.
What ran
IQ4_XS, pinned revision 8be84a0 ran through llama.cpp 9a286ac using CPU in 4 threads, 90 °C guard mode on Youyeetoo X1S.
The selected four-thread row improved generation 32.1% and completed under the disclosed 90 °C ceiling.
The four-thread row used a disclosed 90 °C abort after the first attempt reached the normal 85 °C guard.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 1, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 1.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 1, MTP on mode on Youyeetoo X1S.
MTP was active and ran 46.73% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 2, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 2.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 2, MTP on mode on Youyeetoo X1S.
MTP was active and ran 65.03% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 3, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 3.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 3, MTP on mode on Youyeetoo X1S.
MTP was active and ran 72.06% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 4, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 4.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 4, MTP on mode on Youyeetoo X1S.
MTP was active and ran 76.48% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 1, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 1.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 1, MTP on mode on Youyeetoo X1S.
MTP was active and ran 22.76% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 2, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 2.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 2, MTP on mode on Youyeetoo X1S.
MTP was active and ran 44.65% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 3, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 3.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 3, MTP on mode on Youyeetoo X1S.
MTP was active and ran 46.45% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Depth 4, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 4.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU + draft MTP in Depth 4, MTP on mode on Youyeetoo X1S.
MTP was active and ran 70.28% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Depth 1, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 1.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU + draft MTP in Depth 1, MTP on mode on Youyeetoo X1S.
MTP was active and ran 30.68% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Depth 2, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 2. This pair includes a 90 °C rerun after the first attempt stopped at the normal 85 °C guard.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU + draft MTP in Depth 2, MTP on mode on Youyeetoo X1S.
MTP was active and ran 41.63% slower than its paired baseline. This pair includes a 90 °C rerun after the first attempt stopped at the normal 85 °C guard.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Depth 3, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 3. This pair includes a 90 °C rerun after the first attempt stopped at the normal 85 °C guard.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU + draft MTP in Depth 3, MTP on mode on Youyeetoo X1S.
MTP was active and ran 56.03% slower than its paired baseline. This pair includes a 90 °C rerun after the first attempt stopped at the normal 85 °C guard.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Depth 4, MTP off mode on Youyeetoo X1S.
Paired MTP-off baseline for draft depth 4.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU + draft MTP in Depth 4, MTP on mode on Youyeetoo X1S.
MTP was active and ran 60.75% slower than its paired baseline.
Each on/off pair belongs to its own requested draft depth. Do not compare the baseline from one pair as if it were the same request block as another.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
It is the speed-first CPU option from this result set and shows that active MTP still carried too much overhead on the four-core N5095.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Two-run warm average mode on Youyeetoo X1S.
The fastest normal text-generation result in the X1S work. Throughput does not establish answer quality.
The rows use different repetition counts and artifact states. They are useful operating points, not one controlled model-quality contest.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
It sits between the 0.8B speed result and the slower larger models, making it useful for checking whether MTP behavior changed with model size.
What ran
Q8_0 ran through Ollama 0.32.1 using CPU in Two-run warm average mode on Youyeetoo X1S.
A clean middle-tier CPU result from the normal non-MTP Ollama path.
The rows use different repetition counts and artifact states. They are useful operating points, not one controlled model-quality contest.
Qwen3.5 9B has records from two different projects: fitting a larger model on the X1S and producing structured Home SOC reports on the Minisforum. Generation speed and completed report checks measure different things. Each test keeps its own workload, runtime and outcome.
Family
Qwen3.5
Recorded size
9B
This was a fit-and-completion test for a much larger model on the X1S, not a claim that 1.110 tok/s is comfortable interactive speed.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Five-prompt average mode on Youyeetoo X1S.
The 9B artifact fit in 16 GB and completed five prompts, but generation requires patience.
The rows use different repetition counts and artifact states. They are useful operating points, not one controlled model-quality contest.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
It tested the MTP path that required an external assistant instead of assuming the presence of model tensors meant speculative decoding was active.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Two-run warm average mode on Youyeetoo X1S.
The normal non-MTP Ollama result. The separate MTP sweep is recorded in its own test group.
The rows use different repetition counts and artifact states. They are useful operating points, not one controlled model-quality contest.
This Gemma 4 E4B entry records the original warm Ollama run on the X1S. It is a useful reference when browsing that test, with its original method and status kept on the result row.
Family
Gemma 4
Recorded size
E4B
The row is useful as a recorded X1S operating point, but it does not have enough separate material for an indexable model page yet.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Two-run warm average mode on Youyeetoo X1S.
A clean normal Ollama row retained as an operating point, not a quality ranking.
The rows use different repetition counts and artifact states. They are useful operating points, not one controlled model-quality contest.
Granite 4 Tiny-H has one retained Ollama observation on the X1S. The result links back to that test rather than expanding a single measurement into a broader model comparison.
Family
Granite 4
Recorded size
Tiny-H
One observation is retained for completeness but is not promoted into a deeper standalone page.
What ran
Original Ollama tag ran through Ollama 0.32.1 using CPU in Single original observation mode on Youyeetoo X1S.
One retained observation, not a repeated average.
The rows use different repetition counts and artifact states. They are useful operating points, not one controlled model-quality contest.
This entry keeps the LFM2.5 label from the original Ollama run. It used a mutable latest tag, so the row preserves the dated observation and the artifact limitation together.
Family
LFM
Recorded size
2.5
The tag was not content-pinned, so the row stays visible as a partial result without becoming its own search page.
What ran
Mutable latest tag ran through Ollama 0.32.1 using CPU in Single recorded tag state mode on Youyeetoo X1S.
The mutable latest tag was not content-pinned, so this remains a partial result.
The rows use different repetition counts and artifact states. They are useful operating points, not one controlled model-quality contest.
BitCPM-CANN 1B is recorded here as the official TQ2_0 GGUF. I gave the same verified file to llama.cpp and Ollama, and they did not have the same loading outcome. The useful question here is runtime compatibility before it is generation speed.
Family
BitCPM
Recorded size
1B
This is a runtime-support result, not a speed contest. The same verified file produced inference in one runtime and a tensor-size overflow in the other.
What ran
Official TQ2_0 GGUF ran through llama.cpp 9a286ac using CPU in True CPU, five repetitions mode on Youyeetoo X1S.
Pinned llama.cpp accepted the verified file and completed the controlled true-CPU workload.
BitCPM is not a matched runtime speed contest because Ollama never loaded the model. The Gemma derivative is inspectable but does not repair the Vulkan device-loss path.
BitCPM-CANN 1B is recorded here as the official TQ2_0 GGUF. I gave the same verified file to llama.cpp and Ollama, and they did not have the same loading outcome. The useful question here is runtime compatibility before it is generation speed.
Family
BitCPM
Recorded size
1B
This is a runtime-support result, not a speed contest. The same verified file produced inference in one runtime and a tensor-size overflow in the other.
What ran
Official TQ2_0 GGUF ran through Ollama 0.32.1 using CPU in Model load mode on Youyeetoo X1S.
Ollama rejected the same verified file with a tensor-size overflow before inference.
BitCPM is not a matched runtime speed contest because Ollama never loaded the model. The Gemma derivative is inspectable but does not repair the Vulkan device-loss path.
The Gemma 3 entry includes a separate text-only derivative made from the original artifact. That distinction matters when reproducing the run: the derivative completed true CPU, while the later Vulkan attempt still hit the device-loss path. Keep the file identity with the result.
Family
Gemma 3
Recorded size
4B
It records two different compatibility boundaries: converting the older multimodal artifact for current llama.cpp and the separate Vulkan failure that conversion did not fix.
What ran
Separate text-only derivative ran through llama.cpp 9a286ac using CPU in True CPU, five repetitions mode on Youyeetoo X1S.
The derived copy completed true CPU. The original multimodal source artifact was not overwritten.
BitCPM is not a matched runtime speed contest because Ollama never loaded the model. The Gemma derivative is inspectable but does not repair the Vulkan device-loss path.
Qwen3 0.6B is the smallest Qwen3 model in these tests. The 0.6B label describes its parameter count; Q4_K_M identifies the quantized file I ran. Its small size made it a useful starting point for testing CPU inference and Intel GPU offload on the X1S.
Family
Qwen3
Recorded size
0.6B
It was small enough to show the CPU, mixed host-operation, and full-Vulkan differences without crossing into the larger-model i915 failures.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Mean warm generation mode on Youyeetoo X1S.
Fastest row, least complete response on the single prompt.
These warm Ollama results belong together. They should not be used as the other half of the later llama-bench runtime comparison.
Qwen3 1.7B is the next step up from the 0.6B model in the matched X1S tests. I used the Q4_K_M file in both CPU servers, then ran the same model through the three llama.cpp device modes. The two groups answer different questions, so their results are listed separately.
Family
Qwen3
Recorded size
1.7B
The original X1S work put it in the usable interactive tier, so it became the second clean model for checking whether the Vulkan pattern held beyond 0.6B.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Mean warm generation mode on Youyeetoo X1S.
Best interactive starting tier tested.
These warm Ollama results belong together. They should not be used as the other half of the later llama-bench runtime comparison.
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
It marks the point where the clean small-model Vulkan result stopped scaling into a reliable larger-model run on this software stack.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Mean warm generation mode on Youyeetoo X1S.
Patient local batch use.
These warm Ollama results belong together. They should not be used as the other half of the later llama-bench runtime comparison.
Phi-4 Mini gave me a second model family for the matched CPU test. Both runtimes loaded the Q4_K_M file, so I could compare the same workload beyond the Qwen models. Its later Vulkan failure belongs to the GPU test, not the completed CPU comparison.
Family
Phi-4
Recorded size
Mini
It provides a second model family inside the matched runtime result and a retained failure outside the clean Vulkan pair.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Mean warm generation mode on Youyeetoo X1S.
Patient local batch use.
These warm Ollama results belong together. They should not be used as the other half of the later llama-bench runtime comparison.
The Gemma 3 entry includes a separate text-only derivative made from the original artifact. That distinction matters when reproducing the run: the derivative completed true CPU, while the later Vulkan attempt still hit the device-loss path. Keep the file identity with the result.
Family
Gemma 3
Recorded size
4B
It records two different compatibility boundaries: converting the older multimodal artifact for current llama.cpp and the separate Vulkan failure that conversion did not fix.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Mean warm generation mode on Youyeetoo X1S.
Patient local batch use.
These warm Ollama results belong together. They should not be used as the other half of the later llama-bench runtime comparison.
The 8B Qwen3 entry was useful for testing the upper end of what I could run on the 16 GB X1S. Loading and completing a CPU request was one result; the larger full-Vulkan attempt was a separate test with its own failure record.
Family
Qwen3
Recorded size
8B
It establishes that fitting in 16 GB and completing on CPU did not guarantee a stable full-GPU path.
What ran
Q4_K_M ran through Ollama 0.32.1 using CPU in Mean warm generation mode on Youyeetoo X1S.
Fit comfortably with sub-1 tok/s generation.
These warm Ollama results belong together. They should not be used as the other half of the later llama-bench runtime comparison.
This is the DeepSeek-R1 model distilled into a Qwen 1.5B model, tested as W8A8 on the Nova NPU. I asked for the same state-capitals list used in the other Nova runs and retained what the answer actually did alongside its generation rate.
Family
DeepSeek-R1
Recorded size
1.5B
It is the clearest example in the Hub of why throughput and useful output cannot be treated as the same result.
What ran
W8A8 RKLLM ran through RKLLM 1.2.1 using NPU in 50 state-capitals prompt mode on Indiedroid Nova.
Started fast, then invented cities around state seven.
The three Nova rows share the local test. Raspberry Pi numbers in the article came from outside references with different runtime and quantization.
Recorded setup
The full test page contains the shared controls and comparison boundary for this row.
Qwen 2.5 3B Instruct ran as a W8A8 model through RKLLM on the Nova. The recorded task was a state-capitals list, so this page keeps both the generation rate and the observed answer result instead of treating throughput as answer quality.
Family
Qwen 2.5
Recorded size
3B
It was slower than the smaller DeepSeek row but produced the useful answer on the shared prompt.
What ran
W8A8 RKLLM ran through RKLLM 1.2.1 using NPU in 50 state-capitals prompt mode on Indiedroid Nova.
Returned all 50 state capitals correctly.
The three Nova rows share the local test. Raspberry Pi numbers in the article came from outside references with different runtime and quantization.
Recorded setup
The full test page contains the shared controls and comparison boundary for this row.
Llama 3.1 8B Instruct connects the Nova NPU run with the Pi 5 references from the original article. The Nova measurement is mine; the Pi rows are outside references with their own sources and test conditions.
Family
Llama 3.1
Recorded size
8B
It connects Trevor-measured Nova data with a clearly separated published Pi reference without presenting the two methods as a matched hardware contest.
What ran
W8A8 RKLLM ran through RKLLM 1.2.1 using NPU in 50 state-capitals prompt mode on Indiedroid Nova.
Measured Nova NPU result.
The three Nova rows share the local test. Raspberry Pi numbers in the article came from outside references with different runtime and quantization.
Recorded setup
The full test page contains the shared controls and comparison boundary for this row.
This is the DeepSeek-R1 model distilled into a Qwen 1.5B model, tested as W8A8 on the Nova NPU. I asked for the same state-capitals list used in the other Nova runs and retained what the answer actually did alongside its generation rate.
Family
DeepSeek-R1
Recorded size
1.5B
It is the clearest example in the Hub of why throughput and useful output cannot be treated as the same result.
What ran
Q4 reference ran through llama.cpp using CPU in Published reference range mode on Raspberry Pi 5.
Approximate published range. Not Trevor-measured.
These are external references with different runtime, quantization, and hardware approach. A matched Trevor-run Pi 5 test has not been completed.
Recorded setup
The full test page contains the shared controls and comparison boundary for this row.
Qwen 2.5 3B Instruct ran as a W8A8 model through RKLLM on the Nova. The recorded task was a state-capitals list, so this page keeps both the generation rate and the observed answer result instead of treating throughput as answer quality.
Family
Qwen 2.5
Recorded size
3B
It was slower than the smaller DeepSeek row but produced the useful answer on the shared prompt.
What ran
Q4 reference ran through llama.cpp using CPU in Published reference range mode on Raspberry Pi 5.
Approximate published range. Not Trevor-measured.
These are external references with different runtime, quantization, and hardware approach. A matched Trevor-run Pi 5 test has not been completed.
Recorded setup
The full test page contains the shared controls and comparison boundary for this row.
Llama 3.1 8B Instruct connects the Nova NPU run with the Pi 5 references from the original article. The Nova measurement is mine; the Pi rows are outside references with their own sources and test conditions.
Family
Llama 3.1
Recorded size
8B
It connects Trevor-measured Nova data with a clearly separated published Pi reference without presenting the two methods as a matched hardware contest.
What ran
Q4 reference ran through llama.cpp using CPU in Jeff Geerling published result mode on Raspberry Pi 5.
Cited result. Not Trevor-measured.
These are external references with different runtime, quantization, and hardware approach. A matched Trevor-run Pi 5 test has not been completed.
Recorded setup
The full test page contains the shared controls and comparison boundary for this row.
I followed a harmless request from my Windows client into Suricata, changed only its test signature to drop, and then restored it to alert. HTTPS and DNS controls checked that ordinary traffic still worked while the selected request was blocked.
This is a physical client-to-Vault check with exact connection evidence. The test signature verifies this inspection path; it is not a detection-rate study or inspection of decrypted HTTPS payloads.
Bounded curl request; the timeout alone is not the blocking oracle.
curl timed out after ten seconds without a response body, while the exact corresponding EVE connection was recorded as blocked. Both observations are required for this result.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned Vault test · Physical path · inline Netmap drop
Selected Critical TLS C2 rules; UID restored to alert.
Native validation exited successfully with dependency-complete selected rules. The final snapshot held 37,125 active rules: 37,121 alerts and four drops across ten enabled categories. Feed updates can change these counts.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned Vault test · Final installed snapshot
Listener, rule and console checks confirmed LAN-only HTTPS management, disabled SSH and password-protected console access. The off-device XML matched its device-side SHA256; five boot environments were retained on the SSD. The XML stays private because it contains secrets, and same-disk boot environments do not replace it.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned Vault test · Recorded appliance configuration
I used three temporary FreeBSD VNET stacks on the Vault to check a small PF policy with real packets, captures and counters. The eight cases cover allowed controls, NAT, unsolicited WAN traffic, spoofing and finite bursts.
The authored policy mirrors core LAN pass, WAN deny, NAT and anti-spoof behavior. It does not import every OPNsense anchor or table, and the finite bursts are blocking checks rather than a DoS-capacity measurement.
HTTP and NAT control
Isolated VNET · core PF mirror
Published result
Measured on Protectli V1610 · Virtual packet exchange on the Vault
Response body
39 bytes
Exact body/hash, with observed NAT peer.
HTTP 200 returned the exact 39-byte body/hash; allowed LAN traffic and the NAT peer were verified.
I kept engine regressions separate from replays of the installed policy. That distinction matters here: the upstream regression cases passed, while the installed rules produced zero alerts on 20 unchanged upstream captures.
These are offline engine runs on the Vault. Upstream cases carry their own rules and assertions; installed-policy runs use the appliance snapshot. Offline drop records show rule selection, while physical-wire blocking is documented in the separate inline test.
Allowed SID 2100498 from the original HTTP replay.
The HTTP UID fixture produced one allowed signature alert and other cases produced no alerts. Invalid TCP checksums prevent using this original replay to infer HTTPS stream inspection.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned Vault test · Original invalid TCP checksums retained
All eight corrected replays completed with zero invalid TCP checksums. HTTP produced three allowed alerts including the UID signature; HTTPS produced one TLS event without a UID alert.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned Vault test · Only invalid TCP checksum fields changed
Four suffix negatives and one benign control. (fixtures)
Four exact names produced blocked matches and drop records. Four suffix negatives and one reserved-domain control avoided those matches; all nine met the authored expectations.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned Vault test · Inert offline SNI positive and negative fixtures
I measured short, certificate-validated downloads through the final IDS and IPS configurations using the same fixed endpoint. The individual trials stay here alongside their medians because the Internet path varied between requests.
These rates include connection and TLS setup. Upstream conditions, endpoint behavior and other client traffic were not isolated, so the difference does not measure inspection overhead or maximum forwarding capacity.
PCAP IDS
Routed Internet · three complete 25 MB requests
Published result
Measured on Protectli V1610 · Owned client routed through the Vault
Median rate
537.017 Mbps
Three complete trials, including connection and TLS setup.
Complete trials
3 of 3
25,000,000 bytes each after one 1,000,000-byte warmup. (requests)
Sampled core maximum
60 °C
Separate native resource sampling; may miss short request peaks.
Individual payload rates were 537.017, 533.622, 549.84 Mbps. Every request completed under the same final-platform protocol; the median describes this endpoint/path observation.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned client routed through the Vault · Routed Internet · three complete 25 MB requests
Measured on Protectli V1610 · Owned client routed through the Vault
Median rate
493.027 Mbps
Three complete trials, including connection and TLS setup.
Complete trials
3 of 3
25,000,000 bytes each after one 1,000,000-byte warmup. (requests)
Sampled core maximum
48 °C
Separate native resource sampling; may miss short request peaks.
Individual payload rates were 493.027, 480.639, 623.063 Mbps. Every request completed under the same final-platform protocol; the median describes this endpoint/path observation.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned client routed through the Vault · Routed Internet · three complete 25 MB requests
I also transferred fixed payloads from a bounded server on the Vault to the owned LAN client, then retained the engine counters with their actual observation windows. Successful downloads and complete packet inspection are different measurements here.
This workload terminates on the Vault, so it is not WAN forwarding throughput. PCAP kernel drops span its full runtime without an isolated pre-LAN boundary; the Netmap stream-discard snapshot uses a different window and cannot serve as a matched loss comparison.
Cumulative 962,578 packets across the full PCAP runtime.
Netmap stream discards
188 packets
610,614 packets in the separate 5m14s final snapshot.
PCAP cumulative counters cover 40+ minutes without a pre-LAN boundary. Netmap counters cover 5m14s after restart; its 188 discards were stream errors of unproven cause. No emitted kernel-drop counter is not a zero-loss certificate.
Setup and original evidence
Protectli V1610 · OPNsense 26.7.5 / Suricata 8.0.7_1 · Owned Vault test · Different engine windows · preserve each denominator
The firewall records also need to survive review and a restart. These checks cover Vault event forwarding, the Windows evidence tools, rolling storage and recovery after sign-in, with the maintenance gaps and failed readings kept in the record.
This is a two-machine workflow: the Vault supplies events and native observations, while Windows receives, indexes and serves them. Scheduled recovery requires Windows sign-in, and USB sampling needs an authenticated appliance console after a Vault reboot. The Hub is a saved result record, not a live health monitor.
Historical isolated software suite; distinct from the later 250-check retention suite.
17 cases delivered 153 datagrams with the expected SID/action counts; seven original references verified and the report/source routes returned HTTP 200. These were replay receipts, without model inference. The separate local pipeline/fault lane passed 75 software tests using temporary stores and mocked failure conditions.
Setup and original evidence
Protectli V1610 · Windows Python receiver, evidence builder and dashboard · Two-machine evidence workflow; isolated host replay · Isolated loopback receiver and evidence builder
Current policy · fixture-tested expiry and live activation
Published result
Tested system: Protectli Vault + Windows evidence host · Windows evidence storage
Preserved setup baseline
310110946 bytes
Nine original members, including failures; immutable baseline separate from ordinary expiry.
Hot budget
1024 MiB
1 GiB originals, active rollover at 32 MiB.
Cold budget
4096 MiB
4 GiB compressed sealed segments.
Ordinary sealed-segment window
7 days
After sealing; capacity may shorten it. Disposable-fixture expiry test, not a full live seven-day observation.
The live migration preserved all setup bytes and returned new Vault receipts without a full/storage-error state. Hot originals roll at 32 MiB into a verified compressed cold tier; ordinary sealed segments expire seven days after sealing or earlier under capacity pressure.
Setup and original evidence
Protectli V1610 · Windows Python / rolling-v2 / SQLite · Windows evidence storage · Current policy · fixture-tested expiry and live activation
Tested system: Protectli Vault + Windows evidence host · Windows evidence storage
Pinned-source budget
64 MiB
Separate from hot/cold ordinary storage.
Pinned-record limit
4096 records
New note refused at the limit, without stopping collection.
Pre-activation regression checks
250 tests
Software/fixture checks, not independent incidents.
New operator notes preserve their cited raw records under a separate 64 MiB / 4,096-record budget. A full pin budget refuses another note while collection continues; an expired unpreserved source returns HTTP 410. The pre-activation suite passed 250 tests, including storage and recovery fixtures.
Setup and original evidence
Protectli V1610 · Windows Python / rolling-v2 / SQLite · Windows evidence storage · Separate pins · exact hashes · visible expiry
Tested system: Protectli Vault + Windows evidence host · Vault + Windows sign-in recovery
Vault login prompt
59 seconds
Approximate observed appliance reboot interval, separate from full application readiness.
The Vault reached its login prompt in about 59 seconds. After Windows reboot and sign-in, recovery started the owned components and the next check reused them without duplicates. Original log prefix, notes and backup hashes matched; certificate-valid HTTPS, direct Vault DNS and exact UID/EVE correlation passed.
Setup and original evidence
Protectli V1610 · Vault boot / Windows Task Scheduler and PowerShell · Vault + Windows sign-in recovery · Recorded appliance reboot and newer Windows boot
Later ownership-receipt repair, without a new reboot
Published result
Tested system: Protectli Vault + Windows evidence host · Windows sign-in and five-minute recovery
A manual receiver launch had left a stale recovery-owner receipt. After exact ownership reconciliation, the scheduled task restored the dashboard and reporting runtime, then repeated the restoration after an idle dashboard stop without stopping collection. Original files/prefixes verified. This repair was not a new sign-out/sign-in or reboot test.
Setup and original evidence
Protectli V1610 · Windows Task Scheduler / PowerShell 7 · Windows sign-in and five-minute recovery · Later ownership-receipt repair, without a new reboot
Tested system: Protectli Vault + Windows evidence host · Vault + Windows evidence workflow
Timestamped observations
352 observations
Retained Oct 5–7 snapshot; not independent incidents.
Sampled core minimum
30 °C
Operating observations, separate from guarded load.
Sampled core maximum
56 °C
Operating observations, separate from the 62 °C load peak.
Observation gaps over ten minutes
5 gaps
27 min, 12 min, 11h57, 20 min and 12 min.
352 observations recorded sampled cores at 30–56 °C, with no sampled full/storage-error/capacity-warning state. Five gaps exceeded ten minutes, including an 11h57 host interruption; one native reading had unknown fields and another was stale. Maintenance and UDP receipt uncertainty remain visible.
Setup and original evidence
Protectli V1610 · Vault native sensors / Windows observation journal · Vault + Windows evidence workflow · Historical snapshot, not current live health
Tested system: Protectli Vault + Windows evidence host · Vault + Windows evidence workflow
Post-repair clock difference
46 ms
Approximate independent host/appliance comparison after successful resync.
The Windows time-service repair completed a successful resync and cleared its synchronization warning. A direct comparison then showed about 46 ms difference from the Vault. Earlier records with roughly 20 seconds of skew stayed unchanged; these offsets are not network latency or a clock-drift study.
Setup and original evidence
Protectli V1610 · Windows W32Time / Vault clock · Vault + Windows evidence workflow · Independent clock check after repair
Four owned CPU-load processes completed the 180-second check with five-second core-temperature sampling. The peak was 62 °C, followed by recorded cooling and process cleanup.
Thirty-six samples from one short load, not independent thermal trials. This is a CPU-core observation, not case-surface temperature, thermal equilibrium, longevity or an overheating guarantee for every environment.
Highest core 62 °C, final readings 53/53/53/52 °C, cooldown 47 °C. No thermal stop; all owned children exited. This does not measure thermal equilibrium or case surface.
Four models answered the same saved log summaries. This group contains two tasks: structured reports with a two-minute deadline, and written reports with a five-minute deadline. The sampling settings use each model's defaults.
Each model received 12 structured tasks three times and 6 written tasks twice. Repeats test consistency on those tasks; they are not new incidents. Compare completion and explanations within the same task and settings. These runs do not measure each model's best possible performance.
12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)
Passed full answer checklist
0 of 30
10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)
Expected explanations selected
36 of 63
21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)
6 answers stopped at the 700-token output cap. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.
Setup and original evidence
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native structured profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)
Stopped at the output cap
8 of 12
These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)
Finished answers with a major factual error
0 of 4
No delivered answer was rejected for a major factual error in the recorded review. Other shortcomings are documented; this is not a complete accuracy score. (answers)
4 answers finished and 8 stopped at the output cap. The completed answers still have recorded shortcomings, including citations and explanation quality.
Setup and original evidence
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native written profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)
Passed full answer checklist
3 of 30
10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)
Expected explanations selected
15 of 63
21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)
All 36 reports finished, though most missed expected explanations. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native structured profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)
Stopped at the output cap
4 of 12
These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)
Finished answers with a major factual error
2 of 8
One task produced a major factual error, repeated in both answers. Repeating it twice is not two different incidents. (answers)
The model claimed 14 groups where the supplied summary contained 16. That error repeated twice; unfinished answers were withheld.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native written profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)
Passed full answer checklist
3 of 30
10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)
Expected explanations selected
30 of 63
21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)
6 requests exceeded the 120-second deadline. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.
Setup and original evidence
Qwen3.5 9B has records from two different projects: fitting a larger model on the X1S and producing structured Home SOC reports on the Minisforum. Generation speed and completed report checks measure different things. Each test keeps its own workload, runtime and outcome.
Family
Qwen3.5
Recorded size
9B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native structured profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)
Stopped at the output cap
12 of 12
These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)
Finished answers with a major factual error
Not assessed
No finished answer was available to assess. This is not zero errors or an accuracy pass. (answers)
All 12 written answers stopped at the output cap. Their partial text is retained, but there is no finished answer to grade for factual errors.
Setup and original evidence
Qwen3.5 9B has records from two different projects: fitting a larger model on the X1S and producing structured Home SOC reports on the Minisforum. Generation speed and completed report checks measure different things. Each test keeps its own workload, runtime and outcome.
Family
Qwen3.5
Recorded size
9B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native written profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)
Passed full answer checklist
6 of 30
10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)
Expected explanations selected
6 of 63
21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)
24 requests exceeded the 120-second deadline. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.
Setup and original evidence
Gemma 4 12B QAT was the larger installed alternative in the Home SOC reporting tests. The records show its structured-answer checks, deadlines and unfinished responses, so you can inspect what happened on the actual briefs.
Family
Gemma 4
Recorded size
12B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native structured profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)
Stopped at the output cap
4 of 12
These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)
Finished answers with a major factual error
2 of 8
One task produced a major factual error, repeated in both answers. Repeating it twice is not two different incidents. (answers)
The model claimed 14 allowed events where the supplied summary contained 15 and described traffic as routine/internal without enough evidence. That error repeated twice; unfinished answers were withheld.
Setup and original evidence
Gemma 4 12B QAT was the larger installed alternative in the Home SOC reporting tests. The records show its structured-answer checks, deadlines and unfinished responses, so you can inspect what happened on the actual briefs.
Family
Gemma 4
Recorded size
12B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native written profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
I converted the simulation logs into summaries, then asked each model to review them twice. Some simulations produced no alerts, so completion across all requests is shown separately from explanations on the summaries that contain alerts.
21 source cases produced 9 different summaries: 8 with alerts and 1 empty summary shared by 13 sources. Each model made 42 requests, but only 16 were the alert-bearing tasks (8 summaries, twice). Those repeats and duplicate empty inputs do not represent 42 different incidents.
21 source cases, each asked twice. 13 sources produced the same empty summary, so this total includes duplicate inputs. (answers)
All expected explanations shown
16 of 16
8 different summaries contain alerts, each asked twice. An answer counts only when every expected explanation is visible in the rendered report. (answers)
Passed full answer checklist
6 of 16
The same 16 alert-bearing attempts. The unedited answer must meet explanation, list-length, repetition and review-priority checks as well as finish. (answers)
Completion across all requests includes the empty summaries. Read the explanation and checklist counts for the 16 attempts with alerts. Identical repeated answers reused their source-linked agent review after exact identity checks.
Setup and original evidence
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native simulation-source profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
21 source cases, each asked twice. 13 sources produced the same empty summary, so this total includes duplicate inputs. (answers)
All expected explanations shown
4 of 16
8 different summaries contain alerts, each asked twice. An answer counts only when every expected explanation is visible in the rendered report. (answers)
Passed full answer checklist
4 of 16
The same 16 alert-bearing attempts. The unedited answer must meet explanation, list-length, repetition and review-priority checks as well as finish. (answers)
Completion across all requests includes the empty summaries. Read the explanation and checklist counts for the 16 attempts with alerts. Identical repeated answers reused their source-linked agent review after exact identity checks.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native simulation-source profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
21 source cases, each asked twice. 13 sources produced the same empty summary, so this total includes duplicate inputs. (answers)
All expected explanations shown
16 of 16
8 different summaries contain alerts, each asked twice. An answer counts only when every expected explanation is visible in the rendered report. (answers)
Passed full answer checklist
14 of 16
The same 16 alert-bearing attempts. The unedited answer must meet explanation, list-length, repetition and review-priority checks as well as finish. (answers)
Completion across all requests includes the empty summaries. Read the explanation and checklist counts for the 16 attempts with alerts. Identical repeated answers reused their source-linked agent review after exact identity checks.
Setup and original evidence
Qwen3.5 9B has records from two different projects: fitting a larger model on the X1S and producing structured Home SOC reports on the Minisforum. Generation speed and completed report checks measure different things. Each test keeps its own workload, runtime and outcome.
Family
Qwen3.5
Recorded size
9B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native simulation-source profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
21 source cases, each asked twice. 13 sources produced the same empty summary, so this total includes duplicate inputs. (answers)
All expected explanations shown
14 of 16
8 different summaries contain alerts, each asked twice. An answer counts only when every expected explanation is visible in the rendered report. (answers)
Passed full answer checklist
14 of 16
The same 16 alert-bearing attempts. The unedited answer must meet explanation, list-length, repetition and review-priority checks as well as finish. (answers)
Completion across all requests includes the empty summaries. Read the explanation and checklist counts for the 16 attempts with alerts. Identical repeated answers reused their source-linked agent review after exact identity checks.
Setup and original evidence
Gemma 4 12B QAT was the larger installed alternative in the Home SOC reporting tests. The records show its structured-answer checks, deadlines and unfinished responses, so you can inspect what happened on the actual briefs.
Family
Gemma 4
Recorded size
12B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Native simulation-source profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
Each model answered the same 12 tasks once with the sampling settings made explicit. Read completion first, then check whether the alert explanations and full answer checklist passed.
10 of the 12 tasks contain alerts. Completion uses all 12; explanation and checklist results use those 10. The 21 expected explanations count individual items across the same 10 tasks. This pass supports a choice for the scheduled report under these settings, not a general intelligence ranking.
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
2 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
0 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
12 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)
2 answers stopped at the output cap. Of the 10 tasks with alerts, 2 showed every expected explanation and 0 met the full answer checklist. The 12/21 count tracks individual explanation items within those same tasks.
Setup and original evidence
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
7 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
4 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
18 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)
All 12 finished. Of the 10 tasks with alerts, 7 showed every expected explanation and 4 met the full answer checklist. The 18/21 count tracks individual explanation items within those same tasks.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
5 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
2 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
9 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)
3 requests timed out. Of the 10 tasks with alerts, 5 showed every expected explanation and 2 met the full answer checklist. The 9/21 count tracks individual explanation items within those same tasks.
Setup and original evidence
Qwen3.5 9B has records from two different projects: fitting a larger model on the X1S and producing structured Home SOC reports on the Minisforum. Generation speed and completed report checks measure different things. Each test keeps its own workload, runtime and outcome.
Family
Qwen3.5
Recorded size
9B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
12 fixed summaries asked once; 2 have no alerts. A report must finish within 120 seconds and pass the report validator. (answers)
All expected explanations shown
2 of 10
10 of those summaries contain alerts. Each pass means all expected explanations appear in the rendered report. (answers)
Passed full answer checklist
2 of 10
The same 10 alert-bearing tasks. A pass also requires a finished answer, valid list lengths, no repeated selections/citations and the required review priority, before display cleanup. (answers)
Expected explanations selected
2 of 21
21 individual explanation items across the same 10 tasks. This is an item count, not 21 cases. (selections)
8 requests timed out. Of the 10 tasks with alerts, 2 showed every expected explanation and 2 met the full answer checklist. The 2/21 count tracks individual explanation items within those same tasks.
Setup and original evidence
Gemma 4 12B QAT was the larger installed alternative in the Home SOC reporting tests. The records show its structured-answer checks, deadlines and unfinished responses, so you can inspect what happened on the actual briefs.
Family
Gemma 4
Recorded size
12B
Minisforum MS-S1 MAX · Ollama 0.35.1 · CPU · Explicit common profile
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
The original investigation answers failed. A revised task then exposed two prompts that were too long for the configured context. After fixing the instructions and ensuring the full input reached the model, I reran the same cases.
This is a correction history on 6 constructed cases, each tried in 3 conditions: evidence only, analyst assistance, and assistance with repetition grouping. That produces 18 measured answers, not 18 independent incidents. All 3 corrected conditions passed, so this test shows no accuracy advantage for assistance or grouping.
Some answers copied real groups and then claimed there were no alerts; empty cases invented groups. Practice answers stayed outside the measured denominator.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · Recorded backend in request receipts · Original V1 investigator
Context window: 4,096 tokens for input and output together.
Exposed regression set; two expanded inputs were truncated (answers)
Largest original expanded input
5464 tokens
Truncated to 2050 under context 4096
Two inputs were cut from 5,107 and 5,464 tokens to 2,050. The grouped condition saw different actual evidence from the expanded conditions, so 16/18 cannot support a grouping advantage.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · Recorded backend in request receipts · Intermediate V5; invalid condition comparison
Context window: 4,096 tokens for input and output together.
The same 6 constructed cases, asked once in this condition with complete model input. These cases were already used during development. (answers)
6/6 passed in each of the three conditions. The model answered these known cases correctly after the input fix; assistance and grouping did not score higher than evidence alone. No humans took this test.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · Recorded backend in request receipts · Corrected V6 condition A: evidence only
Context window: 8,192 tokens for input and output together.
The same 6 constructed cases, asked once in this condition with complete model input. These cases were already used during development. (answers)
6/6 passed in each of the three conditions. The model answered these known cases correctly after the input fix; assistance and grouping did not score higher than evidence alone. No humans took this test.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · Recorded backend in request receipts · Corrected V6 condition B: analyst assistance
Context window: 8,192 tokens for input and output together.
The same 6 constructed cases, asked once in this condition with complete model input. These cases were already used during development. (answers)
6/6 passed in each of the three conditions. The model answered these known cases correctly after the input fix; assistance and grouping did not score higher than evidence alone. No humans took this test.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · Recorded backend in request receipts · Corrected V6 condition C: assistance plus grouping
Context window: 8,192 tokens for input and output together.
6 deliberately wrong answers. Only 1 review met every required defect check; the larger model was unreliable as a judge. (answers)
Reviews matching every answer-key category
18 of 24
24 reviews, including practice. 18 matched every independently defined answer-key category; this is separate from the 6 wrong-answer controls. (answers)
The larger reviewer missed defects and raised false alarms. Its agreement never overrides the source score.
Setup and original evidence
Qwen3.5 9B has records from two different projects: fitting a larger model on the X1S and producing structured Home SOC reports on the Minisforum. Generation speed and completed report checks measure different things. Each test keeps its own workload, runtime and outcome.
Family
Qwen3.5
Recorded size
9B
Minisforum MS-S1 MAX · Ollama 0.35.1 · Recorded backend in request receipts · Corrected V6 reviewer controls
Context window: 4,096 tokens for input and output together.
One live report ran on the Minisforum's Radeon 8060S. The runner confirmed all 33 model layers were offloaded, and the returned report passed the existing checker.
The 20.781 seconds is the total HTTP request duration for one report with 309 generated tokens. The input and answer length differ from the CPU study. Use this as evidence that GPU execution worked, not a CPU/GPU speed comparison.
One live request; includes more than token generation
Actual generated tokens
309 tokens
Returned request receipt
Offloaded layers
33 layers
Actual runner identified Radeon 8060S; 33 of 33 layers
The structured report passed validation, but some explanation remained generic. No matched speedup or generation-rate claim.
Setup and original evidence
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
Minisforum MS-S1 MAX · Ollama 0.35.1 · Vulkan / Radeon 8060S · First live GPU acceptance
Context window: 4,096 tokens for input and output together. CPU-thread setting: 2; this is a runtime setting, not the host's core count.
Start with a model to see what I used it for, which files and devices I tested, and where it completed or failed.
Qwen3 0.6B6 result rows · Youyeetoo X1S
Qwen3 0.6B is the smallest Qwen3 model in these tests. The 0.6B label describes its parameter count; Q4_K_M identifies the quantized file I ran. Its small size made it a useful starting point for testing CPU inference and Intel GPU offload on the X1S.
Family
Qwen3
Recorded size
0.6B
The smallest matched X1S model completed both CPU runtimes and all three explicit llama.cpp device modes.
Qwen3 1.7B is the next step up from the 0.6B model in the matched X1S tests. I used the Q4_K_M file in both CPU servers, then ran the same model through the three llama.cpp device modes. The two groups answer different questions, so their results are listed separately.
Family
Qwen3
Recorded size
1.7B
This model completed the matched CPU comparison and the full CPU, mixed, and Vulkan mode test.
Qwen3 4B Instruct8 result rows · Youyeetoo X1S, Minisforum MS-S1 MAX
This is the instruction-tuned 4B Qwen3 entry. It appears in the X1S inference tests and the Home SOC reporting work on the Minisforum. The model name connects those records, while the file, hardware and settings on each row tell you which configuration actually ran.
Family
Qwen3
Recorded size
4B
The Q4_K_M file completed the matched CPU test, then reached an i915 reset timeout during full Vulkan.
Phi-4 Mini gave me a second model family for the matched CPU test. Both runtimes loaded the Q4_K_M file, so I could compare the same workload beyond the Qwen models. Its later Vulkan failure belongs to the GPU test, not the completed CPU comparison.
Family
Phi-4
Recorded size
Mini
Phi-4 Mini completed both matched CPU servers. Its full-Vulkan window coincided with an i915 GPU hang.
The 8B Qwen3 entry was useful for testing the upper end of what I could run on the 16 GB X1S. Loading and completing a CPU request was one result; the larger full-Vulkan attempt was a separate test with its own failure record.
Family
Qwen3
Recorded size
8B
Qwen3 8B fit during the original CPU matrix at 0.924 tok/s. A later full-Vulkan attempt coincided with an i915 GPU hang.
The Gemma 3 entry includes a separate text-only derivative made from the original artifact. That distinction matters when reproducing the run: the derivative completed true CPU, while the later Vulkan attempt still hit the device-loss path. Keep the file identity with the result.
Family
Gemma 3
Recorded size
4B
A separate text-only derivative completed true CPU. The later Vulkan path ended with timeouts, an i915 reset, and device loss.
Ling-mini-2.0 is listed as 16B total parameters with 3B active. For this test I pinned one 8.8 GB IQ4_XS file, then changed the CPU thread count. The model page keeps the three- and four-thread settings, repetitions and thermal limits beside their results.
Family
Ling-mini
Recorded size
16B total / 3B active
The exact pinned IQ4_XS file completed true-CPU tests at three and four threads.
This is Qwen3.5 0.8B, a different family and size from Qwen3 0.6B. I tested normal Ollama generation and paired MTP settings on the X1S. MTP uses a draft path to propose tokens; the off/on rows show what that extra work cost on the N5095.
Family
Qwen3.5
Recorded size
0.8B
Qwen3.5 0.8B produced the fastest normal text-generation result in the X1S work and completed every MTP depth pair.
Qwen3.5 2B sits between the 0.8B and 9B entries in the X1S collection. It has a normal Ollama result and a separate MTP depth sweep, which lets you inspect how draft settings behaved at another model size.
Family
Qwen3.5
Recorded size
2B
Qwen3.5 2B completed the normal Ollama run and MTP depths one through four.
Qwen3.5 9B6 result rows · Youyeetoo X1S, Minisforum MS-S1 MAX
Qwen3.5 9B has records from two different projects: fitting a larger model on the X1S and producing structured Home SOC reports on the Minisforum. Generation speed and completed report checks measure different things. Each test keeps its own workload, runtime and outcome.
Family
Qwen3.5
Recorded size
9B
The official Q4_K_M Ollama artifact fit in 16 GB and averaged 1.110 tok/s across five prompts.
Gemma 4 E2B used a separate assistant model for the X1S MTP tests. I kept the draft-path proof and off/on pairs because loading the main model alone would not establish that speculative decoding was active.
Family
Gemma 4
Recorded size
E2B
Gemma 4 E2B completed the normal Ollama run and a proven MTP sweep with its separate assistant model.
This Gemma 4 E4B entry records the original warm Ollama run on the X1S. It is a useful reference when browsing that test, with its original method and status kept on the result row.
Family
Gemma 4
Recorded size
E4B
Gemma 4 E4B completed the original warm Ollama test at 1.78 tok/s.
BitCPM-CANN 1B is recorded here as the official TQ2_0 GGUF. I gave the same verified file to llama.cpp and Ollama, and they did not have the same loading outcome. The useful question here is runtime compatibility before it is generation speed.
Family
BitCPM
Recorded size
1B
The exact official TQ2_0 GGUF ran in pinned llama.cpp and failed to load in Ollama 0.32.1.
Granite 4 Tiny-H has one retained Ollama observation on the X1S. The result links back to that test rather than expanding a single measurement into a broader model comparison.
Family
Granite 4
Recorded size
Tiny-H
The X1S recorded one 5.43 tok/s Ollama observation.
This entry keeps the LFM2.5 label from the original Ollama run. It used a mutable latest tag, so the row preserves the dated observation and the artifact limitation together.
Family
LFM
Recorded size
2.5
The X1S recorded one 4.95 tok/s result from a mutable latest tag.
DeepSeek-R1 Distill Qwen 1.5B2 result rows · Indiedroid Nova, Raspberry Pi 5
This is the DeepSeek-R1 model distilled into a Qwen 1.5B model, tested as W8A8 on the Nova NPU. I asked for the same state-capitals list used in the other Nova runs and retained what the answer actually did alongside its generation rate.
Family
DeepSeek-R1
Recorded size
1.5B
The Nova NPU run started at 11.5 tok/s, then the state-capitals answer degraded into invented cities.
Qwen 2.5 3B Instruct2 result rows · Indiedroid Nova, Raspberry Pi 5
Qwen 2.5 3B Instruct ran as a W8A8 model through RKLLM on the Nova. The recorded task was a state-capitals list, so this page keeps both the generation rate and the observed answer result instead of treating throughput as answer quality.
Family
Qwen 2.5
Recorded size
3B
The Nova NPU result reached 7.0 tok/s and returned all 50 state capitals correctly.
Llama 3.1 8B Instruct2 result rows · Indiedroid Nova, Raspberry Pi 5
Llama 3.1 8B Instruct connects the Nova NPU run with the Pi 5 references from the original article. The Nova measurement is mine; the Pi rows are outside references with their own sources and test conditions.
Family
Llama 3.1
Recorded size
8B
The Nova NPU measured 3.72 tok/s. A separate 1.99 tok/s Pi 5 value remains labeled as an outside reference.
Qwen3.5 4B is the model I selected for the scheduled Home SOC report under the shared settings I tested. The work runs on the Minisforum using evidence from the Vault. Completion, expected explanations and the full answer checklist are separate measurements on those reporting tasks.
Family
Qwen3.5
Recorded size
4B
I selected Qwen3.5 4B for the scheduled Home SOC report under the tested shared settings. Finishing a report, including the expected explanations and passing the full answer checklist are counted separately.
Gemma 4 12B QAT4 result rows · Minisforum MS-S1 MAX
Gemma 4 12B QAT was the larger installed alternative in the Home SOC reporting tests. The records show its structured-answer checks, deadlines and unfinished responses, so you can inspect what happened on the actual briefs.
Family
Gemma 4
Recorded size
12B
Gemma 4 12B QAT was the larger alternative in the Home SOC tests. Its timeouts, unfinished answers and incorrect counts remain in the results.
A model name identifies the model. A variant such as Q4_K_M or IQ4_XS identifies the quantization used for a particular file. The artifact hash ties a result to the exact bytes I tested.
Runtime and backend
Ollama and llama.cpp run the model. CPU, Vulkan and RKLLM NPU paths describe where or how the computation ran. A successful CPU request and a failed GPU request can belong to the same model.
Generation and prompt processing
Generation measures output tokens per second. Prompt processing measures how quickly the input was processed. Open the test group for the workload, token counts and timing method behind each rate.
Context and report checks
The context window holds input and output together. In the Home SOC tests, completed reports, expected explanations and full answer checks have separate counts and denominators. Read the label beside each value.
Four exact Q4_K_M GGUFs received the same raw prompt, context, thread count, batch settings, sampler, and 96-token output through Ollama 0.32.1 and native llama.cpp commit 9a286ac.
Qwen3 0.6B and 1.7B ran in three explicit modes with the same native binary and pp128/tg96 workload. Larger full-Vulkan models were retained as failure evidence.
Qwen3.5 0.8B, Qwen3.5 2B, and Gemma 4 E2B ran with MTP off and on at every draft depth from one through four. Service logs proved the draft path was active.
The exact BitCPM TQ2_0 file exposed a runtime-support difference. A separate Gemma 3 text-only derivative documented a different conversion problem before the later Vulkan failure.
The values quoted in the Nova article remain ranges or cited results. They are kept separate from Trevor-measured rows and are not converted into invented exact midpoints.
Protectli V1610: Detection, Inline Block and Rollback
I followed a harmless request from my Windows client into Suricata, changed only its test signature to drop, and then restored it to alert. HTTPS and DNS controls checked that ordinary traffic still worked while the selected request was blocked.
I used three temporary FreeBSD VNET stacks on the Vault to check a small PF policy with real packets, captures and counters. The eight cases cover allowed controls, NAT, unsolicited WAN traffic, spoofing and finite bursts.
Protectli V1610: Suricata Regression and Policy Replays
I kept engine regressions separate from replays of the installed policy. That distinction matters here: the upstream regression cases passed, while the installed rules produced zero alerts on 20 unchanged upstream captures.
I measured short, certificate-validated downloads through the final IDS and IPS configurations using the same fixed endpoint. The individual trials stay here alongside their medians because the Internet path varied between requests.
Protectli V1610: LAN Endpoint and Capture Counters
I also transferred fixed payloads from a bounded server on the Vault to the owned LAN client, then retained the engine counters with their actual observation windows. Successful downloads and complete packet inspection are different measurements here.
Four owned CPU-load processes completed the 180-second check with five-second core-temperature sampling. The peak was 62 °C, followed by recorded cooling and process cleanup.
Home SOC: Evidence Collection, Retention and Recovery
The firewall records also need to survive review and a restart. These checks cover Vault event forwarding, the Windows evidence tools, rolling storage and recovery after sign-in, with the maintenance gaps and failed readings kept in the record.
Four models answered the same saved log summaries. This group contains two tasks: structured reports with a two-minute deadline, and written reports with a five-minute deadline. The sampling settings use each model's defaults.
I converted the simulation logs into summaries, then asked each model to review them twice. Some simulations produced no alerts, so completion across all requests is shown separately from explanations on the summaries that contain alerts.
Each model answered the same 12 tasks once with the sampling settings made explicit. Read completion first, then check whether the alert explanations and full answer checklist passed.
The original investigation answers failed. A revised task then exposed two prompts that were too long for the configured context. After fixing the instructions and ensuring the full input reached the model, I reran the same cases.
One live report ran on the Minisforum's Radeon 8060S. The runner confirmed all 33 model layers were offloaded, and the returned report passed the existing checker.
The X1S has the deepest result set here: matched Ollama and llama.cpp CPU requests, explicit CPU and Vulkan modes, thermal data, MTP tests, compatibility failures, and retained i915 evidence.
Three W8A8 models ran through the RK3588S NPU. The result set includes speed and the observed answer-quality outcome from the shared state-capitals prompt.
The Pi 5 entries are external references, not Trevor’s matched rerun. They stay visible because they explain the original Nova comparison, but they do not belong on a controlled cross-device leaderboard.
These are the firewall and evidence-workflow checks from my V1610 Home SOC build: physical detection, inline blocking and rollback, isolated PF tests, Suricata replays, path measurements, temperature, retention and restart recovery. The local models run on the Minisforum, where their results are listed separately.
These are Home SOC tests run on the MS-S1 MAX: comparing four local models, checking investigation answers and confirming Radeon 8060S offload. The Vault supplies the firewall events. These entries do not report results from the separate MS-S1 knowledge-assistant or fine-tuning project.