Home SOC: Four Models with Default Sampling

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

30 of 36

12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)

Passed full answer checklist

0 of 30

10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)

Expected explanations selected

36 of 63

21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)

6 answers stopped at the 700-token output cap. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

4 of 12

6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)

Stopped at the output cap

8 of 12

These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)

Finished answers with a major factual error

0 of 4

No delivered answer was rejected for a major factual error in the recorded review. Other shortcomings are documented; this is not a complete accuracy score. (answers)

4 answers finished and 8 stopped at the output cap. The completed answers still have recorded shortcomings, including citations and explanation quality.

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

36 of 36

12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)

Passed full answer checklist

3 of 30

10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)

Expected explanations selected

15 of 63

21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)

All 36 reports finished, though most missed expected explanations. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

8 of 12

6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)

Stopped at the output cap

4 of 12

These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)

Finished answers with a major factual error

2 of 8

One task produced a major factual error, repeated in both answers. Repeating it twice is not two different incidents. (answers)

The model claimed 14 groups where the supplied summary contained 16. That error repeated twice; unfinished answers were withheld.

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

30 of 36

12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)

Passed full answer checklist

3 of 30

10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)

Expected explanations selected

30 of 63

21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)

6 requests exceeded the 120-second deadline. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

0 of 12

6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)

Stopped at the output cap

12 of 12

These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)

Finished answers with a major factual error

Not assessed

No finished answer was available to assess. This is not zero errors or an accuracy pass. (answers)

All 12 written answers stopped at the output cap. Their partial text is retained, but there is no finished answer to grade for factual errors.

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

12 of 36

12 fixed log summaries, each asked 3 times. A finished report completed within 120 seconds and passed the report validator. (answers)

Passed full answer checklist

6 of 30

10 summaries contain alerts, each asked 3 times. A pass needs a finished answer, expected explanations, valid list lengths, no repeated selections/citations and the required review priority. (answers)

Expected explanations selected

6 of 63

21 expected explanation items across those 10 summaries, repeated 3 times. This counts selected items, not whole answers or new discoveries. (selections)

24 requests exceeded the 120-second deadline. The full answer checklist is stricter than completion; failing it does not mean every part of the answer was wrong.

Measured on Minisforum MS-S1 MAX · CPU

Finished reports

8 of 12

6 summaries with alerts, each asked twice. Only naturally finished answers count here. (answers)

Stopped at the output cap

4 of 12

These answers returned text but reached the 700-token output limit before finishing. None of the written requests exceeded its 300-second deadline. (answers)

Finished answers with a major factual error

2 of 8

One task produced a major factual error, repeated in both answers. Repeating it twice is not two different incidents. (answers)

The model claimed 14 allowed events where the supplied summary contained 15 and described traffic as routine/internal without enough evidence. That error repeated twice; unfinished answers were withheld.