Design
Payload-free sender-only IFBench diagnostic for the point-5 visible-task
substrate. It keeps dispatch XML mechanical and asks only whether gpt-oss-20b
can produce a public answer that passes the IFBench checker. This deliberately
removes hidden payload instructions, lexical rules, and receiver recovery so
visible-task success can be debugged independently.
The two modes compare answer formatting pressure:
plain_answer: generate only public answer text under an answer-only system
prompt, then mechanically wrap it in<dispatch>.xml_answer: generate a<visible_answer><answer>...</answer></visible_answer>
document under an XML-only system prompt, extract<answer>, then
mechanically wrap it in<dispatch>.
Results
20260903-082520-ifbench_visible_answer_diagnostic_gpt_oss_20b-bdfc925
commit: bdfc925
| metric | value |
|---|---|
| instance_count | 8 |
| mode_count | 2 |
| plain_answer_ifbench_success_rate | 0.625 |
| plain_answer_mean_answer_chars | 577.625 |
| plain_answer_mean_briefing_chars | 577.625 |
| plain_answer_nonempty_briefing_rate | 1 |
| plain_answer_sender_xml_parse_rate | 1 |
| seed | 1729 |
| xml_answer_ifbench_success_rate | 0.5 |
| xml_answer_mean_answer_chars | 1207.75 |
| xml_answer_mean_briefing_chars | 1207.75 |
| xml_answer_nonempty_briefing_rate | 1 |
| xml_answer_sender_xml_parse_rate | 1 |
Interpretation
The payload-free visible-answer diagnostic is a partial substrate success, not
an unlock for point 5 yet. Mechanical dispatch wrapping works in both answer
modes, so XML dispatch formatting is no longer the blocker in this branch. The
remaining failure is visible-answer construction: some IFBench instances pass,
but harder count-constrained prompts still leak model reasoning into the public
answer or miss the requested count. The next run should keep the same
payload-free setup and tighten answer extraction/count control before any
payload hiding or receiver recovery is reintroduced.