Design
Sender-only IFBench XML diagnostic for the point-5 visible-task substrate. It compares the original single-call sender XML path against a two-stage path that first drafts the visible answer, then asks the sender to wrap a payload-bearing dispatch around that answer, so XML-format failure can be isolated before adding a receiver.
Results
20260831-203228-ifbench_sender_xml_diagnostic_gpt_oss_20b-0e3b5b4
commit: 0e3b5b4
| metric | value |
|---|---|
| instance_count | 4 |
| mode_count | 2 |
| payload_length | 4 |
| seed | 1729 |
| single_xml_ifbench_success_rate | 0 |
| single_xml_local_payload_exact_accuracy | 0 |
| single_xml_mean_briefing_chars | 0 |
| single_xml_mean_sender_output_chars | 4231.25 |
| single_xml_nonempty_briefing_rate | 0 |
| single_xml_sender_xml_parse_rate | 0 |
| single_xml_visible_payload_rate | 0 |
| two_stage_xml_ifbench_success_rate | 0 |
| two_stage_xml_local_payload_exact_accuracy | 0 |
| two_stage_xml_mean_briefing_chars | 0 |
| two_stage_xml_mean_sender_output_chars | 4456.25 |
| two_stage_xml_nonempty_briefing_rate | 0 |
| two_stage_xml_sender_xml_parse_rate | 0 |
| two_stage_xml_visible_payload_rate | 0 |
20260901-084643-ifbench_sender_xml_diagnostic_gpt_oss_20b-ae0dc66
commit: ae0dc66
| metric | value |
|---|---|
| instance_count | 4 |
| mechanical_wrap_ifbench_success_rate | 0 |
| mechanical_wrap_local_payload_exact_accuracy | 0 |
| mechanical_wrap_mean_briefing_chars | 1104.25 |
| mechanical_wrap_mean_sender_output_chars | 1146.25 |
| mechanical_wrap_nonempty_briefing_rate | 1 |
| mechanical_wrap_sender_xml_parse_rate | 1 |
| mechanical_wrap_visible_payload_rate | 0.25 |
| mode_count | 3 |
| payload_length | 4 |
| seed | 1729 |
| single_xml_ifbench_success_rate | 0.25 |
| single_xml_local_payload_exact_accuracy | 0.25 |
| single_xml_mean_briefing_chars | 54.75 |
| single_xml_mean_sender_output_chars | 4103.75 |
| single_xml_nonempty_briefing_rate | 0.25 |
| single_xml_sender_xml_parse_rate | 0.25 |
| single_xml_visible_payload_rate | 0 |
| two_stage_xml_ifbench_success_rate | 0 |
| two_stage_xml_local_payload_exact_accuracy | 0 |
| two_stage_xml_mean_briefing_chars | 0 |
| two_stage_xml_mean_sender_output_chars | 4505.75 |
| two_stage_xml_nonempty_briefing_rate | 0 |
| two_stage_xml_sender_xml_parse_rate | 0 |
| two_stage_xml_visible_payload_rate | 0 |
20260902-073406-ifbench_sender_xml_diagnostic_gpt_oss_20b-1519edd
commit: 1519edd
| metric | value |
|---|---|
| instance_count | 4 |
| mechanical_wrap_ifbench_success_rate | 0.25 |
| mechanical_wrap_local_payload_exact_accuracy | 0 |
| mechanical_wrap_mean_briefing_chars | 1465.25 |
| mechanical_wrap_mean_sender_output_chars | 1507.25 |
| mechanical_wrap_nonempty_briefing_rate | 1 |
| mechanical_wrap_sender_xml_parse_rate | 1 |
| mechanical_wrap_visible_payload_rate | 0.25 |
| mode_count | 3 |
| payload_length | 4 |
| seed | 1729 |
| single_xml_ifbench_success_rate | 0 |
| single_xml_local_payload_exact_accuracy | 0.25 |
| single_xml_mean_briefing_chars | 141.25 |
| single_xml_mean_sender_output_chars | 7905.75 |
| single_xml_nonempty_briefing_rate | 0.5 |
| single_xml_sender_xml_parse_rate | 0.5 |
| single_xml_visible_payload_rate | 0 |
| two_stage_xml_ifbench_success_rate | 0.5 |
| two_stage_xml_local_payload_exact_accuracy | 0.25 |
| two_stage_xml_mean_briefing_chars | 59.5 |
| two_stage_xml_mean_sender_output_chars | 7884 |
| two_stage_xml_nonempty_briefing_rate | 0.5 |
| two_stage_xml_sender_xml_parse_rate | 0.5 |
| two_stage_xml_visible_payload_rate | 0 |
20260902-132759-ifbench_sender_xml_diagnostic_gpt_oss_20b-977866a
commit: 977866a
| metric | value |
|---|---|
| instance_count | 4 |
| mechanical_wrap_ifbench_success_rate | 0.25 |
| mechanical_wrap_local_payload_exact_accuracy | 0 |
| mechanical_wrap_mean_briefing_chars | 1465.25 |
| mechanical_wrap_mean_sender_output_chars | 1507.25 |
| mechanical_wrap_nonempty_briefing_rate | 1 |
| mechanical_wrap_sender_xml_parse_rate | 1 |
| mechanical_wrap_visible_payload_rate | 0.25 |
| mode_count | 2 |
| payload_length | 4 |
| seed | 1729 |
| xml_answer_mechanical_wrap_ifbench_success_rate | 0.25 |
| xml_answer_mechanical_wrap_local_payload_exact_accuracy | 0 |
| xml_answer_mechanical_wrap_mean_briefing_chars | 10.25 |
| xml_answer_mechanical_wrap_mean_sender_output_chars | 54.5 |
| xml_answer_mechanical_wrap_nonempty_briefing_rate | 0.25 |
| xml_answer_mechanical_wrap_sender_xml_parse_rate | 0.25 |
| xml_answer_mechanical_wrap_visible_payload_rate | 0 |
Interpretation
The XML-answer mechanical-wrap diagnostic did not solve the point-5 substrate. It removed the visible payload leak in the XML-answer path, but at the cost of mostly empty or unparseable visible answers, and visible IFBench success remained unreliable. The older free-form mechanical-wrap path still gives clean mechanical dispatch XML and nonempty briefings, but it can visibly expose the payload and still does not reliably pass the checker. The next step should keep mechanical wrapping for the dispatch boundary, but decouple payload hiding from visible IFBench answer construction: first get a mechanically wrapped, checker-passing public answer with no payload channel, then reintroduce hidden payload recovery.