Autoparser: True streaming

[pwilkin]

In the final changes to the autoparser I modified the atomicity constraint to be pretty restrictive due to models such as GLM 4.7-Flash which put arguments directly after the function name, with no markers in between. Now I'm relaxing that constraint so people can look at their favorite assistant building the tool call live ;)

[aldehir]

These already wrap in atomic:

    common_peg_parser tool_open(const common_peg_parser & p) { return atomic(tag(TOOL_OPEN, p)); }
    common_peg_parser tool_close(const common_peg_parser & p) { return atomic(tag(TOOL_CLOSE, p)); }
    common_peg_parser tool_id(const common_peg_parser & p) { return atomic(tag(TOOL_ID, p)); }
    common_peg_parser tool_name(const common_peg_parser & p) { return atomic(tag(TOOL_NAME, p)); }
    common_peg_parser tool_arg_open(const common_peg_parser & p) { return atomic(tag(TOOL_ARG_OPEN, p)); }
    common_peg_parser tool_arg_close(const common_peg_parser & p) { return atomic(tag(TOOL_ARG_CLOSE, p)); }
    common_peg_parser tool_arg_name(const common_peg_parser & p) { return atomic(tag(TOOL_ARG_NAME, p)); }

[aldehir]

I trust you tested this on multiple models? Again, the consequence of a generalized approach: the impact surface is way larger.

[pwilkin]

@aldehir yeah in most cases this wasn't even a problem to begin with, from real-life models the only one affected was GLM 4.7-Flash since it doesn't wrap its function name in anything. Pure JSON models are unaffected because they have function names atomically in "", so this just covers all the fringe cases.

[bchtrue]

@pwilkin You did it, congratulations and thank you from all of us!