Add --tool-args-object to serve tool call arguments as objects, revert to OpenAI compatible mode by default (return as strings)#20202
Closed
pwilkin wants to merge 2 commits intoggml-org:masterfrom
Closed
Conversation
…ble mode (as strings)
Member
Author
|
Fixes #20198 |
Contributor
|
This should only ever be a string. Why was it changed? |
Member
Author
|
@aldehir because a lot of tools actually expect this to be an object (we had a discussion on it some time ago). |
Contributor
|
It's not the spec. I've yet to experience a client that requires it be an object. We need to revert this entirely ASAP. I was under the impression that we had to parse the string as JSON before feeding it to the template. Not introduce an API change. Who are the offending clients? |
Member
Author
|
Okay, let's do it this way - I'll make the string behavior the default and change the switch, that way it'll make the OAI-compat behavior the default. |
Member
Author
|
You're right, we shouldn't introduce backwards-incompatible changes. |
pwilkin
changed the title
Member
Author
|
Aight, going to just go with the simple revert for now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, due to testing with numerous clients, the behavior of the API has been changed to output tool call arguments as JSON objects. However, this breaks compatibility with several clients that actually do expect OpenAI-compatible behavior. Therefore, I'm adding the option to return the OpenAI-compatible version via
--tool-args-compat(or-tacfor short).