Skip to content

server : implement json-schema-to-grammar.mjs & add grammar param in the UI#2588

Merged
jhen0409 merged 8 commits intoggml-org:masterfrom
jhen0409:grammar-js
Aug 14, 2023
Merged

server : implement json-schema-to-grammar.mjs & add grammar param in the UI#2588
jhen0409 merged 8 commits intoggml-org:masterfrom
jhen0409:grammar-js

Conversation

@jhen0409
Copy link
Copy Markdown
Collaborator

This PR implements json-schema-to-grammar.mjs by followed the Python implementation, it can be used in Web & Node.js.

I'm looking for a simple way to use JSON schema and gbnf, but I'm mostly using JS not Python for both client or server, so I thought it might be possible to have a JS implementation here, it should help others too.

Demo

Use a json schema as functions.json (From PR #2532):

{
  "oneOf": [
    {
      "type": "object",
      "properties": {
        "function": {"const": "create_event"},
        "arguments": {
          "type": "object",
          "properties": {
            "title": {"type": "string"},
            "date": {"type": "string"},
            "time": {"type": "string"}
          }
        }
      }
    },
    {
      "type": "object",
      "properties": {
        "function": {"const": "image_search"},
        "arguments": {
          "type": "object",
          "properties": {
            "query": {"type": "string"}
          }
        }
      }
    }
  ]
}

Usage in Node.js:

node chat.mjs --grammar-json-schema functions.json --grammar-json-schema-prop-order function,arguments
> Find an image of a dog.
{ "function" : "image_search", "arguments": { "query": "dog" }} 
> Schedule a birthday party on Aug 14th 2023 at 8pm.
{ "function" : "create_event", "arguments": { "date": "Aug 14th 2023", "time": "8pm", "title": "Birthday Party" }} 
> 

Usage in the UI, we can just use gbnf or put JSON schema and press the convert button:

2023-08-12 10-59-45 2023-08-12 11_02_09

Completions
Screenshot 2023-08-12 at 10 28 31 AM

@x4080
Copy link
Copy Markdown

x4080 commented Aug 12, 2023

This is very useful for integrating llama and traditional programming

@jhen0409 jhen0409 requested a review from ejones August 14, 2023 02:23
Copy link
Copy Markdown
Collaborator

@ejones ejones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice conversion! Code looks good.

@jhen0409 jhen0409 merged commit 5517d6e into ggml-org:master Aug 14, 2023
@jhen0409 jhen0409 deleted the grammar-js branch August 14, 2023 07:16
@khromov khromov mentioned this pull request Aug 14, 2023
4 tasks
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
…the UI (ggml-org#2588)

* server : implement json-schema-to-grammar.mjs by follow python impl

* server : add grammar support in chat.mjs

* server : implement grammer param in the UI

* server : generate .hpp

* server : remove trailing whitespaces

* server : generate .hpp

* server : fix sort of prop pairs

* server : optimize regex & iteration
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
…the UI (ggml-org#2588)

* server : implement json-schema-to-grammar.mjs by follow python impl

* server : add grammar support in chat.mjs

* server : implement grammer param in the UI

* server : generate .hpp

* server : remove trailing whitespaces

* server : generate .hpp

* server : fix sort of prop pairs

* server : optimize regex & iteration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants