Skip to content

convert : fix Norway problem when parsing YAML#12114

Merged
ngxson merged 3 commits intoggml-org:masterfrom
ngxson:xsn/norway_problem
Feb 28, 2025
Merged

convert : fix Norway problem when parsing YAML#12114
ngxson merged 3 commits intoggml-org:masterfrom
ngxson:xsn/norway_problem

Conversation

@ngxson
Copy link
Copy Markdown
Contributor

@ngxson ngxson commented Feb 28, 2025

Related to #12091 (comment)

The problem is:

- no

The YAML above will be interpreted as "yes/no", which is translated into true/false boolean value.

The fix? Do a replace no --> "no"

CC @mofosyne @compilade if you have a better solution

@ngxson ngxson requested a review from ggerganov February 28, 2025 15:31
@github-actions github-actions Bot added the python python script changes label Feb 28, 2025
@ngxson
Copy link
Copy Markdown
Contributor Author

ngxson commented Feb 28, 2025

The problem seems to be fixed on yaml 1.2 specs, but pyyaml doesn't support 1.2: yaml/pyyaml#116

Comment thread gguf-py/gguf/metadata.py Outdated
Comment thread gguf-py/gguf/metadata.py Outdated
Comment on lines +146 to +149
yaml_content += "\n"
yaml_content = yaml_content.replace("- no\n", "- \"no\"\n")

if yaml_content:
Copy link
Copy Markdown
Collaborator

@compilade compilade Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now this if is always true, because of the extra new line.

Maybe the extra \n should be added only when joining the lines?

diff --git a/gguf-py/gguf/metadata.py b/gguf-py/gguf/metadata.py
index 0629306bb..e807f4346 100644
--- a/gguf-py/gguf/metadata.py
+++ b/gguf-py/gguf/metadata.py
@@ -139,11 +139,10 @@ class Metadata:
                     break # End of frontmatter
                 else:
                     lines_yaml.append(line)
-            yaml_content = "\n".join(lines_yaml)
+            yaml_content = "\n".join(lines_yaml) + "\n"
 
         # Quick hack to fix the Norway problem
         # https://hitchdev.com/strictyaml/why/implicit-typing-removed/
-        yaml_content += "\n"
         yaml_content = yaml_content.replace("- no\n", "- \"no\"\n")
 
         if yaml_content:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes thanks, implemented in bd40850

@ngxson ngxson merged commit 06c2b15 into ggml-org:master Feb 28, 2025
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Mar 19, 2025
* convert : fix Norway problem when parsing YAML

* Update gguf-py/gguf/metadata.py

* add newline at correct place
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* convert : fix Norway problem when parsing YAML

* Update gguf-py/gguf/metadata.py

* add newline at correct place
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python python script changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants