Skip to content

feat: generate jsonl #35

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
awxiaoxian2020 merged 1 commit into from
Oct 17, 2023
Merged

feat: generate jsonl #35

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions scripts/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
**/node_modules/*
!mdict/**/node_modules/js-mdict/
*.mdx
*.jsonl
24 changes: 24 additions & 0 deletions scripts/generate-doc/generate_jsonl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import json
from scripts.custom_config import py_config

# 输入的JSON文件名和输出的JSONL文件名
input_json_file = py_config.jsonFile
output_jsonl_file = 'output.jsonl'

# 读取JSON文件
with open(input_json_file, 'r', encoding='utf-8') as json_file:
data = json.load(json_file)

# 获取JSON对象的第一个键，假设您只有一个对象
obj_name = list(data.keys())[0]

# 获取包含对象的列表
obj_list = data.get(obj_name, [])

# 打开JSONL文件以写入
with open(output_jsonl_file, 'w', encoding='utf-8') as jsonl_file:
# 遍历对象列表并将每个对象以JSONL格式写入JSONL文件
for item in obj_list:
jsonl_file.write(json.dumps(item, ensure_ascii=False) + '\n')

print(f"转换完成，已将数据写入 {output_jsonl_file}")