Skip to content

Ignore tokens if their IDs in added token list are below the vocab's base size #3585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
seungduk-yanolja wants to merge 5 commits into from

move back vocab_size where it was

ff12b8f
Select commit
Loading
Failed to load commit list.
Closed

Ignore tokens if their IDs in added token list are below the vocab's base size #3585

move back vocab_size where it was
ff12b8f
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs