Skip to content

Make loading models easier #359

@pweglik

Description

@pweglik

Right now we need to pass all 3 paths/urls, but they are already premade for most of models configurations, so instead:

 const llama = useLLM({
    modelSource: LLAMA3_2_1B,
    tokenizerSource: LLAMA3_2_TOKENIZER,
    tokenizerConfigSource: LLAMA3_2_TOKENIZER_CONFIG,
  });

we could have:

 const llama = useLLM({
    model: LLAMA3_2_1B,
  });

We should introduce it as a non-breaking change, make all attributes optional and add checks that either model or the previous trio needs to be passed

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions