Quick start with working code in readme

[byjlw]

It would be awesome if we could have a quick start runnable example in the main readme.
Nothing major but start to finish on how to get a model wire it to the react framework and run it to interact with the model.
Ideally less than 5 steps.

[chmjkb]

Hi,
In the v0.2.0 release we'll add support for direct ExecuTorch bindings. This will allow users to load any model as long as it's exported to .pte. When that comes out, we'll add a quick runnable example that shows how to use the API. For now, the useLLM hook is described in the docs.

[nicoache1]

Hi @chmjkb
Are you planning to add support for other uses cases like super resolution or image transfer?
Cause right now the useLLM only allows a text in the generate function right?

[chmjkb]

Hi @nicoache1 ,
We are currently working on adding computer vision models to the library. A detailed description of what's planned is explained in the roadmap. Currently, we only have a text generation model, but that will change in v0.2.0. Also, are there any specific types of models you would like us to support? Thanks for reaching out!

[KDLPro]

Hi @chmjkb , I would love to have support for computer vision models since most are either outdated, or have shit READMEs.

[chmjkb]

Hi @KDLPro,
Thanks for your interest in the library. We are working on bringing CV models in the v0.2.0 release. If there are any particular models you need, please let us know!

[ajmalp5]

Hey @chmjkb , can we able to use this model .gguf format?

https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-IQ3_M.gguf

[chmjkb]

Hi @ajmalp5,
We are not planning to support .gguf models. GGUF is meant to be used in the GGML runtime, and our library is meant to support ExecuTorch models. Therefore, it is only possible to run .pte models. Since you pointed to LLaMA 3.2 1B - that model is available in our library. Also, is there any particular reason you're asking about GGUF? I'm curious about the use case :)
Thank you for checking out our work!

[ajmalp5]

@chmjkb , I used this package to my ongoing project, So they sharing .gguf format models. I can see this package only accept .pte format.

how i can convert https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/blob/main/Llama-3.2-1B-Instruct-IQ3_M.gguf this to .pte format?

This model hase 600mb, So i need this model to use?

[chmjkb]

@ajmalp5, The model that I've sent to you (this one) is basically the same one (LLaMa 3.2 1B Instruct) that you're sending me so I don't really see an advantage of converting the GGUF one to PTE. The only difference i'd say is that the model from our library uses different quantization technique (which is really good by the way). I can also see that the GGUF file is smaller than the PTE, but there is not much we can do about that for now.