Structural Overhaul

[danforbes]

Replaces #141. Includes code from #85 and #125.

• GGML is now a submodule and a single crate
• Support for GPT-2 architecture
• Structural refactor into llm, llm-base, model, and llm-cli crates

[philpax]

Aha, I was just about to reply to the BLOOM PR!

I would suggest keeping BLOOM in - there's no reason to remove that code if someone can fix it with some work (we'll make an issue for it) - but we'll remove it from the list of features, and make a note on the crate itself.

As for the rest, let me have a look...

[philpax]

I'm thinking we'll remove this wording as we grow to accommodate more LLMs. I'll revise the wording on this after this PR lands, so nothing for you to do here - just mentioning it.

[danforbes]

I don't see the value of keeping non-working code in the main branch. I would suggest keeping BLOOM in a feature branch until it's working.

[hhamud]

btw could you add me as a git commit author rather than a mention as you did when you upstreamed the ggml changes. @danforbes

[philpax]

I don't see the value of keeping non-working code in the main branch. I would suggest keeping BLOOM in a feature branch until it's working.

Code that isn't kept in the main branch tends to rot, especially if we start making larger changes. As far as I know, BLOOM is done, we just need to figure out what's causing it to fail generation.

It's much easier for someone to pick it up and fix it if it's present; I think it's also a good way to ensure our abstractions work as expected as we increase the number of models.

---

Apart from that, PR looks great! Well done :) Looking forward to merging this, I think it's a huge improvement.

[hhamud]

I don't see the value of keeping non-working code in the main branch. I would suggest keeping BLOOM in a feature branch until it's working.

Code that isn't kept in the main branch tends to rot, especially if we start making larger changes. As far as I know, BLOOM is done, we just need to figure out what's causing it to fail generation.

It's much easier for someone to pick it up and fix it if it's present; I think it's also a good way to ensure our abstractions work as expected as we increase the number of models.

Apart from that, PR looks great! Well done :) Looking forward to merging this, I think it's a huge improvement.

I personally think we should just merge and place a comment on the issue of it not working. I'd pick it up again but I don't have time for it till next week.

[danforbes]

I've added back the BLOOM crate and tagged @hhamud as a co-author ✅

[philpax]

All that's left are the formatting and restoring bindings generation. Are you OK for me to do that, or do you have changes in the works?

[danforbes]

Please go for it - thank you so much 👯