Relaxing dependencies to work with hf smolagents

[galleon]

Fix #164
Tests are running fine

[ManuelFay]

As is we can't merge, there was a training discrepancy with numpy 2, and we still want to bound transformers version etc.
Best is probably that Tony or I take care of it.
Pretty sure PEFT also is a bit different although we did change what we were using !
This requires a bit more extensive testing for sure though

[galleon]

I agree that more tests are probably needed, but in my opinion, capping versions often causes more problems than it solves. I’d like to fine-tune my own version of ColQwen2—would that be acceptable as part of the additional tests? Of course, I’m also fine with Tony or yourself handling the issue if that works better.

[tonywu71]

Duplicate of #166.