Experimenting with Claude skills #97
Conversation
|
This is great - we should break out skills though so it's not baked into the Claude agent - which is how we test baseline / unaugmented Claude performance. We should have some mechanism to test by agent with / without skills. |
|
@joellabes @jasnonaz - @clkao had an earlier PR for a concept of plugins that I believe incorporated skills. I don't have strong opinions about these, though I do bias a bit towards more general concepts so that they could be extended to other types of products (eg, if gemini adds something like skills, etc). It might be worth looking at that implementation and seeing if there are generalizations for this: #42 |
|
Oh yeah for sure! Would happy adopt @clkao's setup instead - I'll just keep noodling with the skill text in the meantime |
|
Cool, @clkao , you have any thoughts on this? |
I keep seeing the LLMs make dumb assumptions about dbt's behaviour, so wanted to try using Skills to give it extra context.
In theory, claude automatically recognises when a skill would be appropriate. In practice, it doesn't unless you yell at it 🤦. So we shouldn't merge this as-is.