Add a way to use Control Vectors

[minipasila]

Describe the Issue
Since Control Vectors were implemented in llama.cpp a few months ago there still doesn't seem to be any way to actually use them in Koboldcpp. It would be nice if there was a way to use them with Koboldcpp.

Additional Information:
Example readme on how to make Control Vectors.

[dazzling2019]

Seconded, that would be awesome.

[shooflyshoo]

Any headway on this issue? Thanks!

[LostRuins]

Not currently, to be honest I'm not really sure what's the point of this - what can it do that cannot be achieved with a system prompt?

[dazzling2019]

The main ones are:

You can make a model (much) more "extreme" than any system prompting could ever do.
You can apply the control vector all throughout the context so it can't start to ignore it and/or deviate back to its "stock" behaviour.
You can use with models like gemma-2 that have no system prompt in their template.
But you can also:

Mix and match different control vectors using vector addition to have much more refined control over the output.
You can "de-bias" a model to set it to the point which is equidistant between two ends of the most extreme versions of what it can output via system promoting - this would be very hard if not impossible using words alone I think.
and lastly you can keep the system prompt free for specifying other stuff that isn't easily represented as a single negative <---> positive axis.

[LostRuins]

There's a misconception about how the system prompt works. You don't need to "keep it free". You can use as many system prompts as you like, and you can use system prompts on any model. Don't let anyone tell you otherwise. Just try it out on the included KoboldAI Lite UI. You can also place system prompts in the "Memory" segment.