This seems simple, but gets complicated real fast.
After thinking about this and starting to implement it, my two cents are that the complications introduced by this feature by creating non-contiguous blocks of selection might not be worth the benefit of having it. Then again, if we don't implement ctrl-clicking we'll probably get some issues opened requesting it. Perhaps we just work on it for a later milestone?
Currently you can not unselect an item by clicking on it
Proposed behavior: ctrl-click will select an item if it's not selected, and unselect it if it is
Note that this can create non-contiguous blocks of selections. On OS X applications this also resets the tail for future selections, and things can get a little complicated depending on what the user does next.

After a cmd-click, the item below the one that was clicked is set as the new tail for the next selection.
If the user shift-clicks below, then the lines between the tail and the clicked line get selected.
If the user shift-clicks above, then the lines between the tail and the clicked line get selected, AND the lines below the tail get unselected
For comparison, this is what tree-view currently does.
Clicking below item that was unselected:

Clicking above item that was unselected

It seems to always choose the top-most selected item as the tail.

For comparison, Finder resets the tail whenever the user clicks a new item

Other behavior we would have to consider implementing:
Clicking into a block: selects line between item and tail and unselects others in the block

Joining blocks: head jumps to border of selection
