Add general compression for large block arrays

[westonpace]

Currently block compression is limited to bitpacking. This is mainly because the only things we block compress are dictionaries (which are usually small-ish) and repdef levels (which are usually highly bitpackable).

Adding general compression should be a quick win to shore up our support when we have large dictionary arrays.

[lyang24]

give my attempt - in my mind the hardest part is how to integrate with current create_block_compressor call.

currently i had if compression scheme is lz4 or zstd we compress the non bitpackable data - not sure if its ideal.

[Xuanwo]

Has this issue been resolved?

[westonpace]

Yes, I think we have general block compression for dictionaries now.