Automatically zipping/pruning `.skops` files.

[koaning]

I've been playing with skops for a bit and I noticed that I'm able to zip the files to get them smaller on disk. The original .skops file was 96.1 Mb (it contains tfidf vectorizer and svd) while the zipped version, handled by MacOS, is 45.1 on disk. That seems like a big difference and I wonder if this might be handled natively.

Related question: I've manually set all the internal numpy arrays to float16. This led to a difference from 238.2 Mb to 96.1 Mb. Might pruning also be a relevant feature for skops?

[adrinjalali]

So the file is already a zip file. What we could do is to expose the compression arg, or maybe all ZipFile args via **kwargs in dump and dumps.

re:pruning, I don't think this should be in the skope for a persistence module. I think the object should be the same when it goes through a dump/load cycle. I think in this particular case it maybe worth making sure this is at least documented in scikit-learn itself?

The DL space does a good job of having tools to prune models and optimize for serving and stuff, I wonder where that should sit in this space.

cc @skops-dev/maintainers

[koaning]

So the file is already a zip file. What we could do is to expose the compression arg, or maybe all ZipFile args via **kwargs in dump and dumps.

Ah, I wasn't aware. But yeah, adding a compression arg and documenting it seems fine!

I think in this particular case it maybe worth making sure this is at least documented in scikit-learn itself?

I was actually contemplating making a small plugin for this actually! My profile always appreciates MOAR scikit-learn plugins. Something like:

from skprune import prune, PrunedPipeline, make_pruned_pipeline

# Take an existing pipeline and prune all float/int arrays.
pruned = prune(pipeline, to="float16")

# Pipelines like normal, but we prune each step before passing
# it on to the next step. Theorically, this might be more numerically 
# stable and it seems like a fun afternoon to confirm this
pruned_pipe = make_pruned_pipeline(TfIdfVectorizer(), LogisticRegression())

[koaning]

Related question. @adrinjalali is this project stable enough for a calmcode video? It seems to have been around for a while now and it would be a nice extra for the scikit-save series.

[adrinjalali]

# Pipelines like normal, but we prune each step before passing
# it on to the next step. Theorically, this might be more numerically 
# stable and it seems like a fun afternoon to confirm this
pruned_pipe = make_pruned_pipeline(TfIdfVectorizer(), LogisticRegression())

@koaning that's a very interesting idea indeed. Curious. And I think it does make sense to have it somewhere.

Related question. @adrinjalali is this project stable enough for a calmcode video? It seems to have been around for a while now and it would be a nice extra for the scikit-save series.

I'd say yes, and I was checking today realizing that quite a bit of our API is actually now stable, and we can start removing some warnings and let people use them now.

[BenjaminBossan]

When it comes to compression, it's probably worth it to check how much possible gain there is. I could imagine that it depends quite a lot on the model whether compression makes a big difference or not. If we see room for decent gains, we could also investigate other compression formats like zstd.

About pruning: Yes, that's a very interesting topic, but better handled outside of skops persistence. Due to the huge variety of sklearn models, my guess would be that it's going to be more difficult than for typical DL models. What I haven't tested at all is whether there could also be any speed gains by, for instance, reducing precision.

we can start removing some warnings and let people use them now.

Indeed, we should update some of the warnings that we added very early in development.

[koaning]

What I haven't tested at all is whether there could also be any speed gains by, for instance, reducing precision.

I ran some quick tests and the timings are exactly the same. So it's more a disk/memory boost.

[koaning]

@adrinjalali super random, but are you at PyCon Berlin now?

If so, this conversation might be a fun lightning talk because ...

... https://github.com/koaning/scikit-prune

[BenjaminBossan]

If so, this conversation might be a fun lightning talk because ...

I think there are no more lightning talks today.

There was, however, a talk today about reducing size of decision tree-based sklearn models:

https://github.com/pavelzw/slim-trees

Most changes they did I feel could be integrated into sklearn, some where destructive like throwing away info about the samples per node.

Maybe we could provide a way in skops to combine different approaches from different libs to prune model size. Also, we should probably measure size of skops files (like we measure runtime performance), skops is going to be larger but I don't know how much.

[koaning]

Maybe we could provide a way in skops to combine different approaches from different libs to prune model size.

For now it might also be fine to keep it as is. Realistically, I just proposed a library that I wrote in a (fun) evening, but there's probably a bunch of edge-cases that I'm blissfully unaware of. It also feels like "pruning" a scikit-learn model won't matter that much in most use-cases. Only when you're dealing with a very large matrix will it actually make a big difference.

[koaning]

I can also see how it might be better for the ecosystem to have lots of small, yet specific, packages that each try to tackle a different sub-problem. My solution might work well for the matrix-heavy components, while my approach isn't going to do anything for tree-based models.

[BenjaminBossan]

Yes, pruning sklearn models would require quite different approaches depending on what type of model it is. As you mentioned, large matrices are one thing, decision trees another. I'm not sure how much of a real world issue the size is for users, compared to popular deep learning models, sklearn models are typically tiny.

It would be a different thing if we saw speedups with prediction times, but unfortunately that doesn't seem to work. For the DT-example, they convert back to original dtypes on unpickling, so runtime should be exactly the same. I do wonder though why lower floating point precision doesn't help. I did a quick test on my machine:

import time
import numpy as np

# double precision
np.random.seed(0)
x = np.random.random((5_000, 5_000))
xt = np.random.random((5_000, 5_000))
tic = time.time()
x.dot(xt)
toc = time.time()
print(toc - tic)  # 1.6213693618774414

# single precision
np.random.seed(0)
x = np.random.random((5_000, 5_000)).astype(np.float32)
xt = np.random.random((5_000, 5_000)).astype(np.float32)
tic = time.time()
x.dot(xt)
toc = time.time()
print(toc - tic)  # 0.7785041332244873

# half precision
np.random.seed(0)
x = np.random.random((5_000, 5_000)).astype(np.float16)
xt = np.random.random((5_000, 5_000)).astype(np.float16)
tic = time.time()
x.dot(xt)
toc = time.time()
print(toc - tic)  # crashes!

However, I'm not sure how well sklearn supports lower precision in general.

[adrinjalali]

In most cases, when data goes to cython space, it gets converted to float64. There's been work to also support/keep float32, but that's not always the case.