From 918ebfeb8c19589a3d198fc3dc4fe94e8916d8ac Mon Sep 17 00:00:00 2001 From: Benjamin Bossan Date: Fri, 3 Feb 2023 13:44:55 +0100 Subject: [PATCH 1/4] [skip ci] Document compatibility across versions Resolves #270 Add a section to the persistence docs that cautions about using skops to load sklearn models saved with one version while using another version. Refers to sklearn docs on that issue. --- docs/persistence.rst | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/persistence.rst b/docs/persistence.rst index 473b9b6f..7627054d 100644 --- a/docs/persistence.rst +++ b/docs/persistence.rst @@ -160,8 +160,29 @@ Therefore, you should only load a skops file containing a model of any of those libraries if you trust them to be secure. It's not a perfect solution, but it's still better than trusting pickle files, which anyone can tamper with easily. +Compatibility across sklearn versions +------------------------------------- + +Using skops to load a model saved in one sklearn version and loading it with +sklearn version is not recommended, because the behavior of the model may change +across versions. Such changes don't happen very often, but they can happen, thus +you should be cautious. To replicate a model trained with one sklearn version +using a different sklearn version, it is advised to retrain the model on the +same data using the same training process. + +The potential compatibility issue between sklearn versions is not skops +specific. It is general sklearn behavior which skops cannot avoid. According to +the sklearn `docs on model persistence +`_: + + While models saved using one version of scikit-learn might load in other + versions, this is entirely unsupported and inadvisable. It should also be + kept in mind that operations performed on such data could give different and + unexpected results. + Roadmap ------- + There needs to be more testing to harden the loader and make sure we don't run arbitrary code when it's not intended. However, the safety mechanisms already in place should prevent most cases of abuse. From ffabe2fbf78c9402bf6c2c5f1240b83f321a5f53 Mon Sep 17 00:00:00 2001 From: Benjamin Bossan Date: Mon, 27 Feb 2023 17:55:21 +0100 Subject: [PATCH 2/4] Update docs/persistence.rst Co-authored-by: Adrin Jalali --- docs/persistence.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/persistence.rst b/docs/persistence.rst index 7627054d..770c45ef 100644 --- a/docs/persistence.rst +++ b/docs/persistence.rst @@ -165,7 +165,8 @@ Compatibility across sklearn versions Using skops to load a model saved in one sklearn version and loading it with sklearn version is not recommended, because the behavior of the model may change -across versions. Such changes don't happen very often, but they can happen, thus +across versions. In some cases loading the model in a different version might not be possible due to internal changes in scikit-learn. +Such changes don't happen very often, but they can happen, thus you should be cautious. To replicate a model trained with one sklearn version using a different sklearn version, it is advised to retrain the model on the same data using the same training process. From 0d19773e54e3fda33697f26ce3b9f470be85142e Mon Sep 17 00:00:00 2001 From: Benjamin Bossan Date: Mon, 27 Feb 2023 17:55:27 +0100 Subject: [PATCH 3/4] Update docs/persistence.rst Co-authored-by: Adrin Jalali --- docs/persistence.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/persistence.rst b/docs/persistence.rst index 770c45ef..e5674f84 100644 --- a/docs/persistence.rst +++ b/docs/persistence.rst @@ -164,7 +164,7 @@ Compatibility across sklearn versions ------------------------------------- Using skops to load a model saved in one sklearn version and loading it with -sklearn version is not recommended, because the behavior of the model may change +another sklearn version is not recommended, because the behavior of the model may change across versions. In some cases loading the model in a different version might not be possible due to internal changes in scikit-learn. Such changes don't happen very often, but they can happen, thus you should be cautious. To replicate a model trained with one sklearn version From 3d63001704d4fb4791737bd66ffc60a45004a1d8 Mon Sep 17 00:00:00 2001 From: adrinjalali Date: Tue, 28 Feb 2023 12:07:29 +0100 Subject: [PATCH 4/4] wrap lines --- docs/persistence.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/persistence.rst b/docs/persistence.rst index e5674f84..341b8c74 100644 --- a/docs/persistence.rst +++ b/docs/persistence.rst @@ -164,12 +164,13 @@ Compatibility across sklearn versions ------------------------------------- Using skops to load a model saved in one sklearn version and loading it with -another sklearn version is not recommended, because the behavior of the model may change -across versions. In some cases loading the model in a different version might not be possible due to internal changes in scikit-learn. -Such changes don't happen very often, but they can happen, thus -you should be cautious. To replicate a model trained with one sklearn version -using a different sklearn version, it is advised to retrain the model on the -same data using the same training process. +another sklearn version is not recommended, because the behavior of the model +may change across versions. In some cases loading the model in a different +version might not be possible due to internal changes in scikit-learn. Such +changes don't happen very often, but they can happen, thus you should be +cautious. To replicate a model trained with one sklearn version using a +different sklearn version, it is advised to retrain the model on the same data +using the same training process. The potential compatibility issue between sklearn versions is not skops specific. It is general sklearn behavior which skops cannot avoid. According to