Skip to content

MNT: updates to HF Hub, manage fairlearn deprecation #392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adrinjalali merged 6 commits into from
Oct 11, 2023
Merged

MNT: updates to HF Hub, manage fairlearn deprecation #392

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2233253
Ignore FutureWarning stemming from fairlearn
BenjaminBossan Oct 6, 2023
af04046
Make the widget attribute an array
BenjaminBossan Oct 6, 2023
d19871b
InferenceApi is deprecated move to InferenceClient
BenjaminBossan Oct 6, 2023
f2a131a
For testing, increase min hfhub version to 0.18
BenjaminBossan Oct 6, 2023
e004671
[CI inference] Empty commit with inference tests
BenjaminBossan Oct 9, 2023
e957c2c
Change min dependency of HF Hub to 0.17
BenjaminBossan Oct 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ filterwarnings = [
# https://github.com/scikit-learn/scikit-learn/pull/23633
"ignore:Unlike other reduction functions:FutureWarning",
# https://github.com/scikit-learn/scikit-learn/pull/25157
"ignore:\\w+ is deprecated. Use files\\(\\) instead:DeprecationWarning"
"ignore:\\w+ is deprecated. Use files\\(\\) instead:DeprecationWarning",
# comes from fairlearn
"ignore:DataFrame.applymap has been deprecated. Use DataFrame.map instead:FutureWarning",
]
markers = [
"network: marks tests as requiring internet (deselect with '-m \"not network\"')",
Expand Down
2 changes: 1 addition & 1 deletion skops/_min_dependencies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
dependent_packages = {
"scikit-learn": ("0.24", "install", None),
"scikit-learn-intelex": ("2021.7.1", "docs", None),
"huggingface_hub": ("0.10.1", "install", None),
"huggingface_hub": ("0.17.0", "install", None),
"tabulate": ("0.8.8", "install", None),
"quantile-forest": ("1.0.0", "tests", None),
"pytest": (PYTEST_MIN_VERSION, "tests", None),
Expand Down
4 changes: 2 additions & 2 deletions skops/card/_model_card.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ def metadata_from_config(config_path: Union[str, Path]) -> ModelCardData:
# https://huggingface.co/docs/hub/models-widgets-examples
if example_input:
if "tabular" in task:
card_data.widget = {"structuredData": example_input} # type: ignore
card_data.widget = [{"structuredData": example_input}] # type: ignore
# TODO: add text data example here.

return card_data
Expand Down Expand Up @@ -1396,7 +1396,7 @@ def __repr__(self) -> str:
metadata_reprs = []
for key, val in self.metadata.to_dict().items() if self.metadata else {}:
if key == "widget":
metadata_reprs.append("metadata.widget={...},")
metadata_reprs.append("metadata.widget=[{...}],")
continue

metadata_reprs.append(self._format_repr(f"metadata.{key}={val},"))
Expand Down
22 changes: 12 additions & 10 deletions skops/card/tests/test_card.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1186,14 +1186,16 @@ def test_metadata_from_config_tabular_data(
metadata = metadata_load(local_path=Path(destination_path) / "README.md")
assert "widget" in metadata

expected_data = {
"structuredData": {
"petal length (cm)": [1.4, 1.4, 1.3],
"petal width (cm)": [0.2, 0.2, 0.2],
"sepal length (cm)": [5.1, 4.9, 4.7],
"sepal width (cm)": [3.5, 3.0, 3.2],
}
}
expected_data = [
{
"structuredData": {
"petal length (cm)": [1.4, 1.4, 1.3],
"petal width (cm)": [0.2, 0.2, 0.2],
"sepal length (cm)": [5.1, 4.9, 4.7],
"sepal width (cm)": [3.5, 3.0, 3.2],
}
},
]
assert metadata["widget"] == expected_data

for tag in ["sklearn", "skops", "tabular-classification"]:
Expand Down Expand Up @@ -1403,7 +1405,7 @@ def test_with_metadata(self, card: Card, meth, expected_lines):
library_name="sklearn",
tags=["sklearn", "tabular-classification"],
foo={"bar": 123},
widget={"something": "very-long"},
widget=[{"something": "very-long"}],
)
card.metadata = metadata

Expand All @@ -1414,7 +1416,7 @@ def test_with_metadata(self, card: Card, meth, expected_lines):
" metadata.library_name=sklearn,",
" metadata.tags=['sklearn', 'tabular-classification'],",
" metadata.foo={'bar': 123},",
" metadata.widget={...},",
" metadata.widget=[{...}],",
]
expected = "\n".join(expected_lines[:2] + extra_lines + expected_lines[2:])

Expand Down
8 changes: 4 additions & 4 deletions skops/hub_utils/_hf_hub.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from typing import Any, List, Literal, MutableMapping, Optional, Sequence, Union

import numpy as np
from huggingface_hub import HfApi, InferenceApi, snapshot_download
from huggingface_hub import HfApi, InferenceClient, snapshot_download
from sklearn.utils import check_array

SUPPORTED_TASKS = [
Expand Down Expand Up @@ -755,9 +755,9 @@ def get_model_output(repo_id: str, data: Any, token: Optional[str] = None) -> An
inputs = {f"x{i}": data[:, i] for i in range(data.shape[1])}
inputs = {"data": inputs}

res = InferenceApi(repo_id=repo_id, task=model_info.pipeline_tag, token=token)(
inputs=inputs
)
client = InferenceClient(token=token)
res_bytes = client.post(json={"inputs": inputs}, model=repo_id)
res = json.loads(res_bytes.decode("utf-8"))
Copy link
Copy Markdown
Member

@adrinjalali adrinjalali Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we have a test for this?

Copy link
Copy Markdown
Collaborator Author

@BenjaminBossan BenjaminBossan Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I forgot that we require [CI inference] in the commit message to trigger it. This is done now but of course we run into the rate limit again...

Copy link
Copy Markdown
Member

@adrinjalali adrinjalali Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove the functionality then since the rate limit is so low?

Copy link
Copy Markdown
Collaborator Author

@BenjaminBossan BenjaminBossan Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's mostly an issue for our CI, right? Which is probably why we made the test opt-in in the first place. I'd say we leave it, in case some users actually use the feature. If we want to remove it later on, we can do it properly with a deprecation cycle.

Copy link
Copy Markdown
Collaborator Author

@BenjaminBossan BenjaminBossan Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also add a random sleep between 1 and 100 hours to avoid hitting the limit ;-P

Copy link
Copy Markdown
Member

@adrinjalali adrinjalali Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, not happy with a functionality that even our CI hits the limits pretty much every time. But we can merge and figure that out in a separate PR.


if isinstance(res, list):
return np.array(res)
Expand Down