Alternative Card implementation

docs/changes.rst

@@ -21,6 +21,11 @@ v0.4
   :pr:`242` by `Merve Noyan`_.
 - Persistence now supports bytes and bytearrays, added tests to verify that
   LightGBM, XGBoost, and CatBoost work now. :pr:`244` by `Benjamin Bossan`_.
+- :class:`.card.Card` now allows to add content to existing sections, using a
+  ``/`` to separate the subsections. E.g. use ``card.add(**{"Existing
+  section/New section": "content"})`` to add "content" a new subsection called
+  "New section" to an existing section called "Existing section". :pr:`203` by
+  `Benjamin Bossan`_.
 
 v0.3
 ----

docs/model_card.rst

@@ -42,32 +42,30 @@ to touch it yourself.
 
 The markdown part does not necessarily need to follow any specification in
 terms of information passed, which gives the user a lot of flexibility. The
-markdown part of the ``README.md`` file is generated from a Jinja template
-with slots that you can inject your content in. ``skops`` has a default
-template which includes the following slots for free text sections:
-
-- ``"model_description"``: A description of the model.
-- ``"limitations"``: Intended use for the model, limitations and potential
-  biases. This section should also include risks of using models in certain
-  domains if relevant.
-- ``"get_started_code"``: Code the user can run to load and use the model.
-- ``"model_card_authors"``: Authors of the model card. This section includes
-  authors of the model card, while ``"citation_bibtex"`` includes citations
-  related to the model if relevant.
-- ``"model_card_contact"``: Contact information of people whom can be reached
+markdown part of the ``README.md`` file comes with a couple of defaults provided
+by ``skops``, which includes the following slots for free text sections:
+
+- ``"Mode description"``: A description of the model.
+- ``"Intended uses & limitations"``: Intended use for the model, limitations and
+  potential biases. This section should also include risks of using models in
+  certain domains if relevant.
+- ``"How to Get Started with the Model"``: Code the user can run to load and use
+  the model.
+- ``"Model Card Authors"``: Authors of the model card. This section includes
+  authors of the model card
+- ``"Model Card Contact"``: Contact information of people whom can be reached
   out, in case of questions about the model or the model card.
-- ``"citation_bibtex"``: Bibtex style citations for the model or resources used
-  to train the model.
-- ``"eval_methods"``: Details about evaluation process of the model.
-- ``"eval_results"``: Evaluation results that are later parsed as a table by
-  :class:`skops.card.Card`.
+- ``"Citation"``: Bibtex style citations for the model or resources used to
+  train the model.
+- ``"Evaluation Results"``: Evaluation results that are later parsed as a table
+  by :class:`skops.card.Card`.
 
 
 The template also contains the following sections that are automatically
 generated by ``skops``.
 
-- ``"hyperparameter_table"``: Hyperparameters of the model.
-- ``"model_plot"``: A diagram of the model, most relevant in case the model is
+- ``"Hyperparameters"``: Hyperparameters of the model.
+- ``"Model Plot"``: A diagram of the model, most relevant in case the model is
   a complex scikit-learn :class:`~sklearn.pipeline.Pipeline`.
 
 Furthermore, it is possible to add plots and tables to the model card. To add
@@ -77,5 +75,31 @@ dictionaries with the key being the header and the values being list of row
 entries, or a pandas ``DataFrame``; use the :meth:`.Card.add_table` method for
 this.
 
+To add content to an existing subsection, or create a new subsection, use a
+``"/"`` to indicate the subsection. E.g. let's assume you would like to add a
+subsection called ``"Figures"`` to the existing section ``"Model description"``,
+as well as adding some subsections with plots below that, you can call the
+:meth:`Card.add` method like this:
+
+.. code-block:: python
+
+    card.add(**{"Model description/Figures": "Here are some nice figures"})
+    card.add_plot(**{
+        "Model description/Figures/Confusion Matrix": "path-to-confusion-matrix.png",
+        "Model description/Figures/ROC": "path-to-roc.png",
+    })
+
+Furthermore, you can select existing sections (as well as their subsections)
+using :meth:`Card.select`, and you can delete sections using
+:meth:`Card.delete`:
+
+.. code-block:: python
+
+    section = card.select("Model description/Figures")
+    print(section.content)  # 'Here are some nice figures'
+    print(section.subsections)
+    card.delete("Model description/Figures/ROC")
+
+
 To see how you can use the API in ``skops`` to create a model card, please
 refer to :ref:`sphx_glr_auto_examples_plot_model_card.py`.

examples/plot_model_card.py

@@ -29,7 +29,8 @@
 )
 from sklearn.model_selection import HalvingGridSearchCV, train_test_split
 
-from skops import card, hub_utils
+from skops import hub_utils
+from skops.card import Card, metadata_from_config
 
 # %%
 # Data
@@ -91,7 +92,7 @@
 # :func:`.hub_utils.init` above. We will see below how we can populate the model
 # card with useful information.
 
-model_card = card.Card(model, metadata=card.metadata_from_config(Path(local_repo)))
+model_card = Card(model, metadata=metadata_from_config(Path(local_repo)))
 
 # %%
 # Add more information
@@ -103,17 +104,19 @@
 model_card.metadata.license = "mit"
 limitations = "This model is not ready to be used in production."
 model_description = (
-    "This is a HistGradientBoostingClassifier model trained on breast cancer dataset."
-    " It's trained with Halving Grid Search Cross Validation, with parameter grids on"
-    " max_leaf_nodes and max_depth."
+    "This is a `HistGradientBoostingClassifier` model trained on breast cancer "
+    "dataset. It's trained with `HalvingGridSearchCV`, with parameter grids on "
+    "`max_leaf_nodes` and `max_depth`."
 )
 model_card_authors = "skops_user"
-citation_bibtex = "bibtex\n@inproceedings{...,year={2020}}"
+citation_bibtex = "**BibTeX**\n\n```\n@inproceedings{...,year={2020}}\n```"
 model_card.add(
-    citation_bibtex=citation_bibtex,
-    model_card_authors=model_card_authors,
-    limitations=limitations,
-    model_description=model_description,
+    **{
+        "Citation": citation_bibtex,
+        "Model Card Authors": model_card_authors,
+        "Model description": model_description,
+        "Model description/Intended uses & limitations": limitations,
+    }
 )
 
 # %%
@@ -132,10 +135,10 @@
 
 y_pred = model.predict(X_test)
 eval_descr = (
-    "The model is evaluated on test data using accuracy and F1-score with macro"
-    " average."
+    "The model is evaluated on test data using accuracy and F1-score with "
+    "macro average."
 )
-model_card.add(eval_method=eval_descr)
+model_card.add(**{"Model description/Evaluation Results": eval_descr})
 
 accuracy = accuracy_score(y_test, y_pred)
 f1 = f1_score(y_test, y_pred, average="micro")
@@ -146,7 +149,9 @@
 disp.plot()
 
 disp.figure_.savefig(Path(local_repo) / "confusion_matrix.png")
-model_card.add_plot(**{"Confusion matrix": "confusion_matrix.png"})
+model_card.add_plot(
+    **{"Model description/Evaluation Results/Confusion Matrix": "confusion_matrix.png"}
+)
 
 cv_results = model.cv_results_
 clf_report = classification_report(
@@ -160,8 +165,8 @@
 model_card.add_table(
     folded=True,
     **{
-        "Hyperparameter search results": cv_results,
-        "Classification report": clf_report,
+        "Model description/Evaluation Results/Hyperparameter search results": cv_results,
+        "Model description/Evaluation Results/Classification report": clf_report,
     },
 )