Skip to content

XGBoost - Switch to dask API #466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
karlhigley merged 14 commits into from
Jun 13, 2022
Merged

XGBoost - Switch to dask API #466

karlhigley merged 14 commits into from
Jun 13, 2022

Conversation

@oliverholworthy
Copy link
Contributor

@oliverholworthy oliverholworthy commented May 27, 2022 •
edited
Loading

Goals ⚽

Since the Merlin Datasets use dask natively. Using the XGBoost Dask API allows us to pass the underlying dask dataframe without needing to call .compute on the dataframe.

Implementation Details 🚧

  • Refactored fit method. Moving the optional target_columns and qid_column to the constructor.
  • Added new required argument schema to the constructor
  • Added booster argument so that a class can be constructed from a loaded booster. predict can then be called without needing to call fit first.

Testing Details 🔍

  • Added a test for the case where a dask client is not initialised. We get a ValueError in this case.

Example Usage

from merlin.datasets.entertainment import get_movielens
from merlin.core.utils import Distributed
from merlin.models.xgb import XGBoost

train, valid = get_movielens(variant="ml-1m")

with Distributed():
    model = XGBoost(train.schema, objective="binary:logistic")
    model.fit(train)
    metrics = model.evaluate(valid)

With a ranking objective

from merlin.datasets.entertainment import get_movielens
from merlin.core.utils import Distributed
from merlin.models.xgb import XGBoost

train, valid = get_movielens(variant="ml-1m")

with Distributed():
    model = XGBoost(train.schema, target_columns=["rating"], qid_column="userId", objective="rank:ndcg", eval_metric="ndcg")
    model.fit(train)
    metrics = model.evaluate(valid)

@github-actions
Copy link

github-actions bot commented May 27, 2022

Documentation preview

https://nvidia-merlin.github.io/models/review/pr-466

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented May 27, 2022

Click to view CI Results
GitHub pull request #466 of commit 8bc925ab28eb827cc8d1dc96aaed170dc8cfeb11, no merge conflicts.
Running as SYSTEM
Setting status of 8bc925ab28eb827cc8d1dc96aaed170dc8cfeb11 to PENDING with url https://10.20.13.93:8080/job/merlin_models/353/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/466/*:refs/remotes/origin/pr/466/* # timeout=10
 > git rev-parse 8bc925ab28eb827cc8d1dc96aaed170dc8cfeb11^{commit} # timeout=10
Checking out Revision 8bc925ab28eb827cc8d1dc96aaed170dc8cfeb11 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 8bc925ab28eb827cc8d1dc96aaed170dc8cfeb11 # timeout=10
Commit message: "Setup dask_client fixture to use CPU"
 > git rev-list --no-walk 238a3ae48d7198c23cd388336161cd0e925467b6 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins10498418461716840195.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /var/jenkins_home/.local/lib/python3.8/site-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.2.1.post0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0; python_version < "3.9"->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 430 items / 3 skipped

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py .........                                     [  7%]

tests/unit/tf/test_dataset.py ...............                            [ 11%]

tests/unit/tf/test_public_api.py .                                       [ 11%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 14%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 16%]

tests/unit/tf/blocks/test_interactions.py .                              [ 16%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 23%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 25%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 25%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_transformations.py ..F........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 30%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 33%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 33%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 34%]

tests/unit/tf/features/test_continuous.py .....                          [ 36%]

tests/unit/tf/features/test_embedding.py ..............                  [ 39%]

tests/unit/tf/features/test_tabular.py .......                           [ 40%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 50%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py .......                                [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ..............                      [ 60%]

tests/unit/tf/models/test_retrieval.py ..................                [ 64%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 70%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 76%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 80%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 87%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 90%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]


=================================== FAILURES ===================================

_______________________ test_stochastic_swap_noise[0.5] ________________________


replacement_prob = 0.5


@pytest.mark.parametrize("replacement_prob", [0.1, 0.3, 0.5, 0.7])
def test_stochastic_swap_noise(replacement_prob):
    NUM_SEQS = 100
    SEQ_LENGTH = 80
    PAD_TOKEN = 0

    # Creating some input sequences with padding in the end
    # (to emulate sessions with different lengths)
    seq_inputs = {
        "categ_seq_feat": tf.experimental.numpy.tril(
            tf.random.uniform((NUM_SEQS, SEQ_LENGTH), minval=1, maxval=100, dtype=tf.int32), 1
        ),
        "cont_seq_feat": tf.experimental.numpy.tril(tf.random.uniform((NUM_SEQS, SEQ_LENGTH)), 1),
        "categ_feat": tf.random.uniform((NUM_SEQS,), minval=1, maxval=100, dtype=tf.int32),
    }

    tf.random.set_seed(0)
    ssn = ml.StochasticSwapNoise(pad_token=PAD_TOKEN, replacement_prob=replacement_prob)
    mask = seq_inputs["categ_seq_feat"] != PAD_TOKEN
    out_features_ssn = ssn(seq_inputs, input_mask=mask, training=True)

    for fname in seq_inputs:
        replaced_mask = out_features_ssn[fname] != seq_inputs[fname]
        replaced_mask_non_padded = tf.boolean_mask(replaced_mask, seq_inputs[fname] != PAD_TOKEN)
        replacement_rate = tf.reduce_mean(
            tf.cast(replaced_mask_non_padded, dtype=tf.float32)
        ).numpy()





      assert replacement_rate == pytest.approx(replacement_prob, abs=0.15)





E           assert 0.32 == 0.5 ± 1.5e-01

E             comparison failed

E             Obtained: 0.3199999928474426

E             Expected: 0.5 ± 1.5e-01


tests/unit/tf/blocks/core/test_transformations.py:54: AssertionError

=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/tf/blocks/core/test_index.py: 7 warnings

tests/unit/tf/models/test_retrieval.py: 147 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 100 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/autograph_generated_fileyw469w8n.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/features/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/features/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:210.)

return {key: torch.tensor(value) for key, value in data.items()}


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

===== 1 failed, 422 passed, 10 skipped, 288 warnings in 1000.91s (0:16:40) =====

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins8902634672852926279.sh

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented May 27, 2022

Click to view CI Results
GitHub pull request #466 of commit 43820c265c34c5b889098895180cd6f3a3ba9fbe, no merge conflicts.
Running as SYSTEM
Setting status of 43820c265c34c5b889098895180cd6f3a3ba9fbe to PENDING with url https://10.20.13.93:8080/job/merlin_models/354/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/466/*:refs/remotes/origin/pr/466/* # timeout=10
 > git rev-parse 43820c265c34c5b889098895180cd6f3a3ba9fbe^{commit} # timeout=10
Checking out Revision 43820c265c34c5b889098895180cd6f3a3ba9fbe (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 43820c265c34c5b889098895180cd6f3a3ba9fbe # timeout=10
Commit message: "Use cluster_type parameter instead of new_cluster"
 > git rev-list --no-walk 8bc925ab28eb827cc8d1dc96aaed170dc8cfeb11 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins10234576530499459618.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /var/jenkins_home/.local/lib/python3.8/site-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.2.1.post0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.5.1)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.1)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.0.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0; python_version < "3.9"->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 430 items / 3 skipped

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py .........                                     [  7%]

tests/unit/tf/test_dataset.py ...............                            [ 11%]

tests/unit/tf/test_public_api.py .                                       [ 11%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 14%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 16%]

tests/unit/tf/blocks/test_interactions.py .                              [ 16%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 23%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 25%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 25%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 30%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 33%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 33%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 34%]

tests/unit/tf/features/test_continuous.py .....                          [ 36%]

tests/unit/tf/features/test_embedding.py ..............                  [ 39%]

tests/unit/tf/features/test_tabular.py .......                           [ 40%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 50%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py .......                                [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ..............                      [ 60%]

tests/unit/tf/models/test_retrieval.py ..................                [ 64%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 70%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 76%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 80%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 87%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 90%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]


=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/tf/blocks/core/test_index.py: 7 warnings

tests/unit/tf/models/test_retrieval.py: 147 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 100 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/autograph_generated_filex13s4ppw.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/features/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/features/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:210.)

return {key: torch.tensor(value) for key, value in data.items()}


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

========== 423 passed, 10 skipped, 288 warnings in 1006.10s (0:16:46) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins6702196319897970314.sh

oliverholworthy
oliverholworthy commented May 27, 2022
View reviewed changes
@oliverholworthy oliverholworthy marked this pull request as ready for review May 30, 2022 09:31
@oliverholworthy oliverholworthy self-assigned this May 30, 2022
@oliverholworthy oliverholworthy mentioned this pull request Jun 1, 2022
Merged
@oliverholworthy oliverholworthy added the enhancement New feature or request label Jun 8, 2022
@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Jun 8, 2022

Click to view CI Results
GitHub pull request #466 of commit 04d578ef228cf0deaf7b5f0e87345da97866f5b3, no merge conflicts.
Running as SYSTEM
Setting status of 04d578ef228cf0deaf7b5f0e87345da97866f5b3 to PENDING with url https://10.20.13.93:8080/job/merlin_models/405/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/466/*:refs/remotes/origin/pr/466/* # timeout=10
 > git rev-parse 04d578ef228cf0deaf7b5f0e87345da97866f5b3^{commit} # timeout=10
Checking out Revision 04d578ef228cf0deaf7b5f0e87345da97866f5b3 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 04d578ef228cf0deaf7b5f0e87345da97866f5b3 # timeout=10
Commit message: "Merge branch 'main' into xgb-dask"
 > git rev-list --no-walk f803a9fbe2d1e6f78e33a0c96effe769f099b60f # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins15496597499402303802.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /var/jenkins_home/.local/lib/python3.8/site-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.2.2.post1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.6.0)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.3)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.1.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0; python_version < "3.9"->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 435 items / 3 skipped

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ......                                        [  7%]

tests/unit/tf/test_dataset.py ...............                            [ 10%]

tests/unit/tf/test_public_api.py .                                       [ 10%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 13%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 15%]

tests/unit/tf/blocks/test_interactions.py .                              [ 15%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 22%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 24%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 24%]

tests/unit/tf/blocks/core/test_combinators.py ...                        [ 25%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 25%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_tabular.py ...                            [ 28%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 31%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 34%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 34%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 35%]

tests/unit/tf/inputs/test_continuous.py .....                            [ 36%]

tests/unit/tf/inputs/test_embedding.py ..............                    [ 39%]

tests/unit/tf/inputs/test_tabular.py .......                             [ 41%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 50%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py .......                                [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ................                    [ 60%]

tests/unit/tf/models/test_retrieval.py ..................                [ 64%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 65%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 88%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 91%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]


=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/tf/blocks/core/test_index.py: 7 warnings

tests/unit/tf/models/test_retrieval.py: 147 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 100 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/autograph_generated_filekq64pg5w.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:210.)

return {key: torch.tensor(value) for key, value in data.items()}


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

========== 428 passed, 10 skipped, 288 warnings in 1069.48s (0:17:49) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins11626702307956709003.sh

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Jun 8, 2022

Click to view CI Results
GitHub pull request #466 of commit eb163426a4b6808237ca6f4485545dfde4e52f65, no merge conflicts.
Running as SYSTEM
Setting status of eb163426a4b6808237ca6f4485545dfde4e52f65 to PENDING with url https://10.20.13.93:8080/job/merlin_models/406/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/466/*:refs/remotes/origin/pr/466/* # timeout=10
 > git rev-parse eb163426a4b6808237ca6f4485545dfde4e52f65^{commit} # timeout=10
Checking out Revision eb163426a4b6808237ca6f4485545dfde4e52f65 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f eb163426a4b6808237ca6f4485545dfde4e52f65 # timeout=10
Commit message: "Merge branch 'main' into xgb-dask"
 > git rev-list --no-walk 04d578ef228cf0deaf7b5f0e87345da97866f5b3 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins10889167517331971122.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /var/jenkins_home/.local/lib/python3.8/site-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.2.2.post1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.6.0)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.3)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.1.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0; python_version < "3.9"->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 434 items / 3 skipped

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ......                                        [  7%]

tests/unit/tf/test_dataset.py ...............                            [ 10%]

tests/unit/tf/test_public_api.py .                                       [ 10%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 13%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 15%]

tests/unit/tf/blocks/test_interactions.py .                              [ 15%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 22%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 24%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 24%]

tests/unit/tf/blocks/core/test_combinators.py ...                        [ 25%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 26%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_tabular.py ...                            [ 28%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 31%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 34%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 34%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 34%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 35%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 35%]

tests/unit/tf/inputs/test_continuous.py .....                            [ 36%]

tests/unit/tf/inputs/test_embedding.py ..............                    [ 39%]

tests/unit/tf/inputs/test_tabular.py .......                             [ 41%]

tests/unit/tf/layers/test_queue.py ..............                        [ 44%]

tests/unit/tf/losses/test_losses.py .......................              [ 49%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 50%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 54%]

tests/unit/tf/models/test_base.py .......                                [ 56%]

tests/unit/tf/models/test_benchmark.py ..                                [ 56%]

tests/unit/tf/models/test_ranking.py ................                    [ 60%]

tests/unit/tf/models/test_retrieval.py .................                 [ 64%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 64%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 66%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 71%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 88%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 91%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]


=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/tf/blocks/core/test_index.py: 7 warnings

tests/unit/tf/models/test_retrieval.py: 143 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 100 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/autograph_generated_filesg260flf.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:210.)

return {key: torch.tensor(value) for key, value in data.items()}


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

========== 427 passed, 10 skipped, 284 warnings in 1059.48s (0:17:39) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins2608469838436429459.sh

@rnyak
Copy link
Contributor

rnyak commented Jun 8, 2022

rerun tests

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Jun 8, 2022

Click to view CI Results
GitHub pull request #466 of commit dfd44966c2717a5554860323fa4b1248d63a5f21, no merge conflicts.
Running as SYSTEM
Setting status of dfd44966c2717a5554860323fa4b1248d63a5f21 to PENDING with url https://10.20.13.93:8080/job/merlin_models/421/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/466/*:refs/remotes/origin/pr/466/* # timeout=10
 > git rev-parse dfd44966c2717a5554860323fa4b1248d63a5f21^{commit} # timeout=10
Checking out Revision dfd44966c2717a5554860323fa4b1248d63a5f21 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f dfd44966c2717a5554860323fa4b1248d63a5f21 # timeout=10
Commit message: "Merge branch 'main' into xgb-dask"
 > git rev-list --no-walk 189f90b11e79c92496b56797273102b5f5306800 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins5042486095531753891.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /var/jenkins_home/.local/lib/python3.8/site-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.2.2.post1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.6.0)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.3)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.1.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0; python_version < "3.9"->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 444 items / 3 skipped

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ......                                        [  6%]

tests/unit/tf/test_dataset.py ...............                            [ 10%]

tests/unit/tf/test_public_api.py .                                       [ 10%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 13%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 14%]

tests/unit/tf/blocks/test_interactions.py .                              [ 15%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 21%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 23%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 24%]

tests/unit/tf/blocks/core/test_combinators.py ...                        [ 24%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 25%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_tabular.py ...                            [ 27%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 30%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 33%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 33%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 33%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 34%]

tests/unit/tf/inputs/test_continuous.py .....                            [ 35%]

tests/unit/tf/inputs/test_embedding.py ..............                    [ 38%]

tests/unit/tf/inputs/test_tabular.py .......                             [ 40%]

tests/unit/tf/layers/test_queue.py ..............                        [ 43%]

tests/unit/tf/losses/test_losses.py .......................              [ 48%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 49%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 53%]

tests/unit/tf/models/test_base.py .......                                [ 55%]

tests/unit/tf/models/test_benchmark.py ..                                [ 55%]

tests/unit/tf/models/test_ranking.py ................                    [ 59%]

tests/unit/tf/models/test_retrieval.py ...........................       [ 65%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 65%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 67%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 72%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 81%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 88%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 91%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]


=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/tf/blocks/core/test_index.py: 7 warnings

tests/unit/tf/models/test_retrieval.py: 272 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 100 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/autograph_generated_file8cqad373.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:210.)

return {key: torch.tensor(value) for key, value in data.items()}


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

========== 437 passed, 10 skipped, 413 warnings in 1198.88s (0:19:58) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins1374814386544346955.sh

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Jun 8, 2022

Click to view CI Results
GitHub pull request #466 of commit dfd44966c2717a5554860323fa4b1248d63a5f21, no merge conflicts.
Running as SYSTEM
Setting status of dfd44966c2717a5554860323fa4b1248d63a5f21 to PENDING with url https://10.20.13.93:8080/job/merlin_models/424/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/466/*:refs/remotes/origin/pr/466/* # timeout=10
 > git rev-parse dfd44966c2717a5554860323fa4b1248d63a5f21^{commit} # timeout=10
Checking out Revision dfd44966c2717a5554860323fa4b1248d63a5f21 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f dfd44966c2717a5554860323fa4b1248d63a5f21 # timeout=10
Commit message: "Merge branch 'main' into xgb-dask"
 > git rev-list --no-walk dee03ed0ed90b983c593a62b94d7b196390e03ab # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins8289760381729782404.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /var/jenkins_home/.local/lib/python3.8/site-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.2.2.post1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.6.0)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.3)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.1.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0; python_version < "3.9"->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 444 items / 3 skipped

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py .Fsss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ......                                        [  6%]

tests/unit/tf/test_dataset.py ...............                            [ 10%]

tests/unit/tf/test_public_api.py .                                       [ 10%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 13%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 14%]

tests/unit/tf/blocks/test_interactions.py .                              [ 15%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 21%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 23%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 24%]

tests/unit/tf/blocks/core/test_combinators.py ...                        [ 24%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 25%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_tabular.py ...                            [ 27%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 30%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 33%]

tests/unit/tf/examples/test_02_dataschema.py F                           [ 33%]

tests/unit/tf/examples/test_03_exploring_different_models.py F           [ 33%]

tests/unit/tf/examples/test_04_export_ranking_models.py F                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py F               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 34%]

tests/unit/tf/inputs/test_continuous.py .....                            [ 35%]

tests/unit/tf/inputs/test_embedding.py ..............                    [ 38%]

tests/unit/tf/inputs/test_tabular.py .......                             [ 40%]

tests/unit/tf/layers/test_queue.py ..............                        [ 43%]

tests/unit/tf/losses/test_losses.py .......................              [ 48%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 49%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 53%]

tests/unit/tf/models/test_base.py .......                                [ 55%]

tests/unit/tf/models/test_benchmark.py ..                                [ 55%]

tests/unit/tf/models/test_ranking.py ................                    [ 59%]

tests/unit/tf/models/test_retrieval.py ...........................       [ 65%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 65%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 67%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 72%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 81%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 88%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 91%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]


=================================== FAILURES ===================================

________________________ test_synthetic_aliccp_raw_data ________________________


tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-23/test_synthetic_aliccp_raw_data0')


def test_synthetic_aliccp_raw_data(tmp_path):
    dataset = generate_data("aliccp-raw", 100)

    assert isinstance(dataset, merlin.io.Dataset)
    assert dataset.num_rows == 100
    assert len(dataset.schema) == 25
    assert sorted(dataset.to_ddf().compute().columns) == [
        "click",
        "conversion",
        "item_brand",
        "item_category",
        "item_id",
        "item_intention",
        "item_shop",
        "position",
        "user_age",
        "user_brands",
        "user_categories",
        "user_consumption_1",
        "user_consumption_2",
        "user_gender",
        "user_geography",
        "user_group",
        "user_id",
        "user_intentions",
        "user_is_occupied",
        "user_item_brands",
        "user_item_categories",
        "user_item_intentions",
        "user_item_shops",
        "user_profile",
        "user_shops",
    ]





  ecommerce.transform_aliccp((dataset, dataset), tmp_path)





tests/unit/datasets/test_ecommerce.py:56:




merlin/datasets/ecommerce/aliccp/dataset.py:253: in transform_aliccp

nvt_workflow = nvt_workflow or default_aliccp_transformation(**locals())




add_target_encoding = True

kwargs = {'data': (<merlin.io.dataset.Dataset object at 0x7f183533bc10>, <merlin.io.dataset.Dataset object at 0x7f183533bc10>),...{}, 'nvt_workflow': None, 'output_path': PosixPath('/tmp/pytest-of-jenkins/pytest-23/test_synthetic_aliccp_raw_data0')}


def default_aliccp_transformation(add_target_encoding=True, **kwargs):





  import nvtabular as nvt





E       ModuleNotFoundError: No module named 'nvtabular'


merlin/datasets/ecommerce/aliccp/dataset.py:177: ModuleNotFoundError

_______________________ test_example_02_nvt_integration ________________________


tb = <testbook.client.TestbookNotebookClient object at 0x7f18357fa1f0>


@testbook(REPO_ROOT / "examples/02-Merlin-Models-and-NVTabular-integration.ipynb", execute=False)
def test_example_02_nvt_integration(tb):
    tb.inject(
        """
        import os
        os.environ["INPUT_DATA_DIR"] = "/tmp/data/"
        from unittest.mock import patch
        from merlin.datasets.synthetic import generate_data
        mock_train, mock_valid = generate_data(
            input="movielens-1m",
            num_rows=1000,
            set_sizes=(0.8, 0.2)
        )
        p1 = patch(
            "merlin.datasets.entertainment.get_movielens",
            return_value=[mock_train, mock_valid]
        )
        p1.start()
        p2 = patch(
            "merlin.core.utils.download_file",
            return_value=[]
        )
        p2.start()
        import numpy as np
        import pandas
        from pathlib import Path
        from merlin.datasets.synthetic import generate_data
        mock_data = generate_data(
            input="movielens-1m-raw-ratings",
            num_rows=1000
        )
        mock_data = mock_data.to_ddf().compute()
        if not isinstance(mock_data, pandas.core.frame.DataFrame):
            mock_data = mock_data.to_pandas()
        input_path = os.environ.get(
            "INPUT_DATA_DIR",
            os.path.expanduser("~/merlin-models-data/movielens/")
        )
        path = Path(input_path + "ml-1m/")
        path.mkdir(parents=True, exist_ok=True)
        np.savetxt(
            input_path + 'ml-1m/ratings.dat',
            mock_data.values,
            delimiter='::',
            fmt='%s',
            encoding='utf-8'
        )
        """
    )





  tb.execute()





tests/unit/tf/examples/test_02_dataschema.py:55:




../../../.local/lib/python3.8/site-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)




self = <testbook.client.TestbookNotebookClient object at 0x7f18357fa1f0>

cell = {'cell_type': 'code', 'execution_count': 3, 'id': '157fe18f', 'metadata': {'execution': {'iopub.status.busy': '2022-06...ls import download_file\nfrom merlin.datasets.entertainment import get_movielens\nfrom merlin.schema.tags import Tags'}

cell_index = 2

exec_reply = {'buffers': [], 'content': {'ename': 'ModuleNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'ac3042f6-...e, 'engine': 'ac3042f6-7f5b-49da-b08a-92b36843d0dd', 'started': '2022-06-08T17:46:50.187970Z', 'status': 'error'}, ...}


async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:





      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)





E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           import os

E           import pandas as pd

E           import nvtabular as nvt

E           from merlin.models.utils.example_utils import workflow_fit_transform

E           import merlin.io

E

E           import merlin.models.tf as mm

E

E           from nvtabular import ops

E           from merlin.core.utils import download_file

E           from merlin.datasets.entertainment import get_movielens

E           from merlin.schema.tags import Tags

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mModuleNotFoundError�[0m                       Traceback (most recent call last)

E           Input �[0;32mIn [3]�[0m, in �[0;36m<cell line: 3>�[0;34m()�[0m

E           �[1;32m      1�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mos�[39;00m

E           �[1;32m      2�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mpandas�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mpd�[39;00m

E           �[0;32m----> 3�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mnvtabular�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mnvt�[39;00m

E           �[1;32m      4�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mmerlin�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mmodels�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mutils�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mexample_utils�[39;00m �[38;5;28;01mimport�[39;00m workflow_fit_transform

E           �[1;32m      5�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mmerlin�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mio�[39;00m

E

E           �[0;31mModuleNotFoundError�[0m: No module named 'nvtabular'

E           ModuleNotFoundError: No module named 'nvtabular'


../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

__________________ test_example_03_exploring_different_models __________________


self = <testbook.client.TestbookNotebookClient object at 0x7f1835163670>

cell = {'cell_type': 'markdown', 'id': '56ecea23', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': "Let's start with importing the libraries that we'll use in this notebook."}

kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]

executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...ycharm': {'name': '#%% md\n'}}, 'source': "Let's start with importing the libraries that we'll use in this notebook."}]

idx = 4


def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
    """
    Executes a cell or list of cells
    """
    if isinstance(cell, slice):
        start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
        if cell.step is not None:
            raise TestbookError('testbook does not support step argument')

        cell = range(start, stop + 1)
    elif isinstance(cell, str) or isinstance(cell, int):
        cell = [cell]

    cell_indexes = cell

    if all(isinstance(x, str) for x in cell):
        cell_indexes = [self._cell_index(tag) for tag in cell]

    executed_cells = []
    for idx in cell_indexes:
        try:





          cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)





../../../.local/lib/python3.8/site-packages/testbook/client.py:133:




args = (<testbook.client.TestbookNotebookClient object at 0x7f1835163670>, {'cell_type': 'code', 'execution_count': 3, 'outpu...cute_reply': '2022-06-08T17:46:52.924331Z', 'iopub.status.idle': '2022-06-08T17:46:52.925626Z'}}, 'id': 'b23d3c31'}, 4)

kwargs = {}


def wrapped(*args, **kwargs):





  return just_run(coro(*args, **kwargs))





../../../.local/lib/python3.8/site-packages/nbclient/util.py:84:




coro = <coroutine object NotebookClient.async_execute_cell at 0x7f17be02ddc0>


def just_run(coro: Awaitable) -> Any:
    """Make the coroutine run, even if there is an event loop running (using nest_asyncio)"""
    # original from vaex/asyncio.py
    loop = asyncio._get_running_loop()
    if loop is None:
        had_running_loop = False
        try:
            loop = asyncio.get_event_loop()
        except RuntimeError:
            # we can still get 'There is no current event loop in ...'
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
    else:
        had_running_loop = True
    if had_running_loop:
        # if there is a running loop, we patch using nest_asyncio
        # to have reentrant event loops
        check_ipython()
        import nest_asyncio

        nest_asyncio.apply()
        check_patch_tornado()





  return loop.run_until_complete(coro)





../../../.local/lib/python3.8/site-packages/nbclient/util.py:62:




self = <_UnixSelectorEventLoop running=False closed=False debug=False>

future = <Task finished name='Task-104' coro=<NotebookClient.async_execute_cell() done, defined at /var/jenkins_home/.local/lib...x1b[0;31mModuleNotFoundError\x1b[0m: No module named 'nvtabular'\nModuleNotFoundError: No module named 'nvtabular'\n")>


def run_until_complete(self, future):
    """Run until the Future is done.

    If the argument is a coroutine, it is wrapped in a Task.

    WARNING: It would be disastrous to call run_until_complete()
    with the same coroutine twice -- it would wrap it in two
    different Tasks and that can't be good.

    Return the Future's result, or raise its exception.
    """
    self._check_closed()
    self._check_running()

    new_task = not futures.isfuture(future)
    future = tasks.ensure_future(future, loop=self)
    if new_task:
        # An exception is raised if the future didn't complete, so there
        # is no need to log the "destroy pending task" message
        future._log_destroy_pending = False

    future.add_done_callback(_run_until_complete_cb)
    try:
        self.run_forever()
    except:
        if new_task and future.done() and not future.cancelled():
            # The coroutine raised a BaseException. Consume the exception
            # to not log a warning, the caller doesn't have access to the
            # local task.
            future.exception()
        raise
    finally:
        future.remove_done_callback(_run_until_complete_cb)
    if not future.done():
        raise RuntimeError('Event loop stopped before Future completed.')





  return future.result()





/usr/lib/python3.8/asyncio/base_events.py:616:




self = <testbook.client.TestbookNotebookClient object at 0x7f1835163670>

cell = {'cell_type': 'code', 'execution_count': 3, 'outputs': [{'output_type': 'error', 'ename': 'ModuleNotFoundError', 'eval....execute_reply': '2022-06-08T17:46:52.924331Z', 'iopub.status.idle': '2022-06-08T17:46:52.925626Z'}}, 'id': 'b23d3c31'}

cell_index = 4, execution_count = None, store_history = True


async def async_execute_cell(
    self,
    cell: NotebookNode,
    cell_index: int,
    execution_count: t.Optional[int] = None,
    store_history: bool = True,
) -> NotebookNode:
    """
    Executes a single code cell.

    To execute all cells see :meth:`execute`.

    Parameters
    ----------
    cell : nbformat.NotebookNode
        The cell which is currently being processed.
    cell_index : int
        The position of the cell within the notebook object.
    execution_count : int
        The execution count to be assigned to the cell (default: Use kernel response)
    store_history : bool
        Determines if history should be stored in the kernel (default: False).
        Specific to ipython kernels, which can store command histories.

    Returns
    -------
    output : dict
        The execution output payload (or None for no output).

    Raises
    ------
    CellExecutionError
        If execution failed and should raise an exception, this will be raised
        with defaults about the failure.

    Returns
    -------
    cell : NotebookNode
        The cell which was just processed.
    """
    assert self.kc is not None

    await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)

    if cell.cell_type != 'code' or not cell.source.strip():
        self.log.debug("Skipping non-executing cell %s", cell_index)
        return cell

    if self.skip_cells_with_tag in cell.metadata.get("tags", []):
        self.log.debug("Skipping tagged cell %s", cell_index)
        return cell

    if self.record_timing:  # clear execution metadata prior to execution
        cell['metadata']['execution'] = {}

    self.log.debug("Executing cell:\n%s", cell.source)

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
    )

    await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
    parent_msg_id = await ensure_async(
        self.kc.execute(
            cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
        )
    )
    await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
    # We launched a code cell to execute
    self.code_cells_executed += 1
    exec_timeout = self._get_timeout(cell)

    cell.outputs = []
    self.clear_before_next_output = False

    task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
    task_poll_output_msg = asyncio.ensure_future(
        self._async_poll_output_msg(parent_msg_id, cell, cell_index)
    )
    self.task_poll_for_reply = asyncio.ensure_future(
        self._async_poll_for_reply(
            parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
        )
    )
    try:
        exec_reply = await self.task_poll_for_reply
    except asyncio.CancelledError:
        # can only be cancelled by task_poll_kernel_alive when the kernel is dead
        task_poll_output_msg.cancel()
        raise DeadKernelError("Kernel died")
    except Exception as e:
        # Best effort to cancel request if it hasn't been resolved
        try:
            # Check if the task_poll_output is doing the raising for us
            if not isinstance(e, CellControlSignal):
                task_poll_output_msg.cancel()
        finally:
            raise

    if execution_count:
        cell['execution_count'] = execution_count





  await self._check_raise_for_error(cell, cell_index, exec_reply)





../../../.local/lib/python3.8/site-packages/nbclient/client.py:965:




self = <testbook.client.TestbookNotebookClient object at 0x7f1835163670>

cell = {'cell_type': 'code', 'execution_count': 3, 'outputs': [{'output_type': 'error', 'ename': 'ModuleNotFoundError', 'eval....execute_reply': '2022-06-08T17:46:52.924331Z', 'iopub.status.idle': '2022-06-08T17:46:52.925626Z'}}, 'id': 'b23d3c31'}

cell_index = 4

exec_reply = {'buffers': [], 'content': {'ename': 'ModuleNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': '688687a2-...e, 'engine': '688687a2-cc97-4d29-be56-2480a9c1baf7', 'started': '2022-06-08T17:46:52.442866Z', 'status': 'error'}, ...}


async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:





      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)





E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           import os

E           import numpy as np

E

E           from nvtabular.loader.tf_utils import configure_tensorflow

E

E           configure_tensorflow()

E

E           import nvtabular as nvt

E           from nvtabular.ops import *

E           from merlin.models.utils.example_utils import workflow_fit_transform, save_results

E

E           from merlin.schema.tags import Tags

E

E           import merlin.models.tf as mm

E           from merlin.io.dataset import Dataset

E

E           import tensorflow as tf

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mModuleNotFoundError�[0m                       Traceback (most recent call last)

E           Input �[0;32mIn [3]�[0m, in �[0;36m<cell line: 4>�[0;34m()�[0m

E           �[1;32m      1�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mos�[39;00m

E           �[1;32m      2�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mnumpy�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mnp�[39;00m

E           �[0;32m----> 4�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mnvtabular�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mloader�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mtf_utils�[39;00m �[38;5;28;01mimport�[39;00m configure_tensorflow

E           �[1;32m      6�[0m configure_tensorflow()

E           �[1;32m      8�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mnvtabular�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mnvt�[39;00m

E

E           �[0;31mModuleNotFoundError�[0m: No module named 'nvtabular'

E           ModuleNotFoundError: No module named 'nvtabular'


../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError


During handling of the above exception, another exception occurred:


tb = <testbook.client.TestbookNotebookClient object at 0x7f1835163670>


@testbook(REPO_ROOT / "examples/03-Exploring-different-models.ipynb", execute=False)
def test_example_03_exploring_different_models(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/tmp/data/"
        os.environ["NUM_ROWS"] = "999"
        """
    )
    NUM_OF_CELLS = len(tb.cells)





  tb.execute_cell(list(range(0, NUM_OF_CELLS - 5)))





tests/unit/tf/examples/test_03_exploring_different_models.py:18:




self = <testbook.client.TestbookNotebookClient object at 0x7f1835163670>

cell = {'cell_type': 'markdown', 'id': '56ecea23', 'metadata': {'pycharm': {'name': '#%% md\n'}}, 'source': "Let's start with importing the libraries that we'll use in this notebook."}

kwargs = {}, cell_indexes = [0, 1, 2, 3, 4, 5, ...]

executed_cells = [{'cell_type': 'code', 'execution_count': 2, 'id': '5f49a48e', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution':...ycharm': {'name': '#%% md\n'}}, 'source': "Let's start with importing the libraries that we'll use in this notebook."}]

idx = 4


def execute_cell(self, cell, **kwargs) -> Union[Dict, List[Dict]]:
    """
    Executes a cell or list of cells
    """
    if isinstance(cell, slice):
        start, stop = self._cell_index(cell.start), self._cell_index(cell.stop)
        if cell.step is not None:
            raise TestbookError('testbook does not support step argument')

        cell = range(start, stop + 1)
    elif isinstance(cell, str) or isinstance(cell, int):
        cell = [cell]

    cell_indexes = cell

    if all(isinstance(x, str) for x in cell):
        cell_indexes = [self._cell_index(tag) for tag in cell]

    executed_cells = []
    for idx in cell_indexes:
        try:
            cell = super().execute_cell(self.nb['cells'][idx], idx, **kwargs)
        except CellExecutionError as ce:





          raise TestbookRuntimeError(ce.evalue, ce, self._get_error_class(ce.ename))





E               testbook.exceptions.TestbookRuntimeError: An error occurred while executing the following cell:

E               ------------------

E               import os

E               import numpy as np

E

E               from nvtabular.loader.tf_utils import configure_tensorflow

E

E               configure_tensorflow()

E

E               import nvtabular as nvt

E               from nvtabular.ops import *

E               from merlin.models.utils.example_utils import workflow_fit_transform, save_results

E

E               from merlin.schema.tags import Tags

E

E               import merlin.models.tf as mm

E               from merlin.io.dataset import Dataset

E

E               import tensorflow as tf

E               ------------------

E

E               �[0;31m---------------------------------------------------------------------------�[0m

E               �[0;31mModuleNotFoundError�[0m                       Traceback (most recent call last)

E               Input �[0;32mIn [3]�[0m, in �[0;36m<cell line: 4>�[0;34m()�[0m

E               �[1;32m      1�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mos�[39;00m

E               �[1;32m      2�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mnumpy�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mnp�[39;00m

E               �[0;32m----> 4�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mnvtabular�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mloader�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mtf_utils�[39;00m �[38;5;28;01mimport�[39;00m configure_tensorflow

E               �[1;32m      6�[0m configure_tensorflow()

E               �[1;32m      8�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mnvtabular�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mnvt�[39;00m

E

E               �[0;31mModuleNotFoundError�[0m: No module named 'nvtabular'

E               ModuleNotFoundError: No module named 'nvtabular'


../../../.local/lib/python3.8/site-packages/testbook/client.py:135: TestbookRuntimeError

___________________ test_example_04_exporting_ranking_models ___________________


tb = <testbook.client.TestbookNotebookClient object at 0x7f1835106b80>


@testbook(REPO_ROOT / "examples/04-Exporting-ranking-models.ipynb", execute=False)
def test_example_04_exporting_ranking_models(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/tmp/data/"
        os.environ["NUM_ROWS"] = "999"
        """
    )





  tb.execute()





tests/unit/tf/examples/test_04_export_ranking_models.py:17:




../../../.local/lib/python3.8/site-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)




self = <testbook.client.TestbookNotebookClient object at 0x7f1835106b80>

cell = {'cell_type': 'code', 'execution_count': 3, 'id': '37d5020c', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution': ...ema.tags import Tags\n\nimport merlin.models.tf as mm\nfrom merlin.io.dataset import Dataset\nimport tensorflow as tf'}

cell_index = 4

exec_reply = {'buffers': [], 'content': {'ename': 'ModuleNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'c7bb6175-...e, 'engine': 'c7bb6175-d30e-4615-bc91-cd8710b5e65b', 'started': '2022-06-08T17:46:54.353125Z', 'status': 'error'}, ...}


async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:





      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)





E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           import os

E

E           import nvtabular as nvt

E           from nvtabular.ops import *

E

E           from merlin.models.utils.example_utils import workflow_fit_transform

E           from merlin.schema.tags import Tags

E

E           import merlin.models.tf as mm

E           from merlin.io.dataset import Dataset

E           import tensorflow as tf

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mModuleNotFoundError�[0m                       Traceback (most recent call last)

E           Input �[0;32mIn [3]�[0m, in �[0;36m<cell line: 3>�[0;34m()�[0m

E           �[1;32m      1�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mos�[39;00m

E           �[0;32m----> 3�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mnvtabular�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mnvt�[39;00m

E           �[1;32m      4�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mnvtabular�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mops�[39;00m �[38;5;28;01mimport�[39;00m �[38;5;241m*�[39m

E           �[1;32m      6�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mmerlin�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mmodels�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mutils�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mexample_utils�[39;00m �[38;5;28;01mimport�[39;00m workflow_fit_transform

E

E           �[0;31mModuleNotFoundError�[0m: No module named 'nvtabular'

E           ModuleNotFoundError: No module named 'nvtabular'


../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

_______________________ test_example_05_retrieval_models _______________________


tb = <testbook.client.TestbookNotebookClient object at 0x7f1835119280>


@testbook(REPO_ROOT / "examples/05-Retrieval-Model.ipynb", execute=False)
def test_example_05_retrieval_models(tb):
    tb.inject(
        """
        import os
        os.environ["DATA_FOLDER"] = "/tmp/data/"
        os.environ["NUM_ROWS"] = "999"
        """
    )





  tb.execute()





tests/unit/tf/examples/test_05_export_retrieval_model.py:17:




../../../.local/lib/python3.8/site-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)




self = <testbook.client.TestbookNotebookClient object at 0x7f1835119280>

cell = {'cell_type': 'code', 'execution_count': 3, 'id': '92aa8daa', 'metadata': {'pycharm': {'name': '#%%\n'}, 'execution': ...a.tags import Tags\n\nimport merlin.models.tf as mm\nfrom merlin.io.dataset import Dataset\n\nimport tensorflow as tf'}

cell_index = 4

exec_reply = {'buffers': [], 'content': {'ename': 'ModuleNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'd7d67afa-...e, 'engine': 'd7d67afa-15c0-4926-8873-06bed691d3df', 'started': '2022-06-08T17:46:56.117275Z', 'status': 'error'}, ...}


async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:





      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)





E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           import os

E

E           import nvtabular as nvt

E           from nvtabular.ops import *

E           from merlin.models.utils.example_utils import workflow_fit_transform

E

E           from merlin.schema.tags import Tags

E

E           import merlin.models.tf as mm

E           from merlin.io.dataset import Dataset

E

E           import tensorflow as tf

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mModuleNotFoundError�[0m                       Traceback (most recent call last)

E           Input �[0;32mIn [3]�[0m, in �[0;36m<cell line: 3>�[0;34m()�[0m

E           �[1;32m      1�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mos�[39;00m

E           �[0;32m----> 3�[0m �[38;5;28;01mimport�[39;00m �[38;5;21;01mnvtabular�[39;00m �[38;5;28;01mas�[39;00m �[38;5;21;01mnvt�[39;00m

E           �[1;32m      4�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mnvtabular�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mops�[39;00m �[38;5;28;01mimport�[39;00m �[38;5;241m*�[39m

E           �[1;32m      5�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mmerlin�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mmodels�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mutils�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01mexample_utils�[39;00m �[38;5;28;01mimport�[39;00m workflow_fit_transform

E

E           �[0;31mModuleNotFoundError�[0m: No module named 'nvtabular'

E           ModuleNotFoundError: No module named 'nvtabular'


../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../../../usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41

/usr/local/lib/python3.8/dist-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/tf/blocks/core/test_index.py: 7 warnings

tests/unit/tf/models/test_retrieval.py: 272 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 100 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/autograph_generated_file5zmayo6y.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:210.)

return {key: torch.tensor(value) for key, value in data.items()}


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

===== 5 failed, 432 passed, 10 skipped, 412 warnings in 1100.31s (0:18:20) =====

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins3079825408113550607.sh

@rnyak
Copy link
Contributor

rnyak commented Jun 13, 2022

rerun tests

oliverholworthy
oliverholworthy commented Jun 13, 2022
View reviewed changes
merlin/models/xgb/__init__.py
qid_column: Optional[str],
):
"""Convert Merlin Dataset to XGBoost DMatrix"""
df = dataset.to_ddf().compute()
Copy link
Contributor Author

@oliverholworthy oliverholworthy Jun 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching to use dask, avoids the need to call compute on the dataset here

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Jun 13, 2022

Click to view CI Results
GitHub pull request #466 of commit 1da5a45935d3fee133a5df43786f67010fa958b0, no merge conflicts.
Running as SYSTEM
Setting status of 1da5a45935d3fee133a5df43786f67010fa958b0 to PENDING with url https://10.20.13.93:8080/job/merlin_models/455/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/466/*:refs/remotes/origin/pr/466/* # timeout=10
 > git rev-parse 1da5a45935d3fee133a5df43786f67010fa958b0^{commit} # timeout=10
Checking out Revision 1da5a45935d3fee133a5df43786f67010fa958b0 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1da5a45935d3fee133a5df43786f67010fa958b0 # timeout=10
Commit message: "Merge branch 'main' into xgb-dask"
 > git rev-list --no-walk d6ae967f371adcfbd05579225de78f8c62dbc9b5 # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins5401366996627524788.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /var/jenkins_home/.local/lib/python3.8/site-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nbclient>=0.4.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from testbook) (0.5.13)
Requirement already satisfied: traitlets>=5.1 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (5.2.2.post1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.6.0)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.4)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: importlib-resources>=1.4.0; python_version < "3.9" in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (23.1.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: zipp>=3.1.0; python_version < "3.10" in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0; python_version < "3.9"->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 442 items / 3 skipped

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  1%]

tests/unit/datasets/test_ecommerce.py ..sss                              [  2%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  4%]

tests/unit/datasets/test_social.py .                                     [  4%]

tests/unit/datasets/test_synthetic.py .....                              [  5%]

tests/unit/tf/test_core.py ......                                        [  7%]

tests/unit/tf/test_dataset.py ...............                            [ 10%]

tests/unit/tf/test_public_api.py .                                       [ 10%]

tests/unit/tf/blocks/test_cross.py ...........                           [ 13%]

tests/unit/tf/blocks/test_dlrm.py ........                               [ 14%]

tests/unit/tf/blocks/test_interactions.py .                              [ 15%]

tests/unit/tf/blocks/test_mlp.py .............................           [ 21%]

tests/unit/tf/blocks/core/test_aggregation.py .........                  [ 23%]

tests/unit/tf/blocks/core/test_base.py ..                                [ 24%]

tests/unit/tf/blocks/core/test_combinators.py ...                        [ 24%]

tests/unit/tf/blocks/core/test_index.py ...                              [ 25%]

tests/unit/tf/blocks/core/test_masking.py .......                        [ 27%]

tests/unit/tf/blocks/core/test_tabular.py ...                            [ 27%]

tests/unit/tf/blocks/core/test_transformations.py ...........            [ 30%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 30%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ...........             [ 33%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 33%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 33%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 33%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 34%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 34%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 34%]

tests/unit/tf/inputs/test_continuous.py .....                            [ 35%]

tests/unit/tf/inputs/test_embedding.py ..............                    [ 38%]

tests/unit/tf/inputs/test_tabular.py .......                             [ 40%]

tests/unit/tf/layers/test_queue.py ..............                        [ 43%]

tests/unit/tf/losses/test_losses.py .......................              [ 48%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 50%]

tests/unit/tf/metrics/test_metrics_ranking.py .................          [ 53%]

tests/unit/tf/models/test_base.py .....                                  [ 54%]

tests/unit/tf/models/test_benchmark.py ..                                [ 55%]

tests/unit/tf/models/test_ranking.py ................                    [ 59%]

tests/unit/tf/models/test_retrieval.py ...........................       [ 65%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 65%]

tests/unit/tf/prediction_tasks/test_multi_task.py .......                [ 67%]

tests/unit/tf/prediction_tasks/test_next_item.py ....................    [ 71%]

tests/unit/tf/prediction_tasks/test_regression.py ..                     [ 72%]

tests/unit/tf/prediction_tasks/test_sampling.py ....................     [ 76%]

tests/unit/tf/utils/test_batch.py ....                                   [ 77%]

tests/unit/torch/test_dataset.py .........                               [ 79%]

tests/unit/torch/test_public_api.py .                                    [ 79%]

tests/unit/torch/block/test_base.py ....                                 [ 80%]

tests/unit/torch/block/test_mlp.py .                                     [ 80%]

tests/unit/torch/features/test_continuous.py ..                          [ 81%]

tests/unit/torch/features/test_embedding.py ..............               [ 84%]

tests/unit/torch/features/test_tabular.py ....                           [ 85%]

tests/unit/torch/model/test_head.py ............                         [ 88%]

tests/unit/torch/model/test_model.py ..                                  [ 88%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 90%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 91%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 92%]

tests/unit/utils/test_schema_utils.py ................................   [100%]


=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/unit/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/unit/tf/test_dataset.py::test_tf_catname_ordering

tests/unit/tf/test_dataset.py::test_tf_map

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/tf/blocks/core/test_index.py: 7 warnings

tests/unit/tf/models/test_retrieval.py: 272 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 100 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

/tmp/autograph_generated_fileccn5g4s6.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:321: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[True]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

tests/unit/tf/models/test_ranking.py::test_dlrm_model_multi_task[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/numpy/core/numeric.py:2453: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.

return bool(asarray(a1 == a2).all())


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:210.)

return {key: torch.tensor(value) for key, value in data.items()}


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

SKIPPED [1] tests/unit/implicit/init.py:18: could not import 'implicit': No module named 'implicit'

SKIPPED [1] tests/unit/lightfm/init.py:18: could not import 'lightfm': No module named 'lightfm'

SKIPPED [1] tests/unit/xgb/init.py:19: could not import 'xgboost': No module named 'xgboost'

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:62: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:78: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:92: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

========== 435 passed, 10 skipped, 413 warnings in 1132.64s (0:18:52) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins4839035008864506636.sh

@karlhigley karlhigley merged commit 62628d6 into NVIDIA-Merlin:main Jun 13, 2022
@oliverholworthy oliverholworthy linked an issue Jun 15, 2022 that may be closed by this pull request
XGBoost Integration #132
Closed
@oliverholworthy oliverholworthy mentioned this pull request Jun 22, 2022
Closed
4 tasks
mengyao00 pushed a commit that referenced this pull request Jul 15, 2022
@oliverholworthy @karlhigley
@rnyak @benfred @mengyao00
XGBoost - Switch to dask API (#466)
a0ba34f 
* Use raise-from exception chaining for objective ValueError

* Add dask_client fixture to conftest

* Remove unncessary else block in get_targets

* Add schema to XGBoost constructor and move fit parameters.

Moving target_columns and qid_column from fit method to __init__.
This makes it clearer the relationship between these attributes and the schema.

* Switch to using dask API for XGBoost. Requiring use of dask client.

* Remove dask_cudf import and typehints

* Setup dask_client fixture to use CPU

* Use cluster_type parameter instead of new_cluster

Co-authored-by: Karl Higley <kmhigley@gmail.com>
Co-authored-by: rnyak <ronayak@hotmail.com>
Co-authored-by: Ben Frederickson <github@benfrederickson.com>
@oliverholworthy oliverholworthy mentioned this pull request Sep 22, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rnyak rnyak rnyak left review comments

@benfred benfred benfred approved these changes

Assignees

@oliverholworthy oliverholworthy

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

XGBoost Integration

5 participants

@oliverholworthy @nvidia-merlin-bot @rnyak @benfred @karlhigley