Add an Operator for Forest Inference Libary (FIL) Triton backend. #110

oliverholworthy · 2022-05-26T20:10:40Z

Add an Operator for Forest Inference Libary (FIL) Triton backend.

figure out output signature automatically (e.g. classification, + with proba)
allow params to be passed in and take precedence over defaults

karlhigley

Looks good! Left a few minor comments

merlin/systems/dag/ops/fil.py

tests/unit/systems/test_fil.py

.github/workflows/cpu-ci.yml

nvidia-merlin-bot · 2022-05-27T17:46:58Z

Click to view CI Results

GitHub pull request #110 of commit 463af92753d1ffc16a539c253911d652d58f2212, no merge conflicts.
Running as SYSTEM
Setting status of 463af92753d1ffc16a539c253911d652d58f2212 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/54/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse 463af92753d1ffc16a539c253911d652d58f2212^{commit} # timeout=10
Checking out Revision 463af92753d1ffc16a539c253911d652d58f2212 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 463af92753d1ffc16a539c253911d652d58f2212 # timeout=10
Commit message: "Change schema dtype to float32"
 > git rev-list --no-walk 59fd4325f4113a3ac90be3759f5a4021a855d4f0 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins5703828684890627765.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 18 items / 1 error
==================================== ERRORS ====================================

_______________ ERROR collecting tests/unit/systems/test_fil.py ________________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/test_fil.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/test_fil.py:21: in 

import xgboost as xgb

E   ModuleNotFoundError: No module named 'xgboost'

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18

/usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========================== short test summary info ============================

ERROR tests/unit/systems/test_fil.py

!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!

========================= 1 warning, 1 error in 2.68s ==========================

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins7759152048863447843.sh

github-actions · 2022-05-27T17:51:39Z

Documentation preview

https://nvidia-merlin.github.io/systems/review/pr-110

nvidia-merlin-bot · 2022-05-31T14:41:59Z

Click to view CI Results

GitHub pull request #110 of commit 594b22cb25bad63cbc4c97ed5a95de0a1e5d832c, no merge conflicts.
Running as SYSTEM
Setting status of 594b22cb25bad63cbc4c97ed5a95de0a1e5d832c to PENDING with url https://10.20.13.93:8080/job/merlin_systems/56/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse 594b22cb25bad63cbc4c97ed5a95de0a1e5d832c^{commit} # timeout=10
Checking out Revision 594b22cb25bad63cbc4c97ed5a95de0a1e5d832c (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 594b22cb25bad63cbc4c97ed5a95de0a1e5d832c # timeout=10
Commit message: "Move imports for FIL to compat. Update docs"
 > git rev-list --no-walk 933b585ffdffaf3ea777dc8bb07efdff21e21e83 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins6575981776301801970.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 0 items / 1 error
==================================== ERRORS ====================================

________________________ ERROR collecting test session _________________________

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

:1014: in _gcd_import

???

:991: in _find_and_load

???

:975: in _find_and_load_unlocked

???

:671: in _load_unlocked

???

/usr/local/lib/python3.8/dist-packages/_pytest/assertion/rewrite.py:168: in exec_module

exec(co, module.dict)

tests/unit/systems/conftest.py:2: in 

import xgboost as xgb

E   ModuleNotFoundError: No module named 'xgboost'

=========================== short test summary info ============================

ERROR  - ModuleNotFoundError: No module named 'xgboost'

!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!

=============================== 1 error in 0.22s ===============================

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins5530083456866707391.sh

nvidia-merlin-bot · 2022-05-31T15:08:54Z

Click to view CI Results

GitHub pull request #110 of commit 8014a77773347b0320328050937639f9b26431cc, no merge conflicts.
Running as SYSTEM
Setting status of 8014a77773347b0320328050937639f9b26431cc to PENDING with url https://10.20.13.93:8080/job/merlin_systems/57/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse 8014a77773347b0320328050937639f9b26431cc^{commit} # timeout=10
Checking out Revision 8014a77773347b0320328050937639f9b26431cc (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 8014a77773347b0320328050937639f9b26431cc # timeout=10
Commit message: "Add handling for edge case where output dim is zero"
 > git rev-list --no-walk 594b22cb25bad63cbc4c97ed5a95de0a1e5d832c # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins14144870236290845795.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 0 items / 1 error
==================================== ERRORS ====================================

________________________ ERROR collecting test session _________________________

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

:1014: in _gcd_import

???

:991: in _find_and_load

???

:975: in _find_and_load_unlocked

???

:671: in _load_unlocked

???

/usr/local/lib/python3.8/dist-packages/_pytest/assertion/rewrite.py:168: in exec_module

exec(co, module.dict)

tests/unit/systems/conftest.py:2: in 

import xgboost as xgb

E   ModuleNotFoundError: No module named 'xgboost'

=========================== short test summary info ============================

ERROR  - ModuleNotFoundError: No module named 'xgboost'

!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!

=============================== 1 error in 0.23s ===============================

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins12227654162477447954.sh

nvidia-merlin-bot · 2022-05-31T20:15:57Z

Click to view CI Results

GitHub pull request #110 of commit 1bb31c619fdfa32e88203c605f7961197cbacb45, no merge conflicts.
Running as SYSTEM
Setting status of 1bb31c619fdfa32e88203c605f7961197cbacb45 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/58/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse 1bb31c619fdfa32e88203c605f7961197cbacb45^{commit} # timeout=10
Checking out Revision 1bb31c619fdfa32e88203c605f7961197cbacb45 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1bb31c619fdfa32e88203c605f7961197cbacb45 # timeout=10
Commit message: "Remove conftest and add parametrized tests for supported model types"
 > git rev-list --no-walk 8014a77773347b0320328050937639f9b26431cc # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins7274216390433095210.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 18 items / 1 skipped
tests/unit/test_version.py .                                             [  5%]

tests/unit/systems/test_ensemble.py ....                                 [ 27%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 38%]

tests/unit/systems/test_export.py .                                      [ 44%]

tests/unit/systems/test_graph.py .                                       [ 50%]

tests/unit/systems/test_inference_ops.py ..                              [ 61%]

tests/unit/systems/test_op_runner.py ....                                [ 83%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18

/usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

============ 18 passed, 1 skipped, 18 warnings in 62.20s (0:01:02) =============

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins15476178657975103257.sh

nvidia-merlin-bot · 2022-05-31T20:17:05Z

Click to view CI Results

GitHub pull request #110 of commit 789693145ccbe216fc60f58abb689a801bbb3872, no merge conflicts.
Running as SYSTEM
Setting status of 789693145ccbe216fc60f58abb689a801bbb3872 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/59/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse 789693145ccbe216fc60f58abb689a801bbb3872^{commit} # timeout=10
Checking out Revision 789693145ccbe216fc60f58abb689a801bbb3872 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 789693145ccbe216fc60f58abb689a801bbb3872 # timeout=10
Commit message: "Add cuml to installataion for tests"
 > git rev-list --no-walk 1bb31c619fdfa32e88203c605f7961197cbacb45 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins14897791595580929223.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 18 items / 1 skipped
tests/unit/test_version.py .                                             [  5%]

tests/unit/systems/test_ensemble.py ....                                 [ 27%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 38%]

tests/unit/systems/test_export.py .                                      [ 44%]

tests/unit/systems/test_graph.py .                                       [ 50%]

tests/unit/systems/test_inference_ops.py ..                              [ 61%]

tests/unit/systems/test_op_runner.py ....                                [ 83%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18

/usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

============ 18 passed, 1 skipped, 18 warnings in 62.24s (0:01:02) =============

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins10640686869443690921.sh

nvidia-merlin-bot · 2022-05-31T20:32:29Z

Click to view CI Results

GitHub pull request #110 of commit e3c447832afe337f3dbd9c54577cbdc3c64428af, no merge conflicts.
Running as SYSTEM
Setting status of e3c447832afe337f3dbd9c54577cbdc3c64428af to PENDING with url https://10.20.13.93:8080/job/merlin_systems/60/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse e3c447832afe337f3dbd9c54577cbdc3c64428af^{commit} # timeout=10
Checking out Revision e3c447832afe337f3dbd9c54577cbdc3c64428af (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e3c447832afe337f3dbd9c54577cbdc3c64428af # timeout=10
Commit message: "Add channel to conda install"
 > git rev-list --no-walk 789693145ccbe216fc60f58abb689a801bbb3872 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins3721378773267252970.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 18 items / 1 skipped
tests/unit/test_version.py .                                             [  5%]

tests/unit/systems/test_ensemble.py ....                                 [ 27%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 38%]

tests/unit/systems/test_export.py .                                      [ 44%]

tests/unit/systems/test_graph.py .                                       [ 50%]

tests/unit/systems/test_inference_ops.py ..                              [ 61%]

tests/unit/systems/test_op_runner.py ....                                [ 83%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18

/usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

============ 18 passed, 1 skipped, 18 warnings in 63.47s (0:01:03) =============

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins10570489757175345757.sh

nvidia-merlin-bot · 2022-06-01T14:37:18Z

Click to view CI Results

GitHub pull request #110 of commit 6b36d2924bfef26d821b75bf4b08b6be9c323ec1, no merge conflicts.
Running as SYSTEM
Setting status of 6b36d2924bfef26d821b75bf4b08b6be9c323ec1 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/61/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse 6b36d2924bfef26d821b75bf4b08b6be9c323ec1^{commit} # timeout=10
Checking out Revision 6b36d2924bfef26d821b75bf4b08b6be9c323ec1 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6b36d2924bfef26d821b75bf4b08b6be9c323ec1 # timeout=10
Commit message: "Remove importorskip for cuml in fil tests."
 > git rev-list --no-walk e3c447832afe337f3dbd9c54577cbdc3c64428af # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins13808942665572280564.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 18 items / 1 skipped
tests/unit/test_version.py .                                             [  5%]

tests/unit/systems/test_ensemble.py ....                                 [ 27%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 38%]

tests/unit/systems/test_export.py .                                      [ 44%]

tests/unit/systems/test_graph.py .                                       [ 50%]

tests/unit/systems/test_inference_ops.py ..                              [ 61%]

tests/unit/systems/test_op_runner.py ....                                [ 83%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18

/usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

================= 18 passed, 1 skipped, 18 warnings in 59.71s ==================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins3393202208321052611.sh

nvidia-merlin-bot · 2022-06-01T14:42:26Z

Click to view CI Results

GitHub pull request #110 of commit e66e5e1ab72a070f01eeca25e7b91eecfdfd52c7, no merge conflicts.
Running as SYSTEM
Setting status of e66e5e1ab72a070f01eeca25e7b91eecfdfd52c7 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/62/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse e66e5e1ab72a070f01eeca25e7b91eecfdfd52c7^{commit} # timeout=10
Checking out Revision e66e5e1ab72a070f01eeca25e7b91eecfdfd52c7 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e66e5e1ab72a070f01eeca25e7b91eecfdfd52c7 # timeout=10
Commit message: "Remove cuml classifier/regressor in test_fil.py"
 > git rev-list --no-walk 6b36d2924bfef26d821b75bf4b08b6be9c323ec1 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins764982043682396994.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 18 items / 1 skipped
tests/unit/test_version.py .                                             [  5%]

tests/unit/systems/test_ensemble.py ....                                 [ 27%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 38%]

tests/unit/systems/test_export.py .                                      [ 44%]

tests/unit/systems/test_graph.py .                                       [ 50%]

tests/unit/systems/test_inference_ops.py ..                              [ 61%]

tests/unit/systems/test_op_runner.py ....                                [ 83%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18

/usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

============ 18 passed, 1 skipped, 18 warnings in 63.07s (0:01:03) =============

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins13311486798200639831.sh

nvidia-merlin-bot · 2022-06-01T15:09:11Z

Click to view CI Results

GitHub pull request #110 of commit bb98249cfd0f00b4b5ce8e0b130aca93b3834121, no merge conflicts.
Running as SYSTEM
Setting status of bb98249cfd0f00b4b5ce8e0b130aca93b3834121 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/63/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/110/*:refs/remotes/origin/pr/110/* # timeout=10
 > git rev-parse bb98249cfd0f00b4b5ce8e0b130aca93b3834121^{commit} # timeout=10
Checking out Revision bb98249cfd0f00b4b5ce8e0b130aca93b3834121 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f bb98249cfd0f00b4b5ce8e0b130aca93b3834121 # timeout=10
Commit message: "Merge branch 'main' into ops-fil"
 > git rev-list --no-walk e66e5e1ab72a070f01eeca25e7b91eecfdfd52c7 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins363968461104551533.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 18 items / 1 skipped
tests/unit/test_version.py .                                             [  5%]

tests/unit/systems/test_ensemble.py ....                                 [ 27%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 38%]

tests/unit/systems/test_export.py .                                      [ 44%]

tests/unit/systems/test_graph.py .                                       [ 50%]

tests/unit/systems/test_inference_ops.py ..                              [ 61%]

tests/unit/systems/test_op_runner.py ....                                [ 83%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18

/usr/local/lib/python3.8/dist-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
tests/unit/systems/test_ensemble.py: 7 warnings

tests/unit/systems/test_export.py: 1 warning

tests/unit/systems/test_inference_ops.py: 2 warnings

tests/unit/systems/test_op_runner.py: 4 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1292: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

============ 18 passed, 1 skipped, 18 warnings in 62.63s (0:01:02) =============

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins15913411464619615353.sh

oliverholworthy added the status/work-in-progress label May 26, 2022

karlhigley reviewed May 27, 2022

View reviewed changes

oliverholworthy added 6 commits May 27, 2022 18:46

Add an Operator for Forest Inference Libary (FIL) Triton backend.

90aca2c

Add xgboost dependency for running FIL tests

bc44d3e

Use tritonclient for protobuf config instead of merlin systems

e53e754

Move schema creation to schema methods

4469fc8

Update to use tritonclient pb2 in importorskip

9f40384

Change schema dtype to float32

463af92

oliverholworthy force-pushed the ops-fil branch from 1980b1e to 463af92 Compare May 27, 2022 17:46

oliverholworthy added 2 commits May 30, 2022 10:41

Rename FILPredict to FIL

b70e197

Update tests for xgboost data to have only one target

50350a1

oliverholworthy self-assigned this May 30, 2022

oliverholworthy added 3 commits May 31, 2022 14:39

Refactor XGBoost FIL op and add support for LightGBM/Scikit-Learn

ac4ff88

Add test for multiclass XGBoost classifier

7fa02ae

Move imports for FIL to compat. Update docs

594b22c

Add handling for edge case where output dim is zero

8014a77

oliverholworthy added 10 commits May 31, 2022 17:49

Move fil tests to subdirectory and importorskip configuration

1913d9b

Add lightgbm and scikit-learn to test installation setup

dc148ad

Update sklearn import to sklearn_ensemble in compat for fil

4a8e2c3

Add tests for sklearn and lightgbm classifiers

ac11106

Add lightgbm and sklearn to known_third_party in pyproject.toml

9de1afa

Update comment and format of output_dim check

4a823d9

Add support for cuml forest models

cdd526e

Remove conftest and add parametrized tests for supported model types

1bb31c6

Add cuml to importorskip

0f6f4ed

Add cuml to installataion for tests

7896931

oliverholworthy added 2 commits May 31, 2022 21:26

Create workflow for FIL op

91ed848

Add channel to conda install

e3c4478

oliverholworthy added 2 commits June 1, 2022 12:38

Update conda install with channels

4fb679f

Remove tests for cuml - they don't work on CPU

2eff643

karlhigley marked this pull request as ready for review June 1, 2022 14:17

oliverholworthy removed the status/work-in-progress label Jun 1, 2022

Remove importorskip for cuml in fil tests.

6b36d29

Remove cuml classifier/regressor in test_fil.py

e66e5e1

Merge branch 'main' into ops-fil

bb98249

karlhigley approved these changes Jun 1, 2022

View reviewed changes

karlhigley merged commit 8393bbc into NVIDIA-Merlin:main Jun 1, 2022

oliverholworthy added the feature label Jun 8, 2022

This was referenced Jun 13, 2022

Add Forest Inference Operator #118

Merged

Add support for additional ML/DL frameworks #7

Open

oliverholworthy mentioned this pull request Jun 22, 2022

[RMP] Support Tree Ranking Models (like XGBoost) in Merlin Models and Systems NVIDIA-Merlin/Merlin#105

Closed

4 tasks

Add an Operator for Forest Inference Libary (FIL) Triton backend. #110

Add an Operator for Forest Inference Libary (FIL) Triton backend. #110

Uh oh!

Conversation

oliverholworthy commented May 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

karlhigley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nvidia-merlin-bot commented May 27, 2022

Uh oh!

github-actions bot commented May 27, 2022

Documentation preview

Uh oh!

nvidia-merlin-bot commented May 31, 2022

Uh oh!

nvidia-merlin-bot commented May 31, 2022

Uh oh!

nvidia-merlin-bot commented May 31, 2022

Uh oh!

nvidia-merlin-bot commented May 31, 2022

Uh oh!

nvidia-merlin-bot commented May 31, 2022

Uh oh!

nvidia-merlin-bot commented Jun 1, 2022

Uh oh!

nvidia-merlin-bot commented Jun 1, 2022

Uh oh!

nvidia-merlin-bot commented Jun 1, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

oliverholworthy commented May 26, 2022 •

edited

Loading