Test for missing system_libs (KB-H043)

[madebr]

Test whether a recipe has all system libraries listed in cpp_info.system_libs.
This heuristic only run on Linux and Windows and can only be done for shared libraries.

WIKI:

#KB-H041: "MISSING SYSTEM LIBS"

A shared library links to some system libraries which are not set by cpp_info.system_libs in package_info.
This test can only be run on shared libraries.

[madebr]

@uilianries
Using this code, we can also check whether the order of the libraries in cpp_info.libs is correct.
Is that hook-worthy?

[madebr]

I think Windows CI fails because it needs the Visual Studio compiler and it is not set in the test section.
(test_script section in appveyor.yml)

[uilianries]

@madebr

Using this code, we can also check whether the order of the libraries in cpp_info.libs is correct.

What do mean by correct? Anyway, it deserves a separated hook.

[uilianries]

Do think about header-only libraries? I mean, in terms of test. Some of them contain system_libs.

[madebr]

@uilianries

What do mean by correct?

By correct, I mean correct ordering. Such that dependencies of other libraries come last in the list.

Do think about header-only libraries? I mean, in terms of test. Some of them contain system_libs.

header-only libraries require another approach.
This pr analyzes shared libraries for its dependencies, which is a fast operation.

For header-only libraries, we'ld need to grep all headers for occurrences of certain includes, which can be time and cpu intensive for larger projects.

e.g. for glibc: (dl, m, pthread and rt)

r"""#\w*include\w*([<"])((dlfcn.h)|(math.h)|(pthread.h)|(time.h))\g<1>"""

It can be done, but will be non-perfect.
Compilers might inline some function and don't require linking to an external library.

[uilianries]

@uilianries

What do mean by correct?

By correct, I mean correct ordering. Such that dependencies of other libraries come last in the list.

Do think about header-only libraries? I mean, in terms of test. Some of them contain system_libs.

header-only libraries require another approach.
This pr analyzes shared libraries for its dependencies, which is a fast operation.

For header-only libraries, we'ld need to grep all headers for occurrences of certain includes, which can be time and cpu intensive for larger projects.

e.g. for glibc: (dl, m, pthread and rt)

r"""#\w*include\w*([<"])((dlfcn.h)|(math.h)|(pthread.h)|(time.h))\g<1>"""

It can be done, but will be non-perfect.
Compilers might inline some function and don't require linking to an external library.

Okay, let's update it in another occasion.

[madebr]

@uilianries
The tests fail on Appveyor, because it cannot find the Visual Studio installation in the post_package hook.
https://ci.appveyor.com/project/ConanOrgCI/hooks/builds/32505216/job/gs8bxp78pu2y82i3#L238

E   ERROR: [HOOK - C:\projects\hooks\tests\test_hooks\conan-center\..\..\..\hooks\conan-center.py] post_package(): VS non-existing installation: Visual Studio 14

Do you have an idea what might be up?

[SpaceIm]

@madebr Does this hook allow recipes to not list system libs if those system libs are not direct dependencies but instead are coming from system libs of dependencies?

[madebr]

@madebr Does this hook allow recipes to not list system libs if those system libs are not direct dependencies but instead are coming from system libs of dependencies?

Yes!

On Linux, it runs objdump -p $LIBRARY and parses the output.
It shows only the direct dependencies of the executables.
Whereas ldd shows the entire depenency tree.

I've got this from the Security section from the man page of ldd.

[prince-chrismc]

I've used this tool for windows https://docs.microsoft.com/en-us/cpp/windows/understanding-the-dependencies-of-a-visual-cpp-application?view=vs-2019

There's a link to a GitHub repo with the tool

[SSE4]

@madebr I took some look, and it seems like environment is somehow corrupted, e.g. it returns None from:

program_files = get_env("ProgramFiles(x86)") or get_env("ProgramFiles")

and from:

env_var = "vs%s0comntools" % compiler_version
vs_path = os.getenv(env_var)

I am pretty sure it's corrupted somewhere inside the test, probably in _get_environ? could you double-check that?

[uilianries]

LGTM

[madebr]

Nice job @SSE4
How a one line change can fix (almost) everything.

[SSE4]

it took me two years to find it: https://tox.wiki/en/latest/config.html#conf-passenv 🤦

[uilianries]

well, today we learned something new at least 🤓

[uilianries]

LGTM

[SSE4]

@madebr @uilianries I've added some macOS implementation, it's detecting missing frameworks.
P.S. it should be relatively safe to activate the hook - it only prints warnings. ofcourse, there could be some hard errors (exceptions) from it, but I think we can address them quickly.

[madebr]

I'm not 100% confident of this pr, but as @SSE4 said.
This hook only emits warnings and bugs can be fixed.