Support Python/C++ interoperability (native Python modules)

[davidzchen]

According to the documentation for py_library.deps:

The list of other libraries to be linked in to the binary target. See general comments about deps at Attributes common to all build rules. These can be py_binary rules, py_library rules or cc_library rules,

However, when I try to build a py_library target that depends on a cc_library, I get the following error:

❯❯❯ bazel build //external:markupsafe
ERROR: /private/var/tmp/_bazel_dzc/a4c09a1fcefb8b21c5b0e7d7db6f08ba/external/markupsafe_archive/BUILD:9:12: in deps attribute of py_library rule @markupsafe_archive//:markupsafe: cc_library rule '@markupsafe_archive//:speedups' is misplaced here (expected py_binary or py_library).
ERROR: Analysis of target '//external:markupsafe' failed; build aborted.
INFO: Elapsed time: 0.213s

According to BazelPyRuleClasses.java, the only allowed rule types in py_library.deps is, in fact, py_binary and py_library:

  public static final String[] ALLOWED_RULES_IN_DEPS = new String[] {
      "py_binary",
      "py_library",
  };

What is the plan for Python-C interoperability? Should we update the documentation to remove the mention of depending on cc_library in the meantime?

The background for this is that I am trying to use Jinja for generating HTML for the Skylark docgen tool that I am working on. However, Jinja depends on Markupsafe, which is partly implemented in C.

[davidzchen]

Some related discussion here: https://groups.google.com/forum/#!searchin/bazel-discuss/pex/bazel-discuss/-DFOrVqK8aY/-7h4e69hfkEJ

[ulfjack]

Recategorizing this as a feature request to support Python-C++ interop. I marked the other report that asked for that as a dupe.

[evanj]

A workarounds I found: you can include pre-compiled .sos in the py_library's data attribute. No idea if you can somehow get the output of a cc_library to work in this way.

[benley]

The trick to building a .so that works as a python extension seems to be making it a cc_binary instead of a cc_library. I think you have to build it with linkshared = 1 to make it work, but I'd love to be wrong about that.