Add Bazel support back?

[per-gron]

I want to use Halide with Bazel. I'm currently hacking on reverting the PRs that deleted support for it earlier this year.

Is there interested in merging this work back into this repository? I could also keep this separate without too much trouble.

If there is interest in porting the whole build system to Bazel and not just the library user side I could potentially help a bit with that too. I'm a Googler so could potentially help with open sourcing the internal Blaze setup. It seems like Windows support in Bazel has gotten a bit better in the last year. Building Halide with Blaze would be very useful for the project I'm using it in, since we will likely need to hack at least a bit in the Halide code base.

Details:

• I use Linux and am not very interested in working with Windows stuff, but if I could build all of Halide with Bazel maybe I can help there too.
• I don't fully understand how the old Bazel rules worked wrt LLVM; Unlike CMake, Bazel cannot build LLVM so it needs to be fed with a binary distribution of it. Linking directly against the LLVM libraries is a bit sensitive to which toolchain is used by the Bazel build. The official LLVM binary distribution for Ubuntu links against libstdc++, but the Bazel compiler toolchain that seems most well supported is hard coded to use libc++... The approach I am going for at the moment is to have my own custom built LLVM where the LLVM libraries are built against libc++ in order to be able to use the gailbio toolchain setup. All in all it just seems a bit tricky to get hermetic builds up and running in a user friendly way.

[steven-johnson]

I am a huuuuge fan of Bazel and would love to be able to use it. The initial Bazel support was removed partly because apparently literally no one was using it; there were also some technical obstacles at the time:

• Windows support was still buggy and suboptimal
• Bazel didn't provide any reasonable way to produce a static library as a final build product
• Possibly others?

Additionally, adding Bazel support implied a couple of other things:

• Do we really want to support three build systems in the long run? If not, which of [Make, CMake, Bazel] would get dropped? There are many core contributors who greatly prefer Make, so that would be a hard sell to drop (unfortunately, IMHO). One could argue that CMake could be dropped in favor of Bazel (assuming that Windows support is finally acceptable) but I suspect that there are downstream users who rely on CMake.

Re: LLVM, at the time, the requirement was "build LLVM on your system somehow and point the Bazel build system at it"; there was no attempt to add LLVM into the overall dependency tree. I vaguely understand that Bazel has added some sort of support for "foreign" build systems as leaf dependencies since then, but I have no idea what's involved. Re: the libc++ vs libstdc++ issue, all of our canonical build systems (and more core developers) tend to build LLVM from source (rather than relying on binary distros), so I don't know if anyone has actively dealt with this issue; your approach is the best in the short term. (If there is a better longterm approach I'd like to hear it too.)

I would be very happy to support your effort in this in terms of code reviewing and such (as I did the lion's share of the Bazel/Blaze rules for Halide both inside and outside Google), but be aware that I'm unlikely to be able to dedicate any meaningful coding time to it anytime soon.

My vote would be to (initially) merge it back into the main codebase, but with the caveat that it isn't officially supported and may require ongoing tweaks on the part of Bazel users to keep things working well.

(Side note: it was an obvious goal to keep the halide_library, etc rules as close as possible to the ones inside Google, with the hope of possibly being able to merge them someday; that isn't super-likely to happen but is still a clear goal IMHO, as Google probably has the largest single concentration of Halide code anywhere in existence today.)

[per-gron]

@steven-johnson What is the point of having both Make and CMake based build systems when CMake can generate Makefiles?

[steven-johnson]

@steven-johnson What is the point of having both Make and CMake based build systems when CMake can generate Makefiles?

Many core contributors prefer to write directly in Make syntax, rather than using CMake's somewhat idiosyncratic language.

[per-gron]

@steven-johnson What is the point of having both Make and CMake based build systems when CMake can generate Makefiles?

Many core contributors prefer to write directly in Make syntax, rather than using CMake's somewhat idiosyncratic language.

Do you mean when they have to update Halide's own build files? But don't they have to write CMake build files also given that you have both build systems? Also, although CMake isn't exactly nice, Makefiles aren't free of sharp edges either... This is honestly quite surprising to me.

I can understand that dropping CMake in favor of Bazel could be quite painful and suddenly require lots of no-value work for users of the library, but isn't this mere preference, which doesn't help anyways since both systems need to be maintained?

[abadams]

Whenever we do something new or slightly unusual (e.g. the new autoscheduler in the apps folder), we just write Makefiles because it's quick and easy. Once things are mainlined we go through the painful process of bending cmake to our will. It's really not designed for staged compilation. So make is used for faster iteration during development, and cmake is added when something stabilizes.

[per-gron]

@abadams I see. In the context of discussing how to avoid having to maintain three separate build systems, could Bazel serve this purpose? IME it works well for that, and the new configuration stuff can express fairly complicated cross-compilation scenarios relatively cleanly.

[abadams]

I liked blaze when I was at Google, but I'm not convinced it's the right tool here. The thing about make that makes it great for experimentation is that it doesn't try to understand things as code or libraries or binaries. You just have targets, what files they depend on, and the shell command to create them. So makefiles are also great for experiment management, which is often going on at the same time when you are writing something experimental. When you are compiling code in a new way, with makefiles there's no awkward layer of abstraction in between you and the compiler flags that makes it hard to pass a specific set of flags. You just write down exactly what you want it to run. With cmake and bazel I need a mental model of how the build system is going to manipulate and combine things to make an actual command line compiler invocation, and I seem to spend a long time fighting the build system when it passes unwanted flags for reasons I can't determine. Capturing and understanding the precise tool invocation that causes a problem during a large blaze build is always tricky due to the flood of obscure flags it appends to everything. Things like cmake or bazel can support this sort of more direct use with custom rules, but it's awkward. In general they try to more deeply understand what you are doing as building binaries from code. Make can reach a higher level of abstraction too (though not as high as cmake/bazel), but it allows for incremental generalization starting from invoking c++ directly on the command line, to copy-pasting those invocations into a very explicit initial set of rules in a Makefile, to a more general set of rules using wildcards and variables.

Bazel is also very Google(r)-specific. The hygiene requirements and the weird places it hides build intermediates all make a ton of sense in the google3 context, but are unhelpful complexity for smaller ad-hoc things. Plus it's a large dependency (e.g. you need a jvm), and is not likely to be used by non-Googlers just due to familiarity. I have no objection to adding it, because the support has to exist inside Google anyway and it's better for those files to be open than closed, but I wouldn't anticipate heavy usage by non-Googlers.

As far as I can tell, the big downside of make is that by operating at a lower level of abstraction, it's less portable across different build environments. At the very least, we need cmake to generate visual studio solution files.

[steven-johnson]

you need a jvm

irrelevant to your points, but this is actually no longer true (it supplies its own hermetic jvm on installation)...

[abadams]

Not entirely irrelevant - that does negate one of my concerns.

[steven-johnson]

While it's true that Make is generally easier to experiment with, the counterpoint is that it's not uncommon for some experiments to never graduate to CMake (or to take a long time to do so). Requiring (say) CMake for everything would mean more work at experimentation time, but it would also mean that systems that require CMake (eg Windows) wouldn't lag behind. (Is there CMake support for the next-gen autoscheduler yet?)

[abadams]

No, there isn't. Sigh. We should probably just require cmake support for merges into master.

[steven-johnson]

We should probably just require cmake support for merges into master.

... which means we'll have to add CMake build tests for Linux and OSX, at least (currently we only use CMake for Windows and for some limited Travis testing).

(As I've said before: I loathe CMake with the intensity of a million exploding suns, but it might be better in the long run for us to standardize on it everywhere; if LLVM made it work, we probably can too.)

[per-gron]

Despite its flaws, it's hard to deny that CMake is the closest thing to a de facto standard build system for C++.

Anyway. I just managed to accidentally delete all my work so far with this so it will take a little bit of time before I can share something with you, but I think I'll just start with something that lets me use Halide with my Bazel based project, rather than porting everything.

Would you be okay with modifying the main build so that there is an option to make a halide.tgz distribution that includes the LLVM static libraries? For me, that would make things a bit easier to set up on the Bazel side.

[abadams]

The existing libHalide.a actually includes the llvm static libraries too within it. Do you need them separated out?

[per-gron]

Oh, I didn't know that! That makes things easier 😄