Skip to content

[CI] macOS build improvements #2884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fhanau merged 3 commits into from
Oct 10, 2024
Merged

[CI] macOS build improvements #2884

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7ded880
Use lld to reduce macOS release binary size
fhanau Oct 9, 2024
5ca5b4a
[CI] Use macos-15 image for test job, implicitly transitioning to App…
fhanau Oct 9, 2024
a9f950b
[nfc] Drop duplicated macOS experimental_objc_fastbuild_options flag
fhanau Oct 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions .bazelrc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -219,12 +219,7 @@ build:macos --copt='-femit-dwarf-unwind=no-compact-unwind'
# an effect. On macOS we build some ObjC as part of Dawn, set this flag to an empty string so we
# can set the right configuration flags.
# TODO(soon): File a bazel issue for the flag so we can drop this.
build:macos --experimental_objc_fastbuild_options=''

# Unfortunately experimental_objc_fastbuild_options does not respect options set before and
# overwrites them. This causes mysterious binary size bloat for fastbuild-based configurations on
# macOS when enabling some optimizations. Turn it in into a no-op.
build:mac --experimental_objc_fastbuild_options=""
build:macos --experimental_objc_fastbuild_options=""

# On Linux, always link libc++ statically to avoid compatibility issues with different OS versions.
# macOS links with dynamic libc++ by default, which has good backwards compatibility.
Expand Down Expand Up @@ -318,6 +313,17 @@ build:release_macos --config=release_unix
build:release_macos --linkopt="-Wl,-no_data_in_code_info"
build:release_macos --linkopt="-Wl,-no_function_starts"

# On macOS, optionally compile using LLD (19 or higher is compatible with the default flags added by
# apple_support). Requires Homebrew's lld package to be installed. This is less CPU intensive than
# the system linker, but also slightly slower in terms of wall time since it is less parallel. More
# importantly, it allows us to enable LLD's ICF pass, which significantly decreases binary sizes.
# We could use Xcode 16's -Wl,-deduplicate option instead, but LLD's ICF appears to be superior.
# We also want to enable ICF for Linux, but there it causes warnings when dynamically linking with
# libc++.
build:macos_lld --linkopt="-fuse-ld=/opt/homebrew/bin/ld64.lld"
build:macos_lld_icf --config=macos_lld
build:macos_lld_icf --linkopt="-Wl,--icf=safe"

build:release_windows --config=release
# Windows uses /O2 as its preferred optimization setting and enabled by bazel in the opt
# configuration, but for clang-cl this is equivalent to only -O2 and a few other things. -O3 is
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,10 @@ jobs:
if: runner.os == 'macOS'
run: |
sudo xcode-select -s "/Applications/Xcode_15.1.app"
# Install lld
brew install lld
Copy link
Copy Markdown
Contributor

@anonrig anonrig Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just fyi, this installs llvm 19. i'm not sure if it will cause any compatibility problems. leaving it as a small note.

Copy link
Copy Markdown
Contributor Author

@fhanau fhanau Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. This should be fine as the release build job doesn't use homebrew LLVM at all, the bazel toolchain is configured solely based on the clang/llvm available in the Xcode installation. Installing homebrew LLVM 19 instead of the default version should not be an issue (I initially included this command in the test job too to try it out, seems to have worked flawlessly. I primarily want to install LLD to make release builds smaller, so this is not needed for the regular test run).

Copy link
Copy Markdown
Collaborator

@danlapid danlapid Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add this install to the README?

Copy link
Copy Markdown
Contributor Author

@fhanau fhanau Oct 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will only be needed for macOS release builds. Developers generally shouldn't need to build those regularly, but I think it should still be supported without needing more stuff – ended up removing LLD from the release_macos config and manually adding it in the release job instead.

# Enable lld identical code folding to significantly reduce binary size.
echo "build:macos --config=macos_lld_icf" >> .bazelrc
- name: Setup Windows
if: runner.os == 'Windows'
run: |
Expand Down
18 changes: 8 additions & 10 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,16 @@ jobs:
fixup:
if: github.event_name == 'pull_request'
uses: ./.github/workflows/fixup.yml
labels:
labels:
if: github.event_name == 'pull_request'
uses: ./.github/workflows/labels.yml
uses: ./.github/workflows/labels.yml
test:
strategy:
matrix:
os:
[
{ name : linux, image : ubuntu-20.04 },
{ name : macOS, image : macos-13 },
{ name : macOS, image : macos-15 },
{ name : windows, image : windows-2022 }
]
config:
Expand Down Expand Up @@ -62,7 +62,7 @@ jobs:
config: { suffix: -debug, bazel-args: --config=debug }
# due to resource constraints, exclude the macOS-debug runner for now. linux-debug and
# linux-asan should provide sufficient coverage for building in the debug configuration.
- os: { name : macOS, image : macos-13 }
- os: { name : macOS, image : macos-15 }
config: { suffix: -debug, bazel-args: --config=debug }
fail-fast: false
runs-on: ${{ matrix.os.image }}
Expand Down Expand Up @@ -103,17 +103,15 @@ jobs:
sed -i -e "s%llvm-symbolizer%/usr/lib/llvm-16/bin/llvm-symbolizer%" .bazelrc
- name: Setup macOS
if: matrix.os.name == 'macOS'
# TODO: We want to symbolize stacks for crashes on CI. Xcode is currently based on LLVM 16
# but the macos-13 image has llvm@15 installed:
# https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md
# TODO: We want to symbolize stacks for crashes on CI. Xcode is currently based on LLVM 17
# but the macos-15 image has llvm@18 installed:
# https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md
#
# Not enabled because symbolication does not work on workerd macOS builds yet and running
# llvm-symbolizer in the currently broken state causes some tests to time out on the
# runner.
# Use latest available Xcode version – runner still defaults to 15.0.1.
run: |
sudo xcode-select -s "/Applications/Xcode_15.1.app"
# export LLVM_SYMBOLIZER=$(brew --prefix llvm@15)/bin/llvm-symbolizer
# export LLVM_SYMBOLIZER=$(brew --prefix llvm@18)/bin/llvm-symbolizer
# sed -i -e "s%llvm-symbolizer%${LLVM_SYMBOLIZER}%" .bazelrc
- name: Setup Windows
if: matrix.os.name == 'windows'
Expand Down