Alpine-musl: fix mmap64 error

[kassane]

$ /opt/bin/ldmd2 --version | head -n 8
LDC - the LLVM D compiler (1.38.0-git-bc00d4b):
  based on DMD v2.108.0 and LLVM 17.0.5
  built with LDC - the LLVM D compiler (1.33.0)
  Default target: x86_64-alpine-linux-musl
  Host CPU: znver3
  http://dlang.org - http://wiki.dlang.org/LDC

$ /opt/bin/ldmd2 -run hellod-sample.d 
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /opt/lib/libdruntime-ldc.a(fiber.o): in function `_D4core6thread5fiber5Fiber10allocStackMFNbmmZv':
fiber.d:(.text._D4core6thread5fiber5Fiber10allocStackMFNbmmZv+0x7d): undefined reference to `mmap64'
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /opt/lib/libdruntime-ldc.a(os.o): in function `_D4core8internal2gc2os10os_mem_mapFNbNimbZPv':
os.d:(.text._D4core8internal2gc2os10os_mem_mapFNbNimbZPv+0x21): undefined reference to `mmap64'
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /opt/lib/libdruntime-ldc.a(io.o): in function `_D4core8internal3elf2io__T5ElfIOTSQBg3sysQz10Elf64_EhdrTSQCdQxQBu10Elf64_ShdrVhi2Z7ElfFile4openFNbNiPxaJSQDzQDxQDrQDq__TQDqTQDnTQCuVhi2ZQCcZb':
io.d:(.text._D4core8internal3elf2io__T5ElfIOTSQBg3sysQz10Elf64_EhdrTSQCdQxQBu10Elf64_ShdrVhi2Z7ElfFile4openFNbNiPxaJSQDzQDxQDrQDq__TQDqTQDnTQCuVhi2ZQCcZb+0xa6): undefined reference to `mmap64'
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /opt/lib/libdruntime-ldc.a(io.o): in function `_D4core8internal3elf2io__T5ElfIOTSQBg3sysQz10Elf64_EhdrTSQCdQxQBu10Elf64_ShdrVhi2Z13NamedSections7opApplyMFNbNiMDFNbNimAxaSQErQEpQEjQEi__TQEiTQEfTQDmVhi2Z16ElfSectionHeaderZiZi':
io.d:(.text._D4core8internal3elf2io__T5ElfIOTSQBg3sysQz10Elf64_EhdrTSQCdQxQBu10Elf64_ShdrVhi2Z13NamedSections7opApplyMFNbNiMDFNbNimAxaSQErQEpQEjQEi__TQEiTQEfTQDmVhi2Z16ElfSectionHeaderZiZi+0xaf): undefined reference to `mmap64'
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: io.d:(.text._D4core8internal3elf2io__T5ElfIOTSQBg3sysQz10Elf64_EhdrTSQCdQxQBu10Elf64_ShdrVhi2Z13NamedSections7opApplyMFNbNiMDFNbNimAxaSQErQEpQEjQEi__TQEiTQEfTQDmVhi2Z16ElfSectionHeaderZiZi+0x16a): undefined reference to `mmap64'
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /opt/lib/libdruntime-ldc.a(io.o):io.d:(.text._D4core8internal3elf2io__T5ElfIOTSQBg3sysQz10Elf64_EhdrTSQCdQxQBu10Elf64_ShdrVhi2Z13NamedSections7opApplyMFNbNiMDFNbNimAxaSQErQEpQEjQEi__TQEiTQEfTQDmVhi2Z16ElfSectionHeaderZiZi+0x2a5): more undefined references to `mmap64' follow
collect2: error: ld returned 1 exit status
Error: /usr/bin/cc failed with status: 1

cc: @kinke

[dlang-bot]

Thanks for your pull request and interest in making D better, @kassane! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#16361"

[JohanEngelen]

To me this does not look like the correct / complete fix.

gperftools/gperftools#693 (comment)

On musl libc there are no such symbol (mmap64) exported. Instead there is a #define mmap64 mmap, meaning that on musl libc, mmap64 is the same thing as mmap.

So this needs a fix here:

dmd/druntime/src/core/sys/posix/sys/mman.d

Lines 294 to 300 in e9c6884

	else version (CRuntime_Musl)
	{
	static if (__USE_LARGEFILE64) void* mmap64(void*, size_t, int, int, int, off_t);
	static if (__USE_FILE_OFFSET64)
	alias mmap = mmap64;
	else
	void* mmap(void*, size_t, int, int, int, off_t);

[JohanEngelen]

I looked a little bit further into the musl druntime code, and I think this needs fixing in several places.
Basically, this has to be removed completely:

    enum __USE_FILE_OFFSET64 = true;
    enum __USE_LARGEFILE     = __USE_FILE_OFFSET64 && !__REDIRECT;
    enum __USE_LARGEFILE64   = __USE_FILE_OFFSET64 && !__REDIRECT;

And everywhere where these are used, druntime needs adjustment. For example also here:

dmd/druntime/src/core/sys/posix/dirent.d

Lines 388 to 396 in d3db2e7

	static if ( __USE_FILE_OFFSET64 )
	{
	dirent* readdir64(DIR*);
	alias readdir64 readdir;
	}
	else
	{
	dirent* readdir(DIR*);
	}

I have fixed the code for mmap (see commit e48132b).

DMD CI does not test Musl, so please do that manually offline :)

Thanks for the work.

[JohanEngelen]

Needs changes as mentioned in this comment: #16361 (comment)

[kassane]

DMD CI does not test Musl, so please do that manually offline :)

Yeah! My test occurs on https://github.com/kassane/alpine-ldc2-docker.
Rebuilding ldc2(master) and/or dmd (build)

[JohanEngelen]

Thanks for the work, looks good now (some minor tweaks to whitespace needed)

[kassane]

Thanks for the work, looks good now (some minor tweaks to whitespace needed)

Thanks, for help-me.

[JohanEngelen]

lgtm

[JohanEngelen]

I'm changing my approval, because of the large amount of code duplication seen in the extended PR for LDC druntime... Very sorry that my advice lead you down this path...

[dkorpel]

@JohanEngelen Is this good again?

[JohanEngelen]

@JohanEngelen Is this good again?

yes

[sertonix]

This change caused off_t to be 32-bit on 32-bit arches but it needs to be 64-bit on all arches for musl. That causes memory corruptions all over the place. The more complicated approach looks more correct to me.

My knowledge of dlang is very limited (0 to be honest) so I hope someone could help me with fixing this.

Edit: 32-bit musl support seems to be broken in multiple places so I will probably need to create a larger patchset.

[JohanEngelen]

This change caused off_t to be 32-bit on 32-bit arches but it needs to be 64-bit on all arches for musl. That causes memory corruptions all over the place. The more complicated approach looks more correct to me.

My knowledge of dlang is very limited (0 to be honest) so I hope someone could help me with fixing this.

Edit: 32-bit musl support seems to be broken in multiple places so I will probably need to create a larger patchset.

Can you submit a new bug report about this? Then you can discuss the correct fix for it there. Thanks!

[sertonix]

I hope a draft is fine as well: #21249