Reproducible Builds: Ensure object files are sorted by kpcyrd · Pull Request #254 · WebAssembly/wasi-libc

kpcyrd · 2021-08-25T23:14:30Z

This fixes a reproducible builds issue that was found in tinygo that was found by the Reproducible Arch Linux setup.

This patch sorts the .c files, which ensures a deterministic order for .o files after patsubst.

/usr/lib/tinygo/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a:

Details

│ │ ├── file list
│ │ │ @@ -1,95 +1,95 @@
│ │ │  ----------   0        0        0    15022 1970-01-01 00:00:00.000000 /
│ │ │  ----------   0        0        0        0 1970-01-01 00:00:00.000000 //
│ │ │  ?rw-r--r--   0        0        0    14724 1970-01-01 00:00:00.000000 dlmalloc.o
│ │ │ -?rw-r--r--   0        0        0      688 1970-01-01 00:00:00.000000 pread.o
│ │ │ -?rw-r--r--   0        0        0      486 1970-01-01 00:00:00.000000 symlinkat.o
│ │ │ -?rw-r--r--   0        0        0      463 1970-01-01 00:00:00.000000 ftruncate.o
│ │ │ -?rw-r--r--   0        0        0      534 1970-01-01 00:00:00.000000 usleep.o
│ │ │ -?rw-r--r--   0        0        0      416 1970-01-01 00:00:00.000000 close.o
│ │ │ -?rw-r--r--   0        0        0      566 1970-01-01 00:00:00.000000 readlinkat.o
│ │ │ -?rw-r--r--   0        0        0      423 1970-01-01 00:00:00.000000 fsync.o
│ │ │ -?rw-r--r--   0        0        0      775 1970-01-01 00:00:00.000000 faccessat.o
│ │ │ -?rw-r--r--   0        0        0      447 1970-01-01 00:00:00.000000 sleep.o
│ │ │ -?rw-r--r--   0        0        0      537 1970-01-01 00:00:00.000000 lseek.o
│ │ │ -?rw-r--r--   0        0        0      388 1970-01-01 00:00:00.000000 unlinkat.o
│ │ │ -?rw-r--r--   0        0        0      537 1970-01-01 00:00:00.000000 read.o
│ │ │ -?rw-r--r--   0        0        0      491 1970-01-01 00:00:00.000000 linkat.o
│ │ │ -?rw-r--r--   0        0        0      692 1970-01-01 00:00:00.000000 pwrite.o
│ │ │ -?rw-r--r--   0        0        0      540 1970-01-01 00:00:00.000000 write.o
│ │ │ -?rw-r--r--   0        0        0      435 1970-01-01 00:00:00.000000 fdatasync.o
│ │ │ -?rw-r--r--   0        0        0      652 1970-01-01 00:00:00.000000 getrusage.o
│ │ │ -?rw-r--r--   0        0        0      516 1970-01-01 00:00:00.000000 times.o
│ │ │ +?rw-r--r--   0        0        0      340 1970-01-01 00:00:00.000000 closedir.o
│ │ │ +?rw-r--r--   0        0        0      268 1970-01-01 00:00:00.000000 dirfd.o
│ │ │ +?rw-r--r--   0        0        0      355 1970-01-01 00:00:00.000000 fdclosedir.o
│ │ │ +?rw-r--r--   0        0        0      613 1970-01-01 00:00:00.000000 fdopendir.o
│ │ │ +?rw-r--r--   0        0        0      434 1970-01-01 00:00:00.000000 opendirat.o
│ │ │ +?rw-r--r--   0        0        0      938 1970-01-01 00:00:00.000000 readdir.o
│ │ │ +?rw-r--r--   0        0        0      294 1970-01-01 00:00:00.000000 rewinddir.o
│ │ │ +?rw-r--r--   0        0        0     1391 1970-01-01 00:00:00.000000 scandirat.o
│ │ │ +?rw-r--r--   0        0        0      292 1970-01-01 00:00:00.000000 seekdir.o
│ │ │ +?rw-r--r--   0        0        0      270 1970-01-01 00:00:00.000000 telldir.o
│ │ │ +?rw-r--r--   0        0        0      313 1970-01-01 00:00:00.000000 errno.o
│ │ │ +?rw-r--r--   0        0        0      801 1970-01-01 00:00:00.000000 fcntl.o
│ │ │ +?rw-r--r--   0        0        0      891 1970-01-01 00:00:00.000000 openat.o
│ │ │ +?rw-r--r--   0        0        0      385 1970-01-01 00:00:00.000000 posix_fadvise.o
│ │ │ +?rw-r--r--   0        0        0      384 1970-01-01 00:00:00.000000 posix_fallocate.o
│ │ │ +?rw-r--r--   0        0        0     1354 1970-01-01 00:00:00.000000 poll.o
│ │ │ +?rw-r--r--   0        0        0      427 1970-01-01 00:00:00.000000 sched_yield.o
│ │ │ +?rw-r--r--   0        0        0      487 1970-01-01 00:00:00.000000 renameat.o
│ │ │ +?rw-r--r--   0        0        0      357 1970-01-01 00:00:00.000000 _Exit.o
│ │ │  ?rw-r--r--   0        0        0     1066 1970-01-01 00:00:00.000000 ioctl.o
│ │ │ -?rw-r--r--   0        0        0      502 1970-01-01 00:00:00.000000 gettimeofday.o
│ │ │ -?rw-r--r--   0        0        0      563 1970-01-01 00:00:00.000000 pwritev.o
│ │ │ -?rw-r--r--   0        0        0      543 1970-01-01 00:00:00.000000 readv.o
│ │ │ -?rw-r--r--   0        0        0      546 1970-01-01 00:00:00.000000 writev.o
│ │ │ -?rw-r--r--   0        0        0      560 1970-01-01 00:00:00.000000 preadv.o
│ │ │ -?rw-r--r--   0        0        0      547 1970-01-01 00:00:00.000000 select.o
│ │ │ +?rw-r--r--   0        0        0      652 1970-01-01 00:00:00.000000 getrusage.o
│ │ │  ?rw-r--r--   0        0        0     1597 1970-01-01 00:00:00.000000 pselect.o
│ │ │ +?rw-r--r--   0        0        0      547 1970-01-01 00:00:00.000000 select.o
│ │ │ +?rw-r--r--   0        0        0      678 1970-01-01 00:00:00.000000 getsockopt.o
│ │ │ +?rw-r--r--   0        0        0      580 1970-01-01 00:00:00.000000 recv.o
│ │ │ +?rw-r--r--   0        0        0      566 1970-01-01 00:00:00.000000 send.o
│ │ │ +?rw-r--r--   0        0        0      452 1970-01-01 00:00:00.000000 shutdown.o
│ │ │ +?rw-r--r--   0        0        0      984 1970-01-01 00:00:00.000000 fstat.o
│ │ │ +?rw-r--r--   0        0        0     1045 1970-01-01 00:00:00.000000 fstatat.o
│ │ │  ?rw-r--r--   0        0        0      992 1970-01-01 00:00:00.000000 futimens.o
│ │ │  ?rw-r--r--   0        0        0      480 1970-01-01 00:00:00.000000 mkdirat.o
│ │ │ -?rw-r--r--   0        0        0     1045 1970-01-01 00:00:00.000000 fstatat.o
│ │ │ -?rw-r--r--   0        0        0      984 1970-01-01 00:00:00.000000 fstat.o
│ │ │  ?rw-r--r--   0        0        0     1045 1970-01-01 00:00:00.000000 utimensat.o
│ │ │ -?rw-r--r--   0        0        0      580 1970-01-01 00:00:00.000000 recv.o
│ │ │ -?rw-r--r--   0        0        0      678 1970-01-01 00:00:00.000000 getsockopt.o
│ │ │ -?rw-r--r--   0        0        0      452 1970-01-01 00:00:00.000000 shutdown.o
│ │ │ -?rw-r--r--   0        0        0      566 1970-01-01 00:00:00.000000 send.o
│ │ │ -?rw-r--r--   0        0        0      313 1970-01-01 00:00:00.000000 errno.o
│ │ │ -?rw-r--r--   0        0        0      292 1970-01-01 00:00:00.000000 seekdir.o
│ │ │ -?rw-r--r--   0        0        0      268 1970-01-01 00:00:00.000000 dirfd.o
│ │ │ -?rw-r--r--   0        0        0      270 1970-01-01 00:00:00.000000 telldir.o
│ │ │ -?rw-r--r--   0        0        0      938 1970-01-01 00:00:00.000000 readdir.o
│ │ │ -?rw-r--r--   0        0        0     1391 1970-01-01 00:00:00.000000 scandirat.o
│ │ │ -?rw-r--r--   0        0        0      434 1970-01-01 00:00:00.000000 opendirat.o
│ │ │ -?rw-r--r--   0        0        0      613 1970-01-01 00:00:00.000000 fdopendir.o
│ │ │ -?rw-r--r--   0        0        0      355 1970-01-01 00:00:00.000000 fdclosedir.o
│ │ │ -?rw-r--r--   0        0        0      294 1970-01-01 00:00:00.000000 rewinddir.o
│ │ │ -?rw-r--r--   0        0        0      340 1970-01-01 00:00:00.000000 closedir.o
│ │ │ -?rw-r--r--   0        0        0     1354 1970-01-01 00:00:00.000000 poll.o
│ │ │ -?rw-r--r--   0        0        0      793 1970-01-01 00:00:00.000000 clock_nanosleep.o
│ │ │ -?rw-r--r--   0        0        0      481 1970-01-01 00:00:00.000000 time.o
│ │ │ -?rw-r--r--   0        0        0      301 1970-01-01 00:00:00.000000 CLOCK_REALTIME.o
│ │ │ +?rw-r--r--   0        0        0      502 1970-01-01 00:00:00.000000 gettimeofday.o
│ │ │ +?rw-r--r--   0        0        0      516 1970-01-01 00:00:00.000000 times.o
│ │ │ +?rw-r--r--   0        0        0      560 1970-01-01 00:00:00.000000 preadv.o
│ │ │ +?rw-r--r--   0        0        0      563 1970-01-01 00:00:00.000000 pwritev.o
│ │ │ +?rw-r--r--   0        0        0      543 1970-01-01 00:00:00.000000 readv.o
│ │ │ +?rw-r--r--   0        0        0      546 1970-01-01 00:00:00.000000 writev.o
│ │ │ +?rw-r--r--   0        0        0      303 1970-01-01 00:00:00.000000 CLOCK_MONOTONIC.o
│ │ │  ?rw-r--r--   0        0        0      321 1970-01-01 00:00:00.000000 CLOCK_PROCESS_CPUTIME_ID.o
│ │ │ +?rw-r--r--   0        0        0      301 1970-01-01 00:00:00.000000 CLOCK_REALTIME.o
│ │ │  ?rw-r--r--   0        0        0      319 1970-01-01 00:00:00.000000 CLOCK_THREAD_CPUTIME_ID.o
│ │ │  ?rw-r--r--   0        0        0      455 1970-01-01 00:00:00.000000 clock.o
│ │ │  ?rw-r--r--   0        0        0      558 1970-01-01 00:00:00.000000 clock_getres.o
│ │ │ -?rw-r--r--   0        0        0      441 1970-01-01 00:00:00.000000 nanosleep.o
│ │ │ -?rw-r--r--   0        0        0      303 1970-01-01 00:00:00.000000 CLOCK_MONOTONIC.o
│ │ │  ?rw-r--r--   0        0        0      589 1970-01-01 00:00:00.000000 clock_gettime.o
│ │ │ -?rw-r--r--   0        0        0      357 1970-01-01 00:00:00.000000 _Exit.o
│ │ │ -?rw-r--r--   0        0        0      801 1970-01-01 00:00:00.000000 fcntl.o
│ │ │ -?rw-r--r--   0        0        0      385 1970-01-01 00:00:00.000000 posix_fadvise.o
│ │ │ -?rw-r--r--   0        0        0      891 1970-01-01 00:00:00.000000 openat.o
│ │ │ -?rw-r--r--   0        0        0      384 1970-01-01 00:00:00.000000 posix_fallocate.o
│ │ │ -?rw-r--r--   0        0        0      427 1970-01-01 00:00:00.000000 sched_yield.o
│ │ │ -?rw-r--r--   0        0        0      487 1970-01-01 00:00:00.000000 renameat.o
│ │ │ -?rw-r--r--   0        0        0     3016 1970-01-01 00:00:00.000000 posix.o
│ │ │ -?rw-r--r--   0        0        0      428 1970-01-01 00:00:00.000000 reallocarray.o
│ │ │ -?rw-r--r--   0        0        0      329 1970-01-01 00:00:00.000000 complex-builtins.o
│ │ │ -?rw-r--r--   0        0        0      989 1970-01-01 00:00:00.000000 __wasilibc_initialize_environ.o
│ │ │ -?rw-r--r--   0        0        0      442 1970-01-01 00:00:00.000000 __wasilibc_fd_renumber.o
│ │ │ -?rw-r--r--   0        0        0      329 1970-01-01 00:00:00.000000 errno.o
│ │ │ +?rw-r--r--   0        0        0      793 1970-01-01 00:00:00.000000 clock_nanosleep.o
│ │ │ +?rw-r--r--   0        0        0      441 1970-01-01 00:00:00.000000 nanosleep.o
│ │ │ +?rw-r--r--   0        0        0      481 1970-01-01 00:00:00.000000 time.o
│ │ │ +?rw-r--r--   0        0        0      416 1970-01-01 00:00:00.000000 close.o
│ │ │ +?rw-r--r--   0        0        0      775 1970-01-01 00:00:00.000000 faccessat.o
│ │ │ +?rw-r--r--   0        0        0      435 1970-01-01 00:00:00.000000 fdatasync.o
│ │ │ +?rw-r--r--   0        0        0      423 1970-01-01 00:00:00.000000 fsync.o
│ │ │ +?rw-r--r--   0        0        0      463 1970-01-01 00:00:00.000000 ftruncate.o
│ │ │ +?rw-r--r--   0        0        0      491 1970-01-01 00:00:00.000000 linkat.o
│ │ │ +?rw-r--r--   0        0        0      537 1970-01-01 00:00:00.000000 lseek.o
│ │ │ +?rw-r--r--   0        0        0      688 1970-01-01 00:00:00.000000 pread.o
│ │ │ +?rw-r--r--   0        0        0      692 1970-01-01 00:00:00.000000 pwrite.o
│ │ │ +?rw-r--r--   0        0        0      537 1970-01-01 00:00:00.000000 read.o
│ │ │ +?rw-r--r--   0        0        0      566 1970-01-01 00:00:00.000000 readlinkat.o
│ │ │ +?rw-r--r--   0        0        0      447 1970-01-01 00:00:00.000000 sleep.o
│ │ │ +?rw-r--r--   0        0        0      486 1970-01-01 00:00:00.000000 symlinkat.o
│ │ │ +?rw-r--r--   0        0        0      388 1970-01-01 00:00:00.000000 unlinkat.o
│ │ │ +?rw-r--r--   0        0        0      534 1970-01-01 00:00:00.000000 usleep.o
│ │ │ +?rw-r--r--   0        0        0      540 1970-01-01 00:00:00.000000 write.o
│ │ │  ?rw-r--r--   0        0        0      321 1970-01-01 00:00:00.000000 __main_argc_argv.o
│ │ │ +?rw-r--r--   0        0        0      806 1970-01-01 00:00:00.000000 __main_void.o
│ │ │ +?rw-r--r--   0        0        0      321 1970-01-01 00:00:00.000000 __original_main.o
│ │ │ +?rw-r--r--   0        0        0      442 1970-01-01 00:00:00.000000 __wasilibc_fd_renumber.o
│ │ │ +?rw-r--r--   0        0        0      989 1970-01-01 00:00:00.000000 __wasilibc_initialize_environ.o
│ │ │  ?rw-r--r--   0        0        0      497 1970-01-01 00:00:00.000000 __wasilibc_rmdirat.o
│ │ │ -?rw-r--r--   0        0        0      263 1970-01-01 00:00:00.000000 abort.o
│ │ │ -?rw-r--r--   0        0        0     1970 1970-01-01 00:00:00.000000 preopens.o
│ │ │ -?rw-r--r--   0        0        0      458 1970-01-01 00:00:00.000000 getentropy.o
│ │ │  ?rw-r--r--   0        0        0      526 1970-01-01 00:00:00.000000 __wasilibc_tell.o
│ │ │ -?rw-r--r--   0        0        0      806 1970-01-01 00:00:00.000000 __main_void.o
│ │ │ +?rw-r--r--   0        0        0      488 1970-01-01 00:00:00.000000 __wasilibc_unlinkat.o
│ │ │ +?rw-r--r--   0        0        0      263 1970-01-01 00:00:00.000000 abort.o
│ │ │ +?rw-r--r--   0        0        0      329 1970-01-01 00:00:00.000000 complex-builtins.o
│ │ │  ?rw-r--r--   0        0        0      476 1970-01-01 00:00:00.000000 environ.o
│ │ │ -?rw-r--r--   0        0        0      425 1970-01-01 00:00:00.000000 sbrk.o
│ │ │ -?rw-r--r--   0        0        0      518 1970-01-01 00:00:00.000000 truncate.o
│ │ │ +?rw-r--r--   0        0        0      329 1970-01-01 00:00:00.000000 errno.o
│ │ │ +?rw-r--r--   0        0        0      458 1970-01-01 00:00:00.000000 getentropy.o
│ │ │  ?rw-r--r--   0        0        0      549 1970-01-01 00:00:00.000000 isatty.o
│ │ │ -?rw-r--r--   0        0        0      541 1970-01-01 00:00:00.000000 math-builtins.o
│ │ │  ?rw-r--r--   0        0        0      435 1970-01-01 00:00:00.000000 fmin-fmax.o
│ │ │ -?rw-r--r--   0        0        0      488 1970-01-01 00:00:00.000000 __wasilibc_unlinkat.o
│ │ │ -?rw-r--r--   0        0        0      321 1970-01-01 00:00:00.000000 __original_main.o
│ │ │ +?rw-r--r--   0        0        0      541 1970-01-01 00:00:00.000000 math-builtins.o
│ │ │ +?rw-r--r--   0        0        0     3016 1970-01-01 00:00:00.000000 posix.o
│ │ │ +?rw-r--r--   0        0        0     1970 1970-01-01 00:00:00.000000 preopens.o
│ │ │ +?rw-r--r--   0        0        0      428 1970-01-01 00:00:00.000000 reallocarray.o
│ │ │ +?rw-r--r--   0        0        0      425 1970-01-01 00:00:00.000000 sbrk.o
│ │ │ +?rw-r--r--   0        0        0      518 1970-01-01 00:00:00.000000 truncate.o
│ │ │  ?rw-r--r--   0        0        0      976 1970-01-01 00:00:00.000000 a64l.o
│ │ │  ?rw-r--r--   0        0        0      529 1970-01-01 00:00:00.000000 basename.o
│ │ │  ?rw-r--r--   0        0        0      643 1970-01-01 00:00:00.000000 dirname.o
│ │ │  ?rw-r--r--   0        0        0      272 1970-01-01 00:00:00.000000 ffs.o
│ │ │  ?rw-r--r--   0        0        0      273 1970-01-01 00:00:00.000000 ffsl.o
│ │ │  ?rw-r--r--   0        0        0      276 1970-01-01 00:00:00.000000 ffsll.o
│ │ │  ?rw-r--r--   0        0        0     2409 1970-01-01 00:00:00.000000 fmtmsg.o

sbc100

Maybe worth adding a comment about where we are sorting these? Specifically so that the inputs llvm-ar are sort, I think?

It makes me wonder if we should do that sorting instead at the point of calling llvm-ar instead?

sbc100 · 2021-08-25T23:32:10Z

Makefile

 	"$(WASM_CC)" $(CFLAGS) -MD -MP -o $@ -c $<

-include $(shell find $(OBJDIR) -name \*.d)
+-include $(shell find $(OBJDIR) -name \*.d | LC_ALL=C sort)


I don't think this change should be needed.

abrown · 2023-07-11T22:59:18Z

Closing... it looks like #399 already solved this. Feel free to reopen if that didn't cover everything!

Reproducible Builds: Ensure object files are sorted

8f8973a

sbc100 reviewed Aug 25, 2021

View reviewed changes

abrown closed this Jul 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reproducible Builds: Ensure object files are sorted#254

Reproducible Builds: Ensure object files are sorted#254
kpcyrd wants to merge 1 commit intoWebAssembly:mainfrom
kpcyrd:repro-builds

kpcyrd commented Aug 25, 2021

Uh oh!

sbc100 left a comment

Uh oh!

sbc100 Aug 25, 2021

Uh oh!

abrown commented Jul 11, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

kpcyrd commented Aug 25, 2021

/usr/lib/tinygo/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a:

Uh oh!

sbc100 left a comment

Choose a reason for hiding this comment

Uh oh!

sbc100 Aug 25, 2021

Choose a reason for hiding this comment

Uh oh!

abrown commented Jul 11, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants