Try `mustang` on real-world programs!

[sunfishcode]

Mustang is now complete enough to support the current directory, command-line arguments, environment variables, std::fs, std::net, std::thread, std::time,std::alloc, and everything in the standard library that doesn't involve I/O too. It can now run some real-world programs! To try it:

• Identify a program to try. Ideally avoid the known limitations, however reports confirming the known limitations are still useful.

• Follow the usage instructions in the README.

• Post a comment here about whatever happens—success, failure of any kind, or something was unclear.

[jplatte]

I tried to run an async web app using tokio, it failed in tokio initialization because sched_getaffinity (used by num_cpus) is not implemented 🙁

What the error looks like

thread 'main' panicked at 'not implemented: sched_getaffinity', /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/c-scape-0.0.0/src/lib.rs:2289:5
stack backtrace:
   0: rust_begin_unwind
             at /home/jplatte/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:517:5
   1: core::panicking::panic_fmt
             at /home/jplatte/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panicking.rs:100:14
   2: sched_getaffinity
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/c-scape-0.0.0/src/lib.rs:2289:5
   3: num_cpus::linux::logical_cpus
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/num_cpus-1.13.0/src/linux.rs:41:17
   4: num_cpus::linux::get_num_cpus
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/num_cpus-1.13.0/src/linux.rs:35:17
   5: num_cpus::get
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/num_cpus-1.13.0/src/lib.rs:73:5
   6: tokio::loom::std::sys::num_cpus
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.11.0/src/loom/std/mod.rs:87:23
   7: core::ops::function::FnOnce::call_once
             at /home/jplatte/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
   8: core::option::Option<T>::unwrap_or_else
             at /home/jplatte/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:784:21
   9: tokio::runtime::builder::Builder::build_threaded_runtime
             at /home/jplatte/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.11.0/src/runtime/builder.rs:545:32
   [...]

When using the current_thread runtime instead, it doesn't fail to initialize but still doesn't work (worker threads panic and new ones are started constantly when trying to connect to the app AFAICT), due to unimplemented condvar stuff mentioned in the readme.

Also, nm -u does return some output for the binary:

                 U epoll_create1@GLIBC_2.9
                 U epoll_ctl@GLIBC_2.3.2
                 U epoll_wait@GLIBC_2.3.2
                 U eventfd@GLIBC_2.7

[jplatte]

I found an application I wrote that works with mustang: https://github.com/jplatte/i3-workspace-scroll

It's < 50 LOC but it uses command line arguments and some IPC through a socket file.

One thing I was curious about was file sizes. Here's a comparison:

	Debug	Release	Mustang+Debug	Mustang+Release
Original	7.1M	3.7M	25.3M	2.0M
Stripped	691K	447K	4.6M	1.4M

The sizes of the .so files the application links against w/o mustang (libc, libdl, libgcc_s, libm, ld-linux-x86-64) add up to about 4.1M.

[sunfishcode]

Thanks for trying it out! I believe you're the first person to have tried anything with async 😄 . Fortunately, I expect sched_getaffinity, epoll_* and eventfd will be pretty straightforward.

Concerning code size, mustang currently forces its entire libc to be statically linked in, in order to prevent the linker from satisfying any symbol references with the platform libc. I'm hoping to find a better approach there.

[sunfishcode]

With #45, mustang no longer forces its entire libc to be statically linked in, and file sizes are much smaller. For example, unstripped release hello world on x86_64 went from 1.8M to 640K.

[tshepang]

Using latest (main) of mustang and rsix, here's what I get when running ripgrep:

...
     Running `target/x86_64-mustang-linux-gnu/debug/rg foo`
thread '<unnamed>' panicked at 'assertion failed: page_size != 0', /home/tshepang/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/thread.rs:465:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'attempt to create unaligned or null slice', /home/tshepang/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/slice/raw.rs:89:5
stack backtrace:
thread panicked while processing panic. aborting.

[sunfishcode]

@tshepang Ooh, good idea trying ripgrip :-). It looks like one thing ripgrep needs is sched_getaffinity, which should help tokio too. I've now filed #56 to track that.

[sunfishcode]

Thanks to @nivkner implementing sched_getaffinity, ripgrip kind of runs now!

When ripgrep runs on a single file, instead of on a directory, it uses mmap via the memmap2 crate, which calls libc::fstat, which isn't implemented yet (#71).

When ripgrip runs on directories, it appears to run successfully, but it also prints out several messages about unimplemented pthread_cond* functions:

unimplemented: pthread_condattr_init
unimplemented: pthread_condattr_setclock
unimplemented: pthread_cond_init
unimplemented: pthread_condattr_destroy
unimplemented: pthread_cond_broadcast
unimplemented: pthread_cond_destroy

This is #40.

[antonok-edm]

I tried running bat with mustang. Following the README, I installed rust-src, set RUST_TARGET_PATH, added mustang = "0.3.0" to bat's dependencies, added mustang::can_run_this!(); to src/bin/bat/main.rs, and built with cargo build --release -Z build-std --target=x86_64-mustang-linux-gnu.

indexmap appears multiple times in bat's dependency tree, and has #[cfg(has_std)] attributes that don't work correctly with mustang. This issue has more info on how it works. I was able to fix it by adding this to [dependencies]:

indexmap = { version = "*", features = ["std"] }

I also ran into an an unrelated regression on Nightly which was easy enough to work around.

After the above modifications, it successfully passed the compilation step, but failed during linking. The traceback is ~1200 lines long; I'm happy to post it if it's helpful, but it looks generally like this:

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "-nostartfiles" "-Wl,--undefined=_Unwind_Backtrace" <...extremely long list of .o files, -L, .rlib files, etc.>
  = note: /usr/bin/ld: /tmp/bat/target/x86_64-mustang-linux-gnu/debug/deps/liblibgit2_sys-e0d42202f97337e4.rlib(attr.o): in function `git_attr_get_many_with_session':
          /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.12.21+1.1.0/libgit2/src/attr.c:183: undefined reference to `__stack_chk_fail'
          /usr/bin/ld: /tmp/bat/target/x86_64-mustang-linux-gnu/debug/deps/liblibgit2_sys-e0d42202f97337e4.rlib(attr.o): in function `preload_attr_file':
          /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/libgit2-sys-0.12.21+1.1.0/libgit2/src/attr.c:274: undefined reference to `__stack_chk_fail'
          <...many more undefined references to __stack_chk_fail, strcmp, strcasecmp, strrchr, strncmp, vsnprintf, etc.>
          collect2: error: ld returned 1 exit status

  = help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

It seems like I must have configured something incorrectly, but I can't tell from the README.

[sunfishcode]

Cool, thanks for trying bat, that's a fun idea! It looks like you've configured everything correctly; the errors you're seeing there are bugs in mustang.

Specifically, mustang so far has focused on running Rust code, and bat uses libgit2 written in C. Mustang's libc support so far only covers things used by Rust, so it doesn't yet have all the things that are common to use in C programs. #69 is a PR to add a note about this to the README.md.

A possible workaround might be to disable the git feature when building bat. I don't know if there is other C code in bat's dependency tree though.

[antonok-edm]

Neat! That makes sense, I see liblibgit2 in the errors above now. Disabling the git feature helped, then it turns out regex-onig uses C code so I had to swap it out for the pure-Rust regex-fancy feature instead.

That narrowed the errors down to just this one:

  = note: /usr/bin/ld: /tmp/bat/target/x86_64-mustang-linux-gnu/debug/deps/libstd-e475f264d98f5821.rlib(std-e475f264d98f5821.std.a79cc641-cgu.2.rcgu.o): in function `std::sys::unix::os::glibc_version':
          /home/antonok/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/os.rs:644: undefined reference to `gnu_get_libc_version'

That's in the standard library, so I went ahead and manually modified that sys/unix/os.rs in my toolchain directory:

 #[cfg(all(target_os = "linux", target_env = "gnu"))]
 pub fn glibc_version() -> Option<(usize, usize)> {
+    None
-    extern "C" {
-        fn gnu_get_libc_version() -> *const libc::c_char;
-    }
-    let version_cstr = unsafe { CStr::from_ptr(gnu_get_libc_version()) };
-    if let Ok(version_str) = version_cstr.to_str() {
-        parse_glibc_version(version_str)
-    } else {
-        None
-    }
}

After a clean rebuild, it successfully compiled!

It panics when running, which I guess is the point of this issue 😄

RUST_BACKTRACE=1 cargo run -Z build-std --target=x86_64-mustang-linux-gnu -- LICENSE-APACHE
    Finished dev [unoptimized + debuginfo] target(s) in 0.08s
     Running `target/x86_64-mustang-linux-gnu/debug/bat LICENSE-APACHE`
thread 'main' panicked at 'unrecognized ioctl(21523)', /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/c-scape-0.3.0/src/lib.rs:1512:14
stack backtrace:
   0: rust_begin_unwind
             at /home/antonok/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /home/antonok/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panicking.rs:107:14
   2: ioctl
             at /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/c-scape-0.3.0/src/lib.rs:1512:14
   3: terminal_size::unix::terminal_size_using_fd
             at /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/terminal_size-0.1.16/src/unix.rs:31:17
   4: console::unix_term::terminal_size
             at /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/console-0.15.0/src/unix_term.rs:47:5
   5: console::term::Term::size_checked
             at /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/console-0.15.0/src/term.rs:392:9
   6: console::term::Term::size
             at /home/antonok/.config/cargo/registry/src/github.com-1ecc6299db9ec823/console-0.15.0/src/term.rs:384:9
   7: bat::app::App::config
             at ./src/bin/bat/app.rs:182:52
   8: bat::run
             at ./src/bin/bat/main.rs:303:26
   9: bat::main
             at ./src/bin/bat/main.rs:337:18
  10: core::ops::function::FnOnce::call_once
             at /home/antonok/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

So that's coming from https://github.com/eminence/terminal-size/blob/master/src/unix.rs#L31:

ioctl(fd, TIOCGWINSZ.into(), &mut winsize)

On the other hand, if I prevent that code from getting run with a small patch:

diff --git a/src/bin/bat/app.rs b/src/bin/bat/app.rs
index 842eec6..b60276f 100644
--- a/src/bin/bat/app.rs
+++ b/src/bin/bat/app.rs
@@ -179,7 +179,7 @@ impl App {
                     _ => unreachable!("other values for --color are not allowed"),
                 },
             paging_mode,
-            term_width: maybe_term_width.unwrap_or(Term::stdout().size().1 as usize),
+            term_width: maybe_term_width.unwrap_or_else(|| Term::stdout().size().1 as usize),
             loop_through: !(self.interactive_output
                 || self.matches.value_of("color") == Some("always")
                 || self.matches.value_of("decorations") == Some("always")

...and pass the --terminal-width argument manually on the command line, I can actually view all sorts of files with correct syntax highlighting and paging interactions!

It's very slow though, and I still can't get it to link in Release mode. I get ~250 lines of undefined reference errors, which appear to all be originating from c-scape. Some random examples, happy to post a full log if that's helpful as well:

  = note: /usr/bin/ld: /tmp/bat/target/x86_64-mustang-linux-gnu/release/deps/libc_scape-2f6985a442db2685.rlib(c_scape-2f6985a442db2685.c_scape.e6de75ef-cgu.0.rcgu.o): in function `<&T as core::fmt::Debug>::fmt':
          c_scape.e6de75ef-cgu.0:(.text._ZN42_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$3fmt17h184ef3cd73ec6070E+0xf): undefined reference to `core::fmt::Formatter::debug_lower_hex'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text._ZN42_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$3fmt17h184ef3cd73ec6070E+0x26): undefined reference to `core::fmt::num::<impl core::fmt::LowerHex for usize>::fmt'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text._ZN42_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$3fmt17h184ef3cd73ec6070E+0x2f): undefined reference to `core::fmt::Formatter::debug_upper_hex'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text._ZN42_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$3fmt17h184ef3cd73ec6070E+0x46): undefined reference to `core::fmt::num::<impl core::fmt::UpperHex for usize>::fmt'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text._ZN42_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$3fmt17h184ef3cd73ec6070E+0x4f): undefined reference to `core::fmt::num::imp::<impl core::fmt::Display for usize>::fmt'
          ...
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.unlikely._ZN5alloc7raw_vec19RawVec$LT$T$C$A$GT$7reserve21do_reserve_and_handle17hbd196bcaed0b3fbcE+0xca): undefined reference to `alloc::alloc::handle_alloc_error'
          /usr/bin/ld: /tmp/bat/target/x86_64-mustang-linux-gnu/release/deps/libc_scape-2f6985a442db2685.rlib(c_scape-2f6985a442db2685.c_scape.e6de75ef-cgu.0.rcgu.o): in function `origin::program::exit_immediately':
          c_scape.e6de75ef-cgu.0:(.text._ZN6origin7program16exit_immediately17h8677a8b25e83e50cE+0xa): undefined reference to `log::MAX_LOG_LEVEL_FILTER'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text._ZN6origin7program16exit_immediately17h8677a8b25e83e50cE+0x5a): undefined reference to `log::__private_api_log'
          /usr/bin/ld: /tmp/bat/target/x86_64-mustang-linux-gnu/release/deps/libc_scape-2f6985a442db2685.rlib(c_scape-2f6985a442db2685.c_scape.e6de75ef-cgu.0.rcgu.o): in function `__xpg_strerror_r':
          c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x19): undefined reference to `rustix::imp::linux_raw::io::error::Error::from_errno'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x4df): undefined reference to `__rust_alloc'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x556): undefined reference to `__rust_dealloc'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x5f5): undefined reference to `core::fmt::num::imp::<impl core::fmt::Display for i32>::fmt'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x63b): undefined reference to `alloc::fmt::format'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x66f): undefined reference to `core::slice::<impl [T]>::copy_from_slice::len_mismatch_fail'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x683): undefined reference to `core::panicking::panic_bounds_check'
          /usr/bin/ld: c_scape.e6de75ef-cgu.0:(.text.__xpg_strerror_r+0x693): undefined reference to `alloc::alloc::handle_alloc_error'
          ...

[sunfishcode]

Cool! #68 adds gnu_get_libc_version and #70 adds the TIOCGWINSZ ioctl.

I don't know what's going on with those release mode link errors yet though. Could you post the full log? You can use <details> and </details> tags to hide the log behind a click in a github comment. Also, would you be able to try running mustang's hello example in --release mode?