Skip to content

Add CONFIGURATOR_WRAPPER#3464

Merged
rustyrussell merged 2 commits into
ElementsProject:masterfrom
NickeZ:fix-xc-builds
Feb 6, 2020
Merged

Add CONFIGURATOR_WRAPPER#3464
rustyrussell merged 2 commits into
ElementsProject:masterfrom
NickeZ:fix-xc-builds

Conversation

@NickeZ
Copy link
Copy Markdown
Contributor

@NickeZ NickeZ commented Jan 29, 2020

This was necessary for cross-compiling. Otherwise the configurator will be compiled for the host architecture.

Here are my scripts for cross compiling with docker, if you want to reproduce:

NickeZ/bitbox-base@cf7c4e8

Thanks

@cdecker cdecker requested a review from rustyrussell January 29, 2020 19:14
@cdecker cdecker added this to the 0.8.1 milestone Jan 29, 2020
@NickeZ
Copy link
Copy Markdown
Contributor Author

NickeZ commented Jan 29, 2020

After reading the code and some old PRs some more I realize that I might have misunderstood things. Am I supposed to cross compile configurator.c and then run it in QEMU?

@cdecker
Copy link
Copy Markdown
Member

cdecker commented Jan 30, 2020

After reading the code and some old PRs some more I realize that I might have misunderstood things. Am I supposed to cross compile configurator.c and then run it in QEMU?

I'm not sure tbh, I think the idea behind the configurator is to determine the target architecture capabilities. Running it on the host machine is unlikely to return the correct configuration (though most archs don't differ too much afaik). I think running it with the target configuration (natively or through QEMU) is the intended use, but @rustyrussell can likely say more to this :-)

@NickeZ
Copy link
Copy Markdown
Contributor Author

NickeZ commented Jan 30, 2020

I'm not sure tbh, I think the idea behind the configurator is to determine the target architecture capabilities. Running it on the host machine is unlikely to return the correct configuration (though most archs don't differ too much afaik). I think running it with the target configuration (natively or through QEMU) is the intended use, but @rustyrussell can likely say more to this :-)

Right, other config generators typically never run machine code for that reason. You have to use hacks, like generate compiler warnings, to figure out the capabilities of the host system.

But then there needs to be some variable that specifies a wrapper with which the configuration tool should be called so that it can be run in a virtual machine.

@NickeZ NickeZ changed the title Compile configurator with CONFIGURATOR_CC Add CONFIGURATOR_WRAPPER Jan 31, 2020
@NickeZ
Copy link
Copy Markdown
Contributor Author

NickeZ commented Jan 31, 2020

I updated this PR by adding an environment variable for wrapping the calls to configurator and also compile the configurator and all small programs it creates with -static to avoid the dynamic linker. This allows us to run configurator with qemu-aarch64-static.

I tested it like this:

CC=aarch64-linux-gnu-gcc CONFIGURATOR_WRAPPER=qemu-aarch64-static ./configure

@NickeZ
Copy link
Copy Markdown
Contributor Author

NickeZ commented Jan 31, 2020

this didn't help much I think. Doing it like this requires a native aarch64 compiler for the configurator.

@NickeZ NickeZ force-pushed the fix-xc-builds branch 3 times, most recently from 724e734 to d06c0f3 Compare January 31, 2020 18:28
@NickeZ
Copy link
Copy Markdown
Contributor Author

NickeZ commented Jan 31, 2020

new update: Now I build the configurator like before but I added a feature to wrap every call to the test binaries that the configurator compiles. This actually successfully compiles for me and I hope this was how the configurator was meant to be used.

@NickeZ
Copy link
Copy Markdown
Contributor Author

NickeZ commented Feb 1, 2020

Actually, this might not even be necessary. On modern debian (and I guess most distributions) it is actually possible to transparently run binaries for the wrong architecture. Somehow you can register with the kernel that it should launch binaries for certain architectures with qemu-static.

https://wiki.debian.org/QemuUserEmulation

@rustyrussell Maybe for cross-compiling on windows/osx you would still need to run the binaries with some wrapper. WDYT?

@remyers
Copy link
Copy Markdown
Contributor

remyers commented Feb 3, 2020

Maybe for cross-compiling on windows/osx you would still need to run the binaries with some wrapper. WDYT?

From what I can tell, running qemu-user-static on osx is not possible. I assume qemu-user-static would work in the Windows Subsystem for Linux (WSL) on windows, but it's unclear. A docker based alternative is really useful, thanks for that. I'll give it a try soon on osx.

rustyrussell and others added 2 commits February 4, 2020 11:12
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since the probing binaries compiled by the configurator needs to run on
the host machine we provide a variable CONFIGURATOR_WRAPPER that can be
set to anything that you want to wrap the calls with.

One example is `qemu-aarch64-static`.
@rustyrussell
Copy link
Copy Markdown
Contributor

I like this change, but I applied it to upstream (ccan/) instead, and then pulled that back in and used your ./configure patch.

Will apply once Travis acks, thanks!

Ack: 65a72d6

@rustyrussell
Copy link
Copy Markdown
Contributor

Ack 65a72d6

@rustyrussell rustyrussell merged commit de5eaf2 into ElementsProject:master Feb 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants