cygwin support

[corngood]

I had a look at making zig target cygwin, and got most of the way there: cygwin target definition, stdlib changes, etc. but I got stuck on the linker.

The lld COFF linker doesn't support linker scripts, but the cygwin linker script is needed to set up __data_start__ etc.

Any thoughts on where to go from here? Is it a waste of time?

[andrewrk]

I think you might have better luck in the llvm6 branch, but there are some zig additions needed. LLD added a MinGW driver API which is probably what we should use for cygwin.

[andrewrk]

LLVM 6 rc2 just came out today. It's scheduled to be released at the end of February. Zig will release 0.2.0 1 week after that, at which point master branch will depend on LLD 6 and we can focus on cygwin support.

[corngood]

Interesting. I was looking for COFF diffs in llvm6 today, but I didn't notice the MinGW driver.

It looks like in it's current state it's just doing some simple argument translation, so I'm doubtful it'll help too much since zig already deals with msvc-style linker args.

I'll take a look at the llvm6 branch.

[andrewrk]

we could also ask a question on the llvm mailing list. The LLD maintainers are pretty responsive when it comes to COFF. I believe there should be a lot of interest in getting this working since it benefits the entire LLVM community and not just zig.

[andrewrk]

maybe we are supposed to use the ELF linker interface from LLD for cygwin? and the target triple would be x86_64-windows-gnu ?

[corngood]

I think windows-gnu is mingw, and I'm pretty sure lld/ELF is only for elf format objects, but I could be wrong.

https://llvm.org/doxygen/Triple_8h_source.html#l00542

I'll do a bit more digging.

[andrewrk]

Ah right, of course, the cygnus environment. So we can use that to detect that we should link for cygwin. (I'm guessing you are already doing that in your branch.)

I sent an email to the mailing list: http://lists.llvm.org/pipermail/llvm-dev/2018-February/121066.html

[andrewrk]

If you have some specific need, please explain. We may be able to help.

It looks like the LLD team is willing to work with us to support cygwin.

[corngood]

But I don't think that you actually need all the features of the linker script.

I was thinking that this probably is the case, I just need to figure out what's needed. __begin_data__ etc are probably pretty simple to implement. I wasn't sure about interpreting the linker script vs. just implementing this stuff directly in the linker, but it sounds like the latter would be the way to go.

In my branch I added OsCygwin / Os.cygwin, selected for -windows-cygnus. I did that so I wouldn't have to deal with is_windows / OsWindows, since the fallbacks are usually posix friendly. I consider that a hack though, and I think it would be better to use the llvm triple values as-is. We could add some helpers, and then most of the current is_windows stuff would be covered by something like:

   /// Is this a "Windows" OS targeting a "MSVCRT.dll" environment.
   bool isOSMSVCRT() const {
     return isWindowsMSVCEnvironment() || isWindowsGNUEnvironment() ||
            isWindowsItaniumEnvironment();
  }

But even that's not entirely accurate, because we can target:

• win32
• msvcrt -> win32
• libcygwin -> win32

So it would be odd to call the first one MSVC when it doesn't have anything to do with MSVC.

Another source of inspiration is how nixpkgs handles them:
https://github.com/NixOS/nixpkgs/blob/master/lib/systems/parse.nix

[andrewrk]

Thanks, this all makes sense.

Can you show me the linker script that you would want to use, and maybe I can look at it and suggest how we can achieve it in a different way?

[corngood]

Sure, what I have now:

lld: warning: ./zig-cache/hello.o: undefined symbol: __stack_chk_guard
lld: warning: ./zig-cache/hello.o: undefined symbol: __stack_chk_fail
lld: warning: ./zig-cache/hello.o: undefined symbol: __error
lld: warning: libcygwin.a(_cygwin_crt0_common.o): undefined symbol: __data_start__
lld: warning: libcygwin.a(_cygwin_crt0_common.o): undefined symbol: __data_end__
lld: warning: libcygwin.a(_cygwin_crt0_common.o): undefined symbol: __bss_start__
lld: warning: libcygwin.a(_cygwin_crt0_common.o): undefined symbol: __bss_end__
lld: warning: libcygwin.a(_cygwin_crt0_common.o): undefined symbol: __RUNTIME_PSEUDO_RELOC_LIST__
lld: warning: libcygwin.a(_cygwin_crt0_common.o): undefined symbol: __RUNTIME_PSEUDO_RELOC_LIST_END__
lld: warning: libcygwin.a(_cygwin_crt0_common.o): undefined symbol: __image_base__

The hello.o ones are probably just a missing library. The libcywin ones seem to be from the linker script. It's built into ld, so I dumped it with -verbose:

GNU ld (GNU Binutils) 2.29.1.20171006
  Supported emulations:
   i386pep
   i386pe
using internal linker script:
==================================================
/* Default linker script, for normal executables */
/* Copyright (C) 2014-2017 Free Software Foundation, Inc.
   Copying and distribution of this script, with or without modification,
   are permitted in any medium without royalty provided the copyright
   notice and this notice are preserved.  */
OUTPUT_FORMAT(pei-x86-64)
SEARCH_DIR("/usr/x86_64-pc-cygwin/lib"); SEARCH_DIR("/usr/lib"); SEARCH_DIR("/usr/lib/w32api");
SECTIONS
{
  /* Make the virtual address and file offset synced if the alignment is
     lower than the target page size. */
  . = SIZEOF_HEADERS;
  . = ALIGN(__section_alignment__);
  .text  __image_base__ + ( __section_alignment__ < 0x1000 ? . : __section_alignment__ ) :
  {
     KEEP(*(.init))
    *(.text)
    *(SORT(.text$*))
     *(.text.*)
     *(.gnu.linkonce.t.*)
    *(.glue_7t)
    *(.glue_7)
    . = ALIGN(8);
     ___CTOR_LIST__ = .; __CTOR_LIST__ = . ;
			LONG (-1); LONG (-1);
			KEEP (*(.ctors));
			KEEP (*(.ctor));
			KEEP (*(SORT(.ctors.*)));
			LONG (0); LONG (0);
     ___DTOR_LIST__ = .; __DTOR_LIST__ = . ;
			LONG (-1); LONG (-1);
			KEEP (*(.dtors));
			KEEP (*(.dtor));
			KEEP (*(SORT(.dtors.*)));
			LONG (0); LONG (0);
     KEEP (*(.fini))
    /* ??? Why is .gcc_exc here?  */
     *(.gcc_exc)
    PROVIDE (etext = .);
     KEEP (*(.gcc_except_table))
  }
  /* The Cygwin32 library uses a section to avoid copying certain data
     on fork.  This used to be named ".data".  The linker used
     to include this between __data_start__ and __data_end__, but that
     breaks building the cygwin32 dll.  Instead, we name the section
     ".data_cygwin_nocopy" and explicitly include it after __data_end__. */
  .data BLOCK(__section_alignment__) :
  {
    __data_start__ = . ;
    *(.data)
    *(.data2)
    *(SORT(.data$*))
    KEEP(*(.jcr))
    __data_end__ = . ;
    *(.data_cygwin_nocopy)
  }
  .rdata BLOCK(__section_alignment__) :
  {
    *(.rdata)
             *(SORT(.rdata$*))
    __rt_psrelocs_start = .;
    KEEP(*(.rdata_runtime_pseudo_reloc))
    __rt_psrelocs_end = .;
  }
  __rt_psrelocs_size = __rt_psrelocs_end - __rt_psrelocs_start;
  ___RUNTIME_PSEUDO_RELOC_LIST_END__ = .;
  __RUNTIME_PSEUDO_RELOC_LIST_END__ = .;
  ___RUNTIME_PSEUDO_RELOC_LIST__ = . - __rt_psrelocs_size;
  __RUNTIME_PSEUDO_RELOC_LIST__ = . - __rt_psrelocs_size;
  .eh_frame BLOCK(__section_alignment__) :
  {
    KEEP (*(.eh_frame*))
  }
  .pdata BLOCK(__section_alignment__) :
  {
    KEEP(*(.pdata*))
  }
  .xdata BLOCK(__section_alignment__) :
  {
    KEEP(*(.xdata*))
  }
  .bss BLOCK(__section_alignment__) :
  {
    __bss_start__ = . ;
    *(.bss)
    *(COMMON)
    __bss_end__ = . ;
  }
  .edata BLOCK(__section_alignment__) :
  {
    *(.edata)
  }
  /DISCARD/ :
  {
    *(.debug$S)
    *(.debug$T)
    *(.debug$F)
    *(.drectve)
     *(.note.GNU-stack)
     *(.gnu.lto_*)
  }
  .idata BLOCK(__section_alignment__) :
  {
    /* This cannot currently be handled with grouped sections.
	See pep.em:sort_sections.  */
    KEEP (SORT(*)(.idata$2))
    KEEP (SORT(*)(.idata$3))
    /* These zeroes mark the end of the import list.  */
    LONG (0); LONG (0); LONG (0); LONG (0); LONG (0);
    KEEP (SORT(*)(.idata$4))
    __IAT_start__ = .;
    SORT(*)(.idata$5)
    __IAT_end__ = .;
    KEEP (SORT(*)(.idata$6))
    KEEP (SORT(*)(.idata$7))
  }
  .CRT BLOCK(__section_alignment__) :
  {
    ___crt_xc_start__ = . ;
    KEEP (*(SORT(.CRT$XC*)))  /* C initialization */
    ___crt_xc_end__ = . ;
    ___crt_xi_start__ = . ;
    KEEP (*(SORT(.CRT$XI*)))  /* C++ initialization */
    ___crt_xi_end__ = . ;
    ___crt_xl_start__ = . ;
    KEEP (*(SORT(.CRT$XL*)))  /* TLS callbacks */
    /* ___crt_xl_end__ is defined in the TLS Directory support code */
    ___crt_xp_start__ = . ;
    KEEP (*(SORT(.CRT$XP*)))  /* Pre-termination */
    ___crt_xp_end__ = . ;
    ___crt_xt_start__ = . ;
    KEEP (*(SORT(.CRT$XT*)))  /* Termination */
    ___crt_xt_end__ = . ;
  }
  /* Windows TLS expects .tls$AAA to be at the start and .tls$ZZZ to be
     at the end of the .tls section.  This is important because _tls_start MUST
     be at the beginning of the section to enable SECREL32 relocations with TLS
     data.  */
  .tls BLOCK(__section_alignment__) :
  {
    ___tls_start__ = . ;
    KEEP (*(.tls$AAA))
    KEEP (*(.tls))
    KEEP (*(.tls$))
    KEEP (*(SORT(.tls$*)))
    KEEP (*(.tls$ZZZ))
    ___tls_end__ = . ;
  }
  .endjunk BLOCK(__section_alignment__) :
  {
    /* end is deprecated, don't use it */
    PROVIDE (end = .);
    PROVIDE ( _end = .);
     __end__ = .;
  }
  .rsrc BLOCK(__section_alignment__) : SUBALIGN(4)
  {
    KEEP (*(.rsrc))
    KEEP (*(.rsrc$*))
  }
  .reloc BLOCK(__section_alignment__) :
  {
    *(.reloc)
  }
  .stab BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.stab)
  }
  .stabstr BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.stabstr)
  }
  /* DWARF debug sections.
     Symbols in the DWARF debugging sections are relative to the beginning
     of the section.  Unlike other targets that fake this by putting the
     section VMA at 0, the PE format will not allow it.  */
  /* DWARF 1.1 and DWARF 2.  */
  .debug_aranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_aranges)
  }
  .zdebug_aranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_aranges)
  }
  .debug_pubnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_pubnames)
  }
  .zdebug_pubnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_pubnames)
  }
  .debug_pubtypes BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_pubtypes)
  }
  .zdebug_pubtypes BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_pubtypes)
  }
  /* DWARF 2.  */
  .debug_info BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_info .gnu.linkonce.wi.*)
  }
  .zdebug_info BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_info .zdebug.gnu.linkonce.wi.*)
  }
  .debug_abbrev BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_abbrev)
  }
  .zdebug_abbrev BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_abbrev)
  }
  .debug_line BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_line)
  }
  .zdebug_line BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_line)
  }
  .debug_frame BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_frame)
  }
  .zdebug_frame BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_frame)
  }
  .debug_str BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_str)
  }
  .zdebug_str BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_str)
  }
  .debug_loc BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_loc)
  }
  .zdebug_loc BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_loc)
  }
  .debug_macinfo BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_macinfo)
  }
  .zdebug_macinfo BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_macinfo)
  }
  /* SGI/MIPS DWARF 2 extensions.  */
  .debug_weaknames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_weaknames)
  }
  .zdebug_weaknames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_weaknames)
  }
  .debug_funcnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_funcnames)
  }
  .zdebug_funcnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_funcnames)
  }
  .debug_typenames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_typenames)
  }
  .zdebug_typenames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_typenames)
  }
  .debug_varnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_varnames)
  }
  .zdebug_varnames BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_varnames)
  }
  .debug_macro BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_macro)
  }
  .zdebug_macro BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_macro)
  }
  /* DWARF 3.  */
  .debug_ranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_ranges)
  }
  .zdebug_ranges BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_ranges)
  }
  /* DWARF 4.  */
  .debug_types BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_types .gnu.linkonce.wt.*)
  }
  .zdebug_types BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_types .zdebug.gnu.linkonce.wt.*)
  }
  /* For Go and Rust.  */
  .debug_gdb_scripts BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.debug_gdb_scripts)
  }
  .zdebug_gdb_scripts BLOCK(__section_alignment__) (NOLOAD) :
  {
    *(.zdebug_gdb_scripts)
  }
}

[andrewrk]

As you mentioned, the hello.o errors can be fixed by implementing the named functions in std/special/builtin.zig.

I'll send an email to LLVM developers for the others.

[corngood]

Thanks. There are a few examples of synthetic symbols in the COFF linker, so I'll see if I can get something working myself.