From 3f360ded54f57476ab66bb2ae8294d05a4bb1ede Mon Sep 17 00:00:00 2001 From: Luke Diamand Date: Wed, 6 Feb 2019 15:11:53 +0000 Subject: [PATCH 01/16] git-p4: remove ticket expiry test The git-p4 login ticket expiry test causes unreliable test runs. Since the handling of ticket expiry in git-p4 is far from polished anyway, let's remove it for now. A better way to actually run the test is to create a python "fake" version of "p4" which returns whatever expiry results the test requires. Ideally git-p4 would look at the expiry time before starting any long operations, and cleanup gracefully if there is not enough time left. But that's quite hard to do. Signed-off-by: Luke Diamand Signed-off-by: Junio C Hamano --- t/t9833-errors.sh | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/t/t9833-errors.sh b/t/t9833-errors.sh index 277d34701201af..47b312e1c9705a 100755 --- a/t/t9833-errors.sh +++ b/t/t9833-errors.sh @@ -45,33 +45,6 @@ test_expect_success 'ticket logged out' ' ) ' -test_expect_success 'create group with short ticket expiry' ' - P4TICKETS="$cli/tickets" && - echo "newpassword" | p4 login && - p4_add_user short_expiry_user && - p4 -u short_expiry_user passwd -P password && - p4 group -i <<-EOF && - Group: testgroup - Timeout: 3 - Users: short_expiry_user - EOF - - p4 users | grep short_expiry_user -' - -test_expect_success 'git operation with expired ticket' ' - P4TICKETS="$cli/tickets" && - P4USER=short_expiry_user && - echo "password" | p4 login && - ( - cd "$git" && - git p4 sync && - sleep 5 && - test_must_fail git p4 sync 2>errmsg && - grep "failure accessing depot" errmsg - ) -' - test_expect_success 'kill p4d' ' kill_p4d ' From 37c80012f1415bc4ff23def735a87cbd395bca31 Mon Sep 17 00:00:00 2001 From: Jeff King Date: Mon, 4 Mar 2019 23:11:39 -0500 Subject: [PATCH 02/16] fetch: avoid calling write_or_die() The write_or_die() function has one quirk that a caller might not expect: when it sees EPIPE from the write() call, it translates that into a death by SIGPIPE. This doesn't change the overall behavior (the program exits either way), but it does potentially confuse test scripts looking for a non-signal exit code. Let's switch away from using write_or_die() in a few code paths, which will give us more consistent exit codes. It also gives us the opportunity to write more descriptive error messages, since we have context that write_or_die() does not. Note that this won't do much by itself, since we'd typically be killed by SIGPIPE before write_or_die() even gets a chance to do its thing. That will be addressed in the next patch. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- fetch-pack.c | 9 ++++++--- pkt-line.c | 6 ++++-- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 9691046e644583..1dcb0f7cd79da2 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -189,8 +189,10 @@ static void send_request(struct fetch_pack_args *args, if (args->stateless_rpc) { send_sideband(fd, -1, buf->buf, buf->len, LARGE_PACKET_MAX); packet_flush(fd); - } else - write_or_die(fd, buf->buf, buf->len); + } else { + if (write_in_full(fd, buf->buf, buf->len) < 0) + die_errno(_("unable to write to remote")); + } } static void insert_one_alternate_object(struct fetch_negotiator *negotiator, @@ -1178,7 +1180,8 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, /* Send request */ packet_buf_flush(&req_buf); - write_or_die(fd_out, req_buf.buf, req_buf.len); + if (write_in_full(fd_out, req_buf.buf, req_buf.len) < 0) + die_errno(_("unable to write request to remote")); strbuf_release(&req_buf); return ret; diff --git a/pkt-line.c b/pkt-line.c index 04d10bbd037b39..9944c7c75d8963 100644 --- a/pkt-line.c +++ b/pkt-line.c @@ -88,13 +88,15 @@ static void packet_trace(const char *buf, unsigned int len, int write) void packet_flush(int fd) { packet_trace("0000", 4, 1); - write_or_die(fd, "0000", 4); + if (write_in_full(fd, "0000", 4) < 0) + die_errno(_("unable to write flush packet")); } void packet_delim(int fd) { packet_trace("0001", 4, 1); - write_or_die(fd, "0001", 4); + if (write_in_full(fd, "0001", 4) < 0) + die_errno(_("unable to write delim packet")); } int packet_flush_gently(int fd) From 143588949c8a0672302403c722fc433a5bb2ea2a Mon Sep 17 00:00:00 2001 From: Jeff King Date: Sun, 3 Mar 2019 11:58:43 -0500 Subject: [PATCH 03/16] fetch: ignore SIGPIPE during network operation The default SIGPIPE behavior can be useful for a command that generates a lot of output: if the receiver of our output goes away, we'll be notified asynchronously to stop generating it (typically by killing the program). But for a command like fetch, which is primarily concerned with receiving data and writing it to disk, an unexpected SIGPIPE can be awkward. We're already checking the return value of all of our write() calls, and dying due to the signal takes away our chance to gracefully handle the error. On Linux, we wouldn't generally see SIGPIPE at all during fetch. If the other side of the network connection hangs up, we'll see ECONNRESET. But on OS X, we get a SIGPIPE, and the process is killed. This causes t5570 to racily fail, as we sometimes die by signal (instead of the expected die() call) when the server side hangs up. Let's ignore SIGPIPE during the network portion of the fetch, which will cause our write() to return EPIPE, giving us consistent behavior across platforms. This fixes the test flakiness, but note that it stops short of fixing the larger problem. The server side hit a fatal error, sent us an "ERR" packet, and then hung up. We notice the failure because we're trying to write to a closed socket. But by dying immediately, we never actually read the ERR packet and report its content to the user. This is a (racy) problem on all platforms. So this patch lays the groundwork from which that problem might be fixed consistently, but it doesn't actually fix it. Note the placement of the SIGPIPE handling. The absolute minimal change would be to ignore SIGPIPE only when we're writing. But twiddling the signal handler for each write call is inefficient and maintenance burden. On the opposite end of the spectrum, we could simply declare that fetch does not need SIGPIPE handling, since it doesn't generate a lot of output, and we could just ignore it at the start of cmd_fetch(). This patch takes a middle ground. It ignores SIGPIPE during the network operation (which is admittedly most of the program, since the actual network operations are all done under the hood by the transport code). So it's still pretty coarse. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- builtin/fetch.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/fetch.c b/builtin/fetch.c index e0140327aab236..d10a67cc1c1618 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1554,7 +1554,9 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru sigchain_push_common(unlock_pack_on_signal); atexit(unlock_pack); + sigchain_push(SIGPIPE, SIG_IGN); exit_code = do_fetch(gtransport, &rs); + sigchain_pop(SIGPIPE); refspec_clear(&rs); transport_disconnect(gtransport); gtransport = NULL; From 4be16dbc19067245f02c0b34f5360ec5836ec838 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Sun, 14 Apr 2019 14:19:29 -0700 Subject: [PATCH 04/16] macOS: make sure that gettext is found Due to reasons (some XCode versions seem to include gettext, some don't?), Homebrew does not expose the libraries and headers in /usr/local/ by default anymore. Let's help find them again. Note: for some reason, this is a change of behavior caused by the upgrade to Mojave, identified in our Azure Pipeline; it seems that Homebrew used to add the /usr/local/ directories to the include and link search path before, but now it no longer does. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- config.mak.uname | 2 ++ 1 file changed, 2 insertions(+) diff --git a/config.mak.uname b/config.mak.uname index 1bffbe530ce532..d1a340525df4d4 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -133,6 +133,8 @@ ifeq ($(uname_S),Darwin) HAVE_BSD_SYSCTL = YesPlease FREAD_READS_DIRECTORIES = UnfortunatelyYes HAVE_NS_GET_EXECUTABLE_PATH = YesPlease + BASIC_CFLAGS += -I/usr/local/include + BASIC_LDFLAGS += -L/usr/local/lib endif ifeq ($(uname_S),SunOS) NEEDS_SOCKET = YesPlease From 6a2dc809b6840138c659f91273f5dfd97572475a Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Sun, 14 Apr 2019 14:19:29 -0700 Subject: [PATCH 05/16] t9822: skip tests if file names cannot be ISO-8859-1 encoded Most notably, it seems that macOS' APFS does not allow that. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/t9822-git-p4-path-encoding.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/t/t9822-git-p4-path-encoding.sh b/t/t9822-git-p4-path-encoding.sh index c78477c19b4330..de0b46581d3207 100755 --- a/t/t9822-git-p4-path-encoding.sh +++ b/t/t9822-git-p4-path-encoding.sh @@ -7,6 +7,13 @@ test_description='Clone repositories with non ASCII paths' UTF8_ESCAPED="a-\303\244_o-\303\266_u-\303\274.txt" ISO8859_ESCAPED="a-\344_o-\366_u-\374.txt" +ISO8859="$(printf "$ISO8859_ESCAPED")" && +echo content123 >"$ISO8859" && +rm "$ISO8859" || { + skip_all="fs does not accept ISO-8859-1 filenames" + test_done +} + test_expect_success 'start p4d' ' start_p4d ' From 7671a3827b207d0de0ae92a03f867bb71026a83d Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 8 Mar 2019 07:51:19 -0800 Subject: [PATCH 06/16] mingw: allow building with an MSYS2 runtime v3.x Recently the Git for Windows project started the upgrade process to a MSYS2 runtime version based on Cygwin v3.x. This has the very notable consequence that `$(uname -r)` no longer reports a version starting with "2", but a version with "3". That breaks our build, as df5218b4c30b (config.mak.uname: support MSys2, 2016-01-13) simply did not expect the version reported by `uname -r` to depend on the underlying Cygwin version: it expected the reported version to match the "2" in "MSYS2". So let's invert that test case to test for *anything else* than a version starting with "1" (for MSys). That should safeguard us for the future, even if Cygwin ends up releasing versionsl like 314.272.65536. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- config.mak.uname | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config.mak.uname b/config.mak.uname index d1a340525df4d4..fd1ec68e21ea90 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -610,7 +610,7 @@ ifneq (,$(wildcard ../THIS_IS_MSYSGIT)) NO_GETTEXT = YesPlease COMPAT_CLFAGS += -D__USE_MINGW_ACCESS else - ifeq ($(shell expr "$(uname_R)" : '2\.'),2) + ifneq ($(shell expr "$(uname_R)" : '1\.'),2) # MSys2 prefix = /usr/ # Enable DEP From 469f915db86a117f7f69a0979d7e30725d94a954 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Sat, 4 May 2019 23:54:03 +0200 Subject: [PATCH 07/16] t6500(mingw): use the Windows PID of the shell In Git for Windows, we use the MSYS2 Bash which inherits a non-standard PID model from Cygwin's POSIX emulation layer: every MSYS2 process has a regular Windows PID, and in addition it has an MSYS2 PID (which corresponds to a shadow process that emulates Unix-style signal handling). With the upgrade to the MSYS2 runtime v3.x, this shadow process cannot be accessed via `OpenProcess()` any longer, and therefore t6500 thought incorrectly that the process referenced in `gc.pid` (which is not actually a real `gc` process in this context, but the current shell) no longer exists. Let's fix this by making sure that the Windows PID is written into `gc.pid` in this test script soo that `git.exe` is able to understand that that process does indeed still exist. Signed-off-by: Johannes Schindelin --- t/t6500-gc.sh | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index 4684d06552ac2e..53258d45a12553 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -162,7 +162,15 @@ test_expect_success 'background auto gc respects lock for all operations' ' # now fake a concurrent gc that holds the lock; we can use our # shell pid so that it looks valid. hostname=$(hostname || echo unknown) && - printf "$$ %s" "$hostname" >.git/gc.pid && + shell_pid=$$ && + if test_have_prereq MINGW && test -f /proc/$shell_pid/winpid + then + # In Git for Windows, Bash (actually, the MSYS2 runtime) has a + # different idea of PIDs than git.exe (actually Windows). Use + # the Windows PID in this case. + shell_pid=$(cat /proc/$shell_pid/winpid) + fi && + printf "%d %s" "$shell_pid" "$hostname" >.git/gc.pid && # our gc should exit zero without doing anything run_and_wait_for_auto_gc && From de859a79e7276e2c12be8089f8a81d8aece90a85 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Sat, 4 May 2019 23:07:18 +0200 Subject: [PATCH 08/16] ci: use git-sdk-64-minimal This is a (partial) backport of the improvements of the Azure Pipeline made in Git for Windows v2.21.0. The major benefit is that it makes the task to initialize the Git for Windows SDK *a lot* faster. It would also have worked around a problem where an MSYS2 update breaks t6500.13, and it will work around similar problems in the future, as the Azure Pipeline that updates git-sdk-64-minimal will not publish a new artifact unless Git's test suite (as of upstream's current `master`) passes. --- azure-pipelines.yml | 52 ++++++++++----------------------------------- 1 file changed, 11 insertions(+), 41 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 757c43c5f48a76..0eff9a5f833523 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -175,53 +175,23 @@ phases: env: GITFILESHAREPWD: $(gitfileshare.pwd) - powershell: | - # Helper to check the error level of the latest command (exit with error when appropriate) - function c() { if (!$?) { exit(1) } } - - # Add build agent's MinGit to PATH - $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH - - # Helper to initialize (or update) a Git worktree - function init ($path, $url, $set_origin) { - if (Test-Path $path) { - cd $path; c - if (Test-Path .git) { - git init; c - } else { - git status - } - } else { - git init $path; c - cd $path; c - } - git config core.autocrlf false; c - git config core.untrackedCache true; c - if (($set_origin -ne 0) -and !(git config remote.origin.url)) { - git remote add origin $url; c - } - git fetch --depth=1 $url master; c - git reset --hard FETCH_HEAD; c - git clean -df; c - } - - # Initialize Git for Windows' SDK - $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64" - init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0 - init usr\src\build-extra https://github.com/git-for-windows/build-extra 1 - - # Let Git ignore the SDK and the test-cache - "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" - - # Help MSYS2 runtime startup by initializing /etc/passwd - & "$sdk_path\git-cmd" --command=usr\\bin\\bash.exe -lc "mkpasswd -c >>/etc/passwd"; c - displayName: 'Initialize the Git for Windows SDK' + $urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds" + $id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id + $downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl + (New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip") + Expand-Archive git-sdk-64-minimal.zip -DestinationPath "$(Build.SourcesDirectory)" -Force + Remove-Item git-sdk-64-minimal.zip + + # Let Git ignore the SDK and the test-cache + "/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" + displayName: 'Download git-sdk-64-minimal' - powershell: | # Helper to check the error level of the latest command (exit with error when appropriate) function c() { if (!$?) { exit(1) } } cd "$(Build.SourcesDirectory)"; c - git-sdk-64\git-cmd --command=usr\\bin\\bash.exe -lc @" + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" export MAKEFLAGS=-j10 export DEVELOPER=1 export NO_PERL=1 From b1493165d3ee86836ee2ced83f066e4b97d50d4c Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 10 May 2019 21:20:28 +0200 Subject: [PATCH 09/16] ci: use newer Azure Pipelines YAML schema We will need this in the next commit, where we backport the parallelized tests on Windows. --- azure-pipelines.yml | 54 ++++++++++++++++++--------------------------- 1 file changed, 22 insertions(+), 32 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 0eff9a5f833523..1197d03ea91512 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -1,13 +1,11 @@ -resources: -- repo: self - fetchDepth: 1 +variables: + Agent.Source.Git.ShallowFetchDepth: 100 -phases: -- phase: linux_clang +jobs: +- job: linux_clang displayName: linux-clang condition: succeeded() - queue: - name: Hosted Ubuntu 1604 + pool: Hosted Ubuntu 1604 steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -37,11 +35,10 @@ phases: publishRunAttachments: false condition: succeededOrFailed() -- phase: linux_gcc +- job: linux_gcc displayName: linux-gcc condition: succeeded() - queue: - name: Hosted Ubuntu 1604 + pool: Hosted Ubuntu 1604 steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -69,11 +66,10 @@ phases: publishRunAttachments: false condition: succeededOrFailed() -- phase: osx_clang +- job: osx_clang displayName: osx-clang condition: succeeded() - queue: - name: Hosted macOS + pool: Hosted macOS steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -99,11 +95,10 @@ phases: publishRunAttachments: false condition: succeededOrFailed() -- phase: osx_gcc +- job: osx_gcc displayName: osx-gcc condition: succeeded() - queue: - name: Hosted macOS + pool: Hosted macOS steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -127,11 +122,10 @@ phases: publishRunAttachments: false condition: succeededOrFailed() -- phase: gettext_poison +- job: gettext_poison displayName: GETTEXT_POISON condition: succeeded() - queue: - name: Hosted Ubuntu 1604 + pool: Hosted Ubuntu 1604 steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -160,11 +154,10 @@ phases: publishRunAttachments: false condition: succeededOrFailed() -- phase: windows +- job: windows displayName: Windows - queue: - name: Hosted - timeoutInMinutes: 240 + pool: Hosted + timeoutInMinutes: 240 steps: - powershell: | if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { @@ -225,11 +218,10 @@ phases: publishRunAttachments: false condition: succeededOrFailed() -- phase: linux32 +- job: linux32 displayName: Linux32 condition: succeeded() - queue: - name: Hosted Ubuntu 1604 + pool: Hosted Ubuntu 1604 steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -266,11 +258,10 @@ phases: publishRunAttachments: false condition: succeededOrFailed() -- phase: static_analysis +- job: static_analysis displayName: StaticAnalysis condition: succeeded() - queue: - name: Hosted Ubuntu 1604 + pool: Hosted Ubuntu 1604 steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -288,11 +279,10 @@ phases: env: GITFILESHAREPWD: $(gitfileshare.pwd) -- phase: documentation +- job: documentation displayName: Documentation condition: succeeded() - queue: - name: Hosted Ubuntu 1604 + pool: Hosted Ubuntu 1604 steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 From 9d2b5614415d81709893c002d442a1fffc5c5c29 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 10 May 2019 21:14:38 +0200 Subject: [PATCH 10/16] ci: backport parallel tests on Windows It just takes forever, or at least over an hour, to run them in a non-parallel way. That's why we switched to parallel testing in v2.21.0, and hereby we bring that goodness back to vfs-2.20.1. --- azure-pipelines.yml | 101 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 92 insertions(+), 9 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 1197d03ea91512..b3242a6923eebc 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -154,8 +154,8 @@ jobs: publishRunAttachments: false condition: succeededOrFailed() -- job: windows - displayName: Windows +- job: windows_build + displayName: Windows Build pool: Hosted timeoutInMinutes: 240 steps: @@ -188,19 +188,30 @@ jobs: export MAKEFLAGS=-j10 export DEVELOPER=1 export NO_PERL=1 - export NO_SVN_TESTS=1 - export GIT_TEST_SKIP_REBASE_P=1 - ci/run-build-and-tests.sh || { - ci/print-test-failures.sh - exit 1 - } + mkdir -p artifacts && + printf '%s\n%s\n\t%s\n\t%s\n' \ + 'include Makefile' \ + 'a:: `$(ALL_PROGRAMS) `$(SCRIPT_LIB) `$(BUILT_INS) `$(OTHER_PROGRAMS) GIT-BUILD-OPTIONS `$(TEST_PROGRAMS) `$(test_bindir_programs) `$(NO_INSTALL) `$(MOFILES)' \ + '`$(MAKE) -C templates' \ + 'tar czf artifacts/artifacts.tar.gz `$^ templates/blt/' >mak && + make -f mak a "@ c - displayName: 'Build & Test' + displayName: 'Build test artifacts' env: HOME: $(Build.SourcesDirectory) MSYSTEM: MINGW64 + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)\artifacts' + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' - powershell: | if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" @@ -209,6 +220,72 @@ jobs: condition: true env: GITFILESHAREPWD: $(gitfileshare.pwd) + +- job: windows_test + displayName: Windows Test + dependsOn: windows_build + condition: succeeded() + pool: Hosted + timeoutInMinutes: 240 + strategy: + parallel: 10 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)' + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' + - powershell: | + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" + test -f artifacts.tar.gz || { + echo No test artifacts found\; skipping >&2 + exit 0 + } + tar xf artifacts.tar.gz || exit 1 + + # Let Git ignore the SDK and the test-cache + printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude + + cd t && + T=`$(ls -S t[0-9]`*.sh | awk NR%`$SYSTEM_TOTALJOBSINPHASE==`$((`$SYSTEM_JOBPOSITIONINPHASE-1))) && + GIT_TEST_OPTS=\"--verbose-log -x\" GIT_PROVE_OPTS=\"--timer --jobs 10\" eval make -j10 `$T + test 0 != `$? || exit 0 + cd test-results + for t in `$(grep -l '^[1-9]' *.exit | sed -n 's/\.exit`$//p') + do + echo \"##vso[task.logissue type=error]Failed: `$t\" + cat `$t.out + done + exit 1 + "@ + if (!$?) { exit(1) } + displayName: 'Test (parallel)' + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + NO_SVN_TESTS: 1 + GIT_TEST_SKIP_REBASE_P: 1 + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) - task: PublishTestResults@2 displayName: 'Publish Test Results **/TEST-*.xml' inputs: @@ -217,6 +294,12 @@ jobs: platform: Windows publishRunAttachments: false condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts - job: linux32 displayName: Linux32 From 32f719bccf9fb3b9c9a560db663104c9a726a303 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Thu, 27 Dec 2018 11:19:36 +0100 Subject: [PATCH 11/16] fixup! mingw: special-case arguments to `sh` While working on parallelizing the tests in Azure Pipelines, an issue was discovered with the `is_msys2_sh()` function: it expects the path components to be separated by exactly one dir separator. That does not need to be the case, though, e.g. when the components in the `PATH` variable have trailing slashes. Let's make the code much more robust in this respect. This fixes t5580.5. Signed-off-by: Johannes Schindelin --- compat/mingw.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/compat/mingw.c b/compat/mingw.c index c695e5a5a07c91..6003378280ce19 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -1628,6 +1628,21 @@ struct pinfo_t { static struct pinfo_t *pinfo = NULL; CRITICAL_SECTION pinfo_cs; +/* Used to match and chomp off path components */ +static inline int match_last_path_component(const char *path, size_t *len, + const char *component) +{ + size_t component_len = strlen(component); + if (*len < component_len + 1 || + !is_dir_sep(path[*len - component_len - 1]) || + strncasecmp(path + *len - component_len, component, component_len)) + return 0; + *len -= component_len + 1; + while (*len > 0 && is_dir_sep(path[*len - 1])) + (*len)--; + return 1; +} + static int is_msys2_sh(const char *cmd) { if (cmd && !strcmp(cmd, "sh")) { @@ -1642,13 +1657,10 @@ static int is_msys2_sh(const char *cmd) ret = 0; else { size_t len = strlen(p); - ret = len > 15 && - is_dir_sep(p[len - 15]) && - !strncasecmp(p + len - 14, "usr", 3) && - is_dir_sep(p[len - 11]) && - !strncasecmp(p + len - 10, "bin", 3) && - is_dir_sep(p[len - 7]) && - !strcasecmp(p + len - 6, "sh.exe"); + + ret = match_last_path_component(p, &len, "sh.exe") && + match_last_path_component(p, &len, "bin") && + match_last_path_component(p, &len, "usr"); free(p); } return ret; From d68d6c8019e2329c2b64460e5a7495a95d8ff598 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 10 May 2019 21:22:10 +0200 Subject: [PATCH 12/16] ci: move the Windows jobs to the top The tests on Windows arguably take the longest, so let's get the build done with, first thing. --- azure-pipelines.yml | 294 ++++++++++++++++++++++---------------------- 1 file changed, 147 insertions(+), 147 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index b3242a6923eebc..a0dfe0de862bdc 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -2,6 +2,153 @@ variables: Agent.Source.Git.ShallowFetchDepth: 100 jobs: +- job: windows_build + displayName: Windows Build + pool: Hosted + timeoutInMinutes: 240 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - powershell: | + $urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds" + $id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id + $downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl + (New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip") + Expand-Archive git-sdk-64-minimal.zip -DestinationPath "$(Build.SourcesDirectory)" -Force + Remove-Item git-sdk-64-minimal.zip + + # Let Git ignore the SDK and the test-cache + "/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" + displayName: 'Download git-sdk-64-minimal' + - powershell: | + # Helper to check the error level of the latest command (exit with error when appropriate) + function c() { if (!$?) { exit(1) } } + + cd "$(Build.SourcesDirectory)"; c + + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" + export MAKEFLAGS=-j10 + export DEVELOPER=1 + export NO_PERL=1 + + mkdir -p artifacts && + printf '%s\n%s\n\t%s\n\t%s\n' \ + 'include Makefile' \ + 'a:: `$(ALL_PROGRAMS) `$(SCRIPT_LIB) `$(BUILT_INS) `$(OTHER_PROGRAMS) GIT-BUILD-OPTIONS `$(TEST_PROGRAMS) `$(test_bindir_programs) `$(NO_INSTALL) `$(MOFILES)' \ + '`$(MAKE) -C templates' \ + 'tar czf artifacts/artifacts.tar.gz `$^ templates/blt/' >mak && + make -f mak a + "@ + c + displayName: 'Build test artifacts' + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)\artifacts' + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + +- job: windows_test + displayName: Windows Test + dependsOn: windows_build + condition: succeeded() + pool: Hosted + timeoutInMinutes: 240 + strategy: + parallel: 10 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)' + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' + - powershell: | + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" + test -f artifacts.tar.gz || { + echo No test artifacts found\; skipping >&2 + exit 0 + } + tar xf artifacts.tar.gz || exit 1 + + # Let Git ignore the SDK and the test-cache + printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude + + cd t && + T=`$(ls -S t[0-9]`*.sh | awk NR%`$SYSTEM_TOTALJOBSINPHASE==`$((`$SYSTEM_JOBPOSITIONINPHASE-1))) && + GIT_TEST_OPTS=\"--verbose-log -x\" GIT_PROVE_OPTS=\"--timer --jobs 10\" eval make -j10 `$T + test 0 != `$? || exit 0 + cd test-results + for t in `$(grep -l '^[1-9]' *.exit | sed -n 's/\.exit`$//p') + do + echo \"##vso[task.logissue type=error]Failed: `$t\" + cat `$t.out + done + exit 1 + "@ + if (!$?) { exit(1) } + displayName: 'Test (parallel)' + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + NO_SVN_TESTS: 1 + GIT_TEST_SKIP_REBASE_P: 1 + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'windows' + platform: Windows + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + - job: linux_clang displayName: linux-clang condition: succeeded() @@ -154,153 +301,6 @@ jobs: publishRunAttachments: false condition: succeededOrFailed() -- job: windows_build - displayName: Windows Build - pool: Hosted - timeoutInMinutes: 240 - steps: - - powershell: | - if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { - net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no - cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ - } - displayName: 'Mount test-cache' - env: - GITFILESHAREPWD: $(gitfileshare.pwd) - - powershell: | - $urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds" - $id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id - $downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl - (New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip") - Expand-Archive git-sdk-64-minimal.zip -DestinationPath "$(Build.SourcesDirectory)" -Force - Remove-Item git-sdk-64-minimal.zip - - # Let Git ignore the SDK and the test-cache - "/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" - displayName: 'Download git-sdk-64-minimal' - - powershell: | - # Helper to check the error level of the latest command (exit with error when appropriate) - function c() { if (!$?) { exit(1) } } - - cd "$(Build.SourcesDirectory)"; c - - & git-sdk-64-minimal\usr\bin\bash.exe -lc @" - export MAKEFLAGS=-j10 - export DEVELOPER=1 - export NO_PERL=1 - - mkdir -p artifacts && - printf '%s\n%s\n\t%s\n\t%s\n' \ - 'include Makefile' \ - 'a:: `$(ALL_PROGRAMS) `$(SCRIPT_LIB) `$(BUILT_INS) `$(OTHER_PROGRAMS) GIT-BUILD-OPTIONS `$(TEST_PROGRAMS) `$(test_bindir_programs) `$(NO_INSTALL) `$(MOFILES)' \ - '`$(MAKE) -C templates' \ - 'tar czf artifacts/artifacts.tar.gz `$^ templates/blt/' >mak && - make -f mak a - "@ - c - displayName: 'Build test artifacts' - env: - HOME: $(Build.SourcesDirectory) - MSYSTEM: MINGW64 - - task: PublishPipelineArtifact@0 - displayName: 'Publish Pipeline Artifact: test artifacts' - inputs: - artifactName: 'windows-artifacts' - targetPath: '$(Build.SourcesDirectory)\artifacts' - - task: PublishPipelineArtifact@0 - displayName: 'Publish Pipeline Artifact: git-sdk-64-minimal' - inputs: - artifactName: 'git-sdk-64-minimal' - targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' - - powershell: | - if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { - cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" - } - displayName: 'Unmount test-cache' - condition: true - env: - GITFILESHAREPWD: $(gitfileshare.pwd) - -- job: windows_test - displayName: Windows Test - dependsOn: windows_build - condition: succeeded() - pool: Hosted - timeoutInMinutes: 240 - strategy: - parallel: 10 - steps: - - powershell: | - if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { - net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no - cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ - } - displayName: 'Mount test-cache' - env: - GITFILESHAREPWD: $(gitfileshare.pwd) - - task: DownloadPipelineArtifact@0 - displayName: 'Download Pipeline Artifact: test artifacts' - inputs: - artifactName: 'windows-artifacts' - targetPath: '$(Build.SourcesDirectory)' - - task: DownloadPipelineArtifact@0 - displayName: 'Download Pipeline Artifact: git-sdk-64-minimal' - inputs: - artifactName: 'git-sdk-64-minimal' - targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' - - powershell: | - & git-sdk-64-minimal\usr\bin\bash.exe -lc @" - test -f artifacts.tar.gz || { - echo No test artifacts found\; skipping >&2 - exit 0 - } - tar xf artifacts.tar.gz || exit 1 - - # Let Git ignore the SDK and the test-cache - printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude - - cd t && - T=`$(ls -S t[0-9]`*.sh | awk NR%`$SYSTEM_TOTALJOBSINPHASE==`$((`$SYSTEM_JOBPOSITIONINPHASE-1))) && - GIT_TEST_OPTS=\"--verbose-log -x\" GIT_PROVE_OPTS=\"--timer --jobs 10\" eval make -j10 `$T - test 0 != `$? || exit 0 - cd test-results - for t in `$(grep -l '^[1-9]' *.exit | sed -n 's/\.exit`$//p') - do - echo \"##vso[task.logissue type=error]Failed: `$t\" - cat `$t.out - done - exit 1 - "@ - if (!$?) { exit(1) } - displayName: 'Test (parallel)' - env: - HOME: $(Build.SourcesDirectory) - MSYSTEM: MINGW64 - NO_SVN_TESTS: 1 - GIT_TEST_SKIP_REBASE_P: 1 - - powershell: | - if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { - cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" - } - displayName: 'Unmount test-cache' - condition: true - env: - GITFILESHAREPWD: $(gitfileshare.pwd) - - task: PublishTestResults@2 - displayName: 'Publish Test Results **/TEST-*.xml' - inputs: - mergeTestResults: true - testRunTitle: 'windows' - platform: Windows - publishRunAttachments: false - condition: succeededOrFailed() - - task: PublishBuildArtifacts@1 - displayName: 'Publish trash directories of failed tests' - condition: failed() - inputs: - PathtoPublish: t/failed-test-artifacts - ArtifactName: failed-test-artifacts - - job: linux32 displayName: Linux32 condition: succeeded() From 91802d23298e622459f9e1f7cd93ad410176ce6a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?SZEDER=20G=C3=A1bor?= Date: Thu, 17 Jan 2019 02:29:09 +0100 Subject: [PATCH 13/16] compat/obstack: fix -Wcast-function-type warnings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GCC 8 introduced the new -Wcast-function-type warning, which is implied by -Wextra (which, in turn is enabled in our DEVELOPER flags). When building Git with GCC 8 and this warning enabled on a non-glibc platform [1], one is greeted with a screenful of compiler warnings/errors: compat/obstack.c: In function '_obstack_begin': compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type] h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun; ^ compat/obstack.c:163:16: error: cast between incompatible function types from 'void (*)(void *)' to 'void (*)(void *, struct _obstack_chunk *)' [-Werror=cast-function-type] h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun; ^ compat/obstack.c:116:8: error: cast between incompatible function types from 'struct _obstack_chunk * (*)(void *, long int)' to 'struct _obstack_chunk * (*)(long int)' [-Werror=cast-function-type] : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size))) ^ compat/obstack.c:168:22: note: in expansion of macro 'CALL_CHUNKFUN' chunk = h->chunk = CALL_CHUNKFUN (h, h -> chunk_size); ^~~~~~~~~~~~~ 'struct obstack' stores pointers to two functions to allocate and free "chunks", and depending on how obstack is used, these functions take either one parameter (like standard malloc() and free() do; this is how we use it in 'kwset.c') or two parameters. Presumably to reduce memory footprint, a single field is used to store the function pointer for both signatures, and then it's casted to the appropriate signature when the function pointer is accessed. These casts between function pointers with different number of parameters are what trigger those compiler errors. Modify 'struct obstack' to use unions to store function pointers with different signatures, and then use the union member with the appropriate signature when accessing these function pointers. This eliminates the need for those casts, and thus avoids this compiler error. [1] Compiling 'compat/obstack.c' on a platform with glibc is sort of a noop, see the comment before '# define ELIDE_CODE', so this is not an issue on common Linux distros. Signed-off-by: SZEDER Gábor --- compat/obstack.c | 17 +++++++++-------- compat/obstack.h | 18 +++++++++++------- 2 files changed, 20 insertions(+), 15 deletions(-) diff --git a/compat/obstack.c b/compat/obstack.c index 4d1d95beeb509b..27cd5c1ea1f9b3 100644 --- a/compat/obstack.c +++ b/compat/obstack.c @@ -112,15 +112,15 @@ compat_symbol (libc, _obstack_compat, _obstack, GLIBC_2_0); # define CALL_CHUNKFUN(h, size) \ (((h) -> use_extra_arg) \ - ? (*(h)->chunkfun) ((h)->extra_arg, (size)) \ - : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size))) + ? (*(h)->chunkfun.extra) ((h)->extra_arg, (size)) \ + : (*(h)->chunkfun.plain) ((size))) # define CALL_FREEFUN(h, old_chunk) \ do { \ if ((h) -> use_extra_arg) \ - (*(h)->freefun) ((h)->extra_arg, (old_chunk)); \ + (*(h)->freefun.extra) ((h)->extra_arg, (old_chunk)); \ else \ - (*(void (*) (void *)) (h)->freefun) ((old_chunk)); \ + (*(h)->freefun.plain) ((old_chunk)); \ } while (0) @@ -159,8 +159,8 @@ _obstack_begin (struct obstack *h, size = 4096 - extra; } - h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun; - h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun; + h->chunkfun.plain = chunkfun; + h->freefun.plain = freefun; h->chunk_size = size; h->alignment_mask = alignment - 1; h->use_extra_arg = 0; @@ -206,8 +206,9 @@ _obstack_begin_1 (struct obstack *h, int size, int alignment, size = 4096 - extra; } - h->chunkfun = (struct _obstack_chunk * (*)(void *,long)) chunkfun; - h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun; + h->chunkfun.extra = (struct _obstack_chunk * (*)(void *,long)) chunkfun; + h->freefun.extra = (void (*) (void *, struct _obstack_chunk *)) freefun; + h->chunk_size = size; h->alignment_mask = alignment - 1; h->extra_arg = arg; diff --git a/compat/obstack.h b/compat/obstack.h index f0807eaa3b899f..ae36ed6a669d96 100644 --- a/compat/obstack.h +++ b/compat/obstack.h @@ -160,11 +160,15 @@ struct obstack /* control current object in current chunk */ void *tempptr; } temp; /* Temporary for some macros. */ int alignment_mask; /* Mask of alignment for each object. */ - /* These prototypes vary based on `use_extra_arg', and we use - casts to the prototypeless function type in all assignments, - but having prototypes here quiets -Wstrict-prototypes. */ - struct _obstack_chunk *(*chunkfun) (void *, long); - void (*freefun) (void *, struct _obstack_chunk *); + /* These prototypes vary based on `use_extra_arg'. */ + union { + void *(*plain) (long); + struct _obstack_chunk *(*extra) (void *, long); + } chunkfun; + union { + void (*plain) (void *); + void (*extra) (void *, struct _obstack_chunk *); + } freefun; void *extra_arg; /* first arg for chunk alloc/dealloc funcs */ unsigned use_extra_arg:1; /* chunk alloc/dealloc funcs take extra arg */ unsigned maybe_empty_object:1;/* There is a possibility that the current @@ -235,10 +239,10 @@ extern void (*obstack_alloc_failed_handler) (void); (void (*) (void *, void *)) (freefun), (arg)) #define obstack_chunkfun(h, newchunkfun) \ - ((h) -> chunkfun = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun)) + ((h)->chunkfun.extra = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun)) #define obstack_freefun(h, newfreefun) \ - ((h) -> freefun = (void (*)(void *, struct _obstack_chunk *)) (newfreefun)) + ((h)->freefun.extra = (void (*)(void *, struct _obstack_chunk *)) (newfreefun)) #define obstack_1grow_fast(h,achar) (*((h)->next_free)++ = (achar)) From 7df0cdfdbd4206fcbfa7978c3aceb289e0a8387d Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 29 Mar 2019 12:58:27 +0100 Subject: [PATCH 14/16] mingw: allow compiling with GCC 8 and DEVELOPER=1 The return type of the `GetProcAddress()` function is `FARPROC` which evaluates to `long long int (*)()`, i.e. it cannot be cast to the correct function signature by GCC 8. To work around that, we first cast to `void *` and go on with our merry lives. Signed-off-by: Johannes Schindelin --- compat/poll/poll.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/compat/poll/poll.c b/compat/poll/poll.c index a5e879ef30a186..620760a3e4811b 100644 --- a/compat/poll/poll.c +++ b/compat/poll/poll.c @@ -149,7 +149,7 @@ win32_compute_revents (HANDLE h, int *p_sought) case FILE_TYPE_PIPE: if (!once_only) { - NtQueryInformationFile = (PNtQueryInformationFile) + NtQueryInformationFile = (PNtQueryInformationFile)(void (*)(void)) GetProcAddress (GetModuleHandleA ("ntdll.dll"), "NtQueryInformationFile"); once_only = TRUE; From 2f0b391a4c769943dd415adbfa518cd6c8e1846f Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 29 Mar 2019 14:17:03 +0100 Subject: [PATCH 15/16] kwset: allow building with GCC 8 The kwset functionality makes use of the obstack code, which expects to be handed a function that can allocate large chunks of data. It expects that function to accept a `size` parameter of type `long`. This upsets GCC 8 on Windows, because `long` does not have the same bit size as `size_t` there. Now, the proper thing to do would be to switch to `size_t`. But this would make us deviate from the "upstream" code even further, making it hard to synchronize with newer versions, and also it would be quite involved because that `long` type is so invasive in that code. Let's punt, and instead provide a super small wrapper around `xmalloc()`. Signed-off-by: Johannes Schindelin --- kwset.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kwset.c b/kwset.c index 4fb6455acaf129..efc2ff41bc361a 100644 --- a/kwset.c +++ b/kwset.c @@ -38,7 +38,13 @@ #include "compat/obstack.h" #define NCHAR (UCHAR_MAX + 1) -#define obstack_chunk_alloc xmalloc +/* adapter for `xmalloc()`, which takes `size_t`, not `long` */ +static void *obstack_chunk_alloc(long size) +{ + if (size < 0) + BUG("Cannot allocate a negative amount: %ld", size); + return xmalloc(size); +} #define obstack_chunk_free free #define U(c) ((unsigned char) (c)) From 32c18f9c5abe7993da67e8f40aaf76129aa67605 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Wed, 3 Apr 2019 12:47:39 +0200 Subject: [PATCH 16/16] fixup! mingw: kill child processes in a gentler way This is needed to make things compile with GCC 8.x and later. Signed-off-by: Johannes Schindelin --- compat/win32/exit-process.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/compat/win32/exit-process.h b/compat/win32/exit-process.h index 88e3bbc83c499e..d53989884cfb0c 100644 --- a/compat/win32/exit-process.h +++ b/compat/win32/exit-process.h @@ -130,7 +130,8 @@ static int exit_process(HANDLE process, int exit_code) HINSTANCE kernel32 = GetModuleHandleA("kernel32"); if (!kernel32) die("BUG: cannot find kernel32"); - exit_process_address = (LPTHREAD_START_ROUTINE) + exit_process_address = + (LPTHREAD_START_ROUTINE)(void (*)(void)) GetProcAddress(kernel32, "ExitProcess"); initialized = 1; }