Skip to content

Conversation

@dscho
Copy link
Member

@dscho dscho commented Jan 16, 2019

Unfortunately, Travis decided to change some things under the hood that affect the Windows build. As a consequence, master has not been tested in quite a while, even if the test runs pretended that it had been tested :-(

So imagine my surprise when master simply would refuse to pass the test suite cleanly outside Travis, always failing at t6042, despite the fact that Travis passed.

It turns out that two files are written too quickly in succession, running into the issue where Git for Windows chooses not to populate the inode and device numbers in the stat data (this is a noticeable performance optimization). As a consequence, Git thinks the file is unchanged, and fails to pick up a modification. And no, we cannot simply undo the performance optimization, it would make things prohibitively slow in particular in large worktrees, and it is not like the bug is likely to be hit in reality: for Git to be fooled into thinking that a file is unchanged, it has to be written with the same file size, and within a 100ns time bucket (it is pretty improbable that there is any real-world scenario that would run into that, except of course our regression test suite).

This patch works around this issue by forcing Git to recognize the new file versions as new files (which they really are: the patch simply replaces

git mv <old> <new> && mv <file> <new> && git add <new>`

by

git rm <old> && mv <file> <new> && git add <new>`

which is not shorter, but even a performance improvement (an unnoticeable one, of course).

Changes since v1:

  • Clarified the change in the commit message.

Cc: Elijah Newren newren@gmail.com

@dscho
Copy link
Member Author

dscho commented Jan 16, 2019

/submit

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 16, 2019

Submitted as pull.109.git.gitgitgadget@gmail.com

@dscho dscho mentioned this pull request Jan 16, 2019
When Git determines whether a file has changed, it looks at the mtime,
at the file size, and to detect changes even if the mtime is the same
(on Windows, the mtime granularity is 100ns, read: if two files are
written within the same 100ns time slot, they have the same mtime) and
even if the file size is the same, Git also looks at the inode/device
numbers.

This design obviously comes from a Linux background, where `lstat()`
calls were designed to be cheap.

On Windows, there is no `lstat()`. It has to be emulated. And while
obtaining the mtime and the file size is not all that expensive (you can
get both with a single `GetFileAttributesW()` call), obtaining the
equivalent of the inode and device numbers is very expensive (it
requires a call to `GetFileInformationByHandle()`, which in turn
requires a file handle, which is *a lot* more expensive than one might
imagine).

As it is very uncommon for developers to modify files within 100ns time
slots, Git for Windows chooses not to fill inode/device numbers
properly, but simply sets them to 0.

However, in t6042 the files file_v1 and file_v2 are typically written
within the same 100ns time slot, and they do not differ in file size. So
the minor modification is not picked up.

Let's work around this issue by avoiding the `git mv` calls in the
'mod6-setup: chains of rename/rename(1to2) and rename/rename(2to1)' test
case. The target files are overwritten anyway, so it is not like we
really rename those files. This fixes the issue because `git add` will
now add the files as new files (as opposed to existing, just renamed
files).

Functionally, we do not change anything because we replace two `git mv
<old> <new>` calls (where `<new>` is completely overwritten and `git
add`ed later anyway) by `git rm <old>` calls (removing other files, too,
that are also completely overwritten and `git add`ed later).

Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho
Copy link
Member Author

dscho commented Jan 17, 2019

/submit

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 17, 2019

Submitted as pull.109.v2.git.gitgitgadget@gmail.com

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 18, 2019

This branch is now known as js/t6042-timing-fix.

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 18, 2019

This patch series was integrated into pu via git@7c00b1f.

@gitgitgadget gitgitgadget bot added the pu label Jan 18, 2019
@gitgitgadget
Copy link

gitgitgadget bot commented Jan 18, 2019

This patch series was integrated into pu via git@8387ab5.

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 24, 2019

This patch series was integrated into pu via git@3ecf445.

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 28, 2019

This patch series was integrated into pu via git@27eceb6.

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 29, 2019

This patch series was integrated into pu via git@eea38f3.

@gitgitgadget
Copy link

gitgitgadget bot commented Jan 29, 2019

This patch series was integrated into next via git@9543c96.

@gitgitgadget gitgitgadget bot added the next label Jan 29, 2019
@gitgitgadget
Copy link

gitgitgadget bot commented Feb 4, 2019

This patch series was integrated into pu via git@38f56b1.

@gitgitgadget
Copy link

gitgitgadget bot commented Feb 5, 2019

This patch series was integrated into pu via git@3009c8b.

@gitgitgadget
Copy link

gitgitgadget bot commented Feb 5, 2019

This patch series was integrated into next via git@3009c8b.

@gitgitgadget
Copy link

gitgitgadget bot commented Feb 5, 2019

This patch series was integrated into master via git@3009c8b.

@gitgitgadget gitgitgadget bot added the master label Feb 5, 2019
@gitgitgadget gitgitgadget bot closed this Feb 5, 2019
@gitgitgadget
Copy link

gitgitgadget bot commented Feb 5, 2019

Closed via 3009c8b.

@dscho dscho deleted the fix-t6042 branch February 6, 2019 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant