Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
7b29ef4
reset --stdin: trim carriage return from the paths
Apr 5, 2017
9167b96
gvfs: start by adding the -gvfs suffix to the version
sanoursa Jan 24, 2017
758bdf4
gvfs: ensure that the version is based on a GVFS tag
dscho Apr 4, 2017
a18c086
gvfs: add a GVFS-specific header file
Jan 24, 2017
a191ed6
gvfs: add the core.gvfs config setting
Jan 24, 2017
65d8c2a
gvfs: add the feature to skip writing the index' SHA-1
Jan 24, 2017
18135ad
gvfs: add the feature that blobs may be missing
Jan 24, 2017
64985fc
gvfs: prevent files to be deleted outside the sparse checkout
May 18, 2016
4e8499c
gvfs: optionally skip reachability checks/upload pack during fetch
May 30, 2016
63c4477
gvfs: ensure all filters and EOL conversions are blocked
Jun 15, 2016
9ea9339
Add a new run_hook_argv() function
dscho Nov 30, 2016
e00561c
gvfs: allow "virtualizing" objects
Jan 10, 2017
a04f755
Hydrate missing loose objects in check_and_freshen()
Mar 15, 2017
26cf7c2
Add support for read-object as a background process to retrieve missi…
Jul 18, 2017
0e12bee
gvfs: add global command pre and post hook procs
May 24, 2016
5bad178
sha1_file: when writing objects, skip the read_object_hook
dscho Sep 8, 2017
3ff7343
t0400: verify that the hook is called correctly from a subdirectory
dscho Mar 16, 2017
08328ea
Merge core VFS features
dscho Jun 11, 2018
7a27205
status: add status serialization mechanism
jeffhostetler Aug 22, 2017
ecaf772
Teach ahead-behind and serialized status to play nicely together
jamill Jan 10, 2018
146e74d
status: serialize to path
jeffhostetler Feb 2, 2018
7eb4ebe
Pass PID of git process to hooks.
alepauly Apr 10, 2017
ebe420e
status: reject deserialize in V2 and conflicts
jeffhostetler Feb 7, 2018
5df40b3
pre-command: always respect core.hooksPath
dscho Aug 7, 2017
a035fae
status: fix rename reporting when using serialization cache
jeffhostetler Jul 25, 2018
240417a
sparse-checkout: update files with a modify/delete conflict
Feb 22, 2017
b73ece7
gvfs:trace2:data: add trace2 tracing around read_object_process
jeffhostetler Sep 26, 2018
2d40677
serialize-status: serialize global and repo-local exclude file metadata
jeffhostetler Jul 20, 2018
5e4a115
sparse-checkout: avoid writing entries with the skip-worktree bit
Mar 1, 2017
61571d0
gvfs:trace2:data: status deserialization information
jeffhostetler Sep 26, 2018
1cb226a
status: deserialization wait
jeffhostetler Jul 25, 2018
732cc47
Fix reset when using the sparse-checkout feature.
Mar 15, 2017
e14a7a7
gvfs:trace2:data: status serialization
jeffhostetler Jan 7, 2019
ce3f931
merge-recursive: avoid confusing logic in was_dirty()
dscho May 21, 2019
02b3ad1
Do not remove files outside the sparse-checkout
Apr 5, 2017
ea4a601
gvfs:trace2:data: add vfs stats
jeffhostetler Nov 19, 2018
bfaaaa4
merge-recursive: add some defensive coding to was_dirty()
dscho May 21, 2019
3e3a826
gvfs: refactor loading the core.gvfs config value
Apr 14, 2017
59608b9
trace2: refactor setting process starting time
jeffhostetler Apr 15, 2019
4ad0ad9
merge-recursive: teach was_dirty() about the virtualfilesystem
dscho May 21, 2019
2b88924
Add virtual file system settings and hook proc
benpeart Jan 11, 2018
892ddf7
send-pack: do not check for sha1 file when GVFS_MISSING_OK set
Nov 16, 2018
1b6a055
trace2:gvfs:experiment: clear_ce_flags_1
jeffhostetler Apr 30, 2019
b78a803
status: deserialize with -uno does not print correct hint
jeffhostetler Jun 25, 2019
d5e00e2
Update the virtualfilesystem support
benpeart Jun 7, 2018
8f48b09
cache-tree: remove use of strbuf_addf in update_one
Jul 3, 2017
4d00479
trace2:gvfs:experiment: traverse_trees
jeffhostetler Apr 30, 2019
7f49dd2
wt-status-deserialize: fix crash when -v is used
jeffhostetler Apr 10, 2020
513d5ec
virtualfilesystem: don't run the virtual file system hook if the inde…
benpeart Aug 1, 2018
19044c0
gvfs: block unsupported commands when running in a GVFS repo
benpeart Dec 6, 2018
02fecff
trace2:gvfs:experiment: report_tracking
jeffhostetler Apr 30, 2019
cbe2994
fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate
kewillford Nov 21, 2019
13db714
virtualfilesystem: fix bug with symlinks being ignored
benpeart Sep 25, 2018
9bf76c9
gvfs: allow overriding core.gvfs
derrickstolee Apr 15, 2020
c049d39
trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache
jeffhostetler Jun 14, 2019
563d9c2
fsmonitor: add script for debugging and update script for tests
kewillford Nov 21, 2019
1a38880
virtualfilesystem: check if directory is included
Oct 9, 2018
e483af8
BRANCHES.md: Add explanation of branches and using forks
Jul 27, 2018
7e4b0be
trace2:gvfs:experiment: read-cache: time read/write of cache-tree ext…
jeffhostetler Jul 9, 2019
d043db0
status: disable deserialize when verbose output requested.
jeffhostetler Apr 11, 2020
b99bb74
vfs: fix case where directories not handled correctly
jamill Nov 20, 2018
3a8dcbe
vfs: disable `git update-git-for-windows`
dscho Feb 4, 2020
b9df0c9
trace2:gvfs:experiment: add prime_cache_tree region
jeffhostetler Jun 3, 2019
a4568d6
t7524: add test for verbose status deserialzation
jeffhostetler Apr 11, 2020
4451a05
backwards-compatibility: support the post-indexchanged hook
dscho May 28, 2019
17a768d
Merge advanced VFS-specific features
dscho Jun 11, 2018
1fb6b1e
trace2:gvfs:experiment: add region to apply_virtualfilesystem()
jeffhostetler Jul 16, 2019
f9465bb
deserialize-status: silently fallback if we cannot read cache file
jeffhostetler May 13, 2020
af27fd1
Merge virtualfilesystem hook
dscho Jun 11, 2018
91c711b
trace2:gvfs:experiment: add region around unpack_trees()
jeffhostetler Jul 16, 2019
99a88a2
Merge updates to serialized status
dscho Jun 11, 2018
83b1df3
credential: set trace2_child_class for credential manager children
jeffhostetler Oct 3, 2019
871e8d0
sub-process: do not borrow cmd pointer from caller
jeffhostetler Sep 18, 2019
c0be928
sub-process: add subprocess_start_argv()
jeffhostetler Sep 18, 2019
935747e
sha1-file: add function to update existing loose object cache
jeffhostetler Sep 24, 2019
dbc6974
packfile: add install_packed_git_and_mru()
jeffhostetler Sep 25, 2019
d643ee4
index-pack: avoid immediate object fetch while parsing packfile
jeffhostetler Sep 24, 2019
8efa4af
gvfs-helper: create tool to fetch objects using the GVFS Protocol
jeffhostetler Aug 13, 2019
1cf9d05
gvfs-helper: fix race condition when creating loose object dirs
derrickstolee Oct 4, 2019
0cee3ac
sha1-file: create shared-cache directory if it doesn't exist
jeffhostetler Oct 7, 2019
6e47e3d
gvfs-helper: better handling of network errors
jeffhostetler Oct 8, 2019
a52413b
gvfs-helper-client: properly update loose cache with fetched OID
jeffhostetler Oct 8, 2019
0eb1006
gvfs-helper: V2 robust retry and throttling
jeffhostetler Oct 10, 2019
6c5d207
gvfs-helper: add prefetch support
jeffhostetler Nov 11, 2019
c041e73
gvfs-helper: expose gvfs/objects GET and POST semantics
jeffhostetler Oct 21, 2019
2729138
gvfs-helper: add prefetch .keep file for last packfile
jeffhostetler Nov 26, 2019
11be079
gvfs-helper: dramatically reduce progress noise
derrickstolee Oct 24, 2019
076972a
gvfs-helper: do one read in my_copy_fd_len_tail()
derrickstolee Dec 16, 2019
6c2fa6b
gvfs-helper-client.h: define struct object_id
derrickstolee Nov 4, 2019
93c4f14
trace2:gvfs:experiment: add region to cache_tree_fully_valid()
jeffhostetler Jul 16, 2019
86c49e6
gvfs-helper: move content-type warning for prefetch packs
derrickstolee Dec 16, 2019
03cfdaa
gvfs-helper: handle pack-file after single POST request
derrickstolee Nov 11, 2019
bea2a6c
trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() …
jeffhostetler Jul 16, 2019
5da693a
fetch: use gvfs-helper prefetch under config
derrickstolee Dec 17, 2019
6f69b27
test-gvfs-prococol, t5799: tests for gvfs-helper
jeffhostetler Oct 25, 2019
8960222
trace2:gvfs:experiment: increase default event depth for unpack-tree …
jeffhostetler Jul 25, 2019
79fd761
gvfs-helper: better support for concurrent packfile fetches
jeffhostetler Dec 18, 2019
f97ef3b
gvfs-helper: move result-list construction into install functions
jeffhostetler Nov 13, 2019
72c0ef9
trace2:gvfs:experiment: add data for check_updates() in unpack_trees()
jeffhostetler Jul 25, 2019
1a44eec
remote-curl: do not call fetch-pack when using gvfs-helper
derrickstolee Feb 3, 2020
e8543f7
t5799: add support for POST to return either a loose object or packfile
jeffhostetler Nov 13, 2019
9664602
Trace2:gvfs:experiment: capture more 'tracking' details
jeffhostetler Jul 26, 2019
cfe3043
fetch: reprepare packs before checking connectivity
derrickstolee Mar 12, 2020
376b123
t5799: cleanup wc-l and grep-c lines
jeffhostetler Nov 13, 2019
10ece56
Merge trace2 experimental regions
jeffhostetler Apr 23, 2019
3808890
gvfs-helper: fix support for NTLM
jeffhostetler Apr 29, 2020
1b4b87d
gvfs-helper: retry when creating temp files
derrickstolee Dec 26, 2019
b098b57
Merge first wave of gvfs-helper feature
jeffhostetler Nov 14, 2019
f0f0e3f
gvfs-helper: cleanup NTLM fix
jeffhostetler Apr 30, 2020
b0950dc
Merge gvfs-helper prefetch feature
derrickstolee Dec 17, 2019
e401874
upload-pack: fix race condition in error messages
derrickstolee Aug 27, 2019
f5527f0
Merge pull request #265: gvfs-helper: fix support for NTLM
derrickstolee May 1, 2020
26f52f4
Merge upstreamable patches
dscho Oct 18, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
/git-gc
/git-get-tar-commit-id
/git-grep
/git-gvfs-helper
/git-hash-object
/git-help
/git-http-backend
Expand Down
53 changes: 53 additions & 0 deletions BRANCHES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
Branches used in this repo
==========================

The document explains the branching structure that we are using in the VFSForGit repository as well as the forking strategy that we have adopted for contributing.

Repo Branches
-------------

1. master

This will track the Git for Windows repository master branch

2. vfs

Would like to use this branch as an ever-green branch that continually rebases the VFSForGit changes onto a windows ever-green branch that is on the core/master, so that we can detect when the patches for VFSForGit have issues or if we have a new version patches sent upstream git we can regenerate this branch.

3. vs/master

This tracks with the Git for Windows repository vs/master branch and are the generated files for using a Visual Studio solution.

4. vfs-#

These branches are used to track the specific version that match Git for Windows with the VFSForGit specific patches on top. When a new version of Git for Windows is released, the VFSForGit patches will be rebased on that windows version and a new gvfs-# branch created to create pull requests against.

#### Examples

```
vfs-2.20.0
vfs-2.20.1
```

The versions of git for VFSForGit are based on the Git for Windows versions. v2.20.0.vfs.1 will correspond with the v2.20.0.windows.1 with the VFSForGit specific patches applied to the windows version.

Tags
----

We are using annotated tags to build the version number for git. The build will look back through the commit history to find the first tag matching `v[0-9]*vfs*` and build the git version number using that tag.

Forking
-------

A personal fork of this repository and a branch in that repository should be used for development.

These branches should be based on the latest vfs-# branch. If there are work in progress pull requests that you have based on a previous version branch when a new version branch is created, you will need to move your patches to the new branch to get them in that latest version.

#### Example

```
git clone <personal fork repo URL>
git remote add ms https://github.com/Microsoft/git.git
git checkout -b my-changes ms/vfs-2.20.0 --no-track
git push -fu origin HEAD
```
2 changes: 2 additions & 0 deletions Documentation/config.txt
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,8 @@ include::config/gui.txt[]

include::config/guitool.txt[]

include::config/gvfs.txt[]

include::config/help.txt[]

include::config/http.txt[]
Expand Down
57 changes: 57 additions & 0 deletions Documentation/config/core.txt
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,14 @@ core.fsmonitorHookVersion::
something that can be used to determine what files have changed
without race conditions.

core.virtualFilesystem::
If set, the value of this variable is used as a command which
will identify all files and directories that are present in
the working directory. Git will only track and update files
listed in the virtual file system. Using the virtual file system
will supersede the sparse-checkout settings which will be ignored.
See the "virtual file system" section of linkgit:githooks[5].

core.trustctime::
If false, the ctime differences between the index and the
working tree are ignored; useful when the inode change time
Expand Down Expand Up @@ -622,6 +630,55 @@ core.multiPackIndex::
single index. See link:technical/multi-pack-index.html[the
multi-pack-index design document].

core.gvfs::
Enable the features needed for GVFS. This value can be set to true
to indicate all features should be turned on or the bit values listed
below can be used to turn on specific features.
+
--
GVFS_SKIP_SHA_ON_INDEX::
Bit value 1
Disables the calculation of the sha when writing the index
GVFS_MISSING_OK::
Bit value 4
Normally git write-tree ensures that the objects referenced by the
directory exist in the object database. This option disables this check.
GVFS_NO_DELETE_OUTSIDE_SPARSECHECKOUT::
Bit value 8
When marking entries to remove from the index and the working
directory this option will take into account what the
skip-worktree bit was set to so that if the entry has the
skip-worktree bit set it will not be removed from the working
directory. This will allow virtualized working directories to
detect the change to HEAD and use the new commit tree to show
the files that are in the working directory.
GVFS_FETCH_SKIP_REACHABILITY_AND_UPLOADPACK::
Bit value 16
While performing a fetch with a virtual file system we know
that there will be missing objects and we don't want to download
them just because of the reachability of the commits. We also
don't want to download a pack file with commits, trees, and blobs
since these will be downloaded on demand. This flag will skip the
checks on the reachability of objects during a fetch as well as
the upload pack so that extraneous objects don't get downloaded.
GVFS_BLOCK_FILTERS_AND_EOL_CONVERSIONS::
Bit value 64
With a virtual file system we only know the file size before any
CRLF or smudge/clean filters processing is done on the client.
To prevent file corruption due to truncation or expansion with
garbage at the end, these filters must not run when the file
is first accessed and brought down to the client. Git.exe can't
currently tell the first access vs subsequent accesses so this
flag just blocks them from occurring at all.
GVFS_PREFETCH_DURING_FETCH::
Bit value 128
While performing a `git fetch` command, use the gvfs-helper to
perform a "prefetch" of commits and trees.
--

core.useGvfsHelper::
TODO

core.sparseCheckout::
Enable "sparse checkout" feature. See linkgit:git-sparse-checkout[1]
for more information.
Expand Down
5 changes: 5 additions & 0 deletions Documentation/config/gvfs.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
gvfs.cache-server::
TODO

gvfs.sharedcache::
TODO
22 changes: 22 additions & 0 deletions Documentation/config/status.txt
Original file line number Diff line number Diff line change
Expand Up @@ -75,3 +75,25 @@ status.submoduleSummary::
the --ignore-submodules=dirty command-line option or the 'git
submodule summary' command, which shows a similar output but does
not honor these settings.

status.deserializePath::
EXPERIMENTAL, Pathname to a file containing cached status results
generated by `--serialize`. This will be overridden by
`--deserialize=<path>` on the command line. If the cache file is
invalid or stale, git will fall-back and compute status normally.

status.deserializeWait::
EXPERIMENTAL, Specifies what `git status --deserialize` should do
if the serialization cache file is stale and whether it should
fall-back and compute status normally. This will be overridden by
`--deserialize-wait=<value>` on the command line.
+
--
* `fail` - cause git to exit with an error when the status cache file
is stale; this is intended for testing and debugging.
* `block` - cause git to spin and periodically retry the cache file
every 100 ms; this is intended to help coordinate with another git
instance concurrently computing the cache file.
* `no` - to immediately fall-back if cache file is stale. This is the default.
* `<timeout>` - time (in tenths of a second) to spin and retry.
--
35 changes: 35 additions & 0 deletions Documentation/git-status.txt
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,21 @@ ignored, then the directory is not shown, but all contents are shown.
update it afterwards if any changes were detected. Defaults to
`--lock-index`.

--serialize[=<path>]::
(EXPERIMENTAL) Serialize raw status results to a file or stdout
in a format suitable for use by `--deserialize`. If a path is
given, serialize data will be written to that path *and* normal
status output will be written to stdout. If path is omitted,
only binary serialization data will be written to stdout.

--deserialize[=<path>]::
(EXPERIMENTAL) Deserialize raw status results from a file or
stdin rather than scanning the worktree. If `<path>` is omitted
and `status.deserializePath` is unset, input is read from stdin.
--no-deserialize::
(EXPERIMENTAL) Disable implicit deserialization of status results
from the value of `status.deserializePath`.

<pathspec>...::
See the 'pathspec' entry in linkgit:gitglossary[7].

Expand Down Expand Up @@ -405,6 +420,26 @@ quoted as explained for the configuration variable `core.quotePath`
(see linkgit:git-config[1]).


SERIALIZATION and DESERIALIZATION (EXPERIMENTAL)
------------------------------------------------

The `--serialize` option allows git to cache the result of a
possibly time-consuming status scan to a binary file. A local
service/daemon watching file system events could use this to
periodically pre-compute a fresh status result.

Interactive users could then use `--deserialize` to simply
(and immediately) print the last-known-good result without
waiting for the status scan.

The binary serialization file format includes some worktree state
information allowing `--deserialize` to reject the cached data
and force a normal status scan if, for example, the commit, branch,
or status modes/options change. The format cannot, however, indicate
when the cached data is otherwise stale -- that coordination belongs
to the task driving the serializations.


CONFIGURATION
-------------

Expand Down
20 changes: 20 additions & 0 deletions Documentation/githooks.txt
Original file line number Diff line number Diff line change
Expand Up @@ -596,6 +596,26 @@ and "0" meaning they were not.
Only one parameter should be set to "1" when the hook runs. The hook
running passing "1", "1" should not be possible.

virtualFilesystem
~~~~~~~~~~~~~~~~~~

"Virtual File System" allows populating the working directory sparsely.
The projection data is typically automatically generated by an external
process. Git will limit what files it checks for changes as well as which
directories are checked for untracked files based on the path names given.
Git will also only update those files listed in the projection.

The hook is invoked when the configuration option core.virtualFilesystem
is set. It takes one argument, a version (currently 1).

The hook should output to stdout the list of all files in the working
directory that git should track. The paths are relative to the root
of the working directory and are separated by a single NUL. Full paths
('dir1/a.txt') as well as directories are supported (ie 'dir1/').

The exit status determines whether git will use the data from the
hook. On error, git will abort the command with an error message.

GIT
---
Part of the linkgit:git[1] suite
102 changes: 102 additions & 0 deletions Documentation/technical/read-object-protocol.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
Read Object Process
^^^^^^^^^^^^^^^^^^^^^^^^^^^

The read-object process enables Git to read all missing blobs with a
single process invocation for the entire life of a single Git command.
This is achieved by using a packet format (pkt-line, see technical/
protocol-common.txt) based protocol over standard input and standard
output as follows. All packets, except for the "*CONTENT" packets and
the "0000" flush packet, are considered text and therefore are
terminated by a LF.

Git starts the process when it encounters the first missing object that
needs to be retrieved. After the process is started, Git sends a welcome
message ("git-read-object-client"), a list of supported protocol version
numbers, and a flush packet. Git expects to read a welcome response
message ("git-read-object-server"), exactly one protocol version number
from the previously sent list, and a flush packet. All further
communication will be based on the selected version.

The remaining protocol description below documents "version=1". Please
note that "version=42" in the example below does not exist and is only
there to illustrate how the protocol would look with more than one
version.

After the version negotiation Git sends a list of all capabilities that
it supports and a flush packet. Git expects to read a list of desired
capabilities, which must be a subset of the supported capabilities list,
and a flush packet as response:
------------------------
packet: git> git-read-object-client
packet: git> version=1
packet: git> version=42
packet: git> 0000
packet: git< git-read-object-server
packet: git< version=1
packet: git< 0000
packet: git> capability=get
packet: git> capability=have
packet: git> capability=put
packet: git> capability=not-yet-invented
packet: git> 0000
packet: git< capability=get
packet: git< 0000
------------------------
The only supported capability in version 1 is "get".

Afterwards Git sends a list of "key=value" pairs terminated with a flush
packet. The list will contain at least the command (based on the
supported capabilities) and the sha1 of the object to retrieve. Please
note, that the process must not send any response before it received the
final flush packet.

When the process receives the "get" command, it should make the requested
object available in the git object store and then return success. Git will
then check the object store again and this time find it and proceed.
------------------------
packet: git> command=get
packet: git> sha1=0a214a649e1b3d5011e14a3dc227753f2bd2be05
packet: git> 0000
------------------------

The process is expected to respond with a list of "key=value" pairs
terminated with a flush packet. If the process does not experience
problems then the list must contain a "success" status.
------------------------
packet: git< status=success
packet: git< 0000
------------------------

In case the process cannot or does not want to process the content, it
is expected to respond with an "error" status.
------------------------
packet: git< status=error
packet: git< 0000
------------------------

In case the process cannot or does not want to process the content as
well as any future content for the lifetime of the Git process, then it
is expected to respond with an "abort" status at any point in the
protocol.
------------------------
packet: git< status=abort
packet: git< 0000
------------------------

Git neither stops nor restarts the process in case the "error"/"abort"
status is set.

If the process dies during the communication or does not adhere to the
protocol then Git will stop the process and restart it with the next
object that needs to be processed.

After the read-object process has processed an object it is expected to
wait for the next "key=value" list containing a command. Git will close
the command pipe on exit. The process is expected to detect EOF and exit
gracefully on its own. Git will wait until the process has stopped.

A long running read-object process demo implementation can be found in
`contrib/long-running-read-object/example.pl` located in the Git core
repository. If you develop your own long running process then the
`GIT_TRACE_PACKET` environment variables can be very helpful for
debugging (see linkgit:git[1]).
Loading