diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt index 4e23d73cdcadf6..5c86b3648732ec 100644 --- a/Documentation/config/extensions.txt +++ b/Documentation/config/extensions.txt @@ -6,3 +6,10 @@ extensions.objectFormat:: Note that this setting should only be set by linkgit:git-init[1] or linkgit:git-clone[1]. Trying to change it after initialization will not work and will produce hard-to-diagnose issues. + +extensions.sparseIndex:: + When combined with `core.sparseCheckout=true` and + `core.sparseCheckoutCone=true`, the index may contain entries + corresponding to directories outside of the sparse-checkout + definition. Versions of Git that do not understand this extension + do not expect directory entries in the index. diff --git a/Documentation/git-sparse-checkout.txt b/Documentation/git-sparse-checkout.txt index a0eeaeb02ee310..b51b8450cfd981 100644 --- a/Documentation/git-sparse-checkout.txt +++ b/Documentation/git-sparse-checkout.txt @@ -45,6 +45,20 @@ To avoid interfering with other worktrees, it first enables the When `--cone` is provided, the `core.sparseCheckoutCone` setting is also set, allowing for better performance with a limited set of patterns (see 'CONE PATTERN SET' below). ++ +Use the `--[no-]sparse-index` option to toggle the use of the sparse +index format. This reduces the size of the index to be more closely +aligned with your sparse-checkout definition. This can have significant +performance advantages for commands such as `git status` or `git add`. +This feature is still experimental. Some commands might be slower with +a sparse index until they are properly integrated with the feature. ++ +**WARNING:** Using a sparse index requires modifying the index in a way +that is not completely understood by other tools. Enabling sparse index +enables the `extensions.spareseIndex` config value, which might cause +other tools to stop working with your repository. If you have trouble with +this compatibility, then run `git sparse-checkout sparse-index disable` to +remove this config and rewrite your index to not be sparse. 'set':: Write a set of patterns to the sparse-checkout file, as given as diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt index 69edf46c031683..b633482b1bdff1 100644 --- a/Documentation/technical/index-format.txt +++ b/Documentation/technical/index-format.txt @@ -26,7 +26,7 @@ Git index format Extensions are identified by signature. Optional extensions can be ignored if Git does not understand them. - Git currently supports cached tree and resolve undo extensions. + Git currently supports cache tree and resolve undo extensions. 4-byte extension signature. If the first byte is 'A'..'Z' the extension is optional and can be ignored. @@ -136,14 +136,35 @@ Git index format == Extensions -=== Cached tree - - Cached tree extension contains pre-computed hashes for trees that can - be derived from the index. It helps speed up tree object generation - from index for a new commit. - - When a path is updated in index, the path must be invalidated and - removed from tree cache. +=== Cache tree + + Since the index does not record entries for directories, the cache + entries cannot describe tree objects that already exist in the object + database for regions of the index that are unchanged from an existing + commit. The cache tree extension stores a recursive tree structure that + describes the trees that already exist and completely match sections of + the cache entries. This speeds up tree object generation from the index + for a new commit by only computing the trees that are "new" to that + commit. It also assists when comparing the index to another tree, such + as `HEAD^{tree}`, since sections of the index can be skipped when a tree + comparison demonstrates equality. + + The recursive tree structure uses nodes that store a number of cache + entries, a list of subnodes, and an object ID (OID). The OID references + the existing tree for that node, if it is known to exist. The subnodes + correspond to subdirectories that themselves have cache tree nodes. The + number of cache entries corresponds to the number of cache entries in + the index that describe paths within that tree's directory. + + The extension tracks the full directory structure in the cache tree + extension, but this is generally smaller than the full cache entry list. + + When a path is updated in index, Git invalidates all nodes of the + recursive cache tree corresponding to the parent directories of that + path. We store these tree nodes as being "invalid" by using "-1" as the + number of cache entries. Invalid nodes still store a span of index + entries, allowing Git to focus its efforts when reconstructing a full + cache tree. The signature for this extension is { 'T', 'R', 'E', 'E' }. @@ -174,7 +195,8 @@ Git index format first entry represents the root level of the repository, followed by the first subtree--let's call this A--of the root level (with its name relative to the root level), followed by the first subtree of A (with - its name relative to A), ... + its name relative to A), and so on. The specified number of subtrees + indicates when the current level of the recursive stack is complete. === Resolve undo diff --git a/Documentation/technical/racy-git.txt b/Documentation/technical/racy-git.txt index ceda4bbfda4d27..65188e04559e02 100644 --- a/Documentation/technical/racy-git.txt +++ b/Documentation/technical/racy-git.txt @@ -26,7 +26,7 @@ information obtained from the filesystem via `lstat(2)` system call when they were last updated. When checking if they differ, Git first runs `lstat(2)` on the files and compares the result with this information (this is what was originally done by the -`ce_match_stat()` function, but the current code does it in +`ie_match_stat()` function, but the current code does it in `ce_match_stat_basic()` function). If some of these "cached stat information" fields do not match, Git can tell that the files are modified without even looking at their contents. @@ -102,7 +102,7 @@ timestamp as the index file itself. The callers that want to check if an index entry matches the corresponding file in the working tree continue to call -`ce_match_stat()`, but with this change, `ce_match_stat()` uses +`ie_match_stat()`, but with this change, `ie_match_stat()` uses `ce_modified_check_fs()` to see if racily clean ones are actually clean after comparing the cached stat information using `ce_match_stat_basic()`. @@ -128,7 +128,7 @@ Runtime penalty --------------- The runtime penalty of falling back to `ce_modified_check_fs()` -from `ce_match_stat()` can be very expensive when there are many +from `ie_match_stat()` can be very expensive when there are many racily clean entries. An obvious way to artificially create this situation is to give the same timestamp to all the files in the working tree in a large project, run `git update-index` on diff --git a/Makefile b/Makefile index 631fdc1260592f..89d6af61504911 100644 --- a/Makefile +++ b/Makefile @@ -603,9 +603,6 @@ unexport CDPATH SCRIPT_SH += git-bisect.sh SCRIPT_SH += git-difftool--helper.sh SCRIPT_SH += git-filter-branch.sh -SCRIPT_SH += git-merge-octopus.sh -SCRIPT_SH += git-merge-one-file.sh -SCRIPT_SH += git-merge-resolve.sh SCRIPT_SH += git-mergetool.sh SCRIPT_SH += git-quiltimport.sh SCRIPT_SH += git-request-pull.sh @@ -930,6 +927,7 @@ LIB_OBJS += merge-blobs.o LIB_OBJS += merge-ort.o LIB_OBJS += merge-ort-wrappers.o LIB_OBJS += merge-recursive.o +LIB_OBJS += merge-strategies.o LIB_OBJS += merge.o LIB_OBJS += mergesort.o LIB_OBJS += midx.o @@ -1003,6 +1001,7 @@ LIB_OBJS += sha1-name.o LIB_OBJS += shallow.o LIB_OBJS += sideband.o LIB_OBJS += sigchain.o +LIB_OBJS += sparse-index.o LIB_OBJS += split-index.o LIB_OBJS += stable-qsort.o LIB_OBJS += strbuf.o @@ -1120,8 +1119,11 @@ BUILTIN_OBJS += builtin/mailsplit.o BUILTIN_OBJS += builtin/merge-base.o BUILTIN_OBJS += builtin/merge-file.o BUILTIN_OBJS += builtin/merge-index.o +BUILTIN_OBJS += builtin/merge-octopus.o +BUILTIN_OBJS += builtin/merge-one-file.o BUILTIN_OBJS += builtin/merge-ours.o BUILTIN_OBJS += builtin/merge-recursive.o +BUILTIN_OBJS += builtin/merge-resolve.o BUILTIN_OBJS += builtin/merge-tree.o BUILTIN_OBJS += builtin/merge.o BUILTIN_OBJS += builtin/mktag.o diff --git a/apply.c b/apply.c index 67da906a7d0f65..a7a766fde3897c 100644 --- a/apply.c +++ b/apply.c @@ -3541,6 +3541,8 @@ static int load_current(struct apply_state *state, if (!patch->is_new) BUG("patch to %s is not a creation", patch->old_name); + ensure_full_index(state->repo->index); + pos = index_name_pos(state->repo->index, name, strlen(name)); if (pos < 0) return error(_("%s: does not exist in index"), name); @@ -3710,7 +3712,11 @@ static int check_preimage(struct apply_state *state, } if (state->check_index && !previous) { - int pos = index_name_pos(state->repo->index, old_name, + int pos; + + ensure_full_index(state->repo->index); + + pos = index_name_pos(state->repo->index, old_name, strlen(old_name)); if (pos < 0) { if (patch->is_new < 0) @@ -3769,6 +3775,8 @@ static int check_to_create(struct apply_state *state, if (state->check_index && (!ok_if_exists || !state->cached)) { int pos; + ensure_full_index(state->repo->index); + pos = index_name_pos(state->repo->index, new_name, strlen(new_name)); if (pos >= 0) { struct cache_entry *ce = state->repo->index->cache[pos]; diff --git a/archive.c b/archive.c index 5919d9e5050884..49e1cf128aa6e1 100644 --- a/archive.c +++ b/archive.c @@ -309,6 +309,7 @@ int write_archive_entries(struct archiver_args *args, opts.head_idx = -1; opts.src_index = args->repo->index; opts.dst_index = args->repo->index; + ensure_full_index(opts.src_index); opts.fn = oneway_merge; init_tree_desc(&t, args->tree->buffer, args->tree->size); if (unpack_trees(1, &t, &opts)) diff --git a/blame.c b/blame.c index a5044fcfaa6264..0aa368a35cfa4c 100644 --- a/blame.c +++ b/blame.c @@ -108,6 +108,7 @@ static void verify_working_tree_path(struct repository *r, return; } + ensure_full_index(r->index); pos = index_name_pos(r->index, path, strlen(path)); if (pos >= 0) ; /* path is in the index */ @@ -277,7 +278,11 @@ static struct commit *fake_working_tree_commit(struct repository *r, len = strlen(path); if (!mode) { - int pos = index_name_pos(r->index, path, len); + int pos; + + ensure_full_index(r->index); + + pos = index_name_pos(r->index, path, len); if (0 <= pos) mode = r->index->cache[pos]->ce_mode; else diff --git a/builtin.h b/builtin.h index c752d82a2a2400..e28f76624c67b5 100644 --- a/builtin.h +++ b/builtin.h @@ -177,9 +177,12 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix); int cmd_merge(int argc, const char **argv, const char *prefix); int cmd_merge_base(int argc, const char **argv, const char *prefix); int cmd_merge_index(int argc, const char **argv, const char *prefix); +int cmd_merge_octopus(int argc, const char **argv, const char *prefix); int cmd_merge_ours(int argc, const char **argv, const char *prefix); int cmd_merge_file(int argc, const char **argv, const char *prefix); +int cmd_merge_one_file(int argc, const char **argv, const char *prefix); int cmd_merge_recursive(int argc, const char **argv, const char *prefix); +int cmd_merge_resolve(int argc, const char **argv, const char *prefix); int cmd_merge_tree(int argc, const char **argv, const char *prefix); int cmd_mktag(int argc, const char **argv, const char *prefix); int cmd_mktree(int argc, const char **argv, const char *prefix); diff --git a/builtin/add.c b/builtin/add.c index f24a41598c5bbc..14390205f377a7 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -491,6 +491,9 @@ int cmd_add(int argc, const char **argv, const char *prefix) add_new_files = !take_worktree_changes && !refresh_only && !add_renormalize; require_pathspec = !(take_worktree_changes || (0 < addremove_explicit)); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR); /* diff --git a/builtin/am.c b/builtin/am.c index f22c73a05b0472..bea6906877a946 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1874,6 +1874,7 @@ static int fast_forward_to(struct tree *head, struct tree *remote, int reset) init_tree_desc(&t[0], head->buffer, head->size); init_tree_desc(&t[1], remote->buffer, remote->size); + ensure_full_index(opts.src_index); if (unpack_trees(2, t, &opts)) { rollback_lock_file(&lock_file); return -1; @@ -1908,6 +1909,7 @@ static int merge_tree(struct tree *tree) opts.fn = oneway_merge; init_tree_desc(&t[0], tree->buffer, tree->size); + ensure_full_index(opts.src_index); if (unpack_trees(1, t, &opts)) { rollback_lock_file(&lock_file); return -1; diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c index 4bbfc92dce5a0e..24c85b1c1250e3 100644 --- a/builtin/checkout-index.c +++ b/builtin/checkout-index.c @@ -48,11 +48,14 @@ static void write_tempfile_record(const char *name, const char *prefix) static int checkout_file(const char *name, const char *prefix) { int namelen = strlen(name); - int pos = cache_name_pos(name, namelen); + int pos; int has_same_name = 0; int did_checkout = 0; int errs = 0; + ensure_full_index(the_repository->index); + pos = index_name_pos(the_repository->index, name, namelen); + if (pos < 0) pos = -pos - 1; diff --git a/builtin/checkout.c b/builtin/checkout.c index 4c52ace2c1db13..6d9ca7d5fa5957 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -644,6 +644,7 @@ static int reset_tree(struct tree *tree, const struct checkout_opts *o, NULL); parse_tree(tree); init_tree_desc(&tree_desc, tree->buffer, tree->size); + ensure_full_index(opts.src_index); switch (unpack_trees(1, &tree_desc, &opts)) { case -2: *writeout_error = 1; @@ -824,9 +825,6 @@ static int merge_working_tree(const struct checkout_opts *opts, } } - if (!active_cache_tree) - active_cache_tree = cache_tree(); - if (!cache_tree_fully_valid(active_cache_tree)) cache_tree_update(&the_index, WRITE_TREE_SILENT | WRITE_TREE_REPAIR); diff --git a/builtin/clone.c b/builtin/clone.c index e2ab026b7a5f4d..61c09b7c0827dd 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -808,6 +808,7 @@ static int checkout(int submodule_progress) tree = parse_tree_indirect(&oid); parse_tree(tree); init_tree_desc(&t, tree->buffer, tree->size); + ensure_full_index(opts.src_index); if (unpack_trees(1, &t, &opts) < 0) die(_("unable to checkout working tree")); diff --git a/builtin/commit.c b/builtin/commit.c index f415e4808a78f9..0e91094651d6ab 100644 --- a/builtin/commit.c +++ b/builtin/commit.c @@ -428,6 +428,7 @@ static void create_base_index(const struct commit *current_head) die(_("failed to unpack HEAD tree object")); parse_tree(tree); init_tree_desc(&t, tree->buffer, tree->size); + ensure_full_index(opts.src_index); if (unpack_trees(1, &t, &opts)) exit(128); /* We've already reported the error, finish dying */ } @@ -1559,6 +1560,9 @@ int cmd_status(int argc, const char **argv, const char *prefix) if (argc == 2 && !strcmp(argv[1], "-h")) usage_with_options(builtin_status_usage, builtin_status_options); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + status_init_config(&s, git_status_config); argc = parse_options(argc, argv, prefix, builtin_status_options, diff --git a/builtin/grep.c b/builtin/grep.c index ca259af4416318..e53cf817204436 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -506,6 +506,8 @@ static int grep_cache(struct grep_opt *opt, if (repo_read_index(repo) < 0) die(_("index file corrupt")); + ensure_full_index(repo->index); + for (nr = 0; nr < repo->index->cache_nr; nr++) { const struct cache_entry *ce = repo->index->cache[nr]; strbuf_setlen(&name, name_base_len); diff --git a/builtin/ls-files.c b/builtin/ls-files.c index c8eae899b82a83..933e259cdbe93d 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -150,7 +150,7 @@ static void show_other_files(const struct index_state *istate, } } -static void show_killed_files(const struct index_state *istate, +static void show_killed_files(struct index_state *istate, const struct dir_struct *dir) { int i; @@ -159,6 +159,8 @@ static void show_killed_files(const struct index_state *istate, char *cp, *sp; int pos, len, killed = 0; + ensure_full_index(istate); + for (cp = ent->name; cp - ent->name < ent->len; cp = sp + 1) { sp = strchr(cp, '/'); if (!sp) { @@ -313,6 +315,7 @@ static void show_files(struct repository *repo, struct dir_struct *dir) show_killed_files(repo->index, dir); } if (show_cached || show_stage) { + ensure_full_index(repo->index); for (i = 0; i < repo->index->cache_nr; i++) { const struct cache_entry *ce = repo->index->cache[i]; @@ -332,6 +335,7 @@ static void show_files(struct repository *repo, struct dir_struct *dir) } } if (show_deleted || show_modified) { + ensure_full_index(repo->index); for (i = 0; i < repo->index->cache_nr; i++) { const struct cache_entry *ce = repo->index->cache[i]; struct stat st; @@ -368,6 +372,7 @@ static void prune_index(struct index_state *istate, if (!prefix || !istate->cache_nr) return; + ensure_full_index(istate); pos = index_name_pos(istate, prefix, prefixlen); if (pos < 0) pos = -pos-1; @@ -428,6 +433,8 @@ void overlay_tree_on_index(struct index_state *istate, if (!tree) die("bad tree-ish %s", tree_name); + ensure_full_index(istate); + /* Hoist the unmerged entries up to stage #3 to make room */ for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce = istate->cache[i]; diff --git a/builtin/merge-index.c b/builtin/merge-index.c index dbaf8fa7c6973f..fd1f1c86aaabf9 100644 --- a/builtin/merge-index.c +++ b/builtin/merge-index.c @@ -1,75 +1,16 @@ #define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" -#include "run-command.h" #include "config.h" - -static const char *pgm; -static int one_shot, quiet; -static int err; - -static int merge_entry(int pos, const char *path) -{ - int found; - const char *arguments[] = { pgm, "", "", "", path, "", "", "", NULL }; - char hexbuf[4][GIT_MAX_HEXSZ + 1]; - char ownbuf[4][60]; - - if (pos >= active_nr) - die("git merge-index: %s not in the cache", path); - found = 0; - do { - const struct cache_entry *ce = active_cache[pos]; - int stage = ce_stage(ce); - - if (strcmp(ce->name, path)) - break; - found++; - oid_to_hex_r(hexbuf[stage], &ce->oid); - xsnprintf(ownbuf[stage], sizeof(ownbuf[stage]), "%o", ce->ce_mode); - arguments[stage] = hexbuf[stage]; - arguments[stage + 4] = ownbuf[stage]; - } while (++pos < active_nr); - if (!found) - die("git merge-index: %s not in the cache", path); - - if (run_command_v_opt(arguments, 0)) { - if (one_shot) - err++; - else { - if (!quiet) - die("merge program failed"); - exit(1); - } - } - return found; -} - -static void merge_one_path(const char *path) -{ - int pos = cache_name_pos(path, strlen(path)); - - /* - * If it already exists in the cache as stage0, it's - * already merged and there is nothing to do. - */ - if (pos < 0) - merge_entry(-pos-1, path); -} - -static void merge_all(void) -{ - int i; - for (i = 0; i < active_nr; i++) { - const struct cache_entry *ce = active_cache[i]; - if (!ce_stage(ce)) - continue; - i += merge_entry(i, ce->name)-1; - } -} +#include "lockfile.h" +#include "merge-strategies.h" int cmd_merge_index(int argc, const char **argv, const char *prefix) { - int i, force_file = 0; + int i, force_file = 0, err = 0, one_shot = 0, quiet = 0; + const char *pgm; + void *data = NULL; + merge_fn merge_action; + struct lock_file lock = LOCK_INIT; /* Without this we cannot rely on waitpid() to tell * what happened to our children. @@ -83,6 +24,8 @@ int cmd_merge_index(int argc, const char **argv, const char *prefix) read_cache(); + ensure_full_index(&the_index); + i = 1; if (!strcmp(argv[i], "-o")) { one_shot = 1; @@ -92,7 +35,18 @@ int cmd_merge_index(int argc, const char **argv, const char *prefix) quiet = 1; i++; } + pgm = argv[i++]; + setup_work_tree(); + + if (!strcmp(pgm, "git-merge-one-file")) { + merge_action = merge_one_file_func; + hold_locked_index(&lock, LOCK_DIE_ON_ERROR); + } else { + merge_action = merge_one_file_spawn; + data = (void *)pgm; + } + for (; i < argc; i++) { const char *arg = argv[i]; if (!force_file && *arg == '-') { @@ -101,14 +55,23 @@ int cmd_merge_index(int argc, const char **argv, const char *prefix) continue; } if (!strcmp(arg, "-a")) { - merge_all(); + err |= merge_all_index(the_repository, one_shot, quiet, + merge_action, data); continue; } die("git merge-index: unknown option %s", arg); } - merge_one_path(arg); + err |= merge_index_path(the_repository, one_shot, quiet, arg, + merge_action, data); + } + + if (merge_action == merge_one_file_func) { + if (err) { + rollback_lock_file(&lock); + return err; + } + + return write_locked_index(&the_index, &lock, COMMIT_LOCK); } - if (err && !quiet) - die("merge program failed"); return err; } diff --git a/builtin/merge-octopus.c b/builtin/merge-octopus.c new file mode 100644 index 00000000000000..ca8f9f345d4750 --- /dev/null +++ b/builtin/merge-octopus.c @@ -0,0 +1,69 @@ +/* + * Builtin "git merge-octopus" + * + * Copyright (c) 2020 Alban Gruin + * + * Based on git-merge-octopus.sh, written by Junio C Hamano. + * + * Resolve two or more trees. + */ + +#define USE_THE_INDEX_COMPATIBILITY_MACROS +#include "cache.h" +#include "builtin.h" +#include "commit.h" +#include "merge-strategies.h" + +static const char builtin_merge_octopus_usage[] = + "git merge-octopus [...] -- [...]"; + +int cmd_merge_octopus(int argc, const char **argv, const char *prefix) +{ + int i, sep_seen = 0; + struct commit_list *bases = NULL, *remotes = NULL; + struct commit_list **next_base = &bases, **next_remote = &remotes; + const char *head_arg = NULL; + + if (argc < 5) + usage(builtin_merge_octopus_usage); + + setup_work_tree(); + if (read_cache() < 0) + die("invalid index"); + + /* + * The first parameters up to -- are merge bases; the rest are + * heads. + */ + for (i = 1; i < argc; i++) { + if (strcmp(argv[i], "--") == 0) + sep_seen = 1; + else if (strcmp(argv[i], "-h") == 0) + usage(builtin_merge_octopus_usage); + else if (sep_seen && !head_arg) + head_arg = argv[i]; + else { + struct object_id oid; + struct commit *commit; + + if (get_oid(argv[i], &oid)) + die("object %s not found.", argv[i]); + + commit = lookup_commit_or_die(&oid, argv[i]); + + if (sep_seen) + next_remote = commit_list_append(commit, next_remote); + else + next_base = commit_list_append(commit, next_base); + } + } + + /* + * Reject if this is not an octopus -- resolve should be used + * instead. + */ + if (commit_list_count(remotes) < 2) + return 2; + + return merge_strategies_octopus(the_repository, bases, head_arg, remotes); +} diff --git a/builtin/merge-one-file.c b/builtin/merge-one-file.c new file mode 100644 index 00000000000000..9c21778e1df6d9 --- /dev/null +++ b/builtin/merge-one-file.c @@ -0,0 +1,94 @@ +/* + * Builtin "git merge-one-file" + * + * Copyright (c) 2020 Alban Gruin + * + * Based on git-merge-one-file.sh, written by Linus Torvalds. + * + * This is the git per-file merge utility, called with + * + * argv[1] - original file object name (or empty) + * argv[2] - file in branch1 object name (or empty) + * argv[3] - file in branch2 object name (or empty) + * argv[4] - pathname in repository + * argv[5] - original file mode (or empty) + * argv[6] - file in branch1 mode (or empty) + * argv[7] - file in branch2 mode (or empty) + * + * Handle some trivial cases. The _really_ trivial cases have been + * handled already by git read-tree, but that one doesn't do any merges + * that might change the tree layout. + */ + +#define USE_THE_INDEX_COMPATIBILITY_MACROS +#include "cache.h" +#include "builtin.h" +#include "lockfile.h" +#include "merge-strategies.h" + +static const char builtin_merge_one_file_usage[] = + "git merge-one-file " + " \n\n" + "Blob ids and modes should be empty for missing files."; + +static int read_mode(const char *name, const char *arg, unsigned int *mode) +{ + char *last; + int ret = 0; + + *mode = strtol(arg, &last, 8); + + if (*last) + ret = error(_("invalid '%s' mode: expected nothing, got '%c'"), name, *last); + else if (!(S_ISREG(*mode) || S_ISDIR(*mode) || S_ISLNK(*mode))) + ret = error(_("invalid '%s' mode: %o"), name, *mode); + + return ret; +} + +int cmd_merge_one_file(int argc, const char **argv, const char *prefix) +{ + struct object_id orig_blob, our_blob, their_blob, + *p_orig_blob = NULL, *p_our_blob = NULL, *p_their_blob = NULL; + unsigned int orig_mode = 0, our_mode = 0, their_mode = 0, ret = 0; + struct lock_file lock = LOCK_INIT; + + if (argc != 8) + usage(builtin_merge_one_file_usage); + + if (read_cache() < 0) + die("invalid index"); + + hold_locked_index(&lock, LOCK_DIE_ON_ERROR); + + if (!get_oid_hex(argv[1], &orig_blob)) { + p_orig_blob = &orig_blob; + ret = read_mode("orig", argv[5], &orig_mode); + } else if (!*argv[1] && *argv[5]) + ret = error(_("no 'orig' object id given, but a mode was still given.")); + + if (!get_oid_hex(argv[2], &our_blob)) { + p_our_blob = &our_blob; + ret = read_mode("our", argv[6], &our_mode); + } else if (!*argv[2] && *argv[6]) + ret = error(_("no 'our' object id given, but a mode was still given.")); + + if (!get_oid_hex(argv[3], &their_blob)) { + p_their_blob = &their_blob; + ret = read_mode("their", argv[7], &their_mode); + } else if (!*argv[3] && *argv[7]) + ret = error(_("no 'their' object id given, but a mode was still given.")); + + if (ret) + return ret; + + ret = merge_three_way(the_repository, p_orig_blob, p_our_blob, p_their_blob, + argv[4], orig_mode, our_mode, their_mode); + + if (ret) { + rollback_lock_file(&lock); + return !!ret; + } + + return write_locked_index(&the_index, &lock, COMMIT_LOCK); +} diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c index a4bfd8fc51d6b2..972243b5e96ade 100644 --- a/builtin/merge-recursive.c +++ b/builtin/merge-recursive.c @@ -8,18 +8,6 @@ static const char builtin_merge_recursive_usage[] = "git %s ... -- ..."; -static char *better_branch_name(const char *branch) -{ - static char githead_env[8 + GIT_MAX_HEXSZ + 1]; - char *name; - - if (strlen(branch) != the_hash_algo->hexsz) - return xstrdup(branch); - xsnprintf(githead_env, sizeof(githead_env), "GITHEAD_%s", branch); - name = getenv(githead_env); - return xstrdup(name ? name : branch); -} - int cmd_merge_recursive(int argc, const char **argv, const char *prefix) { const struct object_id *bases[21]; @@ -75,8 +63,8 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix) if (get_oid(o.branch2, &h2)) die(_("could not resolve ref '%s'"), o.branch2); - o.branch1 = better1 = better_branch_name(o.branch1); - o.branch2 = better2 = better_branch_name(o.branch2); + o.branch1 = better1 = merge_get_better_branch_name(o.branch1); + o.branch2 = better2 = merge_get_better_branch_name(o.branch2); if (o.verbosity >= 3) printf(_("Merging %s with %s\n"), o.branch1, o.branch2); diff --git a/builtin/merge-resolve.c b/builtin/merge-resolve.c new file mode 100644 index 00000000000000..dca31676b88664 --- /dev/null +++ b/builtin/merge-resolve.c @@ -0,0 +1,73 @@ +/* + * Builtin "git merge-resolve" + * + * Copyright (c) 2020 Alban Gruin + * + * Based on git-merge-resolve.sh, written by Linus Torvalds and Junio C + * Hamano. + * + * Resolve two trees, using enhanced multi-base read-tree. + */ + +#define USE_THE_INDEX_COMPATIBILITY_MACROS +#include "cache.h" +#include "builtin.h" +#include "merge-strategies.h" + +static const char builtin_merge_resolve_usage[] = + "git merge-resolve ... -- "; + +int cmd_merge_resolve(int argc, const char **argv, const char *prefix) +{ + int i, sep_seen = 0; + const char *head = NULL; + struct commit_list *bases = NULL, *remote = NULL; + struct commit_list **next_base = &bases; + + if (argc < 5) + usage(builtin_merge_resolve_usage); + + setup_work_tree(); + if (read_cache() < 0) + die("invalid index"); + + /* + * The first parameters up to -- are merge bases; the rest are + * heads. + */ + for (i = 1; i < argc; i++) { + if (!strcmp(argv[i], "--")) + sep_seen = 1; + else if (!strcmp(argv[i], "-h")) + usage(builtin_merge_resolve_usage); + else if (sep_seen && !head) + head = argv[i]; + else { + struct object_id oid; + struct commit *commit; + + if (get_oid(argv[i], &oid)) + die("object %s not found.", argv[i]); + + commit = lookup_commit_or_die(&oid, argv[i]); + + if (sep_seen) + commit_list_insert(commit, &remote); + else + next_base = commit_list_append(commit, next_base); + } + } + + /* + * Give up if we are given two or more remotes. Not handling + * octopus. + */ + if (remote && remote->next) + return 2; + + /* Give up if this is a baseless merge. */ + if (!bases) + return 2; + + return merge_strategies_resolve(the_repository, bases, head, remote); +} diff --git a/builtin/merge.c b/builtin/merge.c index 1cff730715394f..ada1575bc864d5 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -42,6 +42,7 @@ #include "commit-reach.h" #include "wt-status.h" #include "commit-graph.h" +#include "merge-strategies.h" #define DEFAULT_TWOHEAD (1<<0) #define DEFAULT_OCTOPUS (1<<1) @@ -690,6 +691,7 @@ static int read_tree_trivial(struct object_id *common, struct object_id *head, parse_tree(trees[i]); init_tree_desc(t+i, trees[i]->buffer, trees[i]->size); } + ensure_full_index(opts.src_index); if (unpack_trees(nr_trees, t, &opts)) return -1; return 0; @@ -754,6 +756,12 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common, COMMIT_LOCK | SKIP_IF_UNCHANGED)) die(_("unable to write %s"), get_index_file()); return clean ? 0 : 1; + } else if (!strcmp(strategy, "resolve")) { + return merge_strategies_resolve(the_repository, common, + head_arg, remoteheads); + } else if (!strcmp(strategy, "octopus")) { + return merge_strategies_octopus(the_repository, common, + head_arg, remoteheads); } else { return try_merge_command(the_repository, strategy, xopts_nr, xopts, diff --git a/builtin/mv.c b/builtin/mv.c index 7dac714af90878..7d9c9b541ca21b 100644 --- a/builtin/mv.c +++ b/builtin/mv.c @@ -3,7 +3,6 @@ * * Copyright (C) 2006 Johannes Schindelin */ -#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "config.h" #include "pathspec.h" @@ -75,13 +74,14 @@ static const char *add_slash(const char *path) #define SUBMODULE_WITH_GITDIR ((const char *)1) -static void prepare_move_submodule(const char *src, int first, +static void prepare_move_submodule(struct index_state *istate, + const char *src, int first, const char **submodule_gitfile) { struct strbuf submodule_dotgit = STRBUF_INIT; - if (!S_ISGITLINK(active_cache[first]->ce_mode)) + if (!S_ISGITLINK(istate->cache[first]->ce_mode)) die(_("Directory %s is in index and no submodule?"), src); - if (!is_staging_gitmodules_ok(&the_index)) + if (!is_staging_gitmodules_ok(istate)) die(_("Please stage your changes to .gitmodules or stash them to proceed")); strbuf_addf(&submodule_dotgit, "%s/.git", src); *submodule_gitfile = read_gitfile(submodule_dotgit.buf); @@ -92,19 +92,20 @@ static void prepare_move_submodule(const char *src, int first, strbuf_release(&submodule_dotgit); } -static int index_range_of_same_dir(const char *src, int length, +static int index_range_of_same_dir(struct index_state *istate, + const char *src, int length, int *first_p, int *last_p) { const char *src_w_slash = add_slash(src); int first, last, len_w_slash = length + 1; - first = cache_name_pos(src_w_slash, len_w_slash); + first = index_name_pos(istate, src_w_slash, len_w_slash); if (first >= 0) die(_("%.*s is in index"), len_w_slash, src_w_slash); first = -1 - first; - for (last = first; last < active_nr; last++) { - const char *path = active_cache[last]->name; + for (last = first; last < istate->cache_nr; last++) { + const char *path = istate->cache[last]->name; if (strncmp(path, src_w_slash, len_w_slash)) break; } @@ -133,6 +134,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix) struct string_list src_for_dst = STRING_LIST_INIT_NODUP; struct lock_file lock_file = LOCK_INIT; struct cache_entry *ce; + struct index_state *istate; git_config(git_default_config, NULL); @@ -141,9 +143,12 @@ int cmd_mv(int argc, const char **argv, const char *prefix) if (--argc < 1) usage_with_options(builtin_mv_usage, builtin_mv_options); - hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR); - if (read_cache() < 0) + repo_hold_locked_index(the_repository, &lock_file, LOCK_DIE_ON_ERROR); + if (repo_read_index(the_repository) < 0) die(_("index file corrupt")); + istate = the_repository->index; + + ensure_full_index(istate); source = internal_prefix_pathspec(prefix, argv, argc, 0); modes = xcalloc(argc, sizeof(enum update_mode)); @@ -190,12 +195,13 @@ int cmd_mv(int argc, const char **argv, const char *prefix) && lstat(dst, &st) == 0) bad = _("cannot move directory over file"); else if (src_is_dir) { - int first = cache_name_pos(src, length), last; + int first = index_name_pos(istate, src, length); + int last; if (first >= 0) - prepare_move_submodule(src, first, + prepare_move_submodule(istate, src, first, submodule_gitfile + i); - else if (index_range_of_same_dir(src, length, + else if (index_range_of_same_dir(istate, src, length, &first, &last) < 1) bad = _("source directory is empty"); else { /* last - first >= 1 */ @@ -212,7 +218,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix) dst_len = strlen(dst); for (j = 0; j < last - first; j++) { - const char *path = active_cache[first + j]->name; + const char *path = istate->cache[first + j]->name; source[argc + j] = path; destination[argc + j] = prefix_path(dst, dst_len, path + length + 1); @@ -221,7 +227,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix) } argc += last - first; } - } else if (!(ce = cache_file_exists(src, length, ignore_case))) { + } else if (!(ce = index_file_exists(istate, src, length, ignore_case))) { bad = _("not under version control"); } else if (ce_stage(ce)) { bad = _("conflicted"); @@ -291,15 +297,15 @@ int cmd_mv(int argc, const char **argv, const char *prefix) if (mode == WORKING_DIRECTORY) continue; - pos = cache_name_pos(src, strlen(src)); + pos = index_name_pos(istate, src, strlen(src)); assert(pos >= 0); - rename_cache_entry_at(pos, dst); + rename_index_entry_at(istate, pos, dst); } if (gitmodules_modified) - stage_updated_gitmodules(&the_index); + stage_updated_gitmodules(istate); - if (write_locked_index(&the_index, &lock_file, + if (write_locked_index(istate, &lock_file, COMMIT_LOCK | SKIP_IF_UNCHANGED)) die(_("Unable to write new index file")); diff --git a/builtin/read-tree.c b/builtin/read-tree.c index 485e7b0479488c..b666a58b519cd4 100644 --- a/builtin/read-tree.c +++ b/builtin/read-tree.c @@ -247,6 +247,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix) parse_tree(tree); init_tree_desc(t+i, tree->buffer, tree->size); } + ensure_full_index(opts.src_index); if (unpack_trees(nr_trees, t, &opts)) return 128; diff --git a/builtin/reset.c b/builtin/reset.c index 7aa305439d4310..32c605aae3e3b1 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -97,6 +97,7 @@ static int reset_index(const char *ref, const struct object_id *oid, int reset_t } nr++; + ensure_full_index(opts.src_index); if (unpack_trees(nr, desc, &opts)) goto out; diff --git a/builtin/rm.c b/builtin/rm.c index 4858631e0f02c5..305ae6ca94965d 100644 --- a/builtin/rm.c +++ b/builtin/rm.c @@ -3,7 +3,6 @@ * * Copyright (C) Linus Torvalds 2006 */ -#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "config.h" #include "lockfile.h" @@ -28,12 +27,14 @@ static struct { } *entry; } list; -static int get_ours_cache_pos(const char *path, int pos) +static int get_ours_cache_pos(struct index_state *istate, + const char *path, int pos) { int i = -pos - 1; - while ((i < active_nr) && !strcmp(active_cache[i]->name, path)) { - if (ce_stage(active_cache[i]) == 2) + while ((i < istate->cache_nr) && + !strcmp(istate->cache[i]->name, path)) { + if (ce_stage(istate->cache[i]) == 2) return i; i++; } @@ -61,7 +62,7 @@ static void print_error_files(struct string_list *files_list, } } -static void submodules_absorb_gitdir_if_needed(void) +static void submodules_absorb_gitdir_if_needed(struct index_state *istate) { int i; for (i = 0; i < list.nr; i++) { @@ -69,13 +70,13 @@ static void submodules_absorb_gitdir_if_needed(void) int pos; const struct cache_entry *ce; - pos = cache_name_pos(name, strlen(name)); + pos = index_name_pos(istate, name, strlen(name)); if (pos < 0) { - pos = get_ours_cache_pos(name, pos); + pos = get_ours_cache_pos(istate, name, pos); if (pos < 0) continue; } - ce = active_cache[pos]; + ce = istate->cache[pos]; if (!S_ISGITLINK(ce->ce_mode) || !file_exists(ce->name) || @@ -88,7 +89,8 @@ static void submodules_absorb_gitdir_if_needed(void) } } -static int check_local_mod(struct object_id *head, int index_only) +static int check_local_mod(struct index_state *istate, + struct object_id *head, int index_only) { /* * Items in list are already sorted in the cache order, @@ -114,21 +116,21 @@ static int check_local_mod(struct object_id *head, int index_only) int local_changes = 0; int staged_changes = 0; - pos = cache_name_pos(name, strlen(name)); + pos = index_name_pos(istate, name, strlen(name)); if (pos < 0) { /* * Skip unmerged entries except for populated submodules * that could lose history when removed. */ - pos = get_ours_cache_pos(name, pos); + pos = get_ours_cache_pos(istate, name, pos); if (pos < 0) continue; - if (!S_ISGITLINK(active_cache[pos]->ce_mode) || + if (!S_ISGITLINK(istate->cache[pos]->ce_mode) || is_empty_dir(name)) continue; } - ce = active_cache[pos]; + ce = istate->cache[pos]; if (lstat(ce->name, &st) < 0) { if (!is_missing_file_error(errno)) @@ -165,7 +167,7 @@ static int check_local_mod(struct object_id *head, int index_only) * Is the index different from the file in the work tree? * If it's a submodule, is its work tree modified? */ - if (ce_match_stat(ce, &st, 0) || + if (ie_match_stat(istate, ce, &st, 0) || (S_ISGITLINK(ce->ce_mode) && bad_to_remove_submodule(ce->name, SUBMODULE_REMOVAL_DIE_ON_ERROR | @@ -257,6 +259,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix) int i; struct pathspec pathspec; char *seen; + struct index_state *istate; git_config(git_default_config, NULL); @@ -284,24 +287,27 @@ int cmd_rm(int argc, const char **argv, const char *prefix) if (!index_only) setup_work_tree(); - hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR); + repo_hold_locked_index(the_repository, &lock_file, LOCK_DIE_ON_ERROR); - if (read_cache() < 0) + if (repo_read_index(the_repository) < 0) die(_("index file corrupt")); - refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, &pathspec, NULL, NULL); + istate = the_repository->index; + refresh_index(istate, REFRESH_QUIET|REFRESH_UNMERGED, &pathspec, NULL, NULL); + + ensure_full_index(istate); seen = xcalloc(pathspec.nr, 1); - for (i = 0; i < active_nr; i++) { - const struct cache_entry *ce = active_cache[i]; - if (!ce_path_match(&the_index, ce, &pathspec, seen)) + for (i = 0; i < istate->cache_nr; i++) { + const struct cache_entry *ce = istate->cache[i]; + if (!ce_path_match(istate, ce, &pathspec, seen)) continue; ALLOC_GROW(list.entry, list.nr + 1, list.alloc); list.entry[list.nr].name = xstrdup(ce->name); list.entry[list.nr].is_submodule = S_ISGITLINK(ce->ce_mode); if (list.entry[list.nr++].is_submodule && - !is_staging_gitmodules_ok(&the_index)) + !is_staging_gitmodules_ok(istate)) die(_("please stage your changes to .gitmodules or stash them to proceed")); } @@ -329,7 +335,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix) } if (!index_only) - submodules_absorb_gitdir_if_needed(); + submodules_absorb_gitdir_if_needed(istate); /* * If not forced, the file, the index and the HEAD (if exists) @@ -345,7 +351,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix) struct object_id oid; if (get_oid("HEAD", &oid)) oidclr(&oid); - if (check_local_mod(&oid, index_only)) + if (check_local_mod(istate, &oid, index_only)) exit(1); } @@ -358,7 +364,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix) if (!quiet) printf("rm '%s'\n", path); - if (remove_file_from_cache(path)) + if (remove_file_from_index(istate, path)) die(_("git rm: unable to remove %s"), path); } @@ -398,10 +404,10 @@ int cmd_rm(int argc, const char **argv, const char *prefix) } strbuf_release(&buf); if (gitmodules_modified) - stage_updated_gitmodules(&the_index); + stage_updated_gitmodules(istate); } - if (write_locked_index(&the_index, &lock_file, + if (write_locked_index(istate, &lock_file, COMMIT_LOCK | SKIP_IF_UNCHANGED)) die(_("Unable to write new index file")); diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index e3140db2a0a63f..14022b5e1826ff 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -14,6 +14,7 @@ #include "unpack-trees.h" #include "wt-status.h" #include "quote.h" +#include "sparse-index.h" static const char *empty_base = ""; @@ -22,11 +23,6 @@ static char const * const builtin_sparse_checkout_usage[] = { NULL }; -static char *get_sparse_checkout_filename(void) -{ - return git_pathdup("info/sparse-checkout"); -} - static void write_patterns_to_file(FILE *fp, struct pattern_list *pl) { int i; @@ -115,6 +111,8 @@ static int update_working_directory(struct pattern_list *pl) if (is_index_unborn(r->index)) return UPDATE_SPARSITY_SUCCESS; + r->index->sparse_checkout_patterns = pl; + memset(&o, 0, sizeof(o)); o.verbose_update = isatty(2); o.update = 1; @@ -125,6 +123,7 @@ static int update_working_directory(struct pattern_list *pl) o.pl = pl; setup_work_tree(); + ensure_full_index(r->index); repo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR); @@ -143,6 +142,7 @@ static int update_working_directory(struct pattern_list *pl) else rollback_lock_file(&lock_file); + r->index->sparse_checkout_patterns = NULL; return result; } @@ -285,12 +285,13 @@ static int set_config(enum sparse_checkout_mode mode) } static char const * const builtin_sparse_checkout_init_usage[] = { - N_("git sparse-checkout init [--cone]"), + N_("git sparse-checkout init [--cone] [--[no-]sparse-index]"), NULL }; static struct sparse_checkout_init_opts { int cone_mode; + int sparse_index; } init_opts; static int sparse_checkout_init(int argc, const char **argv) @@ -305,11 +306,15 @@ static int sparse_checkout_init(int argc, const char **argv) static struct option builtin_sparse_checkout_init_options[] = { OPT_BOOL(0, "cone", &init_opts.cone_mode, N_("initialize the sparse-checkout in cone mode")), + OPT_BOOL(0, "sparse-index", &init_opts.sparse_index, + N_("toggle the use of a sparse index")), OPT_END(), }; repo_read_index(the_repository); + init_opts.sparse_index = -1; + argc = parse_options(argc, argv, NULL, builtin_sparse_checkout_init_options, builtin_sparse_checkout_init_usage, 0); @@ -328,6 +333,15 @@ static int sparse_checkout_init(int argc, const char **argv) sparse_filename = get_sparse_checkout_filename(); res = add_patterns_from_file_to_list(sparse_filename, "", 0, &pl, NULL); + if (init_opts.sparse_index >= 0) { + if (set_sparse_index_config(the_repository, init_opts.sparse_index) < 0) + die(_("failed to modify sparse-index config")); + + /* force an index rewrite */ + repo_read_index(the_repository); + the_repository->index->updated_workdir = 1; + } + /* If we already have a sparse-checkout file, use it. */ if (res >= 0) { free(sparse_filename); @@ -522,19 +536,18 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m) { int result; int changed_config = 0; - struct pattern_list pl; - memset(&pl, 0, sizeof(pl)); + struct pattern_list *pl = xcalloc(1, sizeof(*pl)); switch (m) { case ADD: if (core_sparse_checkout_cone) - add_patterns_cone_mode(argc, argv, &pl); + add_patterns_cone_mode(argc, argv, pl); else - add_patterns_literal(argc, argv, &pl); + add_patterns_literal(argc, argv, pl); break; case REPLACE: - add_patterns_from_input(&pl, argc, argv); + add_patterns_from_input(pl, argc, argv); break; } @@ -544,12 +557,13 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m) changed_config = 1; } - result = write_patterns_and_update(&pl); + result = write_patterns_and_update(pl); if (result && changed_config) set_config(MODE_NO_PATTERNS); - clear_pattern_list(&pl); + clear_pattern_list(pl); + free(pl); return result; } diff --git a/builtin/stash.c b/builtin/stash.c index e1f8235fdd3999..50493cb9cd9d0c 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -258,6 +258,7 @@ static int reset_tree(struct object_id *i_tree, int update, int reset) opts.update = update; opts.fn = oneway_merge; + ensure_full_index(opts.src_index); if (unpack_trees(nr_trees, t, &opts)) return -1; diff --git a/builtin/update-index.c b/builtin/update-index.c index 8718e26d5b6a9b..22b2286083077b 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -3,7 +3,6 @@ * * Copyright (C) Linus Torvalds, 2005 */ -#define USE_THE_INDEX_COMPATIBILITY_MACROS #include "cache.h" #include "gvfs.h" #include "config.h" @@ -228,29 +227,30 @@ static int test_if_untracked_cache_is_supported(void) return ret; } -static int mark_ce_flags(const char *path, int flag, int mark) +static int mark_ce_flags(struct index_state *istate, + const char *path, int flag, int mark) { int namelen = strlen(path); - int pos = cache_name_pos(path, namelen); + int pos = index_name_pos(istate, path, namelen); if (0 <= pos) { - mark_fsmonitor_invalid(&the_index, active_cache[pos]); + mark_fsmonitor_invalid(istate, istate->cache[pos]); if (mark) - active_cache[pos]->ce_flags |= flag; + istate->cache[pos]->ce_flags |= flag; else - active_cache[pos]->ce_flags &= ~flag; - active_cache[pos]->ce_flags |= CE_UPDATE_IN_BASE; - cache_tree_invalidate_path(&the_index, path); - active_cache_changed |= CE_ENTRY_CHANGED; + istate->cache[pos]->ce_flags &= ~flag; + istate->cache[pos]->ce_flags |= CE_UPDATE_IN_BASE; + cache_tree_invalidate_path(istate, path); + istate->cache_changed |= CE_ENTRY_CHANGED; return 0; } return -1; } -static int remove_one_path(const char *path) +static int remove_one_path(struct index_state *istate, const char *path) { if (!allow_remove) return error("%s: does not exist and --remove not passed", path); - if (remove_file_from_cache(path)) + if (remove_file_from_index(istate, path)) return error("%s: cannot remove from the index", path); return 0; } @@ -262,37 +262,40 @@ static int remove_one_path(const char *path) * succeeds. * - permission error. That's never ok. */ -static int process_lstat_error(const char *path, int err) +static int process_lstat_error(struct index_state *istate, + const char *path, int err) { if (is_missing_file_error(err)) - return remove_one_path(path); + return remove_one_path(istate, path); return error("lstat(\"%s\"): %s", path, strerror(err)); } -static int add_one_path(const struct cache_entry *old, const char *path, int len, struct stat *st) +static int add_one_path(struct index_state *istate, + const struct cache_entry *old, + const char *path, int len, struct stat *st) { int option; struct cache_entry *ce; /* Was the old index entry already up-to-date? */ - if (old && !ce_stage(old) && !ce_match_stat(old, st, 0)) + if (old && !ce_stage(old) && !ie_match_stat(istate, old, st, 0)) return 0; - ce = make_empty_cache_entry(&the_index, len); + ce = make_empty_cache_entry(istate, len); memcpy(ce->name, path, len); ce->ce_flags = create_ce_flags(0); ce->ce_namelen = len; - fill_stat_cache_info(&the_index, ce, st); + fill_stat_cache_info(istate, ce, st); ce->ce_mode = ce_mode_from_stat(old, st->st_mode); - if (index_path(&the_index, &ce->oid, path, st, + if (index_path(istate, &ce->oid, path, st, info_only ? 0 : HASH_WRITE_OBJECT)) { discard_cache_entry(ce); return -1; } option = allow_add ? ADD_CACHE_OK_TO_ADD : 0; option |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0; - if (add_cache_entry(ce, option)) { + if (add_index_entry(istate, ce, option)) { discard_cache_entry(ce); return error("%s: cannot add to the index - missing --add option?", path); } @@ -322,30 +325,31 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len * - it doesn't exist at all in the index, but it is a valid * git directory, and it should be *added* as a gitlink. */ -static int process_directory(const char *path, int len, struct stat *st) +static int process_directory(struct index_state *istate, + const char *path, int len, struct stat *st) { struct object_id oid; - int pos = cache_name_pos(path, len); + int pos = index_name_pos(istate, path, len); /* Exact match: file or existing gitlink */ if (pos >= 0) { - const struct cache_entry *ce = active_cache[pos]; + const struct cache_entry *ce = istate->cache[pos]; if (S_ISGITLINK(ce->ce_mode)) { /* Do nothing to the index if there is no HEAD! */ if (resolve_gitlink_ref(path, "HEAD", &oid) < 0) return 0; - return add_one_path(ce, path, len, st); + return add_one_path(istate, ce, path, len, st); } /* Should this be an unconditional error? */ - return remove_one_path(path); + return remove_one_path(istate, path); } /* Inexact match: is there perhaps a subdirectory match? */ pos = -pos-1; - while (pos < active_nr) { - const struct cache_entry *ce = active_cache[pos++]; + while (pos < istate->cache_nr) { + const struct cache_entry *ce = istate->cache[pos++]; if (strncmp(ce->name, path, len)) break; @@ -360,13 +364,14 @@ static int process_directory(const char *path, int len, struct stat *st) /* No match - should we add it as a gitlink? */ if (!resolve_gitlink_ref(path, "HEAD", &oid)) - return add_one_path(NULL, path, len, st); + return add_one_path(istate, NULL, path, len, st); /* Error out. */ return error("%s: is a directory - add files inside instead", path); } -static int process_path(const char *path, struct stat *st, int stat_errno) +static int process_path(struct index_state *istate, + const char *path, struct stat *st, int stat_errno) { int pos, len; const struct cache_entry *ce; @@ -375,8 +380,8 @@ static int process_path(const char *path, struct stat *st, int stat_errno) if (has_symlink_leading_path(path, len)) return error("'%s' is beyond a symbolic link", path); - pos = cache_name_pos(path, len); - ce = pos < 0 ? NULL : active_cache[pos]; + pos = index_name_pos(istate, path, len); + ce = pos < 0 ? NULL : istate->cache[pos]; if (ce && ce_skip_worktree(ce)) { /* * working directory version is assumed "good" @@ -384,7 +389,7 @@ static int process_path(const char *path, struct stat *st, int stat_errno) * On the other hand, removing it from index should work */ if (!ignore_skip_worktree_entries && allow_remove && - remove_file_from_cache(path)) + remove_file_from_index(istate, path)) return error("%s: cannot remove from the index", path); return 0; } @@ -394,52 +399,43 @@ static int process_path(const char *path, struct stat *st, int stat_errno) * what to do about the pathname! */ if (stat_errno) - return process_lstat_error(path, stat_errno); + return process_lstat_error(istate, path, stat_errno); if (S_ISDIR(st->st_mode)) - return process_directory(path, len, st); + return process_directory(istate, path, len, st); - return add_one_path(ce, path, len, st); + return add_one_path(istate, ce, path, len, st); } -static int add_cacheinfo(unsigned int mode, const struct object_id *oid, +static int add_cacheinfo(struct index_state *istate, + unsigned int mode, const struct object_id *oid, const char *path, int stage) { - int len, option; - struct cache_entry *ce; - - if (!verify_path(path, mode)) - return error("Invalid path '%s'", path); + int res; - len = strlen(path); - ce = make_empty_cache_entry(&the_index, len); - - oidcpy(&ce->oid, oid); - memcpy(ce->name, path, len); - ce->ce_flags = create_ce_flags(stage); - ce->ce_namelen = len; - ce->ce_mode = create_ce_mode(mode); - if (assume_unchanged) - ce->ce_flags |= CE_VALID; - option = allow_add ? ADD_CACHE_OK_TO_ADD : 0; - option |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0; - if (add_cache_entry(ce, option)) + res = add_to_index_cacheinfo(istate, mode, oid, path, stage, + allow_add, allow_replace, NULL); + if (res == -1) + return res; + if (res == -2) return error("%s: cannot add to the index - missing --add option?", path); + report("add '%s'", path); return 0; } -static void chmod_path(char flip, const char *path) +static void chmod_path(struct index_state *istate, + char flip, const char *path) { int pos; struct cache_entry *ce; - pos = cache_name_pos(path, strlen(path)); + pos = index_name_pos(istate, path, strlen(path)); if (pos < 0) goto fail; - ce = active_cache[pos]; - if (chmod_cache_entry(ce, flip) < 0) + ce = istate->cache[pos]; + if (chmod_index_entry(istate, ce, flip) < 0) goto fail; report("chmod %cx '%s'", flip, path); @@ -448,7 +444,7 @@ static void chmod_path(char flip, const char *path) die("git update-index: cannot chmod %cx '%s'", flip, path); } -static void update_one(const char *path) +static void update_one(struct index_state *istate, const char *path) { int stat_errno = 0; struct stat st; @@ -466,33 +462,36 @@ static void update_one(const char *path) return; } if (mark_valid_only) { - if (mark_ce_flags(path, CE_VALID, mark_valid_only == MARK_FLAG)) + if (mark_ce_flags(istate, path, CE_VALID, + mark_valid_only == MARK_FLAG)) die("Unable to mark file %s", path); return; } if (mark_skip_worktree_only) { - if (mark_ce_flags(path, CE_SKIP_WORKTREE, mark_skip_worktree_only == MARK_FLAG)) + if (mark_ce_flags(istate, path, CE_SKIP_WORKTREE, + mark_skip_worktree_only == MARK_FLAG)) die("Unable to mark file %s", path); return; } if (mark_fsmonitor_only) { - if (mark_ce_flags(path, CE_FSMONITOR_VALID, mark_fsmonitor_only == MARK_FLAG)) + if (mark_ce_flags(istate, path, CE_FSMONITOR_VALID, + mark_fsmonitor_only == MARK_FLAG)) die("Unable to mark file %s", path); return; } if (force_remove) { - if (remove_file_from_cache(path)) + if (remove_file_from_index(istate, path)) die("git update-index: unable to remove %s", path); report("remove '%s'", path); return; } - if (process_path(path, &st, stat_errno)) + if (process_path(istate, path, &st, stat_errno)) die("Unable to process path %s", path); report("add '%s'", path); } -static void read_index_info(int nul_term_line) +static void read_index_info(struct index_state *istate, int nul_term_line) { const int hexsz = the_hash_algo->hexsz; struct strbuf buf = STRBUF_INIT; @@ -565,7 +564,7 @@ static void read_index_info(int nul_term_line) if (!mode) { /* mode == 0 means there is no such path -- remove */ - if (remove_file_from_cache(path_name)) + if (remove_file_from_index(istate, path_name)) die("git update-index: unable to remove %s", ptr); } @@ -575,7 +574,7 @@ static void read_index_info(int nul_term_line) * ptr[-41] is at the beginning of sha1 */ ptr[-(hexsz + 2)] = ptr[-1] = 0; - if (add_cacheinfo(mode, &oid, path_name, stage)) + if (add_cacheinfo(istate, mode, &oid, path_name, stage)) die("git update-index: unable to update %s", path_name); } @@ -596,7 +595,8 @@ static const char * const update_index_usage[] = { static struct object_id head_oid; static struct object_id merge_head_oid; -static struct cache_entry *read_one_ent(const char *which, +static struct cache_entry *read_one_ent(struct repository *repo, + const char *which, struct object_id *ent, const char *path, int namelen, int stage) { @@ -604,7 +604,7 @@ static struct cache_entry *read_one_ent(const char *which, struct object_id oid; struct cache_entry *ce; - if (get_tree_entry(the_repository, ent, path, &oid, &mode)) { + if (get_tree_entry(repo, ent, path, &oid, &mode)) { if (which) error("%s: not in %s branch.", path, which); return NULL; @@ -614,7 +614,7 @@ static struct cache_entry *read_one_ent(const char *which, error("%s: not a blob in %s branch.", path, which); return NULL; } - ce = make_empty_cache_entry(&the_index, namelen); + ce = make_empty_cache_entry(repo->index, namelen); oidcpy(&ce->oid, &oid); memcpy(ce->name, path, namelen); @@ -624,7 +624,8 @@ static struct cache_entry *read_one_ent(const char *which, return ce; } -static int unresolve_one(const char *path) +static int unresolve_one(struct repository *repo, + const char *path) { int namelen = strlen(path); int pos; @@ -632,12 +633,12 @@ static int unresolve_one(const char *path) struct cache_entry *ce_2 = NULL, *ce_3 = NULL; /* See if there is such entry in the index. */ - pos = cache_name_pos(path, namelen); + pos = index_name_pos(repo->index, path, namelen); if (0 <= pos) { /* already merged */ - pos = unmerge_cache_entry_at(pos); - if (pos < active_nr) { - const struct cache_entry *ce = active_cache[pos]; + pos = unmerge_index_entry_at(repo->index, pos); + if (pos < repo->index->cache_nr) { + const struct cache_entry *ce = repo->index->cache[pos]; if (ce_stage(ce) && ce_namelen(ce) == namelen && !memcmp(ce->name, path, namelen)) @@ -650,8 +651,8 @@ static int unresolve_one(const char *path) * want to do anything in the former case. */ pos = -pos-1; - if (pos < active_nr) { - const struct cache_entry *ce = active_cache[pos]; + if (pos < repo->index->cache_nr) { + const struct cache_entry *ce = repo->index->cache[pos]; if (ce_namelen(ce) == namelen && !memcmp(ce->name, path, namelen)) { fprintf(stderr, @@ -666,8 +667,8 @@ static int unresolve_one(const char *path) * stuff HEAD version in stage #2, * stuff MERGE_HEAD version in stage #3. */ - ce_2 = read_one_ent("our", &head_oid, path, namelen, 2); - ce_3 = read_one_ent("their", &merge_head_oid, path, namelen, 3); + ce_2 = read_one_ent(repo, "our", &head_oid, path, namelen, 2); + ce_3 = read_one_ent(repo, "their", &merge_head_oid, path, namelen, 3); if (!ce_2 || !ce_3) { ret = -1; @@ -680,13 +681,13 @@ static int unresolve_one(const char *path) goto free_return; } - remove_file_from_cache(path); - if (add_cache_entry(ce_2, ADD_CACHE_OK_TO_ADD)) { + remove_file_from_index(repo->index, path); + if (add_index_entry(repo->index, ce_2, ADD_CACHE_OK_TO_ADD)) { error("%s: cannot add our version to the index.", path); ret = -1; goto free_return; } - if (!add_cache_entry(ce_3, ADD_CACHE_OK_TO_ADD)) + if (!add_index_entry(repo->index, ce_3, ADD_CACHE_OK_TO_ADD)) return 0; error("%s: cannot add their version to the index.", path); ret = -1; @@ -706,7 +707,8 @@ static void read_head_pointers(void) } } -static int do_unresolve(int ac, const char **av, +static int do_unresolve(struct repository *repo, + int ac, const char **av, const char *prefix, int prefix_length) { int i; @@ -720,13 +722,14 @@ static int do_unresolve(int ac, const char **av, for (i = 1; i < ac; i++) { const char *arg = av[i]; char *p = prefix_path(prefix, prefix_length, arg); - err |= unresolve_one(p); + err |= unresolve_one(repo, p); free(p); } return err; } -static int do_reupdate(int ac, const char **av, +static int do_reupdate(struct repository *repo, + int ac, const char **av, const char *prefix) { /* Read HEAD and run update-index on paths that are @@ -746,16 +749,16 @@ static int do_reupdate(int ac, const char **av, */ has_head = 0; redo: - for (pos = 0; pos < active_nr; pos++) { - const struct cache_entry *ce = active_cache[pos]; + for (pos = 0; pos < repo->index->cache_nr; pos++) { + const struct cache_entry *ce = repo->index->cache[pos]; struct cache_entry *old = NULL; int save_nr; char *path; - if (ce_stage(ce) || !ce_path_match(&the_index, ce, &pathspec, NULL)) + if (ce_stage(ce) || !ce_path_match(repo->index, ce, &pathspec, NULL)) continue; if (has_head) - old = read_one_ent(NULL, &head_oid, + old = read_one_ent(repo, NULL, &head_oid, ce->name, ce_namelen(ce), 0); if (old && ce->ce_mode == old->ce_mode && oideq(&ce->oid, &old->oid)) { @@ -764,30 +767,35 @@ static int do_reupdate(int ac, const char **av, } /* Be careful. The working tree may not have the * path anymore, in which case, under 'allow_remove', - * or worse yet 'allow_replace', active_nr may decrease. + * or worse yet 'allow_replace', repo->index->cache_nr may decrease. */ - save_nr = active_nr; + save_nr = repo->index->cache_nr; path = xstrdup(ce->name); - update_one(path); + update_one(repo->index, path); free(path); discard_cache_entry(old); - if (save_nr != active_nr) + if (save_nr != repo->index->cache_nr) goto redo; } clear_pathspec(&pathspec); return 0; } -struct refresh_params { +struct callback_data { + struct repository *repo; + unsigned int flags; - int *has_errors; + unsigned int has_errors; + unsigned nul_term_line; + unsigned read_from_stdin; }; -static int refresh(struct refresh_params *o, unsigned int flag) +static int refresh(struct callback_data *cd, unsigned int flag) { setup_work_tree(); - read_cache(); - *o->has_errors |= refresh_cache(o->flags | flag); + repo_read_index(cd->repo); + cd->has_errors |= refresh_index(cd->repo->index, cd->flags | flag, + NULL, NULL, NULL); return 0; } @@ -808,7 +816,7 @@ static int really_refresh_callback(const struct option *opt, } static int chmod_callback(const struct option *opt, - const char *arg, int unset) + const char *arg, int unset) { char *flip = opt->value; BUG_ON_OPT_NEG(unset); @@ -819,11 +827,12 @@ static int chmod_callback(const struct option *opt, } static int resolve_undo_clear_callback(const struct option *opt, - const char *arg, int unset) + const char *arg, int unset) { + struct callback_data *cd = opt->value; BUG_ON_OPT_NEG(unset); BUG_ON_OPT_ARG(arg); - resolve_undo_clear(); + resolve_undo_clear_index(cd->repo->index); return 0; } @@ -858,12 +867,13 @@ static enum parse_opt_result cacheinfo_callback( struct object_id oid; unsigned int mode; const char *path; + struct callback_data *cd = opt->value; BUG_ON_OPT_NEG(unset); BUG_ON_OPT_ARG(arg); if (!parse_new_style_cacheinfo(ctx->argv[1], &mode, &oid, &path)) { - if (add_cacheinfo(mode, &oid, path, 0)) + if (add_cacheinfo(cd->repo->index, mode, &oid, path, 0)) die("git update-index: --cacheinfo cannot add %s", path); ctx->argv++; ctx->argc--; @@ -873,7 +883,7 @@ static enum parse_opt_result cacheinfo_callback( return error("option 'cacheinfo' expects ,,"); if (strtoul_ui(*++ctx->argv, 8, &mode) || get_oid_hex(*++ctx->argv, &oid) || - add_cacheinfo(mode, &oid, *++ctx->argv, 0)) + add_cacheinfo(cd->repo->index, mode, &oid, *++ctx->argv, 0)) die("git update-index: --cacheinfo cannot add %s", *ctx->argv); ctx->argc -= 3; return 0; @@ -883,7 +893,7 @@ static enum parse_opt_result stdin_cacheinfo_callback( struct parse_opt_ctx_t *ctx, const struct option *opt, const char *arg, int unset) { - int *nul_term_line = opt->value; + struct callback_data *cd = opt->value; BUG_ON_OPT_NEG(unset); BUG_ON_OPT_ARG(arg); @@ -891,7 +901,7 @@ static enum parse_opt_result stdin_cacheinfo_callback( if (ctx->argc != 1) return error("option '%s' must be the last argument", opt->long_name); allow_add = allow_replace = allow_remove = 1; - read_index_info(*nul_term_line); + read_index_info(cd->repo->index, cd->nul_term_line); return 0; } @@ -899,14 +909,14 @@ static enum parse_opt_result stdin_callback( struct parse_opt_ctx_t *ctx, const struct option *opt, const char *arg, int unset) { - int *read_from_stdin = opt->value; + struct callback_data *cd = opt->value; BUG_ON_OPT_NEG(unset); BUG_ON_OPT_ARG(arg); if (ctx->argc != 1) return error("option '%s' must be the last argument", opt->long_name); - *read_from_stdin = 1; + cd->read_from_stdin = 1; return 0; } @@ -914,17 +924,17 @@ static enum parse_opt_result unresolve_callback( struct parse_opt_ctx_t *ctx, const struct option *opt, const char *arg, int unset) { - int *has_errors = opt->value; const char *prefix = startup_info->prefix; + struct callback_data *cd = opt->value; BUG_ON_OPT_NEG(unset); BUG_ON_OPT_ARG(arg); /* consume remaining arguments. */ - *has_errors = do_unresolve(ctx->argc, ctx->argv, - prefix, prefix ? strlen(prefix) : 0); - if (*has_errors) - active_cache_changed = 0; + cd->has_errors = do_unresolve(cd->repo, ctx->argc, ctx->argv, + prefix, prefix ? strlen(prefix) : 0); + if (cd->has_errors) + cd->repo->index->cache_changed = 0; ctx->argv += ctx->argc - 1; ctx->argc = 1; @@ -935,17 +945,17 @@ static enum parse_opt_result reupdate_callback( struct parse_opt_ctx_t *ctx, const struct option *opt, const char *arg, int unset) { - int *has_errors = opt->value; const char *prefix = startup_info->prefix; + struct callback_data *cd = opt->value; BUG_ON_OPT_NEG(unset); BUG_ON_OPT_ARG(arg); /* consume remaining arguments. */ setup_work_tree(); - *has_errors = do_reupdate(ctx->argc, ctx->argv, prefix); - if (*has_errors) - active_cache_changed = 0; + cd->has_errors = do_reupdate(cd->repo, ctx->argc, ctx->argv, prefix); + if (cd->has_errors) + cd->repo->index->cache_changed = 0; ctx->argv += ctx->argc - 1; ctx->argc = 1; @@ -954,13 +964,13 @@ static enum parse_opt_result reupdate_callback( int cmd_update_index(int argc, const char **argv, const char *prefix) { - int newfd, entries, has_errors = 0, nul_term_line = 0; + struct repository *repo = the_repository; + struct callback_data cd; + int newfd, entries; enum uc_mode untracked_cache = UC_UNSPECIFIED; - int read_from_stdin = 0; int prefix_length = prefix ? strlen(prefix) : 0; int preferred_index_format = 0; char set_executable_bit = 0; - struct refresh_params refresh_args = {0, &has_errors}; int lock_error = 0; int split_index = -1; int force_write = 0; @@ -969,12 +979,12 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) struct parse_opt_ctx_t ctx; strbuf_getline_fn getline_fn; int parseopt_state = PARSE_OPT_UNKNOWN; - struct repository *r = the_repository; + struct option options[] = { - OPT_BIT('q', NULL, &refresh_args.flags, + OPT_BIT('q', NULL, &cd.flags, N_("continue refresh even when index needs update"), REFRESH_QUIET), - OPT_BIT(0, "ignore-submodules", &refresh_args.flags, + OPT_BIT(0, "ignore-submodules", &cd.flags, N_("refresh: ignore submodules"), REFRESH_IGNORE_SUBMODULES), OPT_SET_INT(0, "add", &allow_add, @@ -983,18 +993,18 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) N_("let files replace directories and vice-versa"), 1), OPT_SET_INT(0, "remove", &allow_remove, N_("notice files missing from worktree"), 1), - OPT_BIT(0, "unmerged", &refresh_args.flags, + OPT_BIT(0, "unmerged", &cd.flags, N_("refresh even if index contains unmerged entries"), REFRESH_UNMERGED), - OPT_CALLBACK_F(0, "refresh", &refresh_args, NULL, + OPT_CALLBACK_F(0, "refresh", &cd, NULL, N_("refresh stat information"), PARSE_OPT_NOARG | PARSE_OPT_NONEG, refresh_callback), - OPT_CALLBACK_F(0, "really-refresh", &refresh_args, NULL, + OPT_CALLBACK_F(0, "really-refresh", &cd, NULL, N_("like --refresh, but ignore assume-unchanged setting"), PARSE_OPT_NOARG | PARSE_OPT_NONEG, really_refresh_callback), - {OPTION_LOWLEVEL_CALLBACK, 0, "cacheinfo", NULL, + {OPTION_LOWLEVEL_CALLBACK, 0, "cacheinfo", &cd, N_(",,"), N_("add the specified entry to the index"), PARSE_OPT_NOARG | /* disallow --cacheinfo= form */ @@ -1023,30 +1033,30 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) N_("add to index only; do not add content to object database"), 1), OPT_SET_INT(0, "force-remove", &force_remove, N_("remove named paths even if present in worktree"), 1), - OPT_BOOL('z', NULL, &nul_term_line, + OPT_BOOL('z', NULL, &cd.nul_term_line, N_("with --stdin: input lines are terminated by null bytes")), - {OPTION_LOWLEVEL_CALLBACK, 0, "stdin", &read_from_stdin, NULL, + {OPTION_LOWLEVEL_CALLBACK, 0, "stdin", &cd, NULL, N_("read list of paths to be updated from standard input"), PARSE_OPT_NONEG | PARSE_OPT_NOARG, NULL, 0, stdin_callback}, - {OPTION_LOWLEVEL_CALLBACK, 0, "index-info", &nul_term_line, NULL, + {OPTION_LOWLEVEL_CALLBACK, 0, "index-info", &cd, NULL, N_("add entries from standard input to the index"), PARSE_OPT_NONEG | PARSE_OPT_NOARG, NULL, 0, stdin_cacheinfo_callback}, - {OPTION_LOWLEVEL_CALLBACK, 0, "unresolve", &has_errors, NULL, + {OPTION_LOWLEVEL_CALLBACK, 0, "unresolve", &cd, NULL, N_("repopulate stages #2 and #3 for the listed paths"), PARSE_OPT_NONEG | PARSE_OPT_NOARG, NULL, 0, unresolve_callback}, - {OPTION_LOWLEVEL_CALLBACK, 'g', "again", &has_errors, NULL, + {OPTION_LOWLEVEL_CALLBACK, 'g', "again", &cd, NULL, N_("only update entries that differ from HEAD"), PARSE_OPT_NONEG | PARSE_OPT_NOARG, NULL, 0, reupdate_callback}, - OPT_BIT(0, "ignore-missing", &refresh_args.flags, + OPT_BIT(0, "ignore-missing", &cd.flags, N_("ignore files missing from worktree"), REFRESH_IGNORE_MISSING), OPT_SET_INT(0, "verbose", &verbose, N_("report actions to standard output"), 1), - OPT_CALLBACK_F(0, "clear-resolve-undo", NULL, NULL, + OPT_CALLBACK_F(0, "clear-resolve-undo", &cd, NULL, N_("(for porcelains) forget saved unresolved conflicts"), PARSE_OPT_NOARG | PARSE_OPT_NONEG, resolve_undo_clear_callback), @@ -1079,15 +1089,22 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) git_config(git_default_config, NULL); /* we will diagnose later if it turns out that we need to update it */ - newfd = hold_locked_index(&lock_file, 0); + newfd = repo_hold_locked_index(repo, &lock_file, 0); if (newfd < 0) lock_error = errno; - entries = read_cache(); + entries = repo_read_index(repo); if (entries < 0) die("cache corrupted"); - the_index.updated_skipworktree = 1; + repo->index->updated_skipworktree = 1; + cd.repo = repo; + cd.flags = 0; + cd.has_errors = 0; + cd.nul_term_line = 0; + cd.read_from_stdin = 0; + + ensure_full_index(repo->index); /* * Custom copy of parse_options() because we want to handle @@ -1115,9 +1132,9 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) setup_work_tree(); p = prefix_path(prefix, prefix_length, path); - update_one(p); + update_one(repo->index, p); if (set_executable_bit) - chmod_path(set_executable_bit, p); + chmod_path(repo->index, set_executable_bit, p); free(p); ctx.argc--; ctx.argv++; @@ -1133,7 +1150,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) } argc = parse_options_end(&ctx); - getline_fn = nul_term_line ? strbuf_getline_nul : strbuf_getline_lf; + getline_fn = cd.nul_term_line ? strbuf_getline_nul : strbuf_getline_lf; if (mark_skip_worktree_only && gvfs_config_is_set(GVFS_BLOCK_COMMANDS)) die(_("modifying the skip worktree bit is not supported on a GVFS repo")); @@ -1147,28 +1164,28 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) preferred_index_format, INDEX_FORMAT_LB, INDEX_FORMAT_UB); - if (the_index.version != preferred_index_format) - active_cache_changed |= SOMETHING_CHANGED; - the_index.version = preferred_index_format; + if (repo->index->version != preferred_index_format) + repo->index->cache_changed |= SOMETHING_CHANGED; + repo->index->version = preferred_index_format; } - if (read_from_stdin) { + if (cd.read_from_stdin) { struct strbuf buf = STRBUF_INIT; struct strbuf unquoted = STRBUF_INIT; setup_work_tree(); while (getline_fn(&buf, stdin) != EOF) { char *p; - if (!nul_term_line && buf.buf[0] == '"') { + if (!cd.nul_term_line && buf.buf[0] == '"') { strbuf_reset(&unquoted); if (unquote_c_style(&unquoted, buf.buf, NULL)) die("line is badly quoted"); strbuf_swap(&buf, &unquoted); } p = prefix_path(prefix, prefix_length, buf.buf); - update_one(p); + update_one(repo->index, p); if (set_executable_bit) - chmod_path(set_executable_bit, p); + chmod_path(repo->index, set_executable_bit, p); free(p); } strbuf_release(&unquoted); @@ -1183,28 +1200,28 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) warning(_("core.splitIndex is set to false; " "remove or change it, if you really want to " "enable split index")); - if (the_index.split_index) - the_index.cache_changed |= SPLIT_INDEX_ORDERED; + if (repo->index->split_index) + repo->index->cache_changed |= SPLIT_INDEX_ORDERED; else - add_split_index(&the_index); + add_split_index(repo->index); } else if (!split_index) { if (git_config_get_split_index() == 1) warning(_("core.splitIndex is set to true; " "remove or change it, if you really want to " "disable split index")); - remove_split_index(&the_index); + remove_split_index(repo->index); } - prepare_repo_settings(r); + prepare_repo_settings(repo); switch (untracked_cache) { case UC_UNSPECIFIED: break; case UC_DISABLE: - if (r->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) + if (repo->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) warning(_("core.untrackedCache is set to true; " "remove or change it, if you really want to " "disable the untracked cache")); - remove_untracked_cache(&the_index); + remove_untracked_cache(repo->index); report(_("Untracked cache disabled")); break; case UC_TEST: @@ -1212,11 +1229,11 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) return !test_if_untracked_cache_is_supported(); case UC_ENABLE: case UC_FORCE: - if (r->settings.core_untracked_cache == UNTRACKED_CACHE_REMOVE) + if (repo->settings.core_untracked_cache == UNTRACKED_CACHE_REMOVE) warning(_("core.untrackedCache is set to false; " "remove or change it, if you really want to " "enable the untracked cache")); - add_untracked_cache(&the_index); + add_untracked_cache(repo->index); report(_("Untracked cache enabled for '%s'"), get_git_work_tree()); break; default: @@ -1228,28 +1245,28 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) warning(_("core.fsmonitor is unset; " "set it if you really want to " "enable fsmonitor")); - add_fsmonitor(&the_index); + add_fsmonitor(repo->index); report(_("fsmonitor enabled")); } else if (!fsmonitor) { if (git_config_get_fsmonitor() == 1) warning(_("core.fsmonitor is set; " "remove it if you really want to " "disable fsmonitor")); - remove_fsmonitor(&the_index); + remove_fsmonitor(repo->index); report(_("fsmonitor disabled")); } - if (active_cache_changed || force_write) { + if (repo->index->cache_changed || force_write) { if (newfd < 0) { - if (refresh_args.flags & REFRESH_QUIET) + if (cd.flags & REFRESH_QUIET) exit(128); unable_to_lock_die(get_index_file(), lock_error); } - if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK)) + if (write_locked_index(repo->index, &lock_file, COMMIT_LOCK)) die("Unable to write new index file"); } rollback_lock_file(&lock_file); - return has_errors ? 1 : 0; + return cd.has_errors ? 1 : 0; } diff --git a/cache-tree.c b/cache-tree.c index f38a87d9faffbc..611f7de6078254 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -7,6 +7,7 @@ #include "object-store.h" #include "replace-object.h" #include "promisor-remote.h" +#include "sparse-index.h" #ifndef DEBUG_CACHE_TREE #define DEBUG_CACHE_TREE 0 @@ -46,7 +47,7 @@ static int subtree_name_cmp(const char *one, int onelen, return memcmp(one, two, onelen); } -static int subtree_pos(struct cache_tree *it, const char *path, int pathlen) +int cache_tree_subtree_pos(struct cache_tree *it, const char *path, int pathlen) { struct cache_tree_sub **down = it->down; int lo, hi; @@ -73,7 +74,7 @@ static struct cache_tree_sub *find_subtree(struct cache_tree *it, int create) { struct cache_tree_sub *down; - int pos = subtree_pos(it, path, pathlen); + int pos = cache_tree_subtree_pos(it, path, pathlen); if (0 <= pos) return it->down[pos]; if (!create) @@ -124,7 +125,7 @@ static int do_invalidate_path(struct cache_tree *it, const char *path) it->entry_count = -1; if (!*slash) { int pos; - pos = subtree_pos(it, path, namelen); + pos = cache_tree_subtree_pos(it, path, namelen); if (0 <= pos) { cache_tree_free(&it->down[pos]->cache_tree); free(it->down[pos]); @@ -152,16 +153,15 @@ void cache_tree_invalidate_path(struct index_state *istate, const char *path) istate->cache_changed |= CACHE_TREE_CHANGED; } -static int verify_cache(struct cache_entry **cache, - int entries, int flags) +static int verify_cache(struct index_state *istate, int flags) { - int i, funny; + unsigned i, funny; int silent = flags & WRITE_TREE_SILENT; /* Verify that the tree is merged */ funny = 0; - for (i = 0; i < entries; i++) { - const struct cache_entry *ce = cache[i]; + for (i = 0; i < istate->cache_nr; i++) { + const struct cache_entry *ce = istate->cache[i]; if (ce_stage(ce)) { if (silent) return -1; @@ -181,17 +181,19 @@ static int verify_cache(struct cache_entry **cache, * stage 0 entries. */ funny = 0; - for (i = 0; i < entries - 1; i++) { + for (i = 0; i + 1 < istate->cache_nr; i++) { /* path/file always comes after path because of the way * the cache is sorted. Also path can appear only once, * which means conflicting one would immediately follow. */ - const char *this_name = cache[i]->name; - const char *next_name = cache[i+1]->name; - int this_len = strlen(this_name); - if (this_len < strlen(next_name) && - strncmp(this_name, next_name, this_len) == 0 && - next_name[this_len] == '/') { + const struct cache_entry *this_ce = istate->cache[i]; + const struct cache_entry *next_ce = istate->cache[i + 1]; + const char *this_name = this_ce->name; + const char *next_name = next_ce->name; + int this_len = ce_namelen(this_ce); + if (this_len < ce_namelen(next_ce) && + next_name[this_len] == '/' && + strncmp(this_name, next_name, this_len) == 0) { if (10 < ++funny) { fprintf(stderr, "...\n"); break; @@ -267,6 +269,24 @@ static int update_one(struct cache_tree *it, *skip_count = 0; + /* + * If the first entry of this region is a sparse directory + * entry corresponding exactly to 'base', then this cache_tree + * struct is a "leaf" in the data structure, pointing to the + * tree OID specified in the entry. + */ + if (entries > 0) { + const struct cache_entry *ce = cache[0]; + + if (ce->ce_mode == CE_MODE_SPARSE_DIRECTORY && + ce->ce_namelen == baselen && + !strncmp(ce->name, base, baselen)) { + it->entry_count = 1; + oidcpy(&it->oid, &ce->oid); + return 1; + } + } + if (0 <= it->entry_count && has_object_file(&it->oid)) return it->entry_count; @@ -469,15 +489,23 @@ static int update_one(struct cache_tree *it, int cache_tree_update(struct index_state *istate, int flags) { - struct cache_tree *it = istate->cache_tree; - struct cache_entry **cache = istate->cache; - int entries = istate->cache_nr; - int skip, i = verify_cache(cache, entries, flags); + int skip, i; + + i = verify_cache(istate, flags); if (i) return i; + + ensure_full_index(istate); + + if (!istate->cache_tree) + istate->cache_tree = cache_tree(); + trace_performance_enter(); - i = update_one(it, cache, entries, "", 0, &skip, flags); + trace2_region_enter("cache_tree", "update", the_repository); + i = update_one(istate->cache_tree, istate->cache, istate->cache_nr, + "", 0, &skip, flags); + trace2_region_leave("cache_tree", "update", the_repository); trace_performance_leave("cache_tree_update"); if (i < 0) return i; @@ -527,7 +555,9 @@ static void write_one(struct strbuf *buffer, struct cache_tree *it, void cache_tree_write(struct strbuf *sb, struct cache_tree *root) { + trace2_region_enter("cache_tree", "write", the_repository); write_one(sb, root, "", 0); + trace2_region_leave("cache_tree", "write", the_repository); } static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) @@ -616,9 +646,16 @@ static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) struct cache_tree *cache_tree_read(const char *buffer, unsigned long size) { + struct cache_tree *result; + if (buffer[0]) return NULL; /* not the whole tree */ - return read_one(&buffer, &size); + + trace2_region_enter("cache_tree", "read", the_repository); + result = read_one(&buffer, &size); + trace2_region_leave("cache_tree", "read", the_repository); + + return result; } static struct cache_tree *cache_tree_find(struct cache_tree *it, const char *path) @@ -657,9 +694,6 @@ static int write_index_as_tree_internal(struct object_id *oid, cache_tree_valid = 0; } - if (!index_state->cache_tree) - index_state->cache_tree = cache_tree(); - if (!cache_tree_valid && cache_tree_update(index_state, flags) < 0) return WRITE_TREE_UNMERGED_INDEX; @@ -772,6 +806,7 @@ void prime_cache_tree(struct repository *r, cache_tree_free(&istate->cache_tree); istate->cache_tree = cache_tree(); + prime_cache_tree_rec(r, istate->cache_tree, tree); istate->cache_changed |= CACHE_TREE_CHANGED; diff --git a/cache-tree.h b/cache-tree.h index 639bfa5340e783..8efeccebfc9f0b 100644 --- a/cache-tree.h +++ b/cache-tree.h @@ -27,6 +27,8 @@ void cache_tree_free(struct cache_tree **); void cache_tree_invalidate_path(struct index_state *, const char *); struct cache_tree_sub *cache_tree_sub(struct cache_tree *, const char *); +int cache_tree_subtree_pos(struct cache_tree *it, const char *path, int pathlen); + void cache_tree_write(struct strbuf *, struct cache_tree *root); struct cache_tree *cache_tree_read(const char *buffer, unsigned long size); diff --git a/cache.h b/cache.h index bf31872add5318..871230a9a8a52b 100644 --- a/cache.h +++ b/cache.h @@ -204,6 +204,10 @@ struct cache_entry { #error "CE_EXTENDED_FLAGS out of range" #endif +#define CE_MODE_SPARSE_DIRECTORY 01000755 +#define SPARSE_DIR_MODE 0100 +#define S_ISSPARSEDIR(m) ((m)->ce_mode == CE_MODE_SPARSE_DIRECTORY) + /* Forward structure decls */ struct pathspec; struct child_process; @@ -249,6 +253,8 @@ static inline unsigned int create_ce_mode(unsigned int mode) { if (S_ISLNK(mode)) return S_IFLNK; + if (mode == SPARSE_DIR_MODE) + return CE_MODE_SPARSE_DIRECTORY; if (S_ISDIR(mode) || S_ISGITLINK(mode)) return S_IFGITLINK; return S_IFREG | ce_permissions(mode); @@ -305,6 +311,7 @@ static inline unsigned int canon_mode(unsigned int mode) struct split_index; struct untracked_cache; struct progress; +struct pattern_list; struct index_state { struct cache_entry **cache; @@ -319,7 +326,8 @@ struct index_state { drop_cache_tree : 1, updated_workdir : 1, updated_skipworktree : 1, - fsmonitor_has_run_once : 1; + fsmonitor_has_run_once : 1, + sparse_index : 1; struct hashmap name_hash; struct hashmap dir_hash; struct object_id oid; @@ -328,6 +336,8 @@ struct index_state { struct ewah_bitmap *fsmonitor_dirty; struct mem_pool *ce_mem_pool; struct progress *progress; + struct repository *repo; + struct pattern_list *sparse_checkout_patterns; }; /* Name hashing */ @@ -336,6 +346,7 @@ void add_name_hash(struct index_state *istate, struct cache_entry *ce); void remove_name_hash(struct index_state *istate, struct cache_entry *ce); void free_name_hash(struct index_state *istate); +void ensure_full_index(struct index_state *istate); /* Cache entry creation and cleanup */ @@ -409,7 +420,6 @@ extern struct index_state the_index; #define unmerged_cache() unmerged_index(&the_index) #define cache_name_pos(name, namelen) index_name_pos(&the_index,(name),(namelen)) #define add_cache_entry(ce, option) add_index_entry(&the_index, (ce), (option)) -#define rename_cache_entry_at(pos, new_name) rename_index_entry_at(&the_index, (pos), (new_name)) #define remove_cache_entry_at(pos) remove_index_entry_at(&the_index, (pos)) #define remove_file_from_cache(path) remove_file_from_index(&the_index, (path)) #define add_to_cache(path, st, flags) add_to_index(&the_index, (path), (st), (flags)) @@ -417,13 +427,10 @@ extern struct index_state the_index; #define chmod_cache_entry(ce, flip) chmod_index_entry(&the_index, (ce), (flip)) #define refresh_cache(flags) refresh_index(&the_index, (flags), NULL, NULL, NULL) #define refresh_and_write_cache(refresh_flags, write_flags, gentle) repo_refresh_and_write_index(the_repository, (refresh_flags), (write_flags), (gentle), NULL, NULL, NULL) -#define ce_match_stat(ce, st, options) ie_match_stat(&the_index, (ce), (st), (options)) #define ce_modified(ce, st, options) ie_modified(&the_index, (ce), (st), (options)) #define cache_dir_exists(name, namelen) index_dir_exists(&the_index, (name), (namelen)) -#define cache_file_exists(name, namelen, igncase) index_file_exists(&the_index, (name), (namelen), (igncase)) #define cache_name_is_other(name, namelen) index_name_is_other(&the_index, (name), (namelen)) #define resolve_undo_clear() resolve_undo_clear_index(&the_index) -#define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at) #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec) #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz)) #define hold_locked_index(lock_file, flags) repo_hold_locked_index(the_repository, (lock_file), (flags)) @@ -720,6 +727,8 @@ int read_index_from(struct index_state *, const char *path, const char *gitdir); int is_index_unborn(struct index_state *); +void ensure_full_index(struct index_state *istate); + /* For use with `write_locked_index()`. */ #define COMMIT_LOCK (1 << 0) #define SKIP_IF_UNCHANGED (1 << 1) @@ -830,6 +839,11 @@ int remove_file_from_index(struct index_state *, const char *path); int add_to_index(struct index_state *, const char *path, struct stat *, int flags); int add_file_to_index(struct index_state *, const char *path, int flags); +int add_to_index_cacheinfo(struct index_state *, unsigned int mode, + const struct object_id *oid, const char *path, + int stage, int allow_add, int allow_replace, + struct cache_entry **pce); + int chmod_index_entry(struct index_state *, struct cache_entry *ce, char flip); int ce_same_name(const struct cache_entry *a, const struct cache_entry *b); void set_object_name_for_intent_to_add_entry(struct cache_entry *ce); @@ -1049,6 +1063,7 @@ struct repository_format { int worktree_config; int is_bare; int hash_algo; + int sparse_index; char *work_tree; struct string_list unknown_extensions; struct string_list v1_only_extensions; @@ -1847,7 +1862,7 @@ int checkout_fast_forward(struct repository *r, const struct object_id *from, const struct object_id *to, int overwrite_ignore); - +char *merge_get_better_branch_name(const char *branch); int sane_execvp(const char *file, char *const argv[]); diff --git a/diff.c b/diff.c index 2253ec880298b4..02fafee85879d7 100644 --- a/diff.c +++ b/diff.c @@ -3901,6 +3901,8 @@ static int reuse_worktree_file(struct index_state *istate, if (!want_file && would_convert_to_git(istate, name)) return 0; + ensure_full_index(istate); + len = strlen(name); pos = index_name_pos(istate, name, len); if (pos < 0) diff --git a/dir.c b/dir.c index ea4ce4f869164b..f8a1132d43fbd0 100644 --- a/dir.c +++ b/dir.c @@ -19,6 +19,7 @@ #include "ewah/ewok.h" #include "fsmonitor.h" #include "submodule-config.h" +#include "sparse-index.h" /* * Tells read_directory_recursive how a file or directory should be treated. @@ -893,7 +894,7 @@ void add_pattern(const char *string, const char *base, add_pattern_to_hashsets(pl, pattern); } -static int read_skip_worktree_file_from_index(const struct index_state *istate, +static int read_skip_worktree_file_from_index(struct index_state *istate, const char *path, size_t *size_out, char **data_out, struct oid_stat *oid_stat) @@ -901,6 +902,8 @@ static int read_skip_worktree_file_from_index(const struct index_state *istate, int pos, len; len = strlen(path); + + expand_to_path(istate, path, len, 0); pos = index_name_pos(istate, path, len); if (pos < 0) return -1; @@ -1132,6 +1135,10 @@ static int add_patterns(const char *fname, const char *base, int baselen, close(fd); if (oid_stat) { int pos; + + if (istate) + expand_to_path(istate, fname, strlen(fname), 0); + if (oid_stat->valid && !match_stat_data_racy(istate, &oid_stat->stat, &st)) ; /* no content change, oid_stat->oid still good */ @@ -1433,6 +1440,11 @@ enum pattern_match_result path_matches_pattern_list( strbuf_addch(&parent_pathname, '/'); strbuf_add(&parent_pathname, pathname, pathlen); + /* Directory requests should be added as if they are a file */ + if (parent_pathname.len > 1 && + parent_pathname.buf[parent_pathname.len - 1] == '/') + strbuf_add(&parent_pathname, "-", 1); + if (hashmap_contains_path(&pl->recursive_hashmap, &parent_pathname)) { result = MATCHED_RECURSIVE; @@ -1763,6 +1775,7 @@ static enum exist_status directory_exists_in_index(struct index_state *istate, if (ignore_case) return directory_exists_in_index_icase(istate, dirname, len); + expand_to_path(istate, dirname, len, 0); pos = index_name_pos(istate, dirname, len); if (pos < 0) pos = -pos-1; @@ -2117,6 +2130,8 @@ static int get_index_dtype(struct index_state *istate, int pos; const struct cache_entry *ce; + ensure_full_index(istate); + ce = index_file_exists(istate, path, len, 0); if (ce) { if (!ce_uptodate(ce)) @@ -3069,6 +3084,23 @@ void setup_standard_excludes(struct dir_struct *dir) } } +char *get_sparse_checkout_filename(void) +{ + return git_pathdup("info/sparse-checkout"); +} + +int get_sparse_checkout_patterns(struct pattern_list *pl) +{ + int res; + char *sparse_filename = get_sparse_checkout_filename(); + + pl->use_cone_patterns = core_sparse_checkout_cone; + res = add_patterns_from_file_to_list(sparse_filename, "", 0, pl, NULL); + + free(sparse_filename); + return res; +} + int remove_path(const char *name) { char *slash; @@ -3590,6 +3622,8 @@ static void connect_wt_gitdir_in_nested(const char *sub_worktree, if (repo_read_index(&subrepo) < 0) die(_("index file corrupt in repo %s"), subrepo.gitdir); + ensure_full_index(subrepo.index); + for (i = 0; i < subrepo.index->cache_nr; i++) { const struct cache_entry *ce = subrepo.index->cache[i]; diff --git a/dir.h b/dir.h index a3c40dec516542..850d0c4da20b8a 100644 --- a/dir.h +++ b/dir.h @@ -448,6 +448,8 @@ int is_empty_dir(const char *dir); void setup_standard_excludes(struct dir_struct *dir); +char *get_sparse_checkout_filename(void); +int get_sparse_checkout_patterns(struct pattern_list *pl); /* Constants for remove_dir_recursively: */ @@ -501,7 +503,7 @@ static inline int ce_path_match(const struct index_state *istate, char *seen) { return match_pathspec(istate, pathspec, ce->name, ce_namelen(ce), 0, seen, - S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)); + (ce->ce_mode == CE_MODE_SPARSE_DIRECTORY) || S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)); } static inline int dir_path_match(const struct index_state *istate, diff --git a/entry.c b/entry.c index f04cb5cff6ac81..9cc768524b1f39 100644 --- a/entry.c +++ b/entry.c @@ -415,6 +415,8 @@ static void mark_colliding_entries(const struct checkout *state, ce->ce_flags |= CE_MATCHED; + ensure_full_index(state->istate); + for (i = 0; i < state->istate->cache_nr; i++) { struct cache_entry *dup = state->istate->cache[i]; diff --git a/fsmonitor.c b/fsmonitor.c index f6f8128b09d58d..36b0b4bca85901 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -13,14 +13,19 @@ struct trace_key trace_fsmonitor = TRACE_KEY_INIT(FSMONITOR); +static void assert_index_minimum(struct index_state *istate, size_t pos) +{ + if (pos > istate->cache_nr) + BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", + (uintmax_t)pos, istate->cache_nr); +} + static void fsmonitor_ewah_callback(size_t pos, void *is) { struct index_state *istate = (struct index_state *)is; struct cache_entry *ce; - if (pos >= istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" >= %u)", - (uintmax_t)pos, istate->cache_nr); + assert_index_minimum(istate, pos + 1); ce = istate->cache[pos]; ce->ce_flags &= ~CE_FSMONITOR_VALID; @@ -53,6 +58,9 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, uint64_t timestamp; struct strbuf last_update = STRBUF_INIT; + if (istate->sparse_index) + return 0; + if (sz < sizeof(uint32_t) + 1 + sizeof(uint32_t)) return error("corrupt fsmonitor extension (too short)"); @@ -82,10 +90,8 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, } istate->fsmonitor_dirty = fsmonitor_dirty; - if (!istate->split_index && - istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + if (!istate->split_index) + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); trace_printf_key(&trace_fsmonitor, "read fsmonitor extension successful"); return 0; @@ -94,6 +100,10 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, void fill_fsmonitor_bitmap(struct index_state *istate) { unsigned int i, skipped = 0; + + if (istate->sparse_index) + return; + istate->fsmonitor_dirty = ewah_new(); for (i = 0; i < istate->cache_nr; i++) { if (istate->cache[i]->ce_flags & CE_REMOVE) @@ -110,10 +120,8 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate) uint32_t ewah_size = 0; int fixup = 0; - if (!istate->split_index && - istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + if (!istate->split_index) + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); put_be32(&hdr_version, INDEX_EXTENSION_VERSION2); strbuf_add(sb, &hdr_version, sizeof(uint32_t)); @@ -182,7 +190,8 @@ void refresh_fsmonitor(struct index_state *istate) char *buf; unsigned int i; - if (!core_fsmonitor || istate->fsmonitor_has_run_once) + if (!core_fsmonitor || istate->fsmonitor_has_run_once || + istate->sparse_index) return; hook_version = fsmonitor_hook_version(); @@ -295,6 +304,9 @@ void add_fsmonitor(struct index_state *istate) unsigned int i; struct strbuf last_update = STRBUF_INIT; + if (istate->sparse_index) + return; + if (!istate->fsmonitor_last_update) { trace_printf_key(&trace_fsmonitor, "add fsmonitor"); istate->cache_changed |= FSMONITOR_CHANGED; @@ -330,17 +342,20 @@ void tweak_fsmonitor(struct index_state *istate) unsigned int i; int fsmonitor_enabled = git_config_get_fsmonitor(); + if (istate->sparse_index) + fsmonitor_enabled = 0; + if (istate->fsmonitor_dirty) { if (fsmonitor_enabled) { + ensure_full_index(istate); + /* Mark all entries valid */ for (i = 0; i < istate->cache_nr; i++) { istate->cache[i]->ce_flags |= CE_FSMONITOR_VALID; } /* Mark all previously saved entries as dirty */ - if (istate->fsmonitor_dirty->bit_size > istate->cache_nr) - BUG("fsmonitor_dirty has more entries than the index (%"PRIuMAX" > %u)", - (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr); + assert_index_minimum(istate, istate->fsmonitor_dirty->bit_size); ewah_each_bit(istate->fsmonitor_dirty, fsmonitor_ewah_callback, istate); refresh_fsmonitor(istate); diff --git a/git-merge-octopus.sh b/git-merge-octopus.sh deleted file mode 100755 index 7d19d379512b52..00000000000000 --- a/git-merge-octopus.sh +++ /dev/null @@ -1,112 +0,0 @@ -#!/bin/sh -# -# Copyright (c) 2005 Junio C Hamano -# -# Resolve two or more trees. -# - -. git-sh-setup - -LF=' -' - -# The first parameters up to -- are merge bases; the rest are heads. -bases= head= remotes= sep_seen= -for arg -do - case ",$sep_seen,$head,$arg," in - *,--,) - sep_seen=yes - ;; - ,yes,,*) - head=$arg - ;; - ,yes,*) - remotes="$remotes$arg " - ;; - *) - bases="$bases$arg " - ;; - esac -done - -# Reject if this is not an octopus -- resolve should be used instead. -case "$remotes" in -?*' '?*) - ;; -*) - exit 2 ;; -esac - -# MRC is the current "merge reference commit" -# MRT is the current "merge result tree" - -if ! git diff-index --quiet --cached HEAD -- -then - gettextln "Error: Your local changes to the following files would be overwritten by merge" - git diff-index --cached --name-only HEAD -- | sed -e 's/^/ /' - exit 2 -fi -MRC=$(git rev-parse --verify -q $head) -MRT=$(git write-tree) -NON_FF_MERGE=0 -OCTOPUS_FAILURE=0 -for SHA1 in $remotes -do - case "$OCTOPUS_FAILURE" in - 1) - # We allow only last one to have a hand-resolvable - # conflicts. Last round failed and we still had - # a head to merge. - gettextln "Automated merge did not work." - gettextln "Should not be doing an octopus." - exit 2 - esac - - eval pretty_name=\${GITHEAD_$SHA1:-$SHA1} - if test "$SHA1" = "$pretty_name" - then - SHA1_UP="$(echo "$SHA1" | tr a-z A-Z)" - eval pretty_name=\${GITHEAD_$SHA1_UP:-$pretty_name} - fi - common=$(git merge-base --all $SHA1 $MRC) || - die "$(eval_gettext "Unable to find common commit with \$pretty_name")" - - case "$LF$common$LF" in - *"$LF$SHA1$LF"*) - eval_gettextln "Already up to date with \$pretty_name" - continue - ;; - esac - - if test "$common,$NON_FF_MERGE" = "$MRC,0" - then - # The first head being merged was a fast-forward. - # Advance MRC to the head being merged, and use that - # tree as the intermediate result of the merge. - # We still need to count this as part of the parent set. - - eval_gettextln "Fast-forwarding to: \$pretty_name" - git read-tree -u -m $head $SHA1 || exit - MRC=$SHA1 MRT=$(git write-tree) - continue - fi - - NON_FF_MERGE=1 - - eval_gettextln "Trying simple merge with \$pretty_name" - git read-tree -u -m --aggressive $common $MRT $SHA1 || exit 2 - next=$(git write-tree 2>/dev/null) - if test $? -ne 0 - then - gettextln "Simple merge did not work, trying automatic merge." - git merge-index -o git-merge-one-file -a || - OCTOPUS_FAILURE=1 - next=$(git write-tree 2>/dev/null) - fi - - MRC="$MRC $SHA1" - MRT=$next -done - -exit "$OCTOPUS_FAILURE" diff --git a/git-merge-one-file.sh b/git-merge-one-file.sh deleted file mode 100755 index f6d9852d2f6183..00000000000000 --- a/git-merge-one-file.sh +++ /dev/null @@ -1,167 +0,0 @@ -#!/bin/sh -# -# Copyright (c) Linus Torvalds, 2005 -# -# This is the git per-file merge script, called with -# -# $1 - original file SHA1 (or empty) -# $2 - file in branch1 SHA1 (or empty) -# $3 - file in branch2 SHA1 (or empty) -# $4 - pathname in repository -# $5 - original file mode (or empty) -# $6 - file in branch1 mode (or empty) -# $7 - file in branch2 mode (or empty) -# -# Handle some trivial cases.. The _really_ trivial cases have -# been handled already by git read-tree, but that one doesn't -# do any merges that might change the tree layout. - -USAGE=' ' -USAGE="$USAGE " -LONG_USAGE="usage: git merge-one-file $USAGE - -Blob ids and modes should be empty for missing files." - -SUBDIRECTORY_OK=Yes -. git-sh-setup -cd_to_toplevel -require_work_tree - -if test $# != 7 -then - echo "$LONG_USAGE" - exit 1 -fi - -case "${1:-.}${2:-.}${3:-.}" in -# -# Deleted in both or deleted in one and unchanged in the other -# -"$1.." | "$1.$1" | "$1$1.") - if { test -z "$6" && test "$5" != "$7"; } || - { test -z "$7" && test "$5" != "$6"; } - then - echo "ERROR: File $4 deleted on one branch but had its" >&2 - echo "ERROR: permissions changed on the other." >&2 - exit 1 - fi - - if test -n "$2" - then - echo "Removing $4" - else - # read-tree checked that index matches HEAD already, - # so we know we do not have this path tracked. - # there may be an unrelated working tree file here, - # which we should just leave unmolested. Make sure - # we do not have it in the index, though. - exec git update-index --remove -- "$4" - fi - if test -f "$4" - then - rm -f -- "$4" && - rmdir -p "$(expr "z$4" : 'z\(.*\)/')" 2>/dev/null || : - fi && - exec git update-index --remove -- "$4" - ;; - -# -# Added in one. -# -".$2.") - # the other side did not add and we added so there is nothing - # to be done, except making the path merged. - exec git update-index --add --cacheinfo "$6" "$2" "$4" - ;; -"..$3") - echo "Adding $4" - if test -f "$4" - then - echo "ERROR: untracked $4 is overwritten by the merge." >&2 - exit 1 - fi - git update-index --add --cacheinfo "$7" "$3" "$4" && - exec git checkout-index -u -f -- "$4" - ;; - -# -# Added in both, identically (check for same permissions). -# -".$3$2") - if test "$6" != "$7" - then - echo "ERROR: File $4 added identically in both branches," >&2 - echo "ERROR: but permissions conflict $6->$7." >&2 - exit 1 - fi - echo "Adding $4" - git update-index --add --cacheinfo "$6" "$2" "$4" && - exec git checkout-index -u -f -- "$4" - ;; - -# -# Modified in both, but differently. -# -"$1$2$3" | ".$2$3") - - case ",$6,$7," in - *,120000,*) - echo "ERROR: $4: Not merging symbolic link changes." >&2 - exit 1 - ;; - *,160000,*) - echo "ERROR: $4: Not merging conflicting submodule changes." >&2 - exit 1 - ;; - esac - - src1=$(git unpack-file $2) - src2=$(git unpack-file $3) - case "$1" in - '') - echo "Added $4 in both, but differently." - orig=$(git unpack-file $(git hash-object /dev/null)) - ;; - *) - echo "Auto-merging $4" - orig=$(git unpack-file $1) - ;; - esac - - git merge-file "$src1" "$orig" "$src2" - ret=$? - msg= - if test $ret != 0 || test -z "$1" - then - msg='content conflict' - ret=1 - fi - - # Create the working tree file, using "our tree" version from the - # index, and then store the result of the merge. - git checkout-index -f --stage=2 -- "$4" && cat "$src1" >"$4" || exit 1 - rm -f -- "$orig" "$src1" "$src2" - - if test "$6" != "$7" - then - if test -n "$msg" - then - msg="$msg, " - fi - msg="${msg}permissions conflict: $5->$6,$7" - ret=1 - fi - - if test $ret != 0 - then - echo "ERROR: $msg in $4" >&2 - exit 1 - fi - exec git update-index -- "$4" - ;; - -*) - echo "ERROR: $4: Not handling case $1 -> $2 -> $3" >&2 - ;; -esac -exit 1 diff --git a/git-merge-resolve.sh b/git-merge-resolve.sh deleted file mode 100755 index 343fe7bccd0d64..00000000000000 --- a/git-merge-resolve.sh +++ /dev/null @@ -1,54 +0,0 @@ -#!/bin/sh -# -# Copyright (c) 2005 Linus Torvalds -# Copyright (c) 2005 Junio C Hamano -# -# Resolve two trees, using enhanced multi-base read-tree. - -# The first parameters up to -- are merge bases; the rest are heads. -bases= head= remotes= sep_seen= -for arg -do - case ",$sep_seen,$head,$arg," in - *,--,) - sep_seen=yes - ;; - ,yes,,*) - head=$arg - ;; - ,yes,*) - remotes="$remotes$arg " - ;; - *) - bases="$bases$arg " - ;; - esac -done - -# Give up if we are given two or more remotes -- not handling octopus. -case "$remotes" in -?*' '?*) - exit 2 ;; -esac - -# Give up if this is a baseless merge. -if test '' = "$bases" -then - exit 2 -fi - -git update-index -q --refresh -git read-tree -u -m --aggressive $bases $head $remotes || exit 2 -echo "Trying simple merge." -if result_tree=$(git write-tree 2>/dev/null) -then - exit 0 -else - echo "Simple merge failed, trying Automatic merge." - if git merge-index -o git-merge-one-file -a - then - exit 0 - else - exit 1 - fi -fi diff --git a/git.c b/git.c index 241c451f9123d2..adcb891a236b6a 100644 --- a/git.c +++ b/git.c @@ -609,10 +609,13 @@ static struct cmd_struct commands[] = { { "merge-base", cmd_merge_base, RUN_SETUP }, { "merge-file", cmd_merge_file, RUN_SETUP_GENTLY }, { "merge-index", cmd_merge_index, RUN_SETUP | NO_PARSEOPT }, + { "merge-octopus", cmd_merge_octopus, RUN_SETUP | NO_PARSEOPT }, { "merge-ours", cmd_merge_ours, RUN_SETUP | NO_PARSEOPT }, + { "merge-one-file", cmd_merge_one_file, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, { "merge-recursive", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, { "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, { "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, + { "merge-resolve", cmd_merge_resolve, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, { "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT }, { "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT }, { "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT }, diff --git a/merge-recursive.c b/merge-recursive.c index 93c56e3e777217..816d056cff5468 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -430,6 +430,7 @@ static int unpack_trees_start(struct merge_options *opt, init_tree_desc_from_tree(t+1, head); init_tree_desc_from_tree(t+2, merge); + ensure_full_index(opt->priv->unpack_opts.src_index); rc = unpack_trees(3, t, &opt->priv->unpack_opts); cache_tree_free(&opt->repo->index->cache_tree); @@ -523,6 +524,8 @@ static struct string_list *get_unmerged(struct index_state *istate) unmerged->strdup_strings = 1; + ensure_full_index(istate); + for (i = 0; i < istate->cache_nr; i++) { struct string_list_item *item; struct stage_data *e; @@ -763,6 +766,8 @@ static int dir_in_way(struct index_state *istate, const char *path, strbuf_addstr(&dirpath, path); strbuf_addch(&dirpath, '/'); + ensure_full_index(istate); + pos = index_name_pos(istate, dirpath.buf, dirpath.len); if (pos < 0) @@ -786,9 +791,13 @@ static int dir_in_way(struct index_state *istate, const char *path, static int was_tracked_and_matches(struct merge_options *opt, const char *path, const struct diff_filespec *blob) { - int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path)); + int pos; struct cache_entry *ce; + ensure_full_index(&opt->priv->orig_index); + + pos = index_name_pos(&opt->priv->orig_index, path, strlen(path)); + if (0 > pos) /* we were not tracking this path before the merge */ return 0; @@ -803,7 +812,11 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path, */ static int was_tracked(struct merge_options *opt, const char *path) { - int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path)); + int pos; + + ensure_full_index(&opt->priv->orig_index); + + pos = index_name_pos(&opt->priv->orig_index, path, strlen(path)); if (0 <= pos) /* we were tracking this path before the merge */ @@ -815,6 +828,9 @@ static int was_tracked(struct merge_options *opt, const char *path) static int would_lose_untracked(struct merge_options *opt, const char *path) { struct index_state *istate = opt->repo->index; + int pos; + + ensure_full_index(istate); /* * This may look like it can be simplified to: @@ -833,7 +849,7 @@ static int would_lose_untracked(struct merge_options *opt, const char *path) * update_file()/would_lose_untracked(); see every comment in this * file which mentions "update_stages". */ - int pos = index_name_pos(istate, path, strlen(path)); + pos = index_name_pos(istate, path, strlen(path)); if (pos < 0) pos = -1 - pos; @@ -3086,6 +3102,7 @@ static int handle_content_merge(struct merge_file_info *mfi, * flag to avoid making the file appear as if it were * deleted by the user. */ + ensure_full_index(&opt->priv->orig_index); pos = index_name_pos(&opt->priv->orig_index, path, strlen(path)); ce = opt->priv->orig_index.cache[pos]; if (ce_skip_worktree(ce)) { diff --git a/merge-strategies.c b/merge-strategies.c new file mode 100644 index 00000000000000..4d9dd552962f28 --- /dev/null +++ b/merge-strategies.c @@ -0,0 +1,571 @@ +#include "cache.h" +#include "cache-tree.h" +#include "commit-reach.h" +#include "dir.h" +#include "lockfile.h" +#include "merge-strategies.h" +#include "run-command.h" +#include "unpack-trees.h" +#include "xdiff-interface.h" + +static int checkout_from_index(struct index_state *istate, const char *path, + struct cache_entry *ce) +{ + struct checkout state = CHECKOUT_INIT; + + state.istate = istate; + state.force = 1; + state.base_dir = ""; + state.base_dir_len = 0; + + if (checkout_entry(ce, &state, NULL, NULL) < 0) + return error(_("%s: cannot checkout file"), path); + return 0; +} + +static int merge_one_file_deleted(struct index_state *istate, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode) +{ + if ((our_blob && orig_mode != our_mode) || + (their_blob && orig_mode != their_mode)) + return error(_("File %s deleted on one branch but had its " + "permissions changed on the other."), path); + + if (our_blob) { + printf(_("Removing %s\n"), path); + + if (file_exists(path)) + remove_path(path); + } + + if (remove_file_from_index(istate, path)) + return error("%s: cannot remove from the index", path); + return 0; +} + +static int do_merge_one_file(struct index_state *istate, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode) +{ + int ret, i, dest; + ssize_t written; + mmbuffer_t result = {NULL, 0}; + mmfile_t mmfs[3]; + xmparam_t xmp = {{0}}; + + if (our_mode == S_IFLNK || their_mode == S_IFLNK) + return error(_("%s: Not merging symbolic link changes."), path); + else if (our_mode == S_IFGITLINK || their_mode == S_IFGITLINK) + return error(_("%s: Not merging conflicting submodule changes."), path); + + if (orig_blob) { + printf(_("Auto-merging %s\n"), path); + read_mmblob(mmfs + 0, orig_blob); + } else { + printf(_("Added %s in both, but differently.\n"), path); + read_mmblob(mmfs + 0, &null_oid); + } + + read_mmblob(mmfs + 1, our_blob); + read_mmblob(mmfs + 2, their_blob); + + xmp.level = XDL_MERGE_ZEALOUS_ALNUM; + xmp.style = 0; + xmp.favor = 0; + + ret = xdl_merge(mmfs + 0, mmfs + 1, mmfs + 2, &xmp, &result); + + for (i = 0; i < 3; i++) + free(mmfs[i].ptr); + + if (ret < 0) { + free(result.ptr); + return error(_("Failed to execute internal merge")); + } + + if (ret > 0 || !orig_blob) + ret = error(_("content conflict in %s"), path); + if (our_mode != their_mode) + ret = error(_("permission conflict: %o->%o,%o in %s"), + orig_mode, our_mode, their_mode, path); + + unlink(path); + if ((dest = open(path, O_WRONLY | O_CREAT, our_mode)) < 0) { + free(result.ptr); + return error_errno(_("failed to open file '%s'"), path); + } + + written = write_in_full(dest, result.ptr, result.size); + close(dest); + + free(result.ptr); + + if (written < 0) + return error_errno(_("failed to write to '%s'"), path); + if (ret) + return ret; + + return add_file_to_index(istate, path, 0); +} + +int merge_three_way(struct repository *r, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode) +{ + if (orig_blob && + ((!their_blob && our_blob && oideq(orig_blob, our_blob)) || + (!our_blob && their_blob && oideq(orig_blob, their_blob)))) { + /* Deleted in both or deleted in one and unchanged in the other. */ + return merge_one_file_deleted(r->index, our_blob, their_blob, path, + orig_mode, our_mode, their_mode); + } else if (!orig_blob && our_blob && !their_blob) { + /* + * Added in one. The other side did not add and we + * added so there is nothing to be done, except making + * the path merged. + */ + return add_to_index_cacheinfo(r->index, our_mode, our_blob, + path, 0, 1, 1, NULL); + } else if (!orig_blob && !our_blob && their_blob) { + struct cache_entry *ce; + printf(_("Adding %s\n"), path); + + if (file_exists(path)) + return error(_("untracked %s is overwritten by the merge."), path); + + if (add_to_index_cacheinfo(r->index, their_mode, their_blob, + path, 0, 1, 1, &ce)) + return -1; + return checkout_from_index(r->index, path, ce); + } else if (!orig_blob && our_blob && their_blob && + oideq(our_blob, their_blob)) { + struct cache_entry *ce; + + /* Added in both, identically (check for same permissions). */ + if (our_mode != their_mode) + return error(_("File %s added identically in both branches, " + "but permissions conflict %o->%o."), + path, our_mode, their_mode); + + printf(_("Adding %s\n"), path); + + if (add_to_index_cacheinfo(r->index, our_mode, our_blob, + path, 0, 1, 1, &ce)) + return -1; + return checkout_from_index(r->index, path, ce); + } else if (our_blob && their_blob) { + /* Modified in both, but differently. */ + return do_merge_one_file(r->index, + orig_blob, our_blob, their_blob, path, + orig_mode, our_mode, their_mode); + } else { + char orig_hex[GIT_MAX_HEXSZ] = {0}, our_hex[GIT_MAX_HEXSZ] = {0}, + their_hex[GIT_MAX_HEXSZ] = {0}; + + if (orig_blob) + oid_to_hex_r(orig_hex, orig_blob); + if (our_blob) + oid_to_hex_r(our_hex, our_blob); + if (their_blob) + oid_to_hex_r(their_hex, their_blob); + + return error(_("%s: Not handling case %s -> %s -> %s"), + path, orig_hex, our_hex, their_hex); + } + + return 0; +} + +int merge_one_file_func(struct repository *r, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode, + void *data) +{ + return merge_three_way(r, + orig_blob, our_blob, their_blob, path, + orig_mode, our_mode, their_mode); +} + +int merge_one_file_spawn(struct repository *r, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode, + void *data) +{ + char oids[3][GIT_MAX_HEXSZ + 1] = {{0}}; + char modes[3][10] = {{0}}; + const char *arguments[] = { (char *)data, oids[0], oids[1], oids[2], + path, modes[0], modes[1], modes[2], NULL }; + + if (orig_blob) { + oid_to_hex_r(oids[0], orig_blob); + xsnprintf(modes[0], sizeof(modes[0]), "%06o", orig_mode); + } + + if (our_blob) { + oid_to_hex_r(oids[1], our_blob); + xsnprintf(modes[1], sizeof(modes[1]), "%06o", our_mode); + } + + if (their_blob) { + oid_to_hex_r(oids[2], their_blob); + xsnprintf(modes[2], sizeof(modes[2]), "%06o", their_mode); + } + + return run_command_v_opt(arguments, 0); +} + +static int merge_entry(struct repository *r, int quiet, unsigned int pos, + const char *path, int *err, merge_fn fn, void *data) +{ + int found = 0; + const struct object_id *oids[3] = {NULL}; + unsigned int modes[3] = {0}; + + do { + const struct cache_entry *ce = r->index->cache[pos]; + int stage = ce_stage(ce); + + if (strcmp(ce->name, path)) + break; + found++; + oids[stage - 1] = &ce->oid; + modes[stage - 1] = ce->ce_mode; + } while (++pos < r->index->cache_nr); + if (!found) + return error(_("%s is not in the cache"), path); + + if (fn(r, oids[0], oids[1], oids[2], path, + modes[0], modes[1], modes[2], data)) { + if (!quiet) + error(_("Merge program failed")); + (*err)++; + } + + return found; +} + +int merge_index_path(struct repository *r, int oneshot, int quiet, + const char *path, merge_fn fn, void *data) +{ + int pos = index_name_pos(r->index, path, strlen(path)), ret, err = 0; + + /* + * If it already exists in the cache as stage0, it's + * already merged and there is nothing to do. + */ + if (pos < 0) { + ret = merge_entry(r, quiet || oneshot, -pos - 1, path, &err, fn, data); + if (ret == -1) + return -1; + else if (err) + return 1; + } + return 0; +} + +int merge_all_index(struct repository *r, int oneshot, int quiet, + merge_fn fn, void *data) +{ + int err = 0, ret; + unsigned int i, prev_nr; + + for (i = 0; i < r->index->cache_nr; i++) { + const struct cache_entry *ce = r->index->cache[i]; + if (!ce_stage(ce)) + continue; + + prev_nr = r->index->cache_nr; + ret = merge_entry(r, quiet || oneshot, i, ce->name, &err, fn, data); + if (ret > 0) { + /* Don't bother handling an index that has + grown, since merge_one_file_func() can't grow + it, and merge_one_file_spawn() can't change + it. */ + i += ret - (prev_nr - r->index->cache_nr) - 1; + } else if (ret == -1) + return -1; + + if (err && !oneshot) + return 1; + } + + return err; +} + +static int fast_forward(struct repository *r, struct tree_desc *t, + int nr, int aggressive) +{ + struct unpack_trees_options opts; + struct lock_file lock = LOCK_INIT; + + refresh_index(r->index, REFRESH_QUIET, NULL, NULL, NULL); + repo_hold_locked_index(r, &lock, LOCK_DIE_ON_ERROR); + + memset(&opts, 0, sizeof(opts)); + opts.head_idx = 1; + opts.src_index = r->index; + opts.dst_index = r->index; + opts.merge = 1; + opts.update = 1; + opts.aggressive = aggressive; + + if (nr == 1) + opts.fn = oneway_merge; + else if (nr == 2) { + opts.fn = twoway_merge; + opts.initial_checkout = is_index_unborn(r->index); + } else if (nr >= 3) { + opts.fn = threeway_merge; + opts.head_idx = nr - 1; + } + + if (unpack_trees(nr, t, &opts)) + return -1; + + if (write_locked_index(r->index, &lock, COMMIT_LOCK)) + return error(_("unable to write new index file")); + + return 0; +} + +static int add_tree(struct tree *tree, struct tree_desc *t) +{ + if (parse_tree(tree)) + return -1; + + init_tree_desc(t, tree->buffer, tree->size); + return 0; +} + +int merge_strategies_resolve(struct repository *r, + struct commit_list *bases, const char *head_arg, + struct commit_list *remote) +{ + struct tree_desc t[MAX_UNPACK_TREES]; + struct object_id head, oid; + struct commit_list *i; + int nr = 0; + + if (head_arg) + get_oid(head_arg, &head); + + puts(_("Trying simple merge.")); + + for (i = bases; i && i->item; i = i->next) { + if (add_tree(repo_get_commit_tree(r, i->item), t + (nr++))) + return 2; + } + + if (head_arg) { + struct tree *tree = parse_tree_indirect(&head); + if (add_tree(tree, t + (nr++))) + return 2; + } + + if (remote && add_tree(repo_get_commit_tree(r, remote->item), t + (nr++))) + return 2; + + if (fast_forward(r, t, nr, 1)) + return 2; + + if (write_index_as_tree(&oid, r->index, r->index_file, + WRITE_TREE_SILENT, NULL)) { + int ret; + struct lock_file lock = LOCK_INIT; + + puts(_("Simple merge failed, trying Automatic merge.")); + repo_hold_locked_index(r, &lock, LOCK_DIE_ON_ERROR); + ret = merge_all_index(r, 1, 0, merge_one_file_func, NULL); + + write_locked_index(r->index, &lock, COMMIT_LOCK); + return !!ret; + } + + return 0; +} + +static int write_tree(struct repository *r, struct tree **reference_tree) +{ + struct object_id oid; + int ret; + + if (!(ret = write_index_as_tree(&oid, r->index, r->index_file, + WRITE_TREE_SILENT, NULL))) + *reference_tree = lookup_tree(r, &oid); + + return ret; +} + +static int octopus_fast_forward(struct repository *r, const char *branch_name, + struct tree *tree_head, struct tree *current_tree, + struct tree **reference_tree) +{ + /* + * The first head being merged was a fast-forward. Advance the + * reference commit to the head being merged, and use that tree + * as the intermediate result of the merge. We still need to + * count this as part of the parent set. + */ + struct tree_desc t[2]; + + printf(_("Fast-forwarding to: %s\n"), branch_name); + + init_tree_desc(t, tree_head->buffer, tree_head->size); + if (add_tree(current_tree, t + 1)) + return -1; + if (fast_forward(r, t, 2, 0)) + return -1; + if (write_tree(r, reference_tree)) + return -1; + + return 0; +} + +static int octopus_do_merge(struct repository *r, const char *branch_name, + struct commit_list *common, struct tree *current_tree, + struct tree **reference_tree) +{ + struct tree_desc t[MAX_UNPACK_TREES]; + struct commit_list *j; + int nr = 0, ret = 0; + + printf(_("Trying simple merge with %s\n"), branch_name); + + for (j = common; j; j = j->next) { + struct tree *tree = repo_get_commit_tree(r, j->item); + if (add_tree(tree, t + (nr++))) + return -1; + } + + if (add_tree(*reference_tree, t + (nr++))) + return -1; + if (add_tree(current_tree, t + (nr++))) + return -1; + if (fast_forward(r, t, nr, 1)) + return -1; + + if (write_tree(r, reference_tree)) { + struct lock_file lock = LOCK_INIT; + + puts(_("Simple merge did not work, trying automatic merge.")); + repo_hold_locked_index(r, &lock, LOCK_DIE_ON_ERROR); + ret = merge_all_index(r, 1, 0, merge_one_file_func, NULL); + write_locked_index(r->index, &lock, COMMIT_LOCK); + + write_tree(r, reference_tree); + } + + return ret ? -2 : 0; +} + +int merge_strategies_octopus(struct repository *r, + struct commit_list *bases, const char *head_arg, + struct commit_list *remotes) +{ + int ff_merge = 1, ret = 0, references = 1; + struct commit **reference_commit, *head_commit; + struct tree *reference_tree, *head_tree; + struct commit_list *i; + struct object_id head; + struct strbuf sb = STRBUF_INIT; + + get_oid(head_arg, &head); + head_commit = lookup_commit_reference(r, &head); + head_tree = repo_get_commit_tree(r, head_commit); + + if (parse_tree(head_tree)) + return 2; + + if (repo_index_has_changes(r, head_tree, &sb)) { + error(_("Your local changes to the following files " + "would be overwritten by merge:\n %s"), + sb.buf); + strbuf_release(&sb); + return 2; + } + + reference_commit = xcalloc(commit_list_count(remotes) + 1, + sizeof(struct commit *)); + reference_commit[0] = head_commit; + reference_tree = head_tree; + + for (i = remotes; i && i->item; i = i->next) { + struct commit *c = i->item; + struct object_id *oid = &c->object.oid; + struct tree *current_tree = repo_get_commit_tree(r, c); + struct commit_list *common, *j; + char *branch_name; + int k = 0, up_to_date = 0; + + if (ret) { + /* + * We allow only last one to have a + * hand-resolvable conflicts. Last round failed + * and we still had a head to merge. + */ + puts(_("Automated merge did not work.")); + puts(_("Should not be doing an octopus.")); + + free(reference_commit); + return 2; + } + + branch_name = merge_get_better_branch_name(oid_to_hex(oid)); + common = get_merge_bases_many(c, references, reference_commit); + + if (!common) { + error(_("Unable to find common commit with %s"), branch_name); + + free(branch_name); + free_commit_list(common); + free(reference_commit); + + return 2; + } + + for (j = common; j && !(up_to_date || !ff_merge); j = j->next) { + up_to_date |= oideq(&j->item->object.oid, oid); + + if (k < references) + ff_merge &= oideq(&j->item->object.oid, &reference_commit[k++]->object.oid); + } + + if (up_to_date) { + printf(_("Already up to date with %s\n"), branch_name); + + free(branch_name); + free_commit_list(common); + continue; + } + + if (ff_merge) { + ret = octopus_fast_forward(r, branch_name, head_tree, + current_tree, &reference_tree); + references = 0; + } else { + ret = octopus_do_merge(r, branch_name, common, + current_tree, &reference_tree); + } + + free(branch_name); + free_commit_list(common); + + if (ret == -1) + break; + + reference_commit[references++] = c; + } + + free(reference_commit); + return ret; +} diff --git a/merge-strategies.h b/merge-strategies.h new file mode 100644 index 00000000000000..05c50159ecba4a --- /dev/null +++ b/merge-strategies.h @@ -0,0 +1,46 @@ +#ifndef MERGE_STRATEGIES_H +#define MERGE_STRATEGIES_H + +#include "commit.h" +#include "object.h" + +int merge_three_way(struct repository *r, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode); + +typedef int (*merge_fn)(struct repository *r, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode, + void *data); + +int merge_one_file_func(struct repository *r, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode, + void *data); + +int merge_one_file_spawn(struct repository *r, + const struct object_id *orig_blob, + const struct object_id *our_blob, + const struct object_id *their_blob, const char *path, + unsigned int orig_mode, unsigned int our_mode, unsigned int their_mode, + void *data); + +int merge_index_path(struct repository *r, int oneshot, int quiet, + const char *path, merge_fn fn, void *data); +int merge_all_index(struct repository *r, int oneshot, int quiet, + merge_fn fn, void *data); + +int merge_strategies_resolve(struct repository *r, + struct commit_list *bases, const char *head_arg, + struct commit_list *remote); +int merge_strategies_octopus(struct repository *r, + struct commit_list *bases, const char *head_arg, + struct commit_list *remote); + +#endif /* MERGE_STRATEGIES_H */ diff --git a/merge.c b/merge.c index 5fb88af10254a7..9f407cdf8ded4a 100644 --- a/merge.c +++ b/merge.c @@ -97,6 +97,7 @@ int checkout_fast_forward(struct repository *r, init_checkout_metadata(&opts.meta, NULL, remote, NULL); setup_unpack_trees_porcelain(&opts, "merge"); + ensure_full_index(opts.src_index); if (unpack_trees(nr_trees, t, &opts)) { rollback_lock_file(&lock_file); clear_unpack_trees_porcelain(&opts); @@ -109,3 +110,15 @@ int checkout_fast_forward(struct repository *r, return error(_("unable to write new index file")); return 0; } + +char *merge_get_better_branch_name(const char *branch) +{ + static char githead_env[8 + GIT_MAX_HEXSZ + 1]; + char *name; + + if (strlen(branch) != the_hash_algo->hexsz) + return xstrdup(branch); + xsnprintf(githead_env, sizeof(githead_env), "GITHEAD_%s", branch); + name = getenv(githead_env); + return xstrdup(name ? name : branch); +} diff --git a/name-hash.c b/name-hash.c index 5d3c7b12c1805c..cb0f316f652fb5 100644 --- a/name-hash.c +++ b/name-hash.c @@ -7,6 +7,8 @@ */ #include "cache.h" #include "thread-utils.h" +#include "trace2.h" +#include "sparse-index.h" struct dir_entry { struct hashmap_entry ent; @@ -108,6 +110,12 @@ static void hash_index_entry(struct index_state *istate, struct cache_entry *ce) if (ce->ce_flags & CE_HASHED) return; ce->ce_flags |= CE_HASHED; + + if (ce->ce_mode == CE_MODE_SPARSE_DIRECTORY) { + add_dir_entry(istate, ce); + return; + } + hashmap_entry_init(&ce->ent, memihash(ce->name, ce_namelen(ce))); hashmap_add(&istate->name_hash, &ce->ent); @@ -577,6 +585,7 @@ static void lazy_init_name_hash(struct index_state *istate) if (istate->name_hash_initialized) return; trace_performance_enter(); + trace2_region_enter("index", "name-hash-init", istate->repo); hashmap_init(&istate->name_hash, cache_entry_cmp, NULL, istate->cache_nr); hashmap_init(&istate->dir_hash, dir_entry_cmp, NULL, istate->cache_nr); @@ -597,6 +606,7 @@ static void lazy_init_name_hash(struct index_state *istate) } istate->name_hash_initialized = 1; + trace2_region_leave("index", "name-hash-init", istate->repo); trace_performance_leave("initialize name hash"); } @@ -677,6 +687,7 @@ int index_dir_exists(struct index_state *istate, const char *name, int namelen) struct dir_entry *dir; lazy_init_name_hash(istate); + expand_to_path(istate, name, namelen, 0); dir = find_dir_entry(istate, name, namelen); return dir && dir->nr; } @@ -687,6 +698,7 @@ void adjust_dirname_case(struct index_state *istate, char *name) const char *ptr = startPtr; lazy_init_name_hash(istate); + expand_to_path(istate, name, strlen(name), 0); while (*ptr) { while (*ptr && *ptr != '/') ptr++; @@ -710,6 +722,7 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na unsigned int hash = memihash(name, namelen); lazy_init_name_hash(istate); + expand_to_path(istate, name, namelen, icase); ce = hashmap_get_entry_from_hash(&istate->name_hash, hash, NULL, struct cache_entry, ent); diff --git a/pathspec.c b/pathspec.c index 7a229d8d22f2f6..61dc771aa02bfa 100644 --- a/pathspec.c +++ b/pathspec.c @@ -20,7 +20,7 @@ * to use find_pathspecs_matching_against_index() instead. */ void add_pathspec_matches_against_index(const struct pathspec *pathspec, - const struct index_state *istate, + struct index_state *istate, char *seen) { int num_unmatched = 0, i; @@ -51,7 +51,7 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec, * given pathspecs achieves against all items in the index. */ char *find_pathspecs_matching_against_index(const struct pathspec *pathspec, - const struct index_state *istate) + struct index_state *istate) { char *seen = xcalloc(pathspec->nr, 1); add_pathspec_matches_against_index(pathspec, istate, seen); diff --git a/pathspec.h b/pathspec.h index 454ce364fac776..f19c5dcf022b7d 100644 --- a/pathspec.h +++ b/pathspec.h @@ -150,10 +150,10 @@ static inline int ps_strcmp(const struct pathspec_item *item, } void add_pathspec_matches_against_index(const struct pathspec *pathspec, - const struct index_state *istate, + struct index_state *istate, char *seen); char *find_pathspecs_matching_against_index(const struct pathspec *pathspec, - const struct index_state *istate); + struct index_state *istate); int match_pathspec_attrs(const struct index_state *istate, const char *name, int namelen, const struct pathspec_item *item); diff --git a/preload-index.c b/preload-index.c index 8b7ac1e267ce77..a377f087eb1bff 100644 --- a/preload-index.c +++ b/preload-index.c @@ -57,6 +57,8 @@ static void *preload_thread(void *_data) continue; if (S_ISGITLINK(ce->ce_mode)) continue; + if (ce->ce_mode == CE_MODE_SPARSE_DIRECTORY) + continue; if (ce_uptodate(ce)) continue; if (ce_skip_worktree(ce)) diff --git a/read-cache.c b/read-cache.c index 0e11d0149c22ea..33831fa930640e 100644 --- a/read-cache.c +++ b/read-cache.c @@ -27,6 +27,7 @@ #include "fsmonitor.h" #include "thread-utils.h" #include "progress.h" +#include "sparse-index.h" /* Mask for the name length in ce_flags in the on-disk index */ @@ -103,6 +104,9 @@ static const char *alternate_index_output; static void set_index_entry(struct index_state *istate, int nr, struct cache_entry *ce) { + if (S_ISSPARSEDIR(ce)) + istate->sparse_index = 1; + istate->cache[nr] = ce; add_name_hash(istate, ce); } @@ -631,7 +635,11 @@ void remove_marked_cache_entries(struct index_state *istate, int invalidate) int remove_file_from_index(struct index_state *istate, const char *path) { - int pos = index_name_pos(istate, path, strlen(path)); + int pos; + + ensure_full_index(istate); + + pos = index_name_pos(istate, path, strlen(path)); if (pos < 0) pos = -pos-1; cache_tree_invalidate_path(istate, path); @@ -649,9 +657,12 @@ static int compare_name(struct cache_entry *ce, const char *path, int namelen) static int index_name_pos_also_unmerged(struct index_state *istate, const char *path, int namelen) { - int pos = index_name_pos(istate, path, namelen); + int pos; struct cache_entry *ce; + ensure_full_index(istate); + + pos = index_name_pos(istate, path, namelen); if (pos >= 0) return pos; @@ -733,6 +744,8 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st, return error(_("%s: can only add regular files, symbolic links or git-directories"), path); namelen = strlen(path); + expand_to_path(istate, path, namelen, 0); + if (S_ISDIR(st_mode)) { if (resolve_gitlink_ref(path, "HEAD", &oid) < 0) return error(_("'%s' does not have a commit checked out"), path); @@ -1012,8 +1025,15 @@ int verify_path(const char *path, unsigned mode) c = *path++; if ((c == '.' && !verify_dotfile(path, mode)) || - is_dir_sep(c) || c == '\0') + is_dir_sep(c)) return 0; + /* + * allow terminating directory separators for + * sparse directory enries. + */ + if (c == '\0') + return mode == CE_MODE_SPARSE_DIRECTORY || + mode == SPARSE_DIR_MODE; } else if (c == '\\' && protect_ntfs) { if (is_ntfs_dotgit(path)) return 0; @@ -1097,6 +1117,9 @@ static int has_dir_name(struct index_state *istate, size_t len_eq_last; int cmp_last = 0; + /* TODO: use 'pos' here? */ + expand_to_path(istate, ce->name, ce->ce_namelen, 0); + /* * We are frequently called during an iteration on a sorted * list of pathnames and while building a new index. Therefore, @@ -1340,6 +1363,8 @@ int add_index_entry(struct index_state *istate, struct cache_entry *ce, int opti { int pos; + expand_to_path(istate, ce->name, ce->ce_namelen, 0); + if (option & ADD_CACHE_JUST_APPEND) pos = istate->cache_nr; else { @@ -1363,6 +1388,41 @@ int add_index_entry(struct index_state *istate, struct cache_entry *ce, int opti return 0; } +int add_to_index_cacheinfo(struct index_state *istate, unsigned int mode, + const struct object_id *oid, const char *path, + int stage, int allow_add, int allow_replace, + struct cache_entry **pce) +{ + int len, option; + struct cache_entry *ce = NULL; + + if (!verify_path(path, mode)) + return error(_("Invalid path '%s'"), path); + + len = strlen(path); + ce = make_empty_cache_entry(istate, len); + + oidcpy(&ce->oid, oid); + memcpy(ce->name, path, len); + ce->ce_flags = create_ce_flags(stage); + ce->ce_namelen = len; + ce->ce_mode = create_ce_mode(mode); + if (assume_unchanged) + ce->ce_flags |= CE_VALID; + option = allow_add ? ADD_CACHE_OK_TO_ADD : 0; + option |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0; + + if (add_index_entry(istate, ce, option)) { + discard_cache_entry(ce); + return -2; + } + + if (pce) + *pce = ce; + + return 0; +} + /* * "refresh" does not calculate a new sha1 file or bring the * cache up-to-date for mode/content changes. But what it @@ -1550,6 +1610,7 @@ int refresh_index(struct index_state *istate, unsigned int flags, * we only have to do the special cases that are left. */ preload_index(istate, pathspec, 0); + for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce, *new_entry; int cache_errno = 0; @@ -1560,6 +1621,9 @@ int refresh_index(struct index_state *istate, unsigned int flags, if (ignore_submodules && S_ISGITLINK(ce->ce_mode)) continue; + if (istate->sparse_index && ce->ce_mode == CE_MODE_SPARSE_DIRECTORY) + continue; + if (pathspec && !ce_path_match(istate, ce, pathspec, seen)) filtered = 1; @@ -2290,6 +2354,12 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) trace2_data_intmax("index", the_repository, "read/cache_nr", istate->cache_nr); + if (!istate->repo) + istate->repo = the_repository; + prepare_repo_settings(istate->repo); + if (istate->repo->settings.command_requires_full_index) + ensure_full_index(istate); + return istate->cache_nr; unmap: @@ -3026,7 +3096,8 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile, if (err) return -1; } - if (!strip_extensions && istate->fsmonitor_last_update) { + if (!strip_extensions && istate->fsmonitor_last_update && + !istate->sparse_index) { struct strbuf sb = STRBUF_INIT; write_fsmonitor_extension(&sb, istate); @@ -3096,6 +3167,13 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l { int ret; + ret = convert_to_sparse(istate); + + if (ret) { + warning(_("failed to convert to a sparse-index")); + return ret; + } + /* * TODO trace2: replace "the_repository" with the actual repo instance * that is associated with the given "istate". diff --git a/repo-settings.c b/repo-settings.c index f7fff0f5ab837e..9677d50f9238e7 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -77,4 +77,19 @@ void prepare_repo_settings(struct repository *r) UPDATE_DEFAULT_BOOL(r->settings.core_untracked_cache, UNTRACKED_CACHE_KEEP); UPDATE_DEFAULT_BOOL(r->settings.fetch_negotiation_algorithm, FETCH_NEGOTIATION_DEFAULT); + + /* + * This setting guards all index reads to require a full index + * over a sparse index. After suitable guards are placed in the + * codebase around uses of the index, this setting will be + * removed. + */ + r->settings.command_requires_full_index = 1; + + /* + * Initialize this as off. + */ + r->settings.sparse_index = 0; + if (!repo_config_get_bool(r, "extensions.sparseindex", &value) && value) + r->settings.sparse_index = 1; } diff --git a/repository.c b/repository.c index a4174ddb0629cd..a8acae002f712c 100644 --- a/repository.c +++ b/repository.c @@ -10,6 +10,7 @@ #include "object.h" #include "lockfile.h" #include "submodule-config.h" +#include "sparse-index.h" /* The main repository */ static struct repository the_repo; @@ -261,10 +262,24 @@ void repo_clear(struct repository *repo) int repo_read_index(struct repository *repo) { + int res; + if (!repo->index) repo->index = xcalloc(1, sizeof(*repo->index)); - return read_index_from(repo->index, repo->index_file, repo->gitdir); + /* Complete the double-reference */ + if (!repo->index->repo) + repo->index->repo = repo; + else if (repo->index->repo != repo) + BUG("repo's index should point back at itself"); + + res = read_index_from(repo->index, repo->index_file, repo->gitdir); + + prepare_repo_settings(repo); + if (repo->settings.command_requires_full_index) + ensure_full_index(repo->index); + + return res; } int repo_hold_locked_index(struct repository *repo, diff --git a/repository.h b/repository.h index b385ca3c94b62b..a45f7520fd9e12 100644 --- a/repository.h +++ b/repository.h @@ -41,6 +41,9 @@ struct repo_settings { enum fetch_negotiation_setting fetch_negotiation_algorithm; int core_multi_pack_index; + + unsigned command_requires_full_index:1, + sparse_index:1; }; struct repository { diff --git a/rerere.c b/rerere.c index 9281131a9f10cd..1836a6cfbcf33f 100644 --- a/rerere.c +++ b/rerere.c @@ -962,6 +962,8 @@ static int handle_cache(struct index_state *istate, struct rerere_io_mem io; int marker_size = ll_merge_marker_size(istate, path); + ensure_full_index(istate); + /* * Reproduce the conflicted merge in-core */ diff --git a/reset.c b/reset.c index 2f4fbd07c54b59..6baeaac42954cf 100644 --- a/reset.c +++ b/reset.c @@ -77,6 +77,7 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action, goto leave_reset_head; } + ensure_full_index(unpack_tree_opts.src_index); if (unpack_trees(nr, desc, &unpack_tree_opts)) { ret = -1; goto leave_reset_head; diff --git a/resolve-undo.c b/resolve-undo.c index 236320f179cbf6..a4265834977e8f 100644 --- a/resolve-undo.c +++ b/resolve-undo.c @@ -125,6 +125,8 @@ int unmerge_index_entry_at(struct index_state *istate, int pos) if (!istate->resolve_undo) return pos; + ensure_full_index(istate); + ce = istate->cache[pos]; if (ce_stage(ce)) { /* already unmerged */ @@ -172,6 +174,8 @@ void unmerge_marked_index(struct index_state *istate) if (!istate->resolve_undo) return; + ensure_full_index(istate); + for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; if (ce->ce_flags & CE_MATCHED) @@ -186,6 +190,8 @@ void unmerge_index(struct index_state *istate, const struct pathspec *pathspec) if (!istate->resolve_undo) return; + ensure_full_index(istate); + for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; if (!ce_path_match(istate, ce, pathspec, NULL)) diff --git a/sequencer.c b/sequencer.c index 8909a467700c50..c2f0e1f8276f8e 100644 --- a/sequencer.c +++ b/sequencer.c @@ -34,6 +34,7 @@ #include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "merge-strategies.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -679,9 +680,6 @@ static int do_recursive_merge(struct repository *r, static struct object_id *get_cache_tree_oid(struct index_state *istate) { - if (!istate->cache_tree) - istate->cache_tree = cache_tree(); - if (!cache_tree_fully_valid(istate->cache_tree)) if (cache_tree_update(istate, 0)) { error(_("unable to update cache tree")); @@ -2041,9 +2039,19 @@ static int do_pick_commit(struct repository *r, commit_list_insert(base, &common); commit_list_insert(next, &remotes); - res |= try_merge_command(r, opts->strategy, - opts->xopts_nr, (const char **)opts->xopts, - common, oid_to_hex(&head), remotes); + + if (!strcmp(opts->strategy, "resolve")) { + repo_read_index(r); + res |= merge_strategies_resolve(r, common, oid_to_hex(&head), remotes); + } else if (!strcmp(opts->strategy, "octopus")) { + repo_read_index(r); + res |= merge_strategies_octopus(r, common, oid_to_hex(&head), remotes); + } else { + res |= try_merge_command(r, opts->strategy, + opts->xopts_nr, (const char **)opts->xopts, + common, oid_to_hex(&head), remotes); + } + free_commit_list(common); free_commit_list(remotes); } @@ -3464,6 +3472,7 @@ static int do_reset(struct repository *r, return -1; } + ensure_full_index(unpack_tree_opts.src_index); if (unpack_trees(1, &desc, &unpack_tree_opts)) { rollback_lock_file(&lock); free((void *)desc.buffer); diff --git a/setup.c b/setup.c index c04cd25a30dfe0..cd83945646136f 100644 --- a/setup.c +++ b/setup.c @@ -500,6 +500,9 @@ static enum extension_result handle_extension(const char *var, return error("invalid value for 'extensions.objectformat'"); data->hash_algo = format; return EXTENSION_OK; + } else if (!strcmp(ext, "sparseindex")) { + data->sparse_index = 1; + return EXTENSION_OK; } return EXTENSION_UNKNOWN; } diff --git a/sha1-name.c b/sha1-name.c index 0b23b86ceb4433..c2f17e526ab35f 100644 --- a/sha1-name.c +++ b/sha1-name.c @@ -1734,6 +1734,8 @@ static void diagnose_invalid_index_path(struct repository *r, if (!prefix) prefix = ""; + ensure_full_index(r->index); + /* Wrong stage number? */ pos = index_name_pos(istate, filename, namelen); if (pos < 0) @@ -1854,6 +1856,7 @@ static enum get_oid_result get_oid_with_context_1(struct repository *repo, if (!repo->index || !repo->index->cache) repo_read_index(repo); + ensure_full_index(repo->index); pos = index_name_pos(repo->index, cp, namelen); if (pos < 0) pos = -pos - 1; diff --git a/sparse-index.c b/sparse-index.c new file mode 100644 index 00000000000000..249564155031d1 --- /dev/null +++ b/sparse-index.c @@ -0,0 +1,360 @@ +#include "cache.h" +#include "repository.h" +#include "sparse-index.h" +#include "tree.h" +#include "pathspec.h" +#include "trace2.h" +#include "cache-tree.h" +#include "config.h" +#include "dir.h" +#include "fsmonitor.h" + +static struct cache_entry *construct_sparse_dir_entry( + struct index_state *istate, + const char *sparse_dir, + struct cache_tree *tree) +{ + struct cache_entry *de; + + de = make_cache_entry(istate, SPARSE_DIR_MODE, &tree->oid, sparse_dir, 0, 0); + + de->ce_flags |= CE_SKIP_WORKTREE; + return de; +} + +/* + * Returns the number of entries "inserted" into the index. + */ +static int convert_to_sparse_rec(struct index_state *istate, + int num_converted, + int start, int end, + const char *ct_path, size_t ct_pathlen, + struct cache_tree *ct) +{ + int i, can_convert = 1; + int start_converted = num_converted; + enum pattern_match_result match; + int dtype; + struct strbuf child_path = STRBUF_INIT; + struct pattern_list *pl = istate->sparse_checkout_patterns; + + /* + * Is the current path outside of the sparse cone? + * Then check if the region can be replaced by a sparse + * directory entry (everything is sparse and merged). + */ + match = path_matches_pattern_list(ct_path, ct_pathlen, + NULL, &dtype, pl, istate); + if (match != NOT_MATCHED) + can_convert = 0; + + for (i = start; can_convert && i < end; i++) { + struct cache_entry *ce = istate->cache[i]; + + if (ce_stage(ce) || + S_ISGITLINK(ce->ce_mode) || + !(ce->ce_flags & CE_SKIP_WORKTREE)) + can_convert = 0; + } + + if (can_convert) { + struct cache_entry *se; + se = construct_sparse_dir_entry(istate, ct_path, ct); + + istate->cache[num_converted++] = se; + return 1; + } + + for (i = start; i < end; ) { + int count, span, pos = -1; + const char *base, *slash; + struct cache_entry *ce = istate->cache[i]; + + /* + * Detect if this is a normal entry oustide of any subtree + * entry. + */ + base = ce->name + ct_pathlen; + slash = strchr(base, '/'); + + if (slash) + pos = cache_tree_subtree_pos(ct, base, slash - base); + + if (pos < 0) { + istate->cache[num_converted++] = ce; + i++; + continue; + } + + strbuf_setlen(&child_path, 0); + strbuf_add(&child_path, ce->name, slash - ce->name + 1); + + span = ct->down[pos]->cache_tree->entry_count; + count = convert_to_sparse_rec(istate, + num_converted, i, i + span, + child_path.buf, child_path.len, + ct->down[pos]->cache_tree); + num_converted += count; + i += span; + } + + strbuf_release(&child_path); + return num_converted - start_converted; +} + +static int enable_sparse_index(struct repository *repo) +{ + int res; + + if (upgrade_repository_format(1) < 0) { + warning(_("unable to upgrade repository format to enable sparse-index")); + return -1; + } + res = git_config_set_gently("extensions.sparseindex", "true"); + + prepare_repo_settings(repo); + repo->settings.sparse_index = 1; + return res; +} + +int set_sparse_index_config(struct repository *repo, int enable) +{ + int res; + + if (enable) + return enable_sparse_index(repo); + + /* Don't downgrade repository format, just remove the extension. */ + res = git_config_set_multivar_gently("extensions.sparseindex", NULL, "", + CONFIG_FLAGS_MULTI_REPLACE); + + prepare_repo_settings(repo); + repo->settings.sparse_index = 0; + return res; +} + +int convert_to_sparse(struct index_state *istate) +{ + int test_env; + if (istate->split_index || istate->sparse_index || + !core_apply_sparse_checkout || !core_sparse_checkout_cone) + return 0; + + if (!istate->repo) + istate->repo = the_repository; + + /* + * If GIT_TEST_SPARSE_INDEX=1, then trigger extensions.sparseIndex + * to be fully enabled. If GIT_TEST_SPARSE_INDEX=0 (set explicitly), + * then purposefully disable the setting. + */ + test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1); + if (test_env >= 0) + set_sparse_index_config(istate->repo, test_env); + + /* + * Only convert to sparse if extensions.sparseIndex is set. + */ + prepare_repo_settings(istate->repo); + if (!istate->repo->settings.sparse_index) + return 0; + + if (!istate->sparse_checkout_patterns) { + istate->sparse_checkout_patterns = xcalloc(1, sizeof(struct pattern_list)); + if (get_sparse_checkout_patterns(istate->sparse_checkout_patterns) < 0) + return 0; + } + + if (!istate->sparse_checkout_patterns->use_cone_patterns) { + warning(_("attempting to use sparse-index without cone mode")); + return -1; + } + + if (cache_tree_update(istate, 0)) { + warning(_("unable to update cache-tree, staying full")); + return -1; + } + + remove_fsmonitor(istate); + + trace2_region_enter("index", "convert_to_sparse", istate->repo); + istate->cache_nr = convert_to_sparse_rec(istate, + 0, 0, istate->cache_nr, + "", 0, istate->cache_tree); + + /* Clear and recompute the cache-tree */ + cache_tree_free(&istate->cache_tree); + cache_tree_update(istate, 0); + + istate->sparse_index = 1; + trace2_region_leave("index", "convert_to_sparse", istate->repo); + return 0; +} + +static void set_index_entry(struct index_state *istate, int nr, struct cache_entry *ce) +{ + ALLOC_GROW(istate->cache, nr + 1, istate->cache_alloc); + + istate->cache[nr] = ce; + add_name_hash(istate, ce); +} + +static int add_path_to_index(const struct object_id *oid, + struct strbuf *base, const char *path, + unsigned int mode, int stage, void *context) +{ + struct index_state *istate = (struct index_state *)context; + struct cache_entry *ce; + size_t len = base->len; + + if (S_ISDIR(mode)) + return READ_TREE_RECURSIVE; + + strbuf_addstr(base, path); + + ce = make_cache_entry(istate, mode, oid, base->buf, 0, 0); + ce->ce_flags |= CE_SKIP_WORKTREE; + set_index_entry(istate, istate->cache_nr++, ce); + + strbuf_setlen(base, len); + return 0; +} + +void ensure_full_index(struct index_state *istate) +{ + int i; + struct index_state *full; + + if (!istate || !istate->sparse_index) + return; + + if (!istate->repo) + istate->repo = the_repository; + + trace2_region_enter("index", "ensure_full_index", istate->repo); + + /* initialize basics of new index */ + full = xcalloc(1, sizeof(struct index_state)); + memcpy(full, istate, sizeof(struct index_state)); + + /* then change the necessary things */ + full->sparse_index = 0; + full->cache_alloc = (3 * istate->cache_alloc) / 2; + full->cache_nr = 0; + ALLOC_ARRAY(full->cache, full->cache_alloc); + + for (i = 0; i < istate->cache_nr; i++) { + struct cache_entry *ce = istate->cache[i]; + struct tree *tree; + struct pathspec ps; + + if (!S_ISSPARSEDIR(ce)) { + set_index_entry(full, full->cache_nr++, ce); + continue; + } + if (!(ce->ce_flags & CE_SKIP_WORKTREE)) + warning(_("index entry is a directory, but not sparse (%08x)"), + ce->ce_flags); + + /* recursively walk into cd->name */ + tree = lookup_tree(istate->repo, &ce->oid); + + memset(&ps, 0, sizeof(ps)); + ps.recursive = 1; + ps.has_wildcard = 1; + ps.max_depth = -1; + + read_tree_recursive(istate->repo, tree, + ce->name, strlen(ce->name), + 0, &ps, + add_path_to_index, full); + + /* free directory entries. full entries are re-used */ + discard_cache_entry(ce); + } + + /* Copy back into original index. */ + memcpy(&istate->name_hash, &full->name_hash, sizeof(full->name_hash)); + istate->sparse_index = 0; + istate->cache = full->cache; + istate->cache_nr = full->cache_nr; + istate->cache_alloc = full->cache_alloc; + + free(full); + + /* Clear and recompute the cache-tree */ + cache_tree_free(&istate->cache_tree); + cache_tree_update(istate, 0); + + trace2_region_leave("index", "ensure_full_index", istate->repo); +} + +static int in_expand_to_path = 0; + +void expand_to_path(struct index_state *istate, + const char *path, size_t pathlen, int icase) +{ + struct cache_entry *ce = NULL; + struct strbuf path_as_dir = STRBUF_INIT; + int pos; + + /* prevent extra recursion */ + if (in_expand_to_path) + return; + + if (!istate || !istate->sparse_index) + return; + + if (!istate->repo) + istate->repo = the_repository; + + in_expand_to_path = 1; + + /* + * We only need to actually expand a region if the + * following are both true: + * + * 1. 'path' is not already in the index. + * 2. Some parent directory of 'path' is a sparse directory. + */ + + strbuf_add(&path_as_dir, path, pathlen); + strbuf_addch(&path_as_dir, '/'); + + /* in_expand_to_path prevents infinite recursion here */ + if (index_file_exists(istate, path, pathlen, icase)) + goto cleanup; + + pos = index_name_pos(istate, path_as_dir.buf, path_as_dir.len); + + if (pos < 0) + pos = -pos - 1; + if (pos < istate->cache_nr) + ce = istate->cache[pos]; + + /* + * If we didn't land on a sparse directory, then there is + * nothing to expand. + */ + if (ce && istate->cache[pos]->ce_mode != CE_MODE_SPARSE_DIRECTORY) + goto cleanup; + /* + * If that sparse directory is not a prefix of the path we + * are looking for, then we don't need to expand. + */ + if (ce && + (ce->ce_namelen >= path_as_dir.len || + strncmp(ce->name, path_as_dir.buf, ce->ce_namelen))) + goto cleanup; + + trace2_region_enter("index", "expand_to_path", istate->repo); + + /* for now, do the obviously-correct, slow thing */ + ensure_full_index(istate); + + trace2_region_leave("index", "expand_to_path", istate->repo); + +cleanup: + strbuf_release(&path_as_dir); + in_expand_to_path = 0; +} diff --git a/sparse-index.h b/sparse-index.h new file mode 100644 index 00000000000000..d452b75e70cb99 --- /dev/null +++ b/sparse-index.h @@ -0,0 +1,23 @@ +#ifndef SPARSE_INDEX_H__ +#define SPARSE_INDEX_H__ + +struct index_state; +struct repository; + +int set_sparse_index_config(struct repository *repo, int enable); +int convert_to_sparse(struct index_state *istate); + +/* + * Some places in the codebase expect to search for a specific path. + * This path might be outside of the sparse-checkout definition, in + * which case a sparse-index may not contain a path for that index. + * + * Given an index and a path, check to see if a leading directory for + * 'path' exists in the index as a sparse directory. In that case, + * expand that sparse directory to a full range of cache entries and + * populate the index accordingly. + */ +void expand_to_path(struct index_state *istate, + const char *path, size_t pathlen, int icase); + +#endif \ No newline at end of file diff --git a/split-index.c b/split-index.c index c0e8ad670d0a17..3150fa6476aabe 100644 --- a/split-index.c +++ b/split-index.c @@ -4,6 +4,8 @@ struct split_index *init_split_index(struct index_state *istate) { + ensure_full_index(istate); + if (!istate->split_index) { istate->split_index = xcalloc(1, sizeof(*istate->split_index)); istate->split_index->refcount = 1; diff --git a/submodule.c b/submodule.c index b3bb59f0664473..487d083e4ef9a7 100644 --- a/submodule.c +++ b/submodule.c @@ -33,9 +33,13 @@ static struct oid_array ref_tips_after_fetch; * will be disabled because we can't guess what might be configured in * .gitmodules unless the user resolves the conflict. */ -int is_gitmodules_unmerged(const struct index_state *istate) +int is_gitmodules_unmerged(struct index_state *istate) { - int pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE)); + int pos; + + ensure_full_index(istate); + + pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE)); if (pos < 0) { /* .gitmodules not found or isn't merged */ pos = -1 - pos; if (istate->cache_nr > pos) { /* there is a .gitmodules */ @@ -77,7 +81,11 @@ int is_writing_gitmodules_ok(void) */ int is_staging_gitmodules_ok(struct index_state *istate) { - int pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE)); + int pos; + + ensure_full_index(istate); + + pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE)); if ((pos >= 0) && (pos < istate->cache_nr)) { struct stat st; @@ -301,7 +309,7 @@ int is_submodule_populated_gently(const char *path, int *return_error_code) /* * Dies if the provided 'prefix' corresponds to an unpopulated submodule */ -void die_in_unpopulated_submodule(const struct index_state *istate, +void die_in_unpopulated_submodule(struct index_state *istate, const char *prefix) { int i, prefixlen; @@ -311,6 +319,8 @@ void die_in_unpopulated_submodule(const struct index_state *istate, prefixlen = strlen(prefix); + ensure_full_index(istate); + for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce = istate->cache[i]; int ce_len = ce_namelen(ce); @@ -331,7 +341,7 @@ void die_in_unpopulated_submodule(const struct index_state *istate, /* * Dies if any paths in the provided pathspec descends into a submodule */ -void die_path_inside_submodule(const struct index_state *istate, +void die_path_inside_submodule(struct index_state *istate, const struct pathspec *ps) { int i, j; @@ -1420,6 +1430,8 @@ static int get_next_submodule(struct child_process *cp, { struct submodule_parallel_fetch *spf = data; + ensure_full_index(spf->r->index); + for (; spf->count < spf->r->index->cache_nr; spf->count++) { const struct cache_entry *ce = spf->r->index->cache[spf->count]; const char *default_argv; diff --git a/submodule.h b/submodule.h index 4ac6e31cf1f7dd..84640c49c1149d 100644 --- a/submodule.h +++ b/submodule.h @@ -39,7 +39,7 @@ struct submodule_update_strategy { }; #define SUBMODULE_UPDATE_STRATEGY_INIT {SM_UPDATE_UNSPECIFIED, NULL} -int is_gitmodules_unmerged(const struct index_state *istate); +int is_gitmodules_unmerged(struct index_state *istate); int is_writing_gitmodules_ok(void); int is_staging_gitmodules_ok(struct index_state *istate); int update_path_in_gitmodules(const char *oldpath, const char *newpath); @@ -60,9 +60,9 @@ int is_submodule_active(struct repository *repo, const char *path); * Otherwise the return error code is the same as of resolve_gitdir_gently. */ int is_submodule_populated_gently(const char *path, int *return_error_code); -void die_in_unpopulated_submodule(const struct index_state *istate, +void die_in_unpopulated_submodule(struct index_state *istate, const char *prefix); -void die_path_inside_submodule(const struct index_state *istate, +void die_path_inside_submodule(struct index_state *istate, const struct pathspec *ps); enum submodule_update_type parse_submodule_update_type(const char *value); int parse_submodule_update_strategy(const char *value, diff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c index 244977a29bdfda..3c45dfeb3cb867 100644 --- a/t/helper/test-read-cache.c +++ b/t/helper/test-read-cache.c @@ -1,36 +1,93 @@ #include "test-tool.h" #include "cache.h" #include "config.h" +#include "sparse-index.h" + +static void print_cache_entry(struct cache_entry *ce, unsigned stat) +{ + if (stat) { + /* stat info */ + printf("%08x %08x %08x %08x %08x %08x ", + ce->ce_stat_data.sd_ctime.sec, + ce->ce_stat_data.sd_ctime.nsec, + ce->ce_stat_data.sd_mtime.sec, + ce->ce_stat_data.sd_mtime.nsec, + ce->ce_stat_data.sd_dev, + ce->ce_stat_data.sd_ino); + } + + /* mode in binary */ + printf("0b%d%d%d%d ", + (ce->ce_mode >> 15) & 1, + (ce->ce_mode >> 14) & 1, + (ce->ce_mode >> 13) & 1, + (ce->ce_mode >> 12) & 1); + + /* output permissions? */ + printf("%04o ", ce->ce_mode & 01777); + + printf("%s ", oid_to_hex(&ce->oid)); + + printf("%s\n", ce->name); +} + +static void print_cache(struct index_state *cache, unsigned stat) +{ + int i; + for (i = 0; i < the_index.cache_nr; i++) + print_cache_entry(the_index.cache[i], stat); +} int cmd__read_cache(int argc, const char **argv) { + struct repository *r = the_repository; int i, cnt = 1; const char *name = NULL; + int table = 0; + int stat = 1; + int expand = 0; + + initialize_the_repository(); + prepare_repo_settings(r); + r->settings.command_requires_full_index = 0; - if (argc > 1 && skip_prefix(argv[1], "--print-and-refresh=", &name)) { - argc--; - argv++; + for (++argv, --argc; *argv && starts_with(*argv, "--"); ++argv, --argc) { + if (skip_prefix(*argv, "--print-and-refresh=", &name)) + continue; + if (!strcmp(*argv, "--table")) + table = 1; + else if (!strcmp(*argv, "--no-stat")) + stat = 0; + else if (!strcmp(*argv, "--expand")) + expand = 1; } - if (argc == 2) - cnt = strtol(argv[1], NULL, 0); + if (argc == 1) + cnt = strtol(argv[0], NULL, 0); setup_git_directory(); git_config(git_default_config, NULL); + for (i = 0; i < cnt; i++) { - read_cache(); + repo_read_index(r); + + if (expand) + ensure_full_index(r->index); + if (name) { int pos; - refresh_index(&the_index, REFRESH_QUIET, + refresh_index(r->index, REFRESH_QUIET, NULL, NULL, NULL); - pos = index_name_pos(&the_index, name, strlen(name)); + pos = index_name_pos(r->index, name, strlen(name)); if (pos < 0) die("%s not in index", name); printf("%s is%s up to date\n", name, - ce_uptodate(the_index.cache[pos]) ? "" : " not"); + ce_uptodate(r->index->cache[pos]) ? "" : " not"); write_file(name, "%d\n", i); } - discard_cache(); + if (table) + print_cache(r->index, stat); + discard_index(r->index); } return 0; } diff --git a/t/t0500-progress-display.sh b/t/t0500-progress-display.sh index 1ed1df351cb178..84cce345e7dd3d 100755 --- a/t/t0500-progress-display.sh +++ b/t/t0500-progress-display.sh @@ -303,8 +303,7 @@ test_expect_success 'progress generates traces' ' "Working hard" stderr && # t0212/parse_events.perl intentionally omits regions and data. - grep -e "region_enter" -e "\"category\":\"progress\"" trace.event && - grep -e "region_leave" -e "\"category\":\"progress\"" trace.event && + test_region progress "Working hard" trace.event && grep "\"key\":\"total_objects\",\"value\":\"40\"" trace.event && grep "\"key\":\"total_bytes\",\"value\":\"409600\"" trace.event ' diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh new file mode 100755 index 00000000000000..b33842bb3e6cb3 --- /dev/null +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -0,0 +1,419 @@ +#!/bin/sh + +test_description='compare full workdir to sparse workdir' + +GIT_TEST_CHECK_CACHE_TREE=0 +GIT_TEST_SPLIT_INDEX=0 +GIT_TEST_SPARSE_INDEX= + +. ./test-lib.sh + +test_expect_success 'setup' ' + git init initial-repo && + ( + (GIT_TEST_SPARSE_INDEX=0 && export GIT_TEST_SPARSE_INDEX) && + cd initial-repo && + echo a >a && + echo "after deep" >e && + echo "after folder1" >g && + echo "after x" >z && + mkdir folder1 folder2 deep x && + mkdir deep/deeper1 deep/deeper2 && + mkdir deep/deeper1/deepest && + echo "after deeper1" >deep/e && + echo "after deepest" >deep/deeper1/e && + cp a folder1 && + cp a folder2 && + cp a x && + cp a deep && + cp a deep/deeper1 && + cp a deep/deeper2 && + cp a deep/deeper1/deepest && + cp -r deep/deeper1/deepest deep/deeper2 && + git add . && + git commit -m "initial commit" && + git checkout -b base && + for dir in folder1 folder2 deep + do + git checkout -b update-$dir && + echo "updated $dir" >$dir/a && + git commit -a -m "update $dir" || return 1 + done && + + git checkout -b rename-base base && + echo >folder1/larger-content <<-\EOF && + matching + lines + help + inexact + renames + EOF + cp folder1/larger-content folder2/ && + cp folder1/larger-content deep/deeper1/ && + git add . && + git commit -m "add interesting rename content" && + + git checkout -b rename-out-to-out rename-base && + mv folder1/a folder2/b && + mv folder1/larger-content folder2/edited-content && + echo >>folder2/edited-content && + git add . && + git commit -m "rename folder1/... to folder2/..." && + + git checkout -b rename-out-to-in rename-base && + mv folder1/a deep/deeper1/b && + mv folder1/larger-content deep/deeper1/edited-content && + echo >>deep/deeper1/edited-content && + git add . && + git commit -m "rename folder1/... to deep/deeper1/..." && + + git checkout -b rename-in-to-out rename-base && + mv deep/deeper1/a folder1/b && + mv deep/deeper1/larger-content folder1/edited-content && + echo >>folder1/edited-content && + git add . && + git commit -m "rename deep/deeper1/... to folder1/..." && + + git checkout -b deepest base && + echo "updated deepest" >deep/deeper1/deepest/a && + git commit -a -m "update deepest" && + + git checkout -f base && + git reset --hard + ) +' + +init_repos () { + rm -rf full-checkout sparse-checkout sparse-index && + + # create repos in initial state + cp -r initial-repo full-checkout && + git -C full-checkout reset --hard && + + cp -r initial-repo sparse-checkout && + git -C sparse-checkout reset --hard && + + cp -r initial-repo sparse-index && + git -C sparse-index reset --hard && + + # initialize sparse-checkout definitions + git -C sparse-checkout sparse-checkout init --cone && + git -C sparse-checkout sparse-checkout set deep && + git -C sparse-index sparse-checkout init --cone --sparse-index && + test_cmp_config -C sparse-index true extensions.sparseindex && + git -C sparse-index sparse-checkout set deep +} + +run_on_sparse () { + ( + cd sparse-checkout && + GIT_TEST_SPARSE_INDEX=0 $* >../sparse-checkout-out 2>../sparse-checkout-err + ) && + ( + cd sparse-index && + $* >../sparse-index-out 2>../sparse-index-err + ) +} + +run_on_all () { + ( + cd full-checkout && + GIT_TEST_SPARSE_INDEX=0 $* >../full-checkout-out 2>../full-checkout-err + ) && + run_on_sparse $* +} + +test_all_match () { + run_on_all $* && + test_cmp full-checkout-out sparse-checkout-out && + test_cmp full-checkout-out sparse-index-out && + test_cmp full-checkout-err sparse-checkout-err && + test_cmp full-checkout-err sparse-index-err +} + +test_sparse_match () { + run_on_sparse $* && + test_cmp sparse-checkout-out sparse-index-out && + test_cmp sparse-checkout-err sparse-index-err +} + +test_expect_success 'sparse-index contents' ' + init_repos && + + test-tool -C sparse-index read-cache --table --no-stat >cache && + for dir in folder1 folder2 x + do + TREE=$(git -C sparse-index rev-parse HEAD:$dir) && + grep "0b0000 0755 $TREE $dir/" cache \ + || return 1 + done && + + git -C sparse-index sparse-checkout set folder1 && + + test-tool -C sparse-index read-cache --table --no-stat >cache && + for dir in deep folder2 x + do + TREE=$(git -C sparse-index rev-parse HEAD:$dir) && + grep "0b0000 0755 $TREE $dir/" cache \ + || return 1 + done && + + git -C sparse-index sparse-checkout set deep/deeper1 && + + test-tool -C sparse-index read-cache --table --no-stat >cache && + for dir in deep/deeper2 folder1 folder2 x + do + TREE=$(git -C sparse-index rev-parse HEAD:$dir) && + grep "0b0000 0755 $TREE $dir/" cache \ + || return 1 + done +' + +test_expect_success 'expanded in-memory index matches full index' ' + init_repos && + test_sparse_match test-tool read-cache --expand --table --no-stat +' + +test_expect_success 'status with options' ' + init_repos && + test_sparse_match ls && + test_all_match git status --porcelain=v2 && + test_all_match git status --porcelain=v2 -z -u && + test_all_match git status --porcelain=v2 -uno && + run_on_all "touch README.md" && + test_all_match git status --porcelain=v2 && + test_all_match git status --porcelain=v2 -z -u && + test_all_match git status --porcelain=v2 -uno && + test_all_match git add README.md && + test_all_match git status --porcelain=v2 && + test_all_match git status --porcelain=v2 -z -u && + test_all_match git status --porcelain=v2 -uno +' + +test_expect_success 'status reports sparse-checkout' ' + init_repos && + git -C sparse-checkout status >full && + git -C sparse-index status >sparse && + test_i18ngrep "You are in a sparse checkout with " full && + test_i18ngrep "You are in a sparse checkout." sparse +' + +test_expect_success 'add, commit, checkout' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>$1 + EOF + run_on_all "../edit-contents README.md" && + + test_all_match git add README.md && + test_all_match git status --porcelain=v2 && + test_all_match git commit -m "Add README.md" && + + test_all_match git checkout HEAD~1 && + test_all_match git checkout - && + + run_on_all "../edit-contents README.md" && + + test_all_match git add -A && + test_all_match git status --porcelain=v2 && + test_all_match git commit -m "Extend-README.md" && + + test_all_match git checkout HEAD~1 && + test_all_match git checkout - && + + run_on_all "../edit-contents deep/newfile" && + + test_all_match git status --porcelain=v2 -uno && + test_all_match git status --porcelain=v2 && + test_all_match git add . && + test_all_match git status --porcelain=v2 && + test_all_match git commit -m "add deep/newfile" && + + test_all_match git checkout HEAD~1 && + test_all_match git checkout - +' + +test_expect_success 'checkout and reset --hard' ' + init_repos && + + test_all_match git checkout update-folder1 && + test_all_match git status --porcelain=v2 && + + test_all_match git checkout update-deep && + test_all_match git status --porcelain=v2 && + + test_all_match git checkout -b reset-test && + test_all_match git reset --hard deepest && + test_all_match git reset --hard update-folder1 && + test_all_match git reset --hard update-folder2 +' + +test_expect_success 'diff --staged' ' + init_repos && + + write_script edit-contents <<-\EOF && + echo text >>README.md + EOF + run_on_all "../edit-contents" && + + test_all_match git diff && + test_all_match git diff --staged && + test_all_match git add README.md && + test_all_match git diff && + test_all_match git diff --staged +' + +test_expect_success 'diff with renames' ' + init_repos && + + for branch in rename-out-to-out rename-out-to-in rename-in-to-out + do + # TODO: without this reset, the output of "git checkout" + # differs in sparse-index. + run_on_all git reset --hard && + test_all_match git checkout rename-base && + test_all_match git checkout $branch -- .&& + test_all_match git diff --staged --no-renames && + test_all_match git diff --staged --find-renames || return 1 + done +' + +test_expect_success 'log with pathspec outside sparse definition' ' + init_repos && + + test_all_match git log -- a && + test_all_match git log -- folder1/a && + test_all_match git log -- folder2/a && + test_all_match git log -- deep/a && + test_all_match git log -- deep/deeper1/a && + test_all_match git log -- deep/deeper1/deepest/a && + + test_all_match git checkout update-folder1 && + test_all_match git log -- folder1/a +' + +test_expect_success 'blame with pathspec inside sparse definition' ' + init_repos && + + test_all_match git blame a && + test_all_match git blame deep/a && + test_all_match git blame deep/deeper1/a && + test_all_match git blame deep/deeper1/deepest/a +' + +# TODO: blame currently does not support blaming files outside of the +# sparse definition. It complains that the file doesn't exist locally. +test_expect_failure 'blame with pathspec outside sparse definition' ' + init_repos && + + test_all_match git blame folder1/a && + test_all_match git blame folder2/a && + test_all_match git blame deep/deeper2/a && + test_all_match git blame deep/deeper2/deepest/a +' + +# TODO: works in microsoft/git version, but not git/git version? +test_expect_success 'checkout and reset (mixed)' ' + init_repos && + + test_all_match git checkout -b reset-test update-deep && + test_all_match git reset deepest && + test_all_match git reset update-folder1 && + test_all_match git reset update-folder2 +' + +# Ensure that sparse-index behaves identically to +# sparse-checkout with a full index. +test_expect_success 'checkout and reset (mixed) [sparse]' ' + init_repos && + + test_sparse_match git checkout -b reset-test update-deep && + test_sparse_match git reset deepest && + test_sparse_match git reset update-folder1 && + test_sparse_match git reset update-folder2 +' + +test_expect_success 'merge' ' + init_repos && + + test_all_match git checkout -b merge update-deep && + test_all_match git merge -m "folder1" update-folder1 && + test_all_match git rev-parse HEAD^{tree} && + test_all_match git merge -m "folder2" update-folder2 && + test_all_match git rev-parse HEAD^{tree} +' + +test_expect_success 'merge with outside renames' ' + init_repos && + + for type in out-to-out out-to-in in-to-out + do + test_all_match git reset --hard && + test_all_match git checkout -f -b merge-$type update-deep && + test_all_match git merge -m "$type" rename-$type && + test_all_match git rev-parse HEAD^{tree} || return 1 + done +' + +test_expect_success 'clean' ' + init_repos && + + echo bogus >>.gitignore && + run_on_all cp ../.gitignore . && + test_all_match git add .gitignore && + test_all_match git commit -m ignore-bogus-files && + + run_on_sparse mkdir folder1 && + run_on_all touch folder1/bogus && + + test_all_match git status --porcelain=v2 && + test_all_match git clean -f && + test_all_match git status --porcelain=v2 && + test_sparse_match ls && + test_sparse_match ls folder1 && + + test_all_match git clean -xf && + test_all_match git status --porcelain=v2 && + test_sparse_match ls && + test_sparse_match ls folder1 && + + test_all_match git clean -xdf && + test_all_match git status --porcelain=v2 && + test_sparse_match ls && + test_sparse_match ls folder1 && + + test_sparse_match test_path_is_dir folder1 +' + +test_expect_success 'sparse-index is expanded and converted back' ' + init_repos && + + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + git -C sparse-index reset --hard && + test_region index convert_to_sparse trace2.txt && + test_region index ensure_full_index trace2.txt +' + +test_expect_success 'sparse-index is not expanded' ' + init_repos && + + rm -f trace2.txt && + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + git -C sparse-index status -uno && + test_region ! index ensure_full_index trace2.txt && + + rm trace2.txt && + echo >>sparse-index/README.md && + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + git -C sparse-index add -A && + test_region ! index ensure_full_index trace2.txt && + + rm trace2.txt && + echo >>sparse-index/extra.txt && + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + git -C sparse-index add extra.txt && + test_region ! index ensure_full_index trace2.txt +' + +test_done diff --git a/t/t6060-merge-index.sh b/t/t6060-merge-index.sh index ddf34f0115b08b..9e15ceb9574f30 100755 --- a/t/t6060-merge-index.sh +++ b/t/t6060-merge-index.sh @@ -7,16 +7,19 @@ test_expect_success 'setup diverging branches' ' for i in 1 2 3 4 5 6 7 8 9 10; do echo $i done >file && - git add file && + cp file file2 && + git add file file2 && git commit -m base && git tag base && sed s/2/two/ tmp && mv tmp file && + cp file file2 && git commit -a -m two && git tag two && git checkout -b other HEAD^ && sed s/10/ten/ tmp && mv tmp file && + cp file file2 && git commit -a -m ten && git tag ten ' @@ -35,8 +38,11 @@ ten EOF test_expect_success 'read-tree does not resolve content merge' ' + cat >expect <<-\EOF && + file + file2 + EOF git read-tree -i -m base ten two && - echo file >expect && git diff-files --name-only --diff-filter=U >unmerged && test_cmp expect unmerged ' diff --git a/t/t6407-merge-binary.sh b/t/t6407-merge-binary.sh index 5b96821ece5611..e03dda018f90ff 100755 --- a/t/t6407-merge-binary.sh +++ b/t/t6407-merge-binary.sh @@ -35,33 +35,19 @@ test_expect_success setup ' ' test_expect_success resolve ' - rm -f a* m* && git reset --hard anchor && - - if git merge -s resolve master - then - echo Oops, should not have succeeded - false - else - git ls-files -s >current - test_cmp expect current - fi + test_must_fail git merge -s resolve master && + git ls-files -s >current && + test_cmp expect current ' test_expect_success recursive ' - rm -f a* m* && git reset --hard anchor && - - if git merge -s recursive master - then - echo Oops, should not have succeeded - false - else - git ls-files -s >current - test_cmp expect current - fi + test_must_fail git merge -s recursive master && + git ls-files -s >current && + test_cmp expect current ' test_done diff --git a/t/t6415-merge-dir-to-symlink.sh b/t/t6415-merge-dir-to-symlink.sh index 2eddcc7664e82a..5fb74e39a0d73e 100755 --- a/t/t6415-merge-dir-to-symlink.sh +++ b/t/t6415-merge-dir-to-symlink.sh @@ -94,7 +94,7 @@ test_expect_success SYMLINKS 'a/b was resolved as symlink' ' test -h a/b ' -test_expect_failure 'do not lose untracked in merge (resolve)' ' +test_expect_success 'do not lose untracked in merge (resolve)' ' git reset --hard && git checkout baseline^0 && >a/b/c/e && diff --git a/t/t7104-reset-hard.sh b/t/t7104-reset-hard.sh index 16faa0781373bb..7948ec392b3599 100755 --- a/t/t7104-reset-hard.sh +++ b/t/t7104-reset-hard.sh @@ -33,7 +33,7 @@ test_expect_success 'reset --hard should restore unmerged ones' ' ' -test_expect_success 'reset --hard did not corrupt index or cached-tree' ' +test_expect_success 'reset --hard did not corrupt index or cache-tree' ' T=$(git write-tree) && rm -f .git/index && diff --git a/t/t1092-virtualfilesystem.sh b/t/t9910-virtualfilesystem.sh similarity index 100% rename from t/t1092-virtualfilesystem.sh rename to t/t9910-virtualfilesystem.sh diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh index 685758a6e7dc57..98865ebc3dc4d9 100644 --- a/t/test-lib-functions.sh +++ b/t/test-lib-functions.sh @@ -1589,3 +1589,45 @@ test_subcommand () { grep "\[$expr\]" fi } + +# Check that the given command was invoked as part of the +# trace2-format trace on stdin. +# +# test_region [!]