git/submodule-config.h

139 lines
4.4 KiB
C
Raw Normal View History

submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
#ifndef SUBMODULE_CONFIG_CACHE_H
#define SUBMODULE_CONFIG_CACHE_H
#include "cache.h"
#include "config.h"
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
#include "hashmap.h"
#include "submodule.h"
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
#include "strbuf.h"
branch: add --recurse-submodules option for branch creation To improve the submodules UX, we would like to teach Git to handle branches in submodules. Start this process by teaching "git branch" the --recurse-submodules option so that "git branch --recurse-submodules topic" will create the `topic` branch in the superproject and its submodules. Although this commit does not introduce breaking changes, it does not work well with existing --recurse-submodules commands because "git branch --recurse-submodules" writes to the submodule ref store, but most commands only consider the superproject gitlink and ignore the submodule ref store. For example, "git checkout --recurse-submodules" will check out the commits in the superproject gitlinks (and put the submodules in detached HEAD) instead of checking out the submodule branches. Because of this, this commit introduces a new configuration value, `submodule.propagateBranches`. The plan is for Git commands to prioritize submodule ref store information over superproject gitlinks if this value is true. Because "git branch --recurse-submodules" writes to submodule ref stores, for the sake of clarity, it will not function unless this configuration value is set. This commit also includes changes that support working with submodules from a superproject commit because "branch --recurse-submodules" (and future commands) need to read .gitmodules and gitlinks from the superproject commit, but submodules are typically read from the filesystem's .gitmodules and the index's gitlinks. These changes are: * add a submodules_of_tree() helper that gives the relevant information of an in-tree submodule (e.g. path and oid) and initializes the repository * add is_tree_submodule_active() by adding a treeish_name parameter to is_submodule_active() * add the "submoduleNotUpdated" advice to advise users to update the submodules in their trees Incidentally, fix an incorrect usage string that combined the 'list' usage of git branch (-l) with the 'create' usage; this string has been incorrect since its inception, a8dfd5eac4 (Make builtin-branch.c use parse_options., 2007-10-07). Helped-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Glen Choo <chooglen@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-29 08:04:45 +08:00
#include "tree-walk.h"
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
/**
* The submodule config cache API allows to read submodule
* configurations/information from specified revisions. Internally
* information is lazily read into a cache that is used to avoid
* unnecessary parsing of the same .gitmodules files. Lookups can be done by
* submodule path or name.
*
* Usage
* -----
*
* The caller can look up information about submodules by using the
* `submodule_from_path()` or `submodule_from_name()` functions. They return
* a `struct submodule` which contains the values. The API automatically
* initializes and allocates the needed infrastructure on-demand. If the
* caller does only want to lookup values from revisions the initialization
* can be skipped.
*
* If the internal cache might grow too big or when the caller is done with
* the API, all internally cached values can be freed with submodule_free().
*
*/
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
/*
* Submodule entry containing the information about a certain submodule
* in a certain revision. It is returned by the lookup functions.
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
*/
struct submodule {
const char *path;
const char *name;
const char *url;
int fetch_recurse;
const char *ignore;
const char *branch;
struct submodule_update_strategy update_strategy;
/* the object id of the responsible .gitmodules file */
struct object_id gitmodules_oid;
int recommend_shallow;
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
};
struct submodule_cache;
struct repository;
void submodule_cache_free(struct submodule_cache *cache);
int parse_submodule_fetchjobs(const char *var, const char *value);
int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
struct option;
int option_fetch_parse_recurse_submodules(const struct option *opt,
const char *arg, int unset);
int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
void repo_read_gitmodules(struct repository *repo, int skip_if_read);
void gitmodules_config_oid(const struct object_id *commit_oid);
/**
* Same as submodule_from_path but lookup by name.
*/
const struct submodule *submodule_from_name(struct repository *r,
const struct object_id *commit_or_tree,
const char *name);
/**
* Given a tree-ish in the superproject and a path, return the submodule that
* is bound at the path in the named tree.
*/
const struct submodule *submodule_from_path(struct repository *r,
const struct object_id *commit_or_tree,
const char *path);
/**
* Use these to free the internally cached values.
*/
void submodule_free(struct repository *r);
int print_config_from_gitmodules(struct repository *repo, const char *key);
int config_set_in_gitmodules_file_gently(const char *key, const char *value);
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
submodule-config: verify submodule names as paths Submodule "names" come from the untrusted .gitmodules file, but we blindly append them to $GIT_DIR/modules to create our on-disk repo paths. This means you can do bad things by putting "../" into the name (among other things). Let's sanity-check these names to avoid building a path that can be exploited. There are two main decisions: 1. What should the allowed syntax be? It's tempting to reuse verify_path(), since submodule names typically come from in-repo paths. But there are two reasons not to: a. It's technically more strict than what we need, as we really care only about breaking out of the $GIT_DIR/modules/ hierarchy. E.g., having a submodule named "foo/.git" isn't actually dangerous, and it's possible that somebody has manually given such a funny name. b. Since we'll eventually use this checking logic in fsck to prevent downstream repositories, it should be consistent across platforms. Because verify_path() relies on is_dir_sep(), it wouldn't block "foo\..\bar" on a non-Windows machine. 2. Where should we enforce it? These days most of the .gitmodules reads go through submodule-config.c, so I've put it there in the reading step. That should cover all of the C code. We also construct the name for "git submodule add" inside the git-submodule.sh script. This is probably not a big deal for security since the name is coming from the user anyway, but it would be polite to remind them if the name they pick is invalid (and we need to expose the name-checker to the shell anyway for our test scripts). This patch issues a warning when reading .gitmodules and just ignores the related config entry completely. This will generally end up producing a sensible error, as it works the same as a .gitmodules file which is missing a submodule entry (so "submodule update" will barf, but "git clone --recurse-submodules" will print an error but not abort the clone. There is one minor oddity, which is that we print the warning once per malformed config key (since that's how the config subsystem gives us the entries). So in the new test, for example, the user would see three warnings. That's OK, since the intent is that this case should never come up outside of malicious repositories (and then it might even benefit the user to see the message multiple times). Credit for finding this vulnerability and the proof of concept from which the test script was adapted goes to Etienne Stalmans. Signed-off-by: Jeff King <peff@peff.net>
2018-04-30 15:25:25 +08:00
/*
* Returns 0 if the name is syntactically acceptable as a submodule "name"
* (e.g., that may be found in the subsection of a .gitmodules file) and -1
* otherwise.
*/
int check_submodule_name(const char *name);
/*
* Note: these helper functions exist solely to maintain backward
* compatibility with 'fetch' and 'update_clone' storing configuration in
* '.gitmodules'.
*
* New helpers to retrieve arbitrary configuration from the '.gitmodules' file
* should NOT be added.
*/
void fetch_config_from_gitmodules(int *max_children, int *recurse_submodules);
void update_clone_config_from_gitmodules(int *max_jobs);
branch: add --recurse-submodules option for branch creation To improve the submodules UX, we would like to teach Git to handle branches in submodules. Start this process by teaching "git branch" the --recurse-submodules option so that "git branch --recurse-submodules topic" will create the `topic` branch in the superproject and its submodules. Although this commit does not introduce breaking changes, it does not work well with existing --recurse-submodules commands because "git branch --recurse-submodules" writes to the submodule ref store, but most commands only consider the superproject gitlink and ignore the submodule ref store. For example, "git checkout --recurse-submodules" will check out the commits in the superproject gitlinks (and put the submodules in detached HEAD) instead of checking out the submodule branches. Because of this, this commit introduces a new configuration value, `submodule.propagateBranches`. The plan is for Git commands to prioritize submodule ref store information over superproject gitlinks if this value is true. Because "git branch --recurse-submodules" writes to submodule ref stores, for the sake of clarity, it will not function unless this configuration value is set. This commit also includes changes that support working with submodules from a superproject commit because "branch --recurse-submodules" (and future commands) need to read .gitmodules and gitlinks from the superproject commit, but submodules are typically read from the filesystem's .gitmodules and the index's gitlinks. These changes are: * add a submodules_of_tree() helper that gives the relevant information of an in-tree submodule (e.g. path and oid) and initializes the repository * add is_tree_submodule_active() by adding a treeish_name parameter to is_submodule_active() * add the "submoduleNotUpdated" advice to advise users to update the submodules in their trees Incidentally, fix an incorrect usage string that combined the 'list' usage of git branch (-l) with the 'create' usage; this string has been incorrect since its inception, a8dfd5eac4 (Make builtin-branch.c use parse_options., 2007-10-07). Helped-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Glen Choo <chooglen@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-29 08:04:45 +08:00
/*
* Submodule entry that contains relevant information about a
* submodule in a tree.
*/
struct submodule_tree_entry {
/* The submodule's tree entry. */
struct name_entry *name_entry;
/*
* A struct repository corresponding to the submodule. May be
* NULL if the submodule has not been updated.
*/
struct repository *repo;
/*
* A struct submodule containing the submodule config in the
* tree's .gitmodules.
*/
const struct submodule *submodule;
};
struct submodule_entry_list {
struct submodule_tree_entry *entries;
int entry_nr;
int entry_alloc;
};
/**
* Given a treeish, return all submodules in the tree and its subtrees,
* but excluding nested submodules. Callers that require nested
* submodules are expected to recurse into the submodules themselves.
*/
void submodules_of_tree(struct repository *r,
const struct object_id *treeish_name,
struct submodule_entry_list *ret);
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 08:21:57 +08:00
#endif /* SUBMODULE_CONFIG_H */