git/scalar.c

1003 lines
24 KiB
C
Raw Normal View History

scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03 21:34:16 +08:00
/*
* The Scalar command-line interface.
*/
global: introduce `USE_THE_REPOSITORY_VARIABLE` macro Use of the `the_repository` variable is deprecated nowadays, and we slowly but steadily convert the codebase to not use it anymore. Instead, callers should be passing down the repository to work on via parameters. It is hard though to prove that a given code unit does not use this variable anymore. The most trivial case, merely demonstrating that there is no direct use of `the_repository`, is already a bit of a pain during code reviews as the reviewer needs to manually verify claims made by the patch author. The bigger problem though is that we have many interfaces that implicitly rely on `the_repository`. Introduce a new `USE_THE_REPOSITORY_VARIABLE` macro that allows code units to opt into usage of `the_repository`. The intent of this macro is to demonstrate that a certain code unit does not use this variable anymore, and to keep it from new dependencies on it in future changes, be it explicit or implicit For now, the macro only guards `the_repository` itself as well as `the_hash_algo`. There are many more known interfaces where we have an implicit dependency on `the_repository`, but those are not guarded at the current point in time. Over time though, we should start to add guards as required (or even better, just remove them). Define the macro as required in our code units. As expected, most of our code still relies on the global variable. Nearly all of our builtins rely on the variable as there is no way yet to pass `the_repository` to their entry point. For now, declare the macro in "biultin.h" to keep the required changes at least a little bit more contained. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-14 14:50:23 +08:00
#define USE_THE_REPOSITORY_VARIABLE
#include "git-compat-util.h"
#include "abspath.h"
scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03 21:34:16 +08:00
#include "gettext.h"
#include "parse-options.h"
#include "config.h"
#include "run-command.h"
#include "simple-ipc.h"
#include "fsmonitor-ipc.h"
#include "fsmonitor-settings.h"
#include "refs.h"
#include "dir.h"
#include "packfile.h"
#include "help.h"
#include "setup.h"
#include "trace2.h"
static void setup_enlistment_directory(int argc, const char **argv,
const char * const *usagestr,
const struct option *options,
struct strbuf *enlistment_root)
{
struct strbuf path = STRBUF_INIT;
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
int enlistment_is_repo_parent = 0;
size_t len;
if (startup_info->have_repository)
BUG("gitdir already set up?!?");
if (argc > 1)
usage_with_options(usagestr, options);
/* find the worktree, determine its corresponding root */
if (argc == 1) {
strbuf_add_absolute_path(&path, argv[0]);
if (!is_directory(path.buf))
die(_("'%s' does not exist"), path.buf);
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
if (chdir(path.buf) < 0)
die_errno(_("could not switch to '%s'"), path.buf);
} else if (strbuf_getcwd(&path) < 0)
die(_("need a working directory"));
strbuf_trim_trailing_dir_sep(&path);
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
/* check if currently in enlistment root with src/ workdir */
len = path.len;
strbuf_addstr(&path, "/src");
if (is_nonbare_repository_dir(&path)) {
enlistment_is_repo_parent = 1;
if (chdir(path.buf) < 0)
die_errno(_("could not switch to '%s'"), path.buf);
}
strbuf_setlen(&path, len);
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
setup_git_directory();
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
if (!the_repository->worktree)
die(_("Scalar enlistments require a worktree"));
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
if (enlistment_root) {
if (enlistment_is_repo_parent)
strbuf_addbuf(enlistment_root, &path);
else
strbuf_addstr(enlistment_root, the_repository->worktree);
}
strbuf_release(&path);
}
LAST_ARG_MUST_BE_NULL
static int run_git(const char *arg, ...)
{
struct child_process cmd = CHILD_PROCESS_INIT;
va_list args;
const char *p;
va_start(args, arg);
strvec_push(&cmd.args, arg);
while ((p = va_arg(args, const char *)))
strvec_push(&cmd.args, p);
va_end(args);
cmd.git_cmd = 1;
return run_command(&cmd);
}
struct scalar_config {
const char *key;
const char *value;
int overwrite_on_reconfigure;
};
static int set_scalar_config(const struct scalar_config *config, int reconfigure)
{
char *value = NULL;
int res;
if ((reconfigure && config->overwrite_on_reconfigure) ||
git_config_get_string(config->key, &value)) {
trace2_data_string("scalar", the_repository, config->key, "created");
res = git_config_set_gently(config->key, config->value);
} else {
trace2_data_string("scalar", the_repository, config->key, "exists");
res = 0;
}
free(value);
return res;
}
static int have_fsmonitor_support(void)
{
return fsmonitor_ipc__is_supported() &&
fsm_settings__get_reason(the_repository) == FSMONITOR_REASON_OK;
}
static int set_recommended_config(int reconfigure)
{
struct scalar_config config[] = {
/* Required */
{ "am.keepCR", "true", 1 },
{ "core.FSCache", "true", 1 },
{ "core.multiPackIndex", "true", 1 },
{ "core.preloadIndex", "true", 1 },
#ifndef WIN32
{ "core.untrackedCache", "true", 1 },
#else
/*
* Unfortunately, Scalar's Functional Tests demonstrated
* that the untracked cache feature is unreliable on Windows
* (which is a bummer because that platform would benefit the
* most from it). For some reason, freshly created files seem
* not to update the directory's `lastModified` time
* immediately, but the untracked cache would need to rely on
* that.
*
* Therefore, with a sad heart, we disable this very useful
* feature on Windows.
*/
{ "core.untrackedCache", "false", 1 },
#endif
{ "core.logAllRefUpdates", "true", 1 },
{ "credential.https://dev.azure.com.useHttpPath", "true", 1 },
{ "credential.validate", "false", 1 }, /* GCM4W-only */
{ "gc.auto", "0", 1 },
{ "gui.GCWarning", "false", 1 },
{ "index.skipHash", "false", 1 },
{ "index.threads", "true", 1 },
{ "index.version", "4", 1 },
{ "merge.stat", "false", 1 },
{ "merge.renames", "true", 1 },
{ "pack.useBitmaps", "false", 1 },
{ "pack.useSparse", "true", 1 },
{ "receive.autoGC", "false", 1 },
{ "feature.manyFiles", "false", 1 },
{ "feature.experimental", "false", 1 },
{ "fetch.unpackLimit", "1", 1 },
{ "fetch.writeCommitGraph", "false", 1 },
#ifdef WIN32
{ "http.sslBackend", "schannel", 1 },
#endif
/* Optional */
{ "status.aheadBehind", "false" },
{ "commitGraph.generationVersion", "1" },
{ "core.autoCRLF", "false" },
{ "core.safeCRLF", "false" },
{ "fetch.showForcedUpdates", "false" },
{ NULL, NULL },
};
int i;
char *value;
for (i = 0; config[i].key; i++) {
if (set_scalar_config(config + i, reconfigure))
return error(_("could not configure %s=%s"),
config[i].key, config[i].value);
}
if (have_fsmonitor_support()) {
struct scalar_config fsmonitor = { "core.fsmonitor", "true" };
if (set_scalar_config(&fsmonitor, reconfigure))
return error(_("could not configure %s=%s"),
fsmonitor.key, fsmonitor.value);
}
/*
* The `log.excludeDecoration` setting is special because it allows
* for multiple values.
*/
if (git_config_get_string("log.excludeDecoration", &value)) {
trace2_data_string("scalar", the_repository,
"log.excludeDecoration", "created");
if (git_config_set_multivar_gently("log.excludeDecoration",
"refs/prefetch/*",
CONFIG_REGEX_NONE, 0))
return error(_("could not configure "
"log.excludeDecoration"));
} else {
trace2_data_string("scalar", the_repository,
"log.excludeDecoration", "exists");
free(value);
}
return 0;
}
static int toggle_maintenance(int enable)
{
return run_git("maintenance",
enable ? "start" : "unregister",
enable ? NULL : "--force",
NULL);
}
static int add_or_remove_enlistment(int add)
{
int res;
if (!the_repository->worktree)
die(_("Scalar enlistments require a worktree"));
res = run_git("config", "--global", "--get", "--fixed-value",
"scalar.repo", the_repository->worktree, NULL);
/*
* If we want to add and the setting is already there, then do nothing.
* If we want to remove and the setting is not there, then do nothing.
*/
if ((add && !res) || (!add && res))
return 0;
return run_git("config", "--global", add ? "--add" : "--unset",
add ? "--no-fixed-value" : "--fixed-value",
"scalar.repo", the_repository->worktree, NULL);
}
static int start_fsmonitor_daemon(void)
{
assert(have_fsmonitor_support());
if (fsmonitor_ipc__get_state() != IPC_STATE__LISTENING)
return run_git("fsmonitor--daemon", "start", NULL);
return 0;
}
static int stop_fsmonitor_daemon(void)
{
assert(have_fsmonitor_support());
if (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING)
return run_git("fsmonitor--daemon", "stop", NULL);
return 0;
}
static int register_dir(void)
{
if (add_or_remove_enlistment(1))
return error(_("could not add enlistment"));
if (set_recommended_config(0))
return error(_("could not set recommended config"));
if (toggle_maintenance(1))
warning(_("could not turn on maintenance"));
if (have_fsmonitor_support() && start_fsmonitor_daemon()) {
return error(_("could not start the FSMonitor daemon"));
}
return 0;
}
static int unregister_dir(void)
{
int res = 0;
if (toggle_maintenance(0))
res = error(_("could not turn off maintenance"));
if (add_or_remove_enlistment(0))
res = error(_("could not remove enlistment"));
return res;
}
/* printf-style interface, expects `<key>=<value>` argument */
__attribute__((format (printf, 1, 2)))
static int set_config(const char *fmt, ...)
{
struct strbuf buf = STRBUF_INIT;
char *value;
int res;
va_list args;
va_start(args, fmt);
strbuf_vaddf(&buf, fmt, args);
va_end(args);
value = strchr(buf.buf, '=');
if (value)
*(value++) = '\0';
res = git_config_set_gently(buf.buf, value);
strbuf_release(&buf);
return res;
}
static char *remote_default_branch(const char *url)
{
struct child_process cp = CHILD_PROCESS_INIT;
struct strbuf out = STRBUF_INIT;
cp.git_cmd = 1;
strvec_pushl(&cp.args, "ls-remote", "--symref", url, "HEAD", NULL);
if (!pipe_command(&cp, NULL, 0, &out, 0, NULL, 0)) {
const char *line = out.buf;
while (*line) {
const char *eol = strchrnul(line, '\n'), *p;
size_t len = eol - line;
char *branch;
if (!skip_prefix(line, "ref: ", &p) ||
!strip_suffix_mem(line, &len, "\tHEAD")) {
line = eol + (*eol == '\n');
continue;
}
eol = line + len;
if (skip_prefix(p, "refs/heads/", &p)) {
branch = xstrndup(p, eol - p);
strbuf_release(&out);
return branch;
}
error(_("remote HEAD is not a branch: '%.*s'"),
(int)(eol - p), p);
strbuf_release(&out);
return NULL;
}
}
warning(_("failed to get default branch name from remote; "
"using local default"));
strbuf_reset(&out);
child_process_init(&cp);
cp.git_cmd = 1;
strvec_pushl(&cp.args, "symbolic-ref", "--short", "HEAD", NULL);
if (!pipe_command(&cp, NULL, 0, &out, 0, NULL, 0)) {
strbuf_trim(&out);
return strbuf_detach(&out, NULL);
}
strbuf_release(&out);
error(_("failed to get default branch name"));
return NULL;
}
static int delete_enlistment(struct strbuf *enlistment)
{
struct strbuf parent = STRBUF_INIT;
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
size_t offset;
char *path_sep;
if (unregister_dir())
return error(_("failed to unregister repository"));
/*
* Change the current directory to one outside of the enlistment so
* that we may delete everything underneath it.
*/
scalar: constrain enlistment search Make the search for repository and enlistment root in 'setup_enlistment_directory()' more constrained to simplify behavior and adhere to 'GIT_CEILING_DIRECTORIES'. Previously, 'setup_enlistment_directory()' would check whether the provided path (or current working directory) '<dir>' or its subdirectory '<dir>/src' was a repository root. If not, the process would repeat on the parent of '<dir>' until the repository was found or it reached the root of the filesystem. This meant that a user could specify a path *anywhere* inside an enlistment (including paths not in the repository contained within the enlistment) and it would be found. The downside to this process is that the search would not account for 'GIT_CEILING_DIRECTORIES', so the upward search could result in modifying repository contents past 'GIT_CEILING_DIRECTORIES'. Similarly, operations like 'scalar delete' could end up unintentionally deleting the parent of a repo if its root was named 'src'. To make this 'setup_enlistment_directory()' both adhere to 'GIT_CEILING_DIRECTORIES' and avoid unwanted deletions, the search for an enlistment directory is simplified to: - if '<dir>/src' is a repository root, '<dir>' is the enlistment root - if '<dir>' is either the repository root or contained within a repository, the repository root is the enlistment root Now, only 'setup_git_directory()' (called by 'setup_enlistment_directory()') searches upwards from the 'scalar' specified path, enforcing 'GIT_CEILING_DIRECTORIES' in the process. Additionally, 'scalar delete <dir>/src' will not delete '<dir>' (if users would like to delete it, they can still specify the enlistment root with 'scalar delete <dir>'). This is true of any 'scalar' operation; users can invoke 'scalar' on the enlistment root, but paths must otherwise be inside the repository to be valid. To help clarify the updated behavior, new tests are added to 't9099-scalar.sh'. Finally, this change leaves 'strbuf_parent_directory()' with only a single, WIN32-specific caller in 'delete_enlistment()'. Rather than wrap 'strbuf_parent_directory()' in '#ifdef WIN32' to avoid the "unused function" compiler error, move the contents of 'strbuf_parent_directory()' into 'delete_enlistment()' and remove the function. Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-19 05:40:46 +08:00
offset = offset_1st_component(enlistment->buf);
path_sep = find_last_dir_sep(enlistment->buf + offset);
strbuf_add(&parent, enlistment->buf,
path_sep ? path_sep - enlistment->buf : offset);
if (chdir(parent.buf) < 0) {
int res = error_errno(_("could not switch to '%s'"), parent.buf);
strbuf_release(&parent);
return res;
}
strbuf_release(&parent);
if (have_fsmonitor_support() && stop_fsmonitor_daemon())
return error(_("failed to stop the FSMonitor daemon"));
if (remove_dir_recursively(enlistment, 0))
return error(_("failed to delete enlistment directory"));
return 0;
}
/*
* Dummy implementation; Using `get_version_info()` would cause a link error
* without this.
*/
void load_builtin_commands(const char *prefix UNUSED,
struct cmdnames *cmds UNUSED)
{
die("not implemented");
}
static int cmd_clone(int argc, const char **argv)
{
const char *branch = NULL;
int full_clone = 0, single_branch = 0, show_progress = isatty(2);
int src = 1, tags = 1;
struct option clone_options[] = {
OPT_STRING('b', "branch", &branch, N_("<branch>"),
N_("branch to checkout after clone")),
OPT_BOOL(0, "full-clone", &full_clone,
N_("when cloning, create full working directory")),
OPT_BOOL(0, "single-branch", &single_branch,
N_("only download metadata for the branch that will "
"be checked out")),
OPT_BOOL(0, "src", &src,
N_("create repository within 'src' directory")),
OPT_BOOL(0, "tags", &tags,
N_("specify if tags should be fetched during clone")),
OPT_END(),
};
const char * const clone_usage[] = {
N_("scalar clone [--single-branch] [--branch <main-branch>] [--full-clone]\n"
"\t[--[no-]src] [--[no-]tags] <url> [<enlistment>]"),
NULL
};
const char *url;
char *enlistment = NULL, *dir = NULL;
struct strbuf buf = STRBUF_INIT;
int res;
argc = parse_options(argc, argv, NULL, clone_options, clone_usage, 0);
if (argc == 2) {
url = argv[0];
enlistment = xstrdup(argv[1]);
} else if (argc == 1) {
url = argv[0];
strbuf_addstr(&buf, url);
/* Strip trailing slashes, if any */
while (buf.len > 0 && is_dir_sep(buf.buf[buf.len - 1]))
strbuf_setlen(&buf, buf.len - 1);
/* Strip suffix `.git`, if any */
strbuf_strip_suffix(&buf, ".git");
enlistment = find_last_dir_sep(buf.buf);
if (!enlistment) {
die(_("cannot deduce worktree name from '%s'"), url);
}
enlistment = xstrdup(enlistment + 1);
} else {
usage_msg_opt(_("You must specify a repository to clone."),
clone_usage, clone_options);
}
if (is_directory(enlistment))
die(_("directory '%s' exists already"), enlistment);
if (src)
dir = xstrfmt("%s/src", enlistment);
else
dir = xstrdup(enlistment);
strbuf_reset(&buf);
if (branch)
strbuf_addf(&buf, "init.defaultBranch=%s", branch);
else {
char *b = repo_default_branch_name(the_repository, 1);
strbuf_addf(&buf, "init.defaultBranch=%s", b);
free(b);
}
if ((res = run_git("-c", buf.buf, "init", "--", dir, NULL)))
goto cleanup;
if (chdir(dir) < 0) {
res = error_errno(_("could not switch to '%s'"), dir);
goto cleanup;
}
setup_git_directory();
/* common-main already logs `argv` */
trace2_def_repo(the_repository);
if (!branch && !(branch = remote_default_branch(url))) {
res = error(_("failed to get default branch for '%s'"), url);
goto cleanup;
}
if (set_config("remote.origin.url=%s", url) ||
set_config("remote.origin.fetch="
"+refs/heads/%s:refs/remotes/origin/%s",
single_branch ? branch : "*",
single_branch ? branch : "*") ||
set_config("remote.origin.promisor=true") ||
set_config("remote.origin.partialCloneFilter=blob:none")) {
res = error(_("could not configure remote in '%s'"), dir);
goto cleanup;
}
if (!tags && set_config("remote.origin.tagOpt=--no-tags")) {
res = error(_("could not disable tags in '%s'"), dir);
goto cleanup;
}
if (!full_clone &&
(res = run_git("sparse-checkout", "init", "--cone", NULL)))
goto cleanup;
if (set_recommended_config(0))
return error(_("could not configure '%s'"), dir);
if ((res = run_git("fetch", "--quiet",
show_progress ? "--progress" : "--no-progress",
"origin",
(tags ? NULL : "--no-tags"),
NULL))) {
warning(_("partial clone failed; attempting full clone"));
if (set_config("remote.origin.promisor") ||
set_config("remote.origin.partialCloneFilter")) {
res = error(_("could not configure for full clone"));
goto cleanup;
}
if ((res = run_git("fetch", "--quiet",
show_progress ? "--progress" : "--no-progress",
"origin", NULL)))
goto cleanup;
}
if ((res = set_config("branch.%s.remote=origin", branch)))
goto cleanup;
if ((res = set_config("branch.%s.merge=refs/heads/%s",
branch, branch)))
goto cleanup;
strbuf_reset(&buf);
strbuf_addf(&buf, "origin/%s", branch);
res = run_git("checkout", "-f", "-t", buf.buf, NULL);
if (res)
goto cleanup;
res = register_dir();
cleanup:
free(enlistment);
free(dir);
strbuf_release(&buf);
return res;
}
static int cmd_diagnose(int argc, const char **argv)
{
struct option options[] = {
OPT_END(),
};
const char * const usage[] = {
N_("scalar diagnose [<enlistment>]"),
NULL
};
struct strbuf diagnostics_root = STRBUF_INIT;
int res = 0;
argc = parse_options(argc, argv, NULL, options,
usage, 0);
setup_enlistment_directory(argc, argv, usage, options, &diagnostics_root);
strbuf_addstr(&diagnostics_root, "/.scalarDiagnostics");
res = run_git("diagnose", "--mode=all", "-s", "%Y%m%d_%H%M%S",
"-o", diagnostics_root.buf, NULL);
strbuf_release(&diagnostics_root);
return res;
}
static int cmd_list(int argc, const char **argv UNUSED)
{
if (argc != 1)
die(_("`scalar list` does not take arguments"));
if (run_git("config", "--global", "--get-all", "scalar.repo", NULL) < 0)
return -1;
return 0;
}
static int cmd_register(int argc, const char **argv)
{
struct option options[] = {
OPT_END(),
};
const char * const usage[] = {
N_("scalar register [<enlistment>]"),
NULL
};
argc = parse_options(argc, argv, NULL, options,
usage, 0);
setup_enlistment_directory(argc, argv, usage, options, NULL);
return register_dir();
}
scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03 21:34:16 +08:00
config: add ctx arg to config_fn_t Add a new "const struct config_context *ctx" arg to config_fn_t to hold additional information about the config iteration operation. config_context has a "struct key_value_info kvi" member that holds metadata about the config source being read (e.g. what kind of config source it is, the filename, etc). In this series, we're only interested in .kvi, so we could have just used "struct key_value_info" as an arg, but config_context makes it possible to add/adjust members in the future without changing the config_fn_t signature. We could also consider other ways of organizing the args (e.g. moving the config name and value into config_context or key_value_info), but in my experiments, the incremental benefit doesn't justify the added complexity (e.g. a config_fn_t will sometimes invoke another config_fn_t but with a different config value). In subsequent commits, the .kvi member will replace the global "struct config_reader" in config.c, making config iteration a global-free operation. It requires much more work for the machinery to provide meaningful values of .kvi, so for now, merely change the signature and call sites, pass NULL as a placeholder value, and don't rely on the arg in any meaningful way. Most of the changes are performed by contrib/coccinelle/config_fn_ctx.pending.cocci, which, for every config_fn_t: - Modifies the signature to accept "const struct config_context *ctx" - Passes "ctx" to any inner config_fn_t, if needed - Adds UNUSED attributes to "ctx", if needed Most config_fn_t instances are easily identified by seeing if they are called by the various config functions. Most of the remaining ones are manually named in the .cocci patch. Manual cleanups are still needed, but the majority of it is trivial; it's either adjusting config_fn_t that the .cocci patch didn't catch, or adding forward declarations of "struct config_context ctx" to make the signatures make sense. The non-trivial changes are in cases where we are invoking a config_fn_t outside of config machinery, and we now need to decide what value of "ctx" to pass. These cases are: - trace2/tr2_cfg.c:tr2_cfg_set_fl() This is indirectly called by git_config_set() so that the trace2 machinery can notice the new config values and update its settings using the tr2 config parsing function, i.e. tr2_cfg_cb(). - builtin/checkout.c:checkout_main() This calls git_xmerge_config() as a shorthand for parsing a CLI arg. This might be worth refactoring away in the future, since git_xmerge_config() can call git_default_config(), which can do much more than just parsing. Handle them by creating a KVI_INIT macro that initializes "struct key_value_info" to a reasonable default, and use that to construct the "ctx" arg. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-29 03:26:22 +08:00
static int get_scalar_repos(const char *key, const char *value,
const struct config_context *ctx UNUSED,
void *data)
{
struct string_list *list = data;
if (!strcmp(key, "scalar.repo"))
string_list_append(list, value);
return 0;
}
static int remove_deleted_enlistment(struct strbuf *path)
{
int res = 0;
strbuf_realpath_forgiving(path, path->buf, 1);
if (run_git("config", "--global",
"--unset", "--fixed-value",
"scalar.repo", path->buf, NULL) < 0)
res = -1;
if (run_git("config", "--global",
"--unset", "--fixed-value",
"maintenance.repo", path->buf, NULL) < 0)
res = -1;
return res;
}
static int cmd_reconfigure(int argc, const char **argv)
{
int all = 0;
struct option options[] = {
OPT_BOOL('a', "all", &all,
N_("reconfigure all registered enlistments")),
OPT_END(),
};
const char * const usage[] = {
N_("scalar reconfigure [--all | <enlistment>]"),
NULL
};
struct string_list scalar_repos = STRING_LIST_INIT_DUP;
int i, res = 0;
struct strbuf commondir = STRBUF_INIT, gitdir = STRBUF_INIT;
argc = parse_options(argc, argv, NULL, options,
usage, 0);
if (!all) {
setup_enlistment_directory(argc, argv, usage, options, NULL);
return set_recommended_config(1);
}
if (argc > 0)
usage_msg_opt(_("--all or <enlistment>, but not both"),
usage, options);
git_config(get_scalar_repos, &scalar_repos);
for (i = 0; i < scalar_repos.nr; i++) {
int succeeded = 0;
scalar: avoid segfault in reconfigure --all During the latest v2.45.0 update, 'scalar reconfigure --all' started to segfault on my machine. Breaking it down via the debugger, it was faulting on a NULL reference to the_hash_algo, which is a macro pointing to the_repository->hash_algo. In my case, this is due to one of my repositories having a detached HEAD, which requires get_oid_hex() to parse that the HEAD reference is valid. Another way to cause a failure is to use the "includeIf.onbranch" config key, which will lead to a BUG() statement. My first inclination was to try to refactor cmd_reconfigure() to execute 'git for-each-repo' instead of this loop. In addition to the difficulty of executing 'scalar reconfigure' within 'git for-each-repo', it would be difficult to perform the clean-up logic for non-existent repos if we relied on that child process. Instead, I chose to move the temporary repo to be within the loop and reinstate the_repository to its old value after we are done performing logic on the current array item. Add tests to t9210-scalar.sh to test 'scalar reconfigure --all' with multiple registered repos. There are two different ways that the old use of the_repository could trigger bugs. These issues are being solved independently to be more careful about the_repository being uninitialized, but the change in this patch around the use of the_repository is still a good safety precaution. Co-authored-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-08 08:05:49 +08:00
struct repository *old_repo, r = { NULL };
const char *dir = scalar_repos.items[i].string;
strbuf_reset(&commondir);
strbuf_reset(&gitdir);
if (chdir(dir) < 0) {
struct strbuf buf = STRBUF_INIT;
if (errno != ENOENT) {
warning_errno(_("could not switch to '%s'"), dir);
goto loop_end;
}
strbuf_addstr(&buf, dir);
if (remove_deleted_enlistment(&buf))
error(_("could not remove stale "
"scalar.repo '%s'"), dir);
else {
warning(_("removed stale scalar.repo '%s'"),
dir);
succeeded = 1;
}
strbuf_release(&buf);
goto loop_end;
}
switch (discover_git_directory_reason(&commondir, &gitdir)) {
case GIT_DIR_INVALID_OWNERSHIP:
warning(_("repository at '%s' has different owner"), dir);
goto loop_end;
case GIT_DIR_INVALID_GITFILE:
case GIT_DIR_INVALID_FORMAT:
warning(_("repository at '%s' has a format issue"), dir);
goto loop_end;
case GIT_DIR_DISCOVERED:
succeeded = 1;
break;
default:
warning(_("repository not found in '%s'"), dir);
break;
}
git_config_clear();
scalar: avoid segfault in reconfigure --all During the latest v2.45.0 update, 'scalar reconfigure --all' started to segfault on my machine. Breaking it down via the debugger, it was faulting on a NULL reference to the_hash_algo, which is a macro pointing to the_repository->hash_algo. In my case, this is due to one of my repositories having a detached HEAD, which requires get_oid_hex() to parse that the HEAD reference is valid. Another way to cause a failure is to use the "includeIf.onbranch" config key, which will lead to a BUG() statement. My first inclination was to try to refactor cmd_reconfigure() to execute 'git for-each-repo' instead of this loop. In addition to the difficulty of executing 'scalar reconfigure' within 'git for-each-repo', it would be difficult to perform the clean-up logic for non-existent repos if we relied on that child process. Instead, I chose to move the temporary repo to be within the loop and reinstate the_repository to its old value after we are done performing logic on the current array item. Add tests to t9210-scalar.sh to test 'scalar reconfigure --all' with multiple registered repos. There are two different ways that the old use of the_repository could trigger bugs. These issues are being solved independently to be more careful about the_repository being uninitialized, but the change in this patch around the use of the_repository is still a good safety precaution. Co-authored-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-08 08:05:49 +08:00
if (repo_init(&r, gitdir.buf, commondir.buf))
goto loop_end;
old_repo = the_repository;
the_repository = &r;
if (set_recommended_config(1) >= 0)
succeeded = 1;
scalar: avoid segfault in reconfigure --all During the latest v2.45.0 update, 'scalar reconfigure --all' started to segfault on my machine. Breaking it down via the debugger, it was faulting on a NULL reference to the_hash_algo, which is a macro pointing to the_repository->hash_algo. In my case, this is due to one of my repositories having a detached HEAD, which requires get_oid_hex() to parse that the HEAD reference is valid. Another way to cause a failure is to use the "includeIf.onbranch" config key, which will lead to a BUG() statement. My first inclination was to try to refactor cmd_reconfigure() to execute 'git for-each-repo' instead of this loop. In addition to the difficulty of executing 'scalar reconfigure' within 'git for-each-repo', it would be difficult to perform the clean-up logic for non-existent repos if we relied on that child process. Instead, I chose to move the temporary repo to be within the loop and reinstate the_repository to its old value after we are done performing logic on the current array item. Add tests to t9210-scalar.sh to test 'scalar reconfigure --all' with multiple registered repos. There are two different ways that the old use of the_repository could trigger bugs. These issues are being solved independently to be more careful about the_repository being uninitialized, but the change in this patch around the use of the_repository is still a good safety precaution. Co-authored-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-08 08:05:49 +08:00
the_repository = old_repo;
repo_clear(&r);
scalar: avoid segfault in reconfigure --all During the latest v2.45.0 update, 'scalar reconfigure --all' started to segfault on my machine. Breaking it down via the debugger, it was faulting on a NULL reference to the_hash_algo, which is a macro pointing to the_repository->hash_algo. In my case, this is due to one of my repositories having a detached HEAD, which requires get_oid_hex() to parse that the HEAD reference is valid. Another way to cause a failure is to use the "includeIf.onbranch" config key, which will lead to a BUG() statement. My first inclination was to try to refactor cmd_reconfigure() to execute 'git for-each-repo' instead of this loop. In addition to the difficulty of executing 'scalar reconfigure' within 'git for-each-repo', it would be difficult to perform the clean-up logic for non-existent repos if we relied on that child process. Instead, I chose to move the temporary repo to be within the loop and reinstate the_repository to its old value after we are done performing logic on the current array item. Add tests to t9210-scalar.sh to test 'scalar reconfigure --all' with multiple registered repos. There are two different ways that the old use of the_repository could trigger bugs. These issues are being solved independently to be more careful about the_repository being uninitialized, but the change in this patch around the use of the_repository is still a good safety precaution. Co-authored-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-08 08:05:49 +08:00
if (toggle_maintenance(1) >= 0)
succeeded = 1;
loop_end:
if (!succeeded) {
res = -1;
warning(_("to unregister this repository from Scalar, run\n"
"\tgit config --global --unset --fixed-value scalar.repo \"%s\""),
dir);
}
}
string_list_clear(&scalar_repos, 1);
strbuf_release(&commondir);
strbuf_release(&gitdir);
return res;
}
static int cmd_run(int argc, const char **argv)
{
struct option options[] = {
OPT_END(),
};
struct {
const char *arg, *task;
} tasks[] = {
{ "config", NULL },
{ "commit-graph", "commit-graph" },
{ "fetch", "prefetch" },
{ "loose-objects", "loose-objects" },
{ "pack-files", "incremental-repack" },
{ NULL, NULL }
};
struct strbuf buf = STRBUF_INIT;
const char *usagestr[] = { NULL, NULL };
int i;
strbuf_addstr(&buf, N_("scalar run <task> [<enlistment>]\nTasks:\n"));
for (i = 0; tasks[i].arg; i++)
strbuf_addf(&buf, "\t%s\n", tasks[i].arg);
usagestr[0] = buf.buf;
argc = parse_options(argc, argv, NULL, options,
usagestr, 0);
if (!argc)
usage_with_options(usagestr, options);
if (!strcmp("all", argv[0])) {
i = -1;
} else {
for (i = 0; tasks[i].arg && strcmp(tasks[i].arg, argv[0]); i++)
; /* keep looking for the task */
if (i > 0 && !tasks[i].arg) {
error(_("no such task: '%s'"), argv[0]);
usage_with_options(usagestr, options);
}
}
argc--;
argv++;
setup_enlistment_directory(argc, argv, usagestr, options, NULL);
strbuf_release(&buf);
if (i == 0)
return register_dir();
if (i > 0)
return run_git("maintenance", "run",
"--task", tasks[i].task, NULL);
if (register_dir())
return -1;
for (i = 1; tasks[i].arg; i++)
if (run_git("maintenance", "run",
"--task", tasks[i].task, NULL))
return -1;
return 0;
}
static int cmd_unregister(int argc, const char **argv)
{
struct option options[] = {
OPT_END(),
};
const char * const usage[] = {
N_("scalar unregister [<enlistment>]"),
NULL
};
argc = parse_options(argc, argv, NULL, options,
usage, 0);
/*
* Be forgiving when the enlistment or worktree does not even exist any
* longer; This can be the case if a user deleted the worktree by
* mistake and _still_ wants to unregister the thing.
*/
if (argc == 1) {
struct strbuf src_path = STRBUF_INIT, workdir_path = STRBUF_INIT;
strbuf_addf(&src_path, "%s/src/.git", argv[0]);
strbuf_addf(&workdir_path, "%s/.git", argv[0]);
if (!is_directory(src_path.buf) && !is_directory(workdir_path.buf)) {
/* remove possible matching registrations */
int res = -1;
strbuf_strip_suffix(&src_path, "/.git");
res = remove_deleted_enlistment(&src_path) && res;
strbuf_strip_suffix(&workdir_path, "/.git");
res = remove_deleted_enlistment(&workdir_path) && res;
strbuf_release(&src_path);
strbuf_release(&workdir_path);
return res;
}
strbuf_release(&src_path);
strbuf_release(&workdir_path);
}
setup_enlistment_directory(argc, argv, usage, options, NULL);
return unregister_dir();
}
static int cmd_delete(int argc, const char **argv)
{
char *cwd = xgetcwd();
struct option options[] = {
OPT_END(),
};
const char * const usage[] = {
N_("scalar delete <enlistment>"),
NULL
};
struct strbuf enlistment = STRBUF_INIT;
int res = 0;
argc = parse_options(argc, argv, NULL, options,
usage, 0);
if (argc != 1)
usage_with_options(usage, options);
setup_enlistment_directory(argc, argv, usage, options, &enlistment);
if (dir_inside_of(cwd, enlistment.buf) >= 0)
res = error(_("refusing to delete current working directory"));
else {
close_object_store(the_repository->objects);
res = delete_enlistment(&enlistment);
}
strbuf_release(&enlistment);
free(cwd);
return res;
}
static int cmd_help(int argc, const char **argv)
{
struct option options[] = {
OPT_END(),
};
const char * const usage[] = {
"scalar help",
NULL
};
argc = parse_options(argc, argv, NULL, options,
usage, 0);
if (argc != 0)
usage_with_options(usage, options);
return run_git("help", "scalar", NULL);
}
static int cmd_version(int argc, const char **argv)
{
int verbose = 0, build_options = 0;
struct option options[] = {
OPT__VERBOSE(&verbose, N_("include Git version")),
OPT_BOOL(0, "build-options", &build_options,
N_("include Git's build options")),
OPT_END(),
};
const char * const usage[] = {
N_("scalar verbose [-v | --verbose] [--build-options]"),
NULL
};
struct strbuf buf = STRBUF_INIT;
argc = parse_options(argc, argv, NULL, options,
usage, 0);
if (argc != 0)
usage_with_options(usage, options);
get_version_info(&buf, build_options);
fprintf(stderr, "%s\n", buf.buf);
strbuf_release(&buf);
return 0;
}
scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03 21:34:16 +08:00
static struct {
const char *name;
int (*fn)(int, const char **);
} builtins[] = {
{ "clone", cmd_clone },
{ "list", cmd_list },
{ "register", cmd_register },
{ "unregister", cmd_unregister },
{ "run", cmd_run },
{ "reconfigure", cmd_reconfigure },
{ "delete", cmd_delete },
{ "help", cmd_help },
{ "version", cmd_version },
{ "diagnose", cmd_diagnose },
scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03 21:34:16 +08:00
{ NULL, NULL},
};
int cmd_main(int argc, const char **argv)
{
struct strbuf scalar_usage = STRBUF_INIT;
int i;
while (argc > 1 && *argv[1] == '-') {
if (!strcmp(argv[1], "-C")) {
if (argc < 3)
die(_("-C requires a <directory>"));
if (chdir(argv[2]) < 0)
die_errno(_("could not change to '%s'"),
argv[2]);
argc -= 2;
argv += 2;
} else if (!strcmp(argv[1], "-c")) {
if (argc < 3)
die(_("-c requires a <key>=<value> argument"));
git_config_push_parameter(argv[2]);
argc -= 2;
argv += 2;
} else
break;
}
scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03 21:34:16 +08:00
if (argc > 1) {
argv++;
argc--;
for (i = 0; builtins[i].name; i++)
if (!strcmp(builtins[i].name, argv[0]))
return !!builtins[i].fn(argc, argv);
}
strbuf_addstr(&scalar_usage,
N_("scalar [-C <directory>] [-c <key>=<value>] "
"<command> [<options>]\n\nCommands:\n"));
scalar: create a rudimentary executable The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03 21:34:16 +08:00
for (i = 0; builtins[i].name; i++)
strbuf_addf(&scalar_usage, "\t%s\n", builtins[i].name);
usage(scalar_usage.buf);
}