mirrors/git
0
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-27 12:03:55 +08:00
Code Issues Packages Projects Releases Wiki Activity
bf1feb9e53
git/delta-islands.h

20 lines
602 B
C
Raw Normal View History

Add delta-islands.{c,h} Hosting providers that allow users to "fork" existing repos want those forks to share as much disk space as possible. Alternates are an existing solution to keep all the objects from all the forks into a unique central repo, but this can have some drawbacks. Especially when packing the central repo, deltas will be created between objects from different forks. This can make cloning or fetching a fork much slower and much more CPU intensive as Git might have to compute new deltas for many objects to avoid sending objects from a different fork. Because the inefficiency primarily arises when an object is deltified against another object that does not exist in the same fork, we partition objects into sets that appear in the same fork, and define "delta islands". When finding delta base, we do not allow an object outside the same island to be considered as its base. So "delta islands" is a way to store objects from different forks in the same repo and packfile without having deltas between objects from different forks. This patch implements the delta islands mechanism in "delta-islands.{c,h}", but does not yet make use of it. A few new fields are added in 'struct object_entry' in "pack-objects.h" though. The documentation will follow in a patch that actually uses delta islands in "builtin/pack-objects.c". Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-16 14:13:07 +08:00
#ifndef DELTA_ISLANDS_H
#define DELTA_ISLANDS_H
delta-islands.c: remove the_repository references Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-10 13:49:03 +08:00
struct commit;
delta-islands.h: add missing forward declarations (hdr-check) Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-19 08:14:30 +08:00
struct object_id;
struct packing_data;
delta-islands.c: remove the_repository references Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-10 13:49:03 +08:00
struct repository;
delta-islands.h: add missing forward declarations (hdr-check) Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-19 08:14:30 +08:00
Add delta-islands.{c,h} Hosting providers that allow users to "fork" existing repos want those forks to share as much disk space as possible. Alternates are an existing solution to keep all the objects from all the forks into a unique central repo, but this can have some drawbacks. Especially when packing the central repo, deltas will be created between objects from different forks. This can make cloning or fetching a fork much slower and much more CPU intensive as Git might have to compute new deltas for many objects to avoid sending objects from a different fork. Because the inefficiency primarily arises when an object is deltified against another object that does not exist in the same fork, we partition objects into sets that appear in the same fork, and define "delta islands". When finding delta base, we do not allow an object outside the same island to be considered as its base. So "delta islands" is a way to store objects from different forks in the same repo and packfile without having deltas between objects from different forks. This patch implements the delta islands mechanism in "delta-islands.{c,h}", but does not yet make use of it. A few new fields are added in 'struct object_entry' in "pack-objects.h" though. The documentation will follow in a patch that actually uses delta islands in "builtin/pack-objects.c". Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-16 14:13:07 +08:00
int island_delta_cmp(const struct object_id *a, const struct object_id *b);
int in_same_island(const struct object_id *, const struct object_id *);
delta-islands.c: remove the_repository references Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-10 13:49:03 +08:00
void resolve_tree_islands(struct repository *r,
int progress,
struct packing_data *to_pack);
delta-islands: respect progress flag The delta island code always prints "Marked %d islands", even if progress has been suppressed with --no-progress or by sending stderr to a non-tty. Let's pass a progress boolean to load_delta_islands(). We already do the same thing for the progress meter in resolve_tree_islands(). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-20 16:58:32 +08:00
void load_delta_islands(struct repository *r, int progress);
Add delta-islands.{c,h} Hosting providers that allow users to "fork" existing repos want those forks to share as much disk space as possible. Alternates are an existing solution to keep all the objects from all the forks into a unique central repo, but this can have some drawbacks. Especially when packing the central repo, deltas will be created between objects from different forks. This can make cloning or fetching a fork much slower and much more CPU intensive as Git might have to compute new deltas for many objects to avoid sending objects from a different fork. Because the inefficiency primarily arises when an object is deltified against another object that does not exist in the same fork, we partition objects into sets that appear in the same fork, and define "delta islands". When finding delta base, we do not allow an object outside the same island to be considered as its base. So "delta islands" is a way to store objects from different forks in the same repo and packfile without having deltas between objects from different forks. This patch implements the delta islands mechanism in "delta-islands.{c,h}", but does not yet make use of it. A few new fields are added in 'struct object_entry' in "pack-objects.h" though. The documentation will follow in a patch that actually uses delta islands in "builtin/pack-objects.c". Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-16 14:13:07 +08:00
void propagate_island_marks(struct commit *commit);
int compute_pack_layers(struct packing_data *to_pack);
delta-islands: free island_marks and bitmaps On my mirror of linux.git forkgroup with 780 islands, this saves nearly 4G of heap memory in pack-objects. This savings only benefits delta island users of pack bitmaps, as the process would otherwise be exiting anyways. However, there's probably not many delta island users, but the majority of delta island users would also be pack bitmaps users. Signed-off-by: Eric Wong <e@80x24.org> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-04 07:44:30 +08:00
void free_island_marks(void);
Add delta-islands.{c,h} Hosting providers that allow users to "fork" existing repos want those forks to share as much disk space as possible. Alternates are an existing solution to keep all the objects from all the forks into a unique central repo, but this can have some drawbacks. Especially when packing the central repo, deltas will be created between objects from different forks. This can make cloning or fetching a fork much slower and much more CPU intensive as Git might have to compute new deltas for many objects to avoid sending objects from a different fork. Because the inefficiency primarily arises when an object is deltified against another object that does not exist in the same fork, we partition objects into sets that appear in the same fork, and define "delta islands". When finding delta base, we do not allow an object outside the same island to be considered as its base. So "delta islands" is a way to store objects from different forks in the same repo and packfile without having deltas between objects from different forks. This patch implements the delta islands mechanism in "delta-islands.{c,h}", but does not yet make use of it. A few new fields are added in 'struct object_entry' in "pack-objects.h" though. The documentation will follow in a patch that actually uses delta islands in "builtin/pack-objects.c". Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-16 14:13:07 +08:00
#endif /* DELTA_ISLANDS_H */
Reference in New Issue Copy Permalink