2022-08-09 21:11:40 +08:00
|
|
|
#ifndef BUNDLE_URI_H
|
|
|
|
#define BUNDLE_URI_H
|
|
|
|
|
bundle-uri: create bundle_list struct and helpers
It will likely be rare where a user uses a single bundle URI and expects
that URI to point to a bundle. Instead, that URI will likely be a list
of bundles provided in some format. Alternatively, the Git server could
advertise a list of bundles.
In anticipation of these two ways of advertising multiple bundles,
create a data structure that represents such a list. This will be
populated using a common API, but for now focus on what data can be
represented.
Each list contains a number of remote_bundle_info structs. These contain
an 'id' that is used to uniquely identify them in the list, and also a
'uri' that contains the location of its data. Finally, there is a strbuf
containing the filename used when Git downloads the contents to disk.
The list itself stores these remote_bundle_info structs in a hashtable
using 'id' as the key. The order of the structs in the input is
considered unimportant, but future modifications to the format and these
data structures will place ordering possibilities on the set. The list
also has a few "global" properties, including the version (used when
parsing the list) and the mode. The mode is one of these two options:
1. BUNDLE_MODE_ALL: all listed URIs are intended to be combined
together. The client should download all of the advertised data to
have a complete copy of the data.
2. BUNDLE_MODE_ANY: any one listed item is sufficient to have a complete
copy of the data. The client can choose arbitrarily from these
options. In the future, the client may use pings to find the closest
URI among geodistributed replicas, or use some other heuristic
information added to the format.
This API is currently unused, but will soon be expanded with parsing
logic and then be consumed by the bundle URI download logic.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12 20:52:29 +08:00
|
|
|
#include "hashmap.h"
|
|
|
|
#include "strbuf.h"
|
|
|
|
|
2022-12-22 23:14:07 +08:00
|
|
|
struct packet_reader;
|
2022-08-09 21:11:40 +08:00
|
|
|
struct repository;
|
bundle-uri: create bundle_list struct and helpers
It will likely be rare where a user uses a single bundle URI and expects
that URI to point to a bundle. Instead, that URI will likely be a list
of bundles provided in some format. Alternatively, the Git server could
advertise a list of bundles.
In anticipation of these two ways of advertising multiple bundles,
create a data structure that represents such a list. This will be
populated using a common API, but for now focus on what data can be
represented.
Each list contains a number of remote_bundle_info structs. These contain
an 'id' that is used to uniquely identify them in the list, and also a
'uri' that contains the location of its data. Finally, there is a strbuf
containing the filename used when Git downloads the contents to disk.
The list itself stores these remote_bundle_info structs in a hashtable
using 'id' as the key. The order of the structs in the input is
considered unimportant, but future modifications to the format and these
data structures will place ordering possibilities on the set. The list
also has a few "global" properties, including the version (used when
parsing the list) and the mode. The mode is one of these two options:
1. BUNDLE_MODE_ALL: all listed URIs are intended to be combined
together. The client should download all of the advertised data to
have a complete copy of the data.
2. BUNDLE_MODE_ANY: any one listed item is sufficient to have a complete
copy of the data. The client can choose arbitrarily from these
options. In the future, the client may use pings to find the closest
URI among geodistributed replicas, or use some other heuristic
information added to the format.
This API is currently unused, but will soon be expanded with parsing
logic and then be consumed by the bundle URI download logic.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12 20:52:29 +08:00
|
|
|
struct string_list;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* The remote_bundle_info struct contains information for a single bundle
|
|
|
|
* URI. This may be initialized simply by a given URI or might have
|
|
|
|
* additional metadata associated with it if the bundle was advertised by
|
|
|
|
* a bundle list.
|
|
|
|
*/
|
|
|
|
struct remote_bundle_info {
|
|
|
|
struct hashmap_entry ent;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* The 'id' is a name given to the bundle for reference
|
|
|
|
* by other bundle infos.
|
|
|
|
*/
|
|
|
|
char *id;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* The 'uri' is the location of the remote bundle so
|
|
|
|
* it can be downloaded on-demand. This will be NULL
|
|
|
|
* if there was no table of contents.
|
|
|
|
*/
|
|
|
|
char *uri;
|
2022-10-12 20:52:36 +08:00
|
|
|
|
|
|
|
/**
|
|
|
|
* If the bundle has been downloaded, then 'file' is a
|
|
|
|
* filename storing its contents. Otherwise, 'file' is
|
|
|
|
* NULL.
|
|
|
|
*/
|
|
|
|
char *file;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* If the bundle has been unbundled successfully, then
|
|
|
|
* this boolean is true.
|
|
|
|
*/
|
|
|
|
unsigned unbundled:1;
|
2023-01-31 21:29:13 +08:00
|
|
|
|
|
|
|
/**
|
|
|
|
* If the bundle is part of a list with the creationToken
|
|
|
|
* heuristic, then we use this member for sorting the bundles.
|
|
|
|
*/
|
|
|
|
uint64_t creationToken;
|
bundle-uri: create bundle_list struct and helpers
It will likely be rare where a user uses a single bundle URI and expects
that URI to point to a bundle. Instead, that URI will likely be a list
of bundles provided in some format. Alternatively, the Git server could
advertise a list of bundles.
In anticipation of these two ways of advertising multiple bundles,
create a data structure that represents such a list. This will be
populated using a common API, but for now focus on what data can be
represented.
Each list contains a number of remote_bundle_info structs. These contain
an 'id' that is used to uniquely identify them in the list, and also a
'uri' that contains the location of its data. Finally, there is a strbuf
containing the filename used when Git downloads the contents to disk.
The list itself stores these remote_bundle_info structs in a hashtable
using 'id' as the key. The order of the structs in the input is
considered unimportant, but future modifications to the format and these
data structures will place ordering possibilities on the set. The list
also has a few "global" properties, including the version (used when
parsing the list) and the mode. The mode is one of these two options:
1. BUNDLE_MODE_ALL: all listed URIs are intended to be combined
together. The client should download all of the advertised data to
have a complete copy of the data.
2. BUNDLE_MODE_ANY: any one listed item is sufficient to have a complete
copy of the data. The client can choose arbitrarily from these
options. In the future, the client may use pings to find the closest
URI among geodistributed replicas, or use some other heuristic
information added to the format.
This API is currently unused, but will soon be expanded with parsing
logic and then be consumed by the bundle URI download logic.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12 20:52:29 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
#define REMOTE_BUNDLE_INFO_INIT { 0 }
|
|
|
|
|
|
|
|
enum bundle_list_mode {
|
|
|
|
BUNDLE_MODE_NONE = 0,
|
|
|
|
BUNDLE_MODE_ALL,
|
|
|
|
BUNDLE_MODE_ANY
|
|
|
|
};
|
|
|
|
|
2023-01-31 21:29:12 +08:00
|
|
|
enum bundle_list_heuristic {
|
|
|
|
BUNDLE_HEURISTIC_NONE = 0,
|
|
|
|
BUNDLE_HEURISTIC_CREATIONTOKEN,
|
|
|
|
|
|
|
|
/* Must be last. */
|
|
|
|
BUNDLE_HEURISTIC__COUNT
|
|
|
|
};
|
|
|
|
|
bundle-uri: create bundle_list struct and helpers
It will likely be rare where a user uses a single bundle URI and expects
that URI to point to a bundle. Instead, that URI will likely be a list
of bundles provided in some format. Alternatively, the Git server could
advertise a list of bundles.
In anticipation of these two ways of advertising multiple bundles,
create a data structure that represents such a list. This will be
populated using a common API, but for now focus on what data can be
represented.
Each list contains a number of remote_bundle_info structs. These contain
an 'id' that is used to uniquely identify them in the list, and also a
'uri' that contains the location of its data. Finally, there is a strbuf
containing the filename used when Git downloads the contents to disk.
The list itself stores these remote_bundle_info structs in a hashtable
using 'id' as the key. The order of the structs in the input is
considered unimportant, but future modifications to the format and these
data structures will place ordering possibilities on the set. The list
also has a few "global" properties, including the version (used when
parsing the list) and the mode. The mode is one of these two options:
1. BUNDLE_MODE_ALL: all listed URIs are intended to be combined
together. The client should download all of the advertised data to
have a complete copy of the data.
2. BUNDLE_MODE_ANY: any one listed item is sufficient to have a complete
copy of the data. The client can choose arbitrarily from these
options. In the future, the client may use pings to find the closest
URI among geodistributed replicas, or use some other heuristic
information added to the format.
This API is currently unused, but will soon be expanded with parsing
logic and then be consumed by the bundle URI download logic.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12 20:52:29 +08:00
|
|
|
/**
|
|
|
|
* A bundle_list contains an unordered set of remote_bundle_info structs,
|
|
|
|
* as well as information about the bundle listing, such as version and
|
|
|
|
* mode.
|
|
|
|
*/
|
|
|
|
struct bundle_list {
|
|
|
|
int version;
|
|
|
|
enum bundle_list_mode mode;
|
|
|
|
struct hashmap bundles;
|
2022-12-22 23:14:15 +08:00
|
|
|
|
|
|
|
/**
|
|
|
|
* The baseURI of a bundle_list is the URI that provided the list.
|
|
|
|
*
|
|
|
|
* In the case of the 'bundle-uri' protocol v2 command, the base
|
|
|
|
* URI is the URI of the Git remote.
|
|
|
|
*
|
|
|
|
* Otherwise, the bundle list was downloaded over HTTP from some
|
|
|
|
* known URI. 'baseURI' is set to that value.
|
|
|
|
*
|
|
|
|
* The baseURI is used as the base for any relative URIs
|
|
|
|
* advertised by the bundle list at that location.
|
|
|
|
*/
|
|
|
|
char *baseURI;
|
2023-01-31 21:29:12 +08:00
|
|
|
|
|
|
|
/**
|
|
|
|
* A list can have a heuristic, which helps reduce the number of
|
|
|
|
* downloaded bundles.
|
|
|
|
*/
|
|
|
|
enum bundle_list_heuristic heuristic;
|
bundle-uri: create bundle_list struct and helpers
It will likely be rare where a user uses a single bundle URI and expects
that URI to point to a bundle. Instead, that URI will likely be a list
of bundles provided in some format. Alternatively, the Git server could
advertise a list of bundles.
In anticipation of these two ways of advertising multiple bundles,
create a data structure that represents such a list. This will be
populated using a common API, but for now focus on what data can be
represented.
Each list contains a number of remote_bundle_info structs. These contain
an 'id' that is used to uniquely identify them in the list, and also a
'uri' that contains the location of its data. Finally, there is a strbuf
containing the filename used when Git downloads the contents to disk.
The list itself stores these remote_bundle_info structs in a hashtable
using 'id' as the key. The order of the structs in the input is
considered unimportant, but future modifications to the format and these
data structures will place ordering possibilities on the set. The list
also has a few "global" properties, including the version (used when
parsing the list) and the mode. The mode is one of these two options:
1. BUNDLE_MODE_ALL: all listed URIs are intended to be combined
together. The client should download all of the advertised data to
have a complete copy of the data.
2. BUNDLE_MODE_ANY: any one listed item is sufficient to have a complete
copy of the data. The client can choose arbitrarily from these
options. In the future, the client may use pings to find the closest
URI among geodistributed replicas, or use some other heuristic
information added to the format.
This API is currently unused, but will soon be expanded with parsing
logic and then be consumed by the bundle URI download logic.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-12 20:52:29 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
void init_bundle_list(struct bundle_list *list);
|
|
|
|
void clear_bundle_list(struct bundle_list *list);
|
|
|
|
|
|
|
|
typedef int (*bundle_iterator)(struct remote_bundle_info *bundle,
|
|
|
|
void *data);
|
|
|
|
|
|
|
|
int for_all_bundles_in_list(struct bundle_list *list,
|
|
|
|
bundle_iterator iter,
|
|
|
|
void *data);
|
2022-08-09 21:11:40 +08:00
|
|
|
|
2022-10-12 20:52:32 +08:00
|
|
|
struct FILE;
|
|
|
|
void print_bundle_list(FILE *fp, struct bundle_list *list);
|
|
|
|
|
2022-10-12 20:52:33 +08:00
|
|
|
/**
|
|
|
|
* A bundle URI may point to a bundle list where the key=value
|
|
|
|
* pairs are provided in config file format. This method is
|
|
|
|
* exposed publicly for testing purposes.
|
|
|
|
*/
|
|
|
|
int bundle_uri_parse_config_format(const char *uri,
|
|
|
|
const char *filename,
|
|
|
|
struct bundle_list *list);
|
|
|
|
|
2022-08-09 21:11:40 +08:00
|
|
|
/**
|
|
|
|
* Fetch data from the given 'uri' and unbundle the bundle data found
|
|
|
|
* based on that information.
|
|
|
|
*
|
|
|
|
* Returns non-zero if no bundle information is found at the given 'uri'.
|
2023-01-31 21:29:15 +08:00
|
|
|
*
|
|
|
|
* If the pointer 'has_heuristic' is non-NULL, then the value it points to
|
|
|
|
* will be set to be non-zero if and only if the fetched list has a
|
|
|
|
* heuristic value. Such a value indicates that the list was designed for
|
|
|
|
* incremental fetches.
|
2022-08-09 21:11:40 +08:00
|
|
|
*/
|
2023-01-31 21:29:15 +08:00
|
|
|
int fetch_bundle_uri(struct repository *r, const char *uri,
|
|
|
|
int *has_heuristic);
|
2022-08-09 21:11:40 +08:00
|
|
|
|
2022-12-22 23:14:16 +08:00
|
|
|
/**
|
|
|
|
* Given a bundle list that was already advertised (likely by the
|
|
|
|
* bundle-uri protocol v2 verb) at the given uri, fetch and unbundle the
|
|
|
|
* bundles according to the bundle strategy of that list.
|
|
|
|
*
|
|
|
|
* It is expected that the given 'list' is initialized, including its
|
|
|
|
* 'baseURI' value.
|
|
|
|
*
|
|
|
|
* Returns non-zero if there was an error trying to download the list
|
|
|
|
* or any of its advertised bundles.
|
|
|
|
*/
|
|
|
|
int fetch_bundle_list(struct repository *r,
|
|
|
|
struct bundle_list *list);
|
|
|
|
|
2022-12-22 23:14:07 +08:00
|
|
|
/**
|
|
|
|
* API for serve.c.
|
|
|
|
*/
|
|
|
|
int bundle_uri_advertise(struct repository *r, struct strbuf *value);
|
|
|
|
int bundle_uri_command(struct repository *r, struct packet_reader *request);
|
|
|
|
|
2022-10-12 20:52:31 +08:00
|
|
|
/**
|
|
|
|
* General API for {transport,connect}.c etc.
|
|
|
|
*/
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Parse a "key=value" packet line from the bundle-uri verb.
|
|
|
|
*
|
|
|
|
* Returns 0 on success and non-zero on error.
|
|
|
|
*/
|
|
|
|
int bundle_uri_parse_line(struct bundle_list *list,
|
|
|
|
const char *line);
|
|
|
|
|
2022-08-09 21:11:40 +08:00
|
|
|
#endif
|