buildroot/support/download/svn
Yann E. MORIN 7dd27cbe5b support/download: add support to exclude svn externals
Like git which can have submodules, subversion can have externals. The
default behaviour for subversion is to retrieve all the externals,
unless told otherwise.

For some repositories, the externals may be huge (e.g. a dataset or some
assets) and may not be required for building the package. In such a
case, retrieving the externals is both a waste of network bandwitdh and
time, and a waste of disk storage.

Like for git submodules and git lfs, add an option that packages can set
to specify whether they want externals or not.

Since we've so far been retrieving externals, we keep that the default,
and packages can opt-out (rather than the opt-in for git submodules or
git lfs).

We must only set it when the package is actually hosted on svn, to avoid
passing -r when the package is not hosted by svn; otherwise, -r would
also be passed e.g. to a git-hosted package, triggering the download of
git submodules even when they are not requested. We need to do so,
because we have a default value, which we usually do not have in other
download options.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-08-06 16:35:52 +02:00

90 lines
3.0 KiB
Bash
Executable File

#!/usr/bin/env bash
# NOTE: if the output of this backend has to change (e.g. we change what gets
# included in the archive, or we change the format of the archive (e.g. tar
# options, compression ratio or method)), we MUST update the format version
# in the variable BR_FTM_VERSION_svn, in package/pkg-download.mk.
# We want to catch any unexpected failure, and exit immediately
set -e
# Download helper for svn, to be called from the download wrapper script
#
# Options:
# -q Be quiet.
# -o FILE Generate archive in FILE.
# -u URI Checkout from repository at URI.
# -c REV Use revision REV.
# -n NAME Use basename NAME.
# -r Recursive, i.e. use externals
#
# Environment:
# SVN : the svn command to call
# shellcheck disable=SC1090 # Only provides mk_tar_gz()
. "${0%/*}/helpers"
quiet=
externals=--ignore-externals
while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
case "${OPT}" in
q) quiet=-q;;
o) output="${OPTARG}";;
u) uri="${OPTARG}";;
c) rev="${OPTARG}";;
n) basename="${OPTARG}";;
r) externals=;;
:) printf "option '%s' expects a mandatory argument\n" "${OPTARG}"; exit 1;;
\?) printf "unknown option '%s'\n" "${OPTARG}" >&2; exit 1;;
esac
done
shift $((OPTIND-1)) # Get rid of our options
# Caller needs to single-quote its arguments to prevent them from
# being expanded a second time (in case there are spaces in them)
_svn() {
if [ -z "${quiet}" ]; then
printf '%s ' "${SVN}" "${@}"; printf '\n'
fi
_plain_svn "$@"
}
# Note: please keep command below aligned with what is printed above
_plain_svn() {
# shellcheck disable=SC2086 # We want word-splitting for SVN
eval ${SVN} "${@}"
}
# shellcheck disable=SC2086 # externals and quiet may be empty
_svn export --ignore-keywords ${quiet} ${externals} "${@}" "'${uri}@${rev}'" "'${basename}'"
# For 'svn info', we only need the credentials, if any; other options
# would be invalid, as they are intended for 'svn export'.
# We can also consume the positional parameters, as we'll no longer
# be calling any other remote-reaching svn command.
creds=
while [ ${#} -gt 0 ]; do
case "${1}" in
--username=*) creds+=" ${1}"; shift;;
--password=*) creds+=" ${1}"; shift;;
--username) creds+=" ${1} ${2}"; shift 2;;
--password) creds+=" ${1} ${2}"; shift 2;;
*) shift;;
esac
done
# Get the date of the revision, to generate reproducible archives.
# The output format is YYYY-MM-DDTHH:MM:SS.mmmuuuZ (i.e. always in the
# UTC timezone), which we can feed as-is to the --mtime option for tar.
# In case there is a redirection (e.g. http -> https), just keep the
# last line (svn outputs everything on stdout)
# shellcheck disable=SC2086 # creds may be empty
date="$( _plain_svn info ${creds} "'${uri}@${rev}'" \
|sed -r -e '/^Last Changed Date: /!d; s///'
)"
# Generate the archive.
# We did a 'svn export' above, so it's not a working copy (there is no .svn
# directory or file to ignore).
mk_tar_gz "${basename}" "${basename}" "${date}" "${output}"