package/pkg-generic.mk: local and override support in legal-info

Sources of packages which are using the 'local' _METHOD_SITE or
which declare an _OVERRIDE_SRC_DIR are not archived when generating
the legal-info target.
This is problematic since those packages are the most likely to
require being published, because it's highly probable that their
sources have been modified. A typical use case is a package (e.g. linux)
that is maintained as a git submodule of the BR2_EXTERNAL and with the
LINUX_OVERRIDE_SRCDIR set appropriately, or with a tool like repo
putting it in a well-known location.

This patch generates a tar.gz archive on the fly, containing the
result of the rsync of the package's source directory, using the
same arguments as during the preparation of a build.
Note that is not possible to archive the builddir, since it may
contain artifacts of previous compilations.

In order to create a tarball that is reproducible, we reuse the
mk_tar_gz function from the download helpers. We have to provide it with
the TAR variable. We also have to cd to the build directory first,
because the script generates a temporary file in the current directory.
That script needs a date. We unforuntately don't have a good
reproducible data. The timestamps of the files in the override srcdir
itself aren't useful either: if they come from git, they typically have
the time of checkout. Therefore, we fix the date at Jan 1, 1970 (which
we can do with @0).

The mk_tar_gz function could actually work directly in the override
srcdir without the need for an rsync - except for one limitation: it
doesn't have the equivalent of RSYNC_VCS_EXCLUSIONS. It would be nice to
refactor the script so it could accept a list of exclusions instead of a
single one and also work at any level instead of just the toplevel. But
that is for future improvement.

It is not possible to consider rsync as a "normal" download method
either (which would have solved the initial problem), because if we
do that:
 * the DL dir's content would be altered improperly
 * the rsync + tar gz overhead would be paid on compilation and is
   significant, for example, for a linux kernel
 * a hash file would be necessary, which would be a burden to
   maintain, for sources frequently modified

Signed-off-by: Nicolas Carrier <carrier.nicolas0@gmail.com>
[Arnout:
 - Drop the auxiliary variables: their use is contained within one code
   block and it's easier to read with being explicit.
 - Put it in an additional subdirectory .legal-info-rsync of the build
   directory, to reduce the chance of conflict with an existing
   file/directory.
 - Archive the directory (e.g. linux-custom) rather than its contents,
   so that it matches a normal tarball.
 - Use the mk_tar_gz function from download helpers to create a
   reproducible tarball.
 - Split long lines.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>

package/pkg-generic.mk

@@ -1155,13 +1155,20 @@ else
 endif # license files
 
 ifeq ($$($(2)_REDISTRIBUTE),YES)
-ifeq ($$($(2)_SITE_METHOD),local)
-# Packages without a tarball: don't save and warn
-	@$$(call legal-warning-nosource,$$($(2)_RAWNAME),local)
-
-else ifneq ($$($(2)_OVERRIDE_SRCDIR),)
-	@$$(call legal-warning-nosource,$$($(2)_RAWNAME),override)
-
+ifneq ($$($(2)_OVERRIDE_SRCDIR),)
+# local/override packages: copy it and archive the copy
+	@echo "Package is of type local or override, archive sources"
+	$$(Q)rm -rf  $$($(2)_BUILDDIR)/.legal-info-rsync
+	$$(Q)mkdir -p  $$($(2)_BUILDDIR)/.legal-info-rsync
+	$$(Q)rsync -au --chmod=u=rwX,go=rX $$(RSYNC_VCS_EXCLUSIONS) \
+		$(call qstrip,$$($(2)_OVERRIDE_SRCDIR))/ \
+		 $$($(2)_BUILDDIR)/.legal-info-rsync/
+	$$(Q). support/download/helpers; set -x; cd $$($(2)_BUILDDIR); TAR=$$(TAR) mk_tar_gz \
+		$$($(2)_BUILDDIR)/.legal-info-rsync/ \
+		$$($(2)_BASENAME_RAW) \
+		@0 \
+		$$($(2)_REDIST_SOURCES_DIR)/$$($(2)_BASENAME_RAW).tar.gz
+	$$(Q)rm -rf $$($(2)_BUILDDIR)/.legal-info-rsync
 else
 # Other packages