freedreno: add CI for envytools tools

This also tunes `.freedreno-rules` a bit so that it isn't triggered by
various tools that don't effect the driver build.

The .gitlab-ci directory is kept separate from the toplevel one so that
updates to (for example) reference decode output do not trigger all the
other-driver jobs to run.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6070>

.gitlab-ci.yml

@@ -221,7 +221,7 @@ x86_build:
   extends:
     - .use-x86_build-base
   variables:
-    FDO_DISTRIBUTION_TAG: &x86_build "2020-07-21-tracie"
+    FDO_DISTRIBUTION_TAG: &x86_build "2020-07-27-fdtools"
 
 .use-x86_build:
   variables:
@@ -554,6 +554,7 @@ meson-gallium:
   script:
     - .gitlab-ci/meson-build.sh
     - .gitlab-ci/run-shader-db.sh
+    - src/freedreno/.gitlab-ci/run-fdtools.sh
 
 meson-classic:
   extends: .meson-build

.gitlab-ci/container/x86_build.sh

@@ -26,6 +26,9 @@ apt-get update
 
 apt-get install -y --no-remove \
       $STABLE_EPHEMERAL \
+      libarchive-dev \
+      liblua5.3-dev \
+      libxml2-dev \
       wine-development \
       wine32-development
 

.gitlab-ci/test-source-dep.yml

@@ -89,7 +89,17 @@
         *gallium_core_file_list
       when: on_success
     - changes:
-      - src/freedreno/**/*
+      # Note: when https://gitlab.com/gitlab-org/gitlab/-/issues/198688
+      # is supported, we can change the src/freedreno/ rule to explicitly
+      # exclude tools, rather than having to explicitly list everything
+      # else
+      - src/freedreno/common/**/*
+      - src/freedreno/drm/**/*
+      - src/freedreno/fdl/**/*
+      - src/freedreno/ir3/**/*
+      - src/freedreno/perfcntrs/**/*
+      - src/freedreno/registers/**/*
+      - src/freedreno/vulkan/**/*
       - src/gallium/drivers/freedreno/**/*
       - src/gallium/winsys/freedreno/**/*
       when: on_success

src/freedreno/.gitlab-ci/README.rst

@@ -0,0 +1,33 @@
+Continuous Integration
+======================
+
+In addition to a build step, the CI setup has a basic test stage
+to ensure that we don't break functionality of various tools.  The
+basic idea is to decode various files and compare the output to a
+reference.  This means that some changes, like renaming registers
+or bitfields in the rnndb xml requires updating the reference
+output.
+
+Layout:
+ - .gitlab-ci/
+
+   - traces/ - reference devcoredump and cmdstream traces.  The trace files should be kept small, and .rd files (which are already binary) should be compressed.
+
+   - reference/ - reference output
+
+   - genoutput.sh - script to generate output from the traces, used both by the CI test job, but it can also be used to update the reference output
+
+Note on paths:
+--------------
+
+Gitlab CI uses an install-path of \`pwd\`/install.  If you use something
+different, then setup a symlink.  Once that is done, to update reference
+decodes (ie. to account for register .xml changes) run:
+
+  ./src/freedreno/.gitlab-ci/genoutput.sh --update
+
+TODO
+----
+- Maybe we could filter out some differences, like a new definition of a previously unknown register?
+- It would be nice to add a test for afuc.. we probably cannot add a "real" fw file to this tree, but maybe could either fetch it from the linux-firmware git tree, or create our own dummy fw.
+

src/freedreno/.gitlab-ci/genoutput.sh

@@ -0,0 +1,50 @@
+#!/bin/bash
+
+# This file contains all of the cmdlines used to generate output
+# for the test step in the CI pipeline.  It can also be used to
+# regenerate reference output
+
+set -x
+set -e
+
+# input/output directories:
+base=src/freedreno
+traces=$base/.gitlab-ci/traces
+output=$base/.gitlab-ci/out
+
+# use the --update arg to update reference output:
+if [ "$1" = "--update" ]; then
+	output=src/freedreno/.gitlab-ci/reference
+fi
+
+mkdir -p $output
+
+# binary locations:
+cffdump=./install/bin/cffdump
+crashdec=./install/bin/crashdec
+
+# helper to filter out paths that can change depending on
+# who is building:
+basepath=`dirname $0`
+basepath=`dirname $basepath`
+basepath=`pwd $basepath`
+filter() {
+	out=$1
+	grep -vF "$basepath" > $out
+}
+
+#
+# The Tests:
+#
+
+# dump only a single frame, and single tile pass, to keep the
+# reference output size managable
+$cffdump --frame 0 --once $traces/fd-clouds.rd.gz | filter $output/fd-clouds.log
+$cffdump --frame 0 --once $traces/es2gears-a320.rd.gz | filter $output/es2gears-a320.log
+$cffdump --frame 1 --once $traces/glxgears-a420.rd.gz | filter $output/glxgears-a420.log
+
+# test a lua script to ensure we don't break scripting API:
+$cffdump --script $base/decode/scripts/parse-submits.lua $traces/shadow.rd.gz | filter $output/shadow.log
+
+$crashdec -sf $traces/crash.devcore | filter $output/crash.log
+

src/freedreno/.gitlab-ci/reference/shadow.log

@@ -0,0 +1,48 @@
+Analyzing Data...
+Reading src/freedreno/.gitlab-ci/traces/shadow.rd.gz...
+Parsing src/freedreno/.gitlab-ci/traces/shadow.rd.gz
+
+Blit:
+-----
+  MRT[0x28b4000:0x0]:	256x144		FMT6_16_UNORM (MSAA_ONE)	CLEARED
+
+
+
+NULL BATCH!
+
+Blit:
+-----
+  MRT[0x20ae000:0x0]:	480x272		FMT6_16_UNORM (MSAA_ONE)	CLEARED
+
+Batch:
+-------
+  # of draws: 1
+  mode: RM6_GMEM
+  bin size: 768x320 (35 bins)
+  DEPTHTEST DEPTHWRITE
+  MRT[0x4715000:0x0]:	3840x2160		FMT6_Z24_UNORM_S8_UINT (MSAA_ONE)	CLEARED	RESOLVED
+
+
+
+Blit:
+-----
+  MRT[0x27d1000:0x0]:	256x144		FMT6_16_UNORM (MSAA_ONE)	CLEARED
+
+
+
+Blit:
+-----
+  MRT[0x28a0000:0x0]:	256x144		FMT6_16_UNORM (MSAA_ONE)	CLEARED
+
+Batch:
+-------
+  # of draws: 2
+  mode: RM6_GMEM
+  bin size: 640x192 (18 bins)
+  DEPTHTEST DEPTHWRITE
+  MRT[0x28fc000:0x0]:	1920x1080		FMT6_Z24_UNORM_S8_UINT (MSAA_ONE)	CLEARED	RESOLVED
+  MRT[0x18ae000:0x0]:	1920x1080		FMT6_8_8_8_8_UNORM (MSAA_ONE)	CLEARED	RESOLVED
+  MRT[0x0:0x0]:	1920x1080		FMT6_8_8_8_8_UNORM (MSAA_ONE)	CLEARED	RESOLVED
+  SRC[0x4715000:0x46f3000]:	3840x2160		FMT6_Z24_UNORM_S8_UINT (MSAA_ONE)
+
+

src/freedreno/.gitlab-ci/run-fdtools.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+set -e
+set -x
+
+p=`dirname $0`
+
+# Run cffdump/crashdec tests:
+$p/genoutput.sh
+diff -r $p/reference $p/out
+
+# Also, while we are still using headergen2 for generating kernel
+# headers, make sure that doesn't break:
+headergen="_build/src/freedreno/rnn/headergen2"
+$headergen adreno.xml
+$headergen msm.xml