A2DP spec allow bitpool changes midstream which is why sbc configuration
has a range of values for bitpool that the encoder can use and decoder
must support.
Bitpool changes do not affect the state of encoder/decoder so they don't
need to be reinitialize when this happens, so the impact is fairly small,
what it does change is the frame length so encoders may change the
bitpool to use the link more efficiently.
In the case of scale factors calculation optimizations, the inline
assembly code has instructions which update flags register, but
"cc" was not mentioned in the clobber list. When optimizing code,
gcc theoretically is allowed to do a comparison before the inline
assembly block, and a conditional branch after it which would lead
to a problem if the flags register gets clobbered. While this is
apparently not happening in practice with the current versions of
gcc, the clobber list needs to be corrected.
Regarding the other inline assembly blocks. While most likely it
is actually unnecessary based on quick review, "cc" is also added
there to the clobber list because it should have no impact on
performance in practice. It's kind of cargo cult, but relieves
us from the need to track the potential updates of flags register
in all these places.
The optimized filter gets enabled when the code is compiled
with -mcpu=/-march options set to target the processors which
support ARMv6 instructions. This code is also disabled when
NEON is used (which is a lot better alternative). For additional
safety ARM EABI is required and thumb mode should not be used.
Benchmarks from ARM11:
== 8 subbands ==
$ time ./sbcenc -b53 -s8 -j test.au > /dev/null
real 0m 35.65s
user 0m 34.17s
sys 0m 1.28s
$ time ./sbcenc.armv6 -b53 -s8 -j test.au > /dev/null
real 0m 17.29s
user 0m 15.47s
sys 0m 0.67s
== 4 subbands ==
$ time ./sbcenc -b53 -s4 -j test.au > /dev/null
real 0m 25.28s
user 0m 23.76s
sys 0m 1.32s
$ time ./sbcenc.armv6 -b53 -s4 -j test.au > /dev/null
real 0m 18.64s
user 0m 15.78s
sys 0m 2.22s
By using SBC_ALWAYS_INLINE trick, the implementation of 'sbc_calculate_bits'
function is split into two branches, each having 'subband' variable value
known at compile time. It helps the compiler to generate more optimal code
by saving at least one extra register, and also provides more obvious
opportunities for loops unrolling.
Benchmarked on ARM Cortex-A8:
== Before: ==
$ time ./sbcenc -b53 -s8 -j test.au > /dev/null
real 0m3.989s
user 0m3.602s
sys 0m0.391s
samples % image name symbol name
26057 32.6128 sbcenc sbc_pack_frame
20003 25.0357 sbcenc sbc_analyze_4b_8s_neon
14220 17.7977 sbcenc sbc_calculate_bits
8498 10.6361 no-vmlinux /no-vmlinux
5300 6.6335 sbcenc sbc_calc_scalefactors_j_neon
3235 4.0489 sbcenc sbc_enc_process_input_8s_be_neon
2172 2.7185 sbcenc sbc_encode
== After: ==
$ time ./sbcenc -b53 -s8 -j test.au > /dev/null
real 0m3.652s
user 0m3.195s
sys 0m0.445s
samples % image name symbol name
26207 36.0095 sbcenc sbc_pack_frame
19820 27.2335 sbcenc sbc_analyze_4b_8s_neon
8629 11.8566 no-vmlinux /no-vmlinux
6988 9.6018 sbcenc sbc_calculate_bits
5094 6.9994 sbcenc sbc_calc_scalefactors_j_neon
3351 4.6044 sbcenc sbc_enc_process_input_8s_be_neon
2182 2.9982 sbcenc sbc_encode
Previous variant was basically derived from C and MMX implementations.
Now new variant makes use of 'vmax' instruction, which is available in
NEON and can do this job faster. The same method for calculating scale
factors is also used in 'sbc_calc_scalefactors_j_neon'.
Benchmarked without joint stereo on ARM Cortex-A8:
== Before: ==
$ time ./sbcenc -b53 -s8 test.au > /dev/null
real 0m3.851s
user 0m3.375s
sys 0m0.469s
samples % image name symbol name
26260 34.2672 sbcenc sbc_pack_frame
20013 26.1154 sbcenc sbc_analyze_4b_8s_neon
13796 18.0027 sbcenc sbc_calculate_bits
8388 10.9457 no-vmlinux /no-vmlinux
3229 4.2136 sbcenc sbc_enc_process_input_8s_be_neon
2408 3.1422 sbcenc sbc_calc_scalefactors_neon
2093 2.7312 sbcenc sbc_encode
== After: ==
$ time ./sbcenc -b53 -s8 test.au > /dev/null
real 0m3.796s
user 0m3.344s
sys 0m0.438s
samples % image name symbol name
26582 34.8726 sbcenc sbc_pack_frame
20032 26.2797 sbcenc sbc_analyze_4b_8s_neon
13808 18.1146 sbcenc sbc_calculate_bits
8374 10.9858 no-vmlinux /no-vmlinux
3187 4.1810 sbcenc sbc_enc_process_input_8s_be_neon
2027 2.6592 sbcenc sbc_encode
1766 2.3168 sbcenc sbc_calc_scalefactors_neon
The code for scale factors calculation with joint stereo support has
been moved to a separate function. It can get platform-specific
SIMD optimizations later for best possible performance.
But even this change in C code improves performance because of the
use of __builtin_clz() instead of loops similar to what was done
to sbc_calc_scalefactors earlier. Also technically it does loop
unrolling by processing two channels at once, which might be either
good or bad for performance (if the registers pressure is increased
and more data is spilled to memory). But the benchmark from 32-bit
x86 system (pentium-m) shows that it got clearly faster:
$ time ./sbcenc.old -b53 -s8 -j test.au > /dev/null
real 0m1.868s
user 0m1.808s
sys 0m0.048s
$ time ./sbcenc.new -b53 -s8 -j test.au > /dev/null
real 0m1.742s
user 0m1.668s
sys 0m0.064s
This prevents overflows and audible artefacts for the audio files which
originally had loudness maximized. Music from audio CD disks is an
example of such files, see http://en.wikipedia.org/wiki/Loudness_war
Buffer position in X array was not always 16-bytes aligned.
Strict 16-byte alignment is strictly required for powerpc altivec
simd optimizations because altivec does not have support for
unaligned vector loads at all.
Count leading zeros operation is often implemented using a special
instruction for it on various architectures (at least this is true
for ARM and x86). Using __builtin_clz gcc intrinsic allows to
eliminate innermost loop in scale factors calculation and improve
performance. Also scale factors calculation can be optimized even
more using SIMD instructions.
Channels deinterleaving, endian conversion and samples reordering
is done in one pass, avoiding the use of intermediate buffer. Also
this code is implemented as a new "performance primitive", which
allows further platform specific optimizations (ARMv6 and ARM NEON
should gain quite a lot from assembly optimizations here).
Added the use of -funroll-loops gcc option for SBC. Also in
order to gain better effect, 'sbc_pack_frame' function
body moved to an inline function, which gets instantiated
for 4 different subbands/channels combinations. So that
'frame_subbands' and 'frame_channels' arguments become compile
time constants and can be better optimized by the compiler.
Multiplying the first part of the analysis filter constant tables
by some coefficients and dividing the second part by the same
coefficients is a transformation which should produce the same
results if rounding errors are not taken into account. These
additional C0/C1/... coefficients can be varied in a certain
range (the requirement is that we still do not get overflows).
The 'magic' values for these coefficients are selected in such
a way that the rounding errors are minimized (rounding errors
are unavoidable when putting all the floating constants into
16-bit tables and losing some of the fractional part).
Also non-SIMD variant of the analysis filter is dropped because
keeping it would require applying a similar change to its tables,
which is a bit tricky and just increases maintenance overhead.
Read and write buffers sizes increased, memmove overhead eliminated.
Nonportable cast from 'unsigned char *' to 'struct au_header *' is
now also resolved as part of the changes.
Most SIMD instruction sets benefit from data being naturally aligned.
And even if it is not strictly required, performance is usually better
with the aligned data. ARM NEON and SSE2 have different instruction
variants for aligned/unaligned memory accesses.
Added SIMD-friendly C implementation of SBC analysis filter (the
structure of code had to be changed a bit and constants in the
tables reordered). This code can be used as a reference for
developing platform specific SIMD optimizations. These functions
are put into a new file 'sbc_primitives.c', which is going to
contain all the basic stuff for SBC codec.
The result of 32x32->64 unsigned long multiplication is returned
in two registers (high and low 32-bit parts) for many 32-bit
architectures. For these architectures constant right shift by
32 bits is optimized out by the compiler to just taking the high
32-bit part. Also some data needed at the quantization stage is
precalculated beforehand to improve performance.
This change is needed for SIMD optimizations which will follow
shortly. And even for non-SIMD capable platforms it still may
be useful to have possibility to merge several analyzing functions
together into one for better code scheduling or reusing loaded
constants. Also analysis filter functions are now called using
function pointers, which allows the default implementation to be
overrided at runtime (with high precision variant or MMX/SSE2/NEON
optimized code).
This code is heavily based on the patch submitted by Jaska Uimonen.
Additional changes include preserving extra bits in the output of
filter function for better precision, support for both 16-bit and
32-bit fixed point implementation. Sign of some table values was
changed in order to preserve a regular code structure and have
multiply-accumulate oparations only. No additional optimizations
were applied as this code is intended to be some kind of "reference"
implementation. Platform specific optimizations may require
different tricks and can be branched off from this implementation.
Some extra information about this code can be found in linux-bluetooth
mailing list archive for December 2008.
