linux/crypto
Nathan Huckleberry 34f7f6c301 crypto: x86/polyval - Add PCLMULQDQ accelerated implementation of POLYVAL
Add hardware accelerated version of POLYVAL for x86-64 CPUs with
PCLMULQDQ support.

This implementation is accelerated using PCLMULQDQ instructions to
perform the finite field computations.  For added efficiency, 8 blocks
of the message are processed simultaneously by precomputing the first
8 powers of the key.

Schoolbook multiplication is used instead of Karatsuba multiplication
because it was found to be slightly faster on x86-64 machines.
Montgomery reduction must be used instead of Barrett reduction due to
the difference in modulus between POLYVAL's field and other finite
fields.

More information on POLYVAL can be found in the HCTR2 paper:
"Length-preserving encryption with HCTR2":
https://eprint.iacr.org/2021/1441.pdf

Signed-off-by: Nathan Huckleberry <nhuck@google.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-06-10 16:40:17 +08:00
..
asymmetric_keys certs: Factor out the blacklist hash creation 2022-05-23 18:47:49 +03:00
async_tx crypto: cleanup comments 2022-03-03 10:49:20 +12:00
842.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
acompress.c crypto: api - permit users to specify numa node of acomp hardware 2020-07-09 18:25:23 +10:00
adiantum.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
aead.c crypto: algapi - enforce that all instances have a ->free() method 2020-01-09 11:30:58 +08:00
aegis128-core.c crypto: aegis128 - Move simd prototypes into aegis.h 2021-03-19 21:59:45 +11:00
aegis128-neon-inner.c crypto: aegis128/neon - move final tag check to SIMD domain 2020-11-27 17:13:40 +11:00
aegis128-neon.c crypto: aegis128 - Move simd prototypes into aegis.h 2021-03-19 21:59:45 +11:00
aegis.h crypto: aegis128 - Move simd prototypes into aegis.h 2021-03-19 21:59:45 +11:00
aes_generic.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
aes_ti.c crypto: aes - create AES library based on the fixed time AES code 2019-07-26 14:55:33 +10:00
af_alg.c crypto: af_alg - get rid of alg_memory_allocated 2022-02-15 14:29:04 +00:00
ahash.c crypto: ahash - Add init_tfm/exit_tfm 2020-08-28 16:58:28 +10:00
akcipher.c crypto: algapi - enforce that all instances have a ->free() method 2020-01-09 11:30:58 +08:00
algapi.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2022-03-21 16:02:36 -07:00
algboss.c exit: Rename module_put_and_exit to module_put_and_kthread_exit 2021-12-13 12:04:45 -06:00
algif_aead.c crypto: algif_aead - Do not set MAY_BACKLOG on the async path 2020-08-21 14:45:27 +10:00
algif_hash.c net: make ->{get,set}sockopt in proto_ops optional 2020-07-19 18:16:41 -07:00
algif_rng.c crypto: af_alg - add extra parameters for DRBG interface 2020-09-25 17:48:52 +10:00
algif_skcipher.c crypto: algif_skcipher - EBUSY on aio should be an error 2020-08-21 14:45:26 +10:00
ansi_cprng.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
anubis.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
api.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2022-03-21 16:02:36 -07:00
arc4.c crypto: arc4 - mark ecb(arc4) skcipher as obsolete 2020-09-11 14:39:16 +10:00
authenc.c crypto: authenc - Fix sleep in atomic context in decrypt_tail 2022-01-31 11:21:44 +11:00
authencesn.c crypto: algapi - use common mechanism for inheriting flags 2020-07-16 21:49:08 +10:00
blake2b_generic.c crypto: blake2b - update file comment 2021-01-03 08:41:39 +11:00
blake2s_generic.c lib/crypto: blake2s: avoid indirect calls to compression function for Clang CFI 2022-02-04 19:22:32 +01:00
blowfish_common.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
blowfish_generic.c crypto: blowfish - use unaligned accessors instead of alignmask 2021-02-10 17:55:56 +11:00
camellia_generic.c crypto: camellia - use unaligned accessors instead of alignmask 2021-02-10 17:55:56 +11:00
cast5_generic.c crypto: cast5 - use unaligned accessors instead of alignmask 2021-02-10 17:55:56 +11:00
cast6_generic.c crypto: cast6 - use unaligned accessors instead of alignmask 2021-02-10 17:55:56 +11:00
cast_common.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 44 2019-05-24 17:27:12 +02:00
cbc.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
ccm.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
cfb.c crypto: cleanup comments 2022-03-03 10:49:20 +12:00
chacha20poly1305.c crypto: algapi - use common mechanism for inheriting flags 2020-07-16 21:49:08 +10:00
chacha_generic.c crypto: chacha_generic - remove unnecessary setkey() functions 2019-11-22 18:48:39 +08:00
cipher.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
cmac.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
compress.c crypto: compress - remove crt_u.compress (struct compress_tfm) 2019-12-11 16:37:01 +08:00
crc32_generic.c crypto: crc32-generic - Use SPDX-License-Identifier 2021-04-16 21:24:27 +10:00
crc32c_generic.c crypto: crc32c_generic - delete and fix duplicated words 2020-08-21 14:45:25 +10:00
crc64_rocksoft_generic.c crypto: add rocksoft 64b crc guard tag framework 2022-03-07 12:48:35 -07:00
crct10dif_common.c
crct10dif_generic.c crypto: crct10dif_generic - fix duplicated words 2020-08-21 14:45:25 +10:00
cryptd.c crypto: cryptd - Protect per-CPU resource by disabling BH. 2022-05-13 17:24:48 +08:00
crypto_engine.c crypto: engine - Add parameter description in crypto_transfer_request() kernel-doc comment 2022-04-08 16:13:30 +08:00
crypto_null.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2019-07-08 20:57:08 -07:00
crypto_user_base.c crypto: algapi - make unregistration functions return void 2019-12-20 14:58:35 +08:00
crypto_user_stat.c crypto: skcipher - remove the "blkcipher" algorithm type 2019-11-01 13:38:32 +08:00
ctr.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
cts.c crypto: algapi - use common mechanism for inheriting flags 2020-07-16 21:49:08 +10:00
curve25519-generic.c crypto: curve25519 - implement generic KPP driver 2019-11-17 09:02:43 +08:00
deflate.c mm, treewide: rename kzfree() to kfree_sensitive() 2020-08-07 11:33:22 -07:00
des_generic.c crypto: remove CRYPTO_TFM_RES_WEAK_KEY 2020-01-09 11:30:53 +08:00
dh_helper.c crypto: dh - split out deserialization code from crypto_dh_decode() 2022-03-03 10:47:50 +12:00
dh.c crypto: dh - Remove the unused function dh_safe_prime_dh_alg() 2022-03-14 14:45:45 +12:00
drbg.c crypto: drbg - ignore jitterentropy errors if not in FIPS mode 2021-12-11 16:48:06 +11:00
ecb.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
ecc_curve_defs.h Merge branch 'ecc' 2021-03-26 19:55:55 +11:00
ecc.c crypto: ecc - Export additional helper functions 2021-10-29 21:04:03 +08:00
ecdh_helper.c crypto: ecdh - move curve_id of ECDH from the key to algorithm name 2021-03-13 00:04:03 +11:00
ecdh.c crypto: ecc - Move ecc.h to include/crypto/internal 2021-10-29 21:04:03 +08:00
ecdsa.c crypto: ecc - Move ecc.h to include/crypto/internal 2021-10-29 21:04:03 +08:00
ecdsasignature.asn1 crypto: ecdsa - Add support for ECDSA signature verification 2021-03-26 19:41:58 +11:00
echainiv.c crypto: geniv - remove unneeded arguments from aead_geniv_alloc() 2020-07-16 21:49:07 +10:00
ecrdsa_defs.h crypto: ecc - Move ecc.h to include/crypto/internal 2021-10-29 21:04:03 +08:00
ecrdsa_params.asn1 crypto: ecrdsa - add EC-RDSA (GOST 34.10) algorithm 2019-04-18 22:15:02 +08:00
ecrdsa_pub_key.asn1 crypto: ecrdsa - add EC-RDSA (GOST 34.10) algorithm 2019-04-18 22:15:02 +08:00
ecrdsa.c crypto: ecrdsa - Fix incorrect use of vli_cmp 2022-04-29 13:44:58 +08:00
essiv.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
fcrypt.c crypto: fcrypt - Remove 'do while(0)' loop for single statement macro 2021-04-02 18:28:13 +11:00
fips.c crypto: fips - add FIPS test failure notification chain 2019-07-26 14:51:57 +10:00
gcm.c mm, treewide: rename kzfree() to kfree_sensitive() 2020-08-07 11:33:22 -07:00
geniv.c crypto: algapi - use common mechanism for inheriting flags 2020-07-16 21:49:08 +10:00
gf128mul.c mm, treewide: rename kzfree() to kfree_sensitive() 2020-08-07 11:33:22 -07:00
ghash-generic.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
hash_info.c crypto: rename sm3-256 to sm3 in hash_algo_name 2020-02-18 07:35:49 -05:00
hctr2.c crypto: hctr2 - Add HCTR2 support 2022-06-10 16:40:17 +08:00
hmac.c crypto: hmac - disallow keys < 112 bits in FIPS mode 2022-02-11 20:22:01 +11:00
internal.h crypto: api - Fix built-in testing dependency failures 2021-09-24 16:03:05 +08:00
jitterentropy-kcapi.c crypto: jitter - don't limit ->health_failure check to FIPS mode 2021-12-11 16:48:06 +11:00
jitterentropy.c crypto: jitter - add oversampling of noise source 2021-12-31 18:10:54 +11:00
jitterentropy.h crypto: jitter - don't limit ->health_failure check to FIPS mode 2021-12-11 16:48:06 +11:00
Kconfig crypto: x86/polyval - Add PCLMULQDQ accelerated implementation of POLYVAL 2022-06-10 16:40:17 +08:00
kdf_sp800108.c crypto: kdf - add SP800-108 counter key derivation function 2021-11-26 16:25:17 +11:00
keywrap.c crypto: keywrap - Remove else after break statement 2021-04-02 18:28:13 +11:00
khazad.c crypto: khazad,wp512 - remove leading spaces before tabs 2021-05-28 15:11:44 +08:00
kpp.c crypto: kpp - provide support for KPP spawns 2022-03-03 10:47:50 +12:00
lrw.c crypto: lrw - Add dependency on ecb 2022-02-18 16:21:08 +11:00
lz4.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2019-07-08 20:57:08 -07:00
lz4hc.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2019-07-08 20:57:08 -07:00
lzo-rle.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2019-07-08 20:57:08 -07:00
lzo.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2019-07-08 20:57:08 -07:00
Makefile crypto: hctr2 - Add HCTR2 support 2022-06-10 16:40:17 +08:00
md4.c crypto: make all generic algorithms set cra_driver_name 2019-06-13 14:31:39 +08:00
md5.c crypto: md5 - remove unused macros 2020-02-28 08:43:21 +08:00
memneq.c crypto: memneq - avoid implicit unaligned accesses 2022-01-31 11:21:44 +11:00
michael_mic.c crypto: michael_mic - fix broken misalignment handling 2021-02-10 17:55:55 +11:00
nhpoly1305.c crypto: poly1305 - add new 32 and 64-bit generic versions 2020-01-16 15:18:12 +08:00
ofb.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
pcbc.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
pcrypt.c crypto: pcrypt - Delay write to padata->info 2021-10-29 21:04:04 +08:00
poly1305_generic.c crypto: poly1305 - add new 32 and 64-bit generic versions 2020-01-16 15:18:12 +08:00
polyval-generic.c crypto: x86/polyval - Add PCLMULQDQ accelerated implementation of POLYVAL 2022-06-10 16:40:17 +08:00
proc.c crypto: proc - Removing some useless only space lines 2020-09-25 17:48:15 +10:00
ripemd.h crypto: rmd320 - remove RIPE-MD 320 hash algorithm 2021-01-29 16:07:04 +11:00
rmd160.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2019-07-08 20:57:08 -07:00
rng.c crypto: rng - fix crypto_rng_reset() refcounting when !CRYPTO_STATS 2021-04-02 18:28:12 +11:00
rsa_helper.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
rsa-pkcs1pad.c crypto: rsa-pkcs1pad - use clearer variable names 2022-01-31 11:21:44 +11:00
rsa.c crypto: rsa - limit key size to 2048 in FIPS mode 2021-11-26 16:25:18 +11:00
rsaprivkey.asn1 crypto: rsa - Store rest of the private key components 2016-07-05 23:05:26 +08:00
rsapubkey.asn1 crypto: akcipher - Changes to asymmetric key API 2015-10-14 22:23:16 +08:00
scatterwalk.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
scompress.c crypto: algapi - make unregistration functions return void 2019-12-20 14:58:35 +08:00
seed.c crypto: seed - remove trailing semicolon in macro definition 2020-12-04 18:13:16 +11:00
seqiv.c mm, treewide: rename kzfree() to kfree_sensitive() 2020-08-07 11:33:22 -07:00
serpent_generic.c crypto: serpent - Fix sparse byte order warnings 2021-03-07 15:13:15 +11:00
sha1_generic.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha3_generic.c crypto: Replace HTTP links with HTTPS ones 2020-07-23 17:34:20 +10:00
sha256_generic.c crypto: sha256 - remove duplicate generic hash init function 2021-12-31 18:10:54 +11:00
sha512_generic.c crypto: sha512 - remove imaginary and mystifying clearing of variables 2021-08-27 16:30:19 +08:00
shash.c crypto: shash - avoid comparing pointers to exported functions under CFI 2021-06-17 15:07:31 +08:00
simd.c crypto: algapi - use common mechanism for inheriting flags 2020-07-16 21:49:08 +10:00
skcipher.c crypto: skcipher - in_irq() cleanup 2021-08-21 15:44:58 +08:00
sm2.c crypto: cleanup comments 2022-03-03 10:49:20 +12:00
sm2signature.asn1 crypto: sm2 - introduce OSCCA SM2 asymmetric cipher algorithm 2020-09-25 17:48:54 +10:00
sm3_generic.c crypto: sm3 - make dependent on sm3 library 2022-01-28 16:51:11 +11:00
sm3.c crypto: sm3,sm4 - move into crypto directory 2022-04-08 16:11:48 +08:00
sm4_generic.c crypto: arm64/sm4-ce - Make dependent on sm4 library instead of sm4-generic 2021-07-30 10:58:30 +08:00
sm4.c crypto: sm4 - export sm4 constant arrays 2022-04-08 16:12:46 +08:00
streebog_generic.c crypto: streebog - remove two unused variables 2019-08-15 21:52:14 +10:00
tcrypt.c crypto: hctr2 - Add HCTR2 support 2022-06-10 16:40:17 +08:00
tcrypt.h crypto: tcrypt - Add support for hash speed testing with keys 2020-08-21 14:45:27 +10:00
tea.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2019-07-08 20:57:08 -07:00
testmgr.c crypto: hctr2 - Add HCTR2 support 2022-06-10 16:40:17 +08:00
testmgr.h crypto: hctr2 - Add HCTR2 support 2022-06-10 16:40:17 +08:00
twofish_common.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
twofish_generic.c crypto: twofish - use unaligned accessors instead of alignmask 2021-02-10 17:55:57 +11:00
vmac.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
wp512.c crypto: wp512 - correct a non-kernel-doc comment 2021-08-12 19:32:17 +08:00
xcbc.c crypto: remove cipher routines from public crypto API 2021-01-03 08:41:35 +11:00
xctr.c crypto: xctr - Add XCTR support 2022-06-10 16:40:16 +08:00
xor.c Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2021-02-21 17:23:56 -08:00
xts.c crypto: xts - Add softdep on ecb 2022-02-18 16:21:08 +11:00
xxhash_generic.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
zstd.c lib: zstd: Add kernel-specific API 2021-11-08 16:55:21 -08:00