Joerg Roedel 763802b53a x86/mm: split vmalloc_sync_all() ... Commit 3f8fd02b1b ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()") introduced a call to vmalloc_sync_all() in the vunmap() code-path. While this change was necessary to maintain correctness on x86-32-pae kernels, it also adds additional cycles for architectures that don't need it. Specifically on x86-64 with CONFIG_VMAP_STACK=y some people reported severe performance regressions in micro-benchmarks because it now also calls the x86-64 implementation of vmalloc_sync_all() on vunmap(). But the vmalloc_sync_all() implementation on x86-64 is only needed for newly created mappings. To avoid the unnecessary work on x86-64 and to gain the performance back, split up vmalloc_sync_all() into two functions: * vmalloc_sync_mappings(), and * vmalloc_sync_unmappings() Most call-sites to vmalloc_sync_all() only care about new mappings being synchronized. The only exception is the new call-site added in the above mentioned commit. Shile Zhang directed us to a report of an 80% regression in reaim throughput. Fixes: 3f8fd02b1b ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()") Reported-by: kernel test robot <oliver.sang@intel.com> Reported-by: Shile Zhang <shile.zhang@linux.alibaba.com> Signed-off-by: Joerg Roedel <jroedel@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Tested-by: Borislav Petkov <bp@suse.de> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> [GHES] Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Andy Lutomirski <luto@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: <stable@vger.kernel.org> Link: http://lkml.kernel.org/r/20191009124418.8286-1-joro@8bytes.org Link: https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/4D3JPPHBNOSPFK2KEPC6KGKS6J25AIDB/ Link: http://lkml.kernel.org/r/20191113095530.228959-1-shile.zhang@linux.alibaba.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2020-03-21 18:56:06 -07:00
..
acpica	ACPI: PM: s2idle: Check fixed wakeup events in acpi_s2idle_wake()	2020-02-21 10:01:25 -08:00
apei	x86/mm: split vmalloc_sync_all()	2020-03-21 18:56:06 -07:00
arm64	IOMMU Updates for Linux v5.6	2020-02-05 17:49:54 +00:00
dptf	ACPI: DPTF: Add Tiger Lake ACPI device IDs	2019-12-19 22:51:55 +01:00
nfit	libnvdimm: Move nvdimm_bus_attribute_group to device_type	2019-11-19 09:52:12 -08:00
numa	ACPI: HMAT: use %u instead of %d to print u32 values	2019-11-12 10:05:57 +01:00
pmic	ACPI / PMIC: Add Cherry Trail Crystal Cove PMIC OpRegion driver	2019-10-25 11:43:08 +02:00
x86	x86/intel: Aggregate big core mobile naming	2019-08-28 11:29:31 +02:00
ac.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157	2019-05-30 11:26:37 -07:00
acpi_adxl.c	ACPI/ADXL: Add address translation interface using an ACPI DSM	2018-10-16 10:03:00 +02:00
acpi_amba.c	Merge 5.2-rc6 into char-misc-next	2019-06-23 09:23:33 +02:00
acpi_apd.c	ACPI / APD: Add clock frequency for Hisilicon Hip08-Lite I2C controller	2020-02-03 12:03:18 +01:00
acpi_cmos_rtc.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500	2019-06-19 17:09:55 +02:00
acpi_configfs.c	ACPICA: make acpi_load_table() return table index	2019-10-28 22:12:46 +01:00
acpi_dbg.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500	2019-06-19 17:09:55 +02:00
acpi_extlog.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 437	2019-06-05 17:37:17 +02:00
acpi_ipmi.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157	2019-05-30 11:26:37 -07:00
acpi_lpat.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174	2019-05-30 11:26:41 -07:00
acpi_lpit.c	remove ioremap_nocache and devm_ioremap_nocache	2020-01-06 09:45:59 +01:00
acpi_lpss.c	ACPI / LPSS: Rename pwm_backlight pwm-lookup to pwm_soc_backlight	2019-12-17 11:24:47 +01:00
acpi_memhotplug.c	mm/memory_hotplug: rename walk_memory_range() and pass start+size instead of pfns	2019-07-18 17:08:06 -07:00
acpi_pad.c	ACPI, x86: Add Zhaoxin processors support for NONSTOP TSC	2019-06-22 11:45:57 +02:00
acpi_platform.c	ACPI: platform: Unregister stale platform devices	2019-10-17 23:48:39 +02:00
acpi_pnp.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500	2019-06-19 17:09:55 +02:00
acpi_processor.c	ACPI: processor: Export acpi_processor_evaluate_cst()	2019-12-27 11:02:07 +01:00
acpi_tad.c	ACPI: TAD: Add low-level support for real time capability	2018-10-18 09:11:53 +02:00
acpi_video.c	ACPI: video: fix typo in comment	2019-12-16 11:19:58 +01:00
acpi_watchdog.c	ACPI: watchdog: Fix gas->access_width usage	2020-02-13 23:51:30 +01:00
battery.c	proc: convert everything to "struct proc_ops"	2020-02-04 03:05:26 +00:00
bgrt.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500	2019-06-19 17:09:55 +02:00
blacklist.c	ACPI: blacklist: fix clang warning for unused DMI table	2019-07-11 22:45:00 +02:00
bus.c	ACPI: bus: Fix NULL pointer check in acpi_bus_get_private_data()	2019-11-29 11:42:10 +01:00
button.c	ACPI: button: Add DMI quirk for Razer Blade Stealth 13 late 2019 lid switch	2020-01-07 11:51:41 +01:00
cm_sbs.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157	2019-05-30 11:26:37 -07:00
container.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157	2019-05-30 11:26:37 -07:00
cppc_acpi.c	ACPI: CPPC: Set pcc_data[pcc_ss_id] to NULL in acpi_cppc_processor_exit()	2019-10-18 10:36:37 +02:00
custom_method.c	Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security	2019-09-28 08:14:15 -07:00
debugfs.c	treewide: Add SPDX license identifier for missed files	2019-05-21 10:50:45 +02:00
device_pm.c	ACPI: fan: Add Tiger Lake ACPI device ID	2019-12-19 22:51:55 +01:00
device_sysfs.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174	2019-05-30 11:26:41 -07:00
dock.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157	2019-05-30 11:26:37 -07:00
ec_sys.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 482	2019-06-19 17:09:52 +02:00
ec.c	ACPI: EC: Fix flushing of pending work	2020-02-11 10:07:43 +01:00
event.c	ACPI: event: replace strcpy() by strscpy()	2019-04-23 10:54:26 +02:00
evged.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 284	2019-06-05 17:36:37 +02:00
fan.c	Merge branches 'acpi-battery', 'acpi-video', 'acpi-fan' and 'acpi-drivers'	2020-01-27 10:57:09 +01:00
glue.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428	2019-06-05 17:37:16 +02:00
hed.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174	2019-05-30 11:26:41 -07:00
internal.h	ACPI: EC: add support for hardware-reduced systems	2019-10-28 16:00:09 +01:00
ioapic.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500	2019-06-19 17:09:55 +02:00
irq.c	Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2019-07-08 11:01:13 -07:00
Kconfig	ACPI: processor: Make ACPI_PROCESSOR_CSTATE depend on ACPI_PROCESSOR	2019-12-27 11:01:45 +01:00
Makefile	Merge branch 'acpi-mm'	2019-11-26 10:31:02 +01:00
nvs.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428	2019-06-05 17:37:16 +02:00
osi.c	ACPI: OSI: Shoot duplicate word	2019-11-13 11:12:08 +01:00
osl.c	ACPI: OSL: only free map once in osl.c	2019-11-29 10:31:12 +01:00
pci_irq.c	ACPI / PCI: fix acpi_pci_irq_enable() memory leak	2019-09-03 09:41:25 +02:00
pci_link.c	ACPI/PCI: Remove surplus parentheses from a return statement	2019-08-21 00:05:20 +02:00
pci_mcfg.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 459	2019-06-19 17:09:09 +02:00
pci_root.c	PCI: Move ASPM declarations to linux/pci.h	2019-08-28 08:28:39 -05:00
pci_slot.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 491	2019-06-19 17:09:52 +02:00
power.c	ACPI / PM: Introduce concept of a _PR0 dependent device	2019-06-27 12:31:57 +02:00
pptt.c	ACPI: PPTT: Consistently use unsigned int as parameter type	2020-01-07 11:46:36 +01:00
proc.c	proc: convert everything to "struct proc_ops"	2020-02-04 03:05:26 +00:00
processor_core.c	treewide: Add SPDX license identifier for missed files	2019-05-21 10:50:45 +02:00
processor_driver.c	cpufreq: Use per-policy frequency QoS	2019-10-21 02:05:21 +02:00
processor_idle.c	ACPI: processor: Export acpi_processor_evaluate_cst()	2019-12-27 11:02:07 +01:00
processor_pdc.c	License cleanup: add SPDX GPL-2.0 license identifier to files with no license	2017-11-02 11:10:55 +01:00
processor_perflib.c	ACPI: processor: Add QoS requests for all CPUs	2019-10-25 11:33:48 +02:00
processor_thermal.c	ACPI: processor: Add QoS requests for all CPUs	2019-10-25 11:33:48 +02:00
processor_throttling.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157	2019-05-30 11:26:37 -07:00
property.c	device property: Add a function to obtain a node's prefix	2019-10-11 11:26:55 +02:00
reboot.c	ACPI: Allow CONFIG_PCI to be unset for reboot	2018-12-20 10:19:49 +01:00
resource.c	acpi: Use pr_warn instead of pr_warning	2019-10-18 15:00:19 +02:00
sbs.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157	2019-05-30 11:26:37 -07:00
sbshc.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 372	2019-06-05 17:37:10 +02:00
sbshc.h	ACPI: SBS: remove unused const variable 'SMBUS_PEC'	2019-08-23 12:55:09 +02:00
scan.c	treewide: remove redundant IS_ERR() before error code check	2020-02-04 03:05:27 +00:00
sleep.c	ACPI: PM: s2idle: Check fixed wakeup events in acpi_s2idle_wake()	2020-02-21 10:01:25 -08:00
sleep.h	License cleanup: add SPDX GPL-2.0 license identifier to files with no license	2017-11-02 11:10:55 +01:00
spcr.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500	2019-06-19 17:09:55 +02:00
sysfs.c	ACPI: sysfs: Change ACPI_MASKABLE_GPE_MAX to 0x100	2019-11-19 09:40:16 +01:00
tables.c	Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security	2019-09-28 08:14:15 -07:00
thermal.c	ACPI: thermal: switch to use <linux/units.h> helpers	2020-01-31 10:30:39 -08:00
utils.c	ACPI / utils: Introduce acpi_dev_hid_uid_match() helper	2019-10-15 12:06:23 +02:00
video_detect.c	ACPI: video: Do not export a non working backlight interface on MSI MS-7721 boards	2019-12-19 23:01:52 +01:00
wakeup.c	License cleanup: add SPDX GPL-2.0 license identifier to files with no license	2017-11-02 11:10:55 +01:00