Go to file
Kirill A. Shutemov 342ee87758 mm, x86/mm: Fix performance regression in get_user_pages_fast()
[ Upstream commit 5b65c4677a ]

The 0-day test bot found a performance regression that was tracked down to
switching x86 to the generic get_user_pages_fast() implementation:

  http://lkml.kernel.org/r/20170710024020.GA26389@yexl-desktop

The regression was caused by the fact that we now use local_irq_save() +
local_irq_restore() in get_user_pages_fast() to disable interrupts.
In x86 implementation local_irq_disable() + local_irq_enable() was used.

The fix is to make get_user_pages_fast() use local_irq_disable(),
leaving local_irq_save() for __get_user_pages_fast() that can be called
with interrupts disabled.

Numbers for pinning a gigabyte of memory, one page a time, 20 repeats:

  Before:  Average: 14.91 ms, stddev: 0.45 ms
  After:   Average: 10.76 ms, stddev: 0.18 ms

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Huang Ying <ying.huang@intel.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Thorsten Leemhuis <regressions@leemhuis.info>
Cc: linux-mm@kvack.org
Fixes: e585513b76 ("x86/mm/gup: Switch GUP to the generic get_user_page_fast() implementation")
Link: http://lkml.kernel.org/r/20170908215603.9189-3-kirill.shutemov@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-12-10 13:40:43 +01:00
arch s390/ptrace: fix guarded storage regset handling 2017-12-10 13:40:43 +01:00
block block: Fix a race between blk_cleanup_queue() and timeout handling 2017-11-30 08:40:52 +00:00
certs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
crypto crypto: skcipher - Fix skcipher_walk_aead_common 2017-12-05 11:26:30 +01:00
Documentation dt-bindings: timer: renesas, cmt: Fix SoC-specific compatible values 2017-12-10 13:40:41 +01:00
drivers staging: fsl-mc/dpio: Fix incorrect comparison 2017-12-10 13:40:42 +01:00
firmware License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fs lockd: lost rollback of set_grace_period() in lockd_down_net() 2017-12-05 11:26:30 +01:00
include locking/refcounts, x86/asm: Use unique .text section for refcount exceptions 2017-12-10 13:40:42 +01:00
init License cleanup: add SPDX license identifiers to some files 2017-11-02 10:04:46 -07:00
ipc License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
kernel kprobes: Use synchronize_rcu_tasks() for optprobe with CONFIG_PREEMPT=y 2017-12-10 13:40:40 +01:00
lib lib/mpi: call cond_resched() from mpi_powm() loop 2017-11-30 08:40:39 +00:00
mm mm, x86/mm: Fix performance regression in get_user_pages_fast() 2017-12-10 13:40:43 +01:00
net NFC: fix device-allocation error return 2017-11-30 08:40:55 +00:00
samples License cleanup: add SPDX license identifiers to some files 2017-11-02 10:04:46 -07:00
scripts scripts: add leaking_addresses.pl 2017-11-06 11:46:42 -08:00
security ima: fix hash algorithm initialization 2017-12-10 13:40:38 +01:00
sound ASoC: sun8i-codec: Set the BCLK divider 2017-11-30 08:40:49 +00:00
tools perf tools: Fix leaking rec_argv in error cases 2017-12-10 13:40:43 +01:00
usr initramfs: fix initramfs rebuilds w/ compression after disabling 2017-11-03 07:39:19 -07:00
virt Fixes for interrupt controller emulation in ARM/ARM64 and x86, plus a one-liner 2017-11-04 11:44:55 -07:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore
.gitattributes .gitattributes: set git diff driver for C source code files 2016-10-07 18:46:30 -07:00
.gitignore kbuild: Add support to generate LLVM assembly files 2017-04-25 08:13:52 +09:00
.mailmap .mailmap: Add Maciej W. Rozycki's Imagination e-mail address 2017-11-10 12:16:15 -08:00
COPYING
CREDITS MAINTAINERS: update TPM driver infrastructure changes 2017-11-09 17:58:40 -08:00
Kbuild License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Kconfig License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
MAINTAINERS Merge branch 'akpm' (patches from Andrew) 2017-11-09 18:26:51 -08:00
Makefile Linux 4.14.4 2017-12-05 11:26:38 +01:00
README README: add a new README file, pointing to the Documentation/ 2016-10-24 08:12:35 -02:00

Linux kernel
============

This file was moved to Documentation/admin-guide/README.rst

Please notice that there are several guides for kernel developers and users.
These guides can be rendered in a number of formats, like HTML and PDF.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.
See Documentation/00-INDEX for a list of what is contained in each file.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.