This patch adds omap_reserve functionality to board-igep0030.c.
This patch is in similar lines to commit id 71ee7dad9b, from
Russell king
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Enric Balletbo i Serra <eballetbo@gmail.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
GPIO reset line for Beagle XM is different from vanilla beagle
so we populate it as part of gpio update routine.
This in part fixes the issue of display not functioning on beagle XM
platform.
[nm@ti.com: split up, added descriptive changelogs]
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Koen Kooi <koen@beagleboard.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>
EHCI enable power pin is inverted (active high) in comparison
to vanilla beagle which is active low. Handle this case conditionally.
Without this fix, Beagle XM 4 port EHCI will not function and no
networking will be available
[nm@ti.com: split up, added descriptive changelogs]
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Koen Kooi <koen@beagleboard.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>
Support twl4030 keypad and gpio keys on IGEP v2.
Signed-off-by: Enric Balletbo i Serra <eballetbo@gmail.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
The second MMC channel (used by the WLAN/BT module) is not linked to
power regulator. This causes the WLAN/BT module to fail being detected if
CONFIG_REGULATOR_DUMMY is not set.
This patch adds the two regulators that actually feed the WLAN/BT module
(1v8 from the TWL4030 VIO LDO, and a fixed 3v3). With that patch, the
second channel is properly detected.
Also change vmmc1 to use symbolic names instead of direct device
reference.
Signed-off-by: Enric Balletbo i Serra <eballetbo@gmail.com>
Acked-by: Marc Zyngier <maz@misterjones.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>
The OMAP3 IGEP module has one EHCI interface on board using
USB2HS port. GPIO183 is used as PHY reset.
Signed-off-by: Enric Balletbo i Serra <eballetbo@gmail.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
At latest mainline commit 0c21e3aaf6, omap2plus build is broken. This
patch is trivial fix for the missed usb clock node for CK_3430ES2PLUS
flag update.
CHK include/generated/compile.h
CC arch/arm/mach-omap2/clock3xxx_data.o
arch/arm/mach-omap2/clock3xxx_data.c:3289: error: 'CK_3430ES2' undeclared
here (not in a function)
make[1]: *** [arch/arm/mach-omap2/clock3xxx_data.o] Error 1
make: *** [arch/arm/mach-omap2] Error 2
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Anand Gadiyar <gadiyar@ti.com>
Acked-by: Paul Walmsley <paul@pwsan.com>
[tony@atomide.com: updated mask to include CK_36XX]
Signed-off-by: Tony Lindgren <tony@atomide.com>
The SMSC 3320 USB PHY on the OMAP4 Pandaboard needs a 19.2 MHz
reference clock. This clock is provided from the OMAP4's fref_clk3
pad.
Recent changes to clock44xx_data.c made the clock framework aware
of the existence of these fref_clk[i] lines. If the option
CONFIG_OMAP_RESET_CLOCKS is enabled in the kernel, then the
clock framework will turn these clocks off during bootup.
Explicitly request and keep this clock enabled at init for the
Pandaboard, so that the PHY receives this clock at all times.
Reported-by: Ming Lei <tom.leiming@gmail.com>
Signed-off-by: Anand Gadiyar <gadiyar@ti.com>
Cc: Benoit Cousson <b-cousson@ti.com>
Reviewed-by: Paul Walmsley <paul@pwsan.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
omap2plus_defocnfig build breaks when customised with only ARCH_OMAP4
selected. This is because common files make references to the functions
which are defined only for omap2xxx and omap3xxx.
LD .tmp_vmlinux1
arch/arm/mach-omap2/built-in.o: In function `pm_dbg_regset_store':
arch/arm/mach-omap2/pm-debug.c:335: undefined reference to `omap2_prm_read_mod_reg'
arch/arm/mach-omap2/built-in.o: In function `omap2_pm_dump':
arch/arm/mach-omap2/pm-debug.c:121: undefined reference to `omap2_prm_read_mod_reg'
arch/arm/mach-omap2/pm-debug.c:123: undefined reference to `omap2_prm_read_mod_reg'
arch/arm/mach-omap2/pm-debug.c:124: undefined reference to `omap2_prm_read_mod_reg'
arch/arm/mach-omap2/pm-debug.c:125: undefined reference to `omap2_prm_read_mod_reg'
arch/arm/mach-omap2/built-in.o: In function `omap_prcm_arch_reset':
arch/arm/mach-omap2/prcm.c:106: undefined reference to `omap2_prm_set_mod_reg_bits'
arch/arm/mach-omap2/prcm.c:108: undefined reference to `omap2_prm_read_mod_reg'
arch/arm/mach-omap2/built-in.o: In function `omap_prcm_get_reset_sources':
arch/arm/mach-omap2/prcm.c:53: undefined reference to `omap2_prm_read_mod_reg'
arch/arm/mach-omap2/built-in.o: In function `clkdm_clear_all_wkdeps':
arch/arm/mach-omap2/clockdomain.c:545: undefined reference to `omap2_prm_clear_mod_reg_bits'
arch/arm/mach-omap2/built-in.o: In function `clkdm_del_wkdep':
arch/arm/mach-omap2/clockdomain.c:475: undefined reference to `omap2_prm_clear_mod_reg_bits'
arch/arm/mach-omap2/built-in.o: In function `clkdm_read_wkdep':
arch/arm/mach-omap2/clockdomain.c:511: undefined reference to `omap2_prm_read_mod_bits_shift'
arch/arm/mach-omap2/built-in.o: In function `clkdm_add_wkdep':
arch/arm/mach-omap2/clockdomain.c:440: undefined reference to `omap2_prm_set_mod_reg_bits'
make: *** [.tmp_vmlinux1] Error 1
This patch adds stubs for these functions so that build continues to work.
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Paul Walmsley <paul@pwsan.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
CC arch/arm/mach-omap2/omap_hwmod_common_data.o
In file included from arch/arm/plat-omap/include/plat/omap_hwmod.h:38,
from arch/arm/mach-omap2/omap_hwmod_common_data.c:20:
arch/arm/plat-omap/include/plat/voltage.h: In function 'omap_voltage_late_init':
arch/arm/plat-omap/include/plat/voltage.h:145: error: 'EINVAL' undeclared (first use in this function)
arch/arm/plat-omap/include/plat/voltage.h:145: error: (Each undeclared identifier is reported only once
arch/arm/plat-omap/include/plat/voltage.h:145: error: for each function it appears in.)
make[1]: *** [arch/arm/mach-omap2/omap_hwmod_common_data.o] Error 1
make: *** [arch/arm/mach-omap2] Error 2
The error is reported when omap2plus_defconfig built with CONFIG_PM disabled
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Thara Gopinath <thara@ti.com>
Cc: Kevin Hilman <khilman@deeprootsystems.com>
Cc: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
LD init/built-in.o
LD .tmp_vmlinux1
arch/arm/mach-omap2/built-in.o: In function `omap2_set_init_voltage':
arch/arm/mach-omap2/pm.c:181: undefined reference to `omap_voltage_domain_lookup'
arch/arm/mach-omap2/built-in.o: In function `omap4_twl_init':
arch/arm/mach-omap2/omap_twl.c:244: undefined reference to `omap_voltage_domain_lookup'
arch/arm/mach-omap2/omap_twl.c:247: undefined reference to `omap_voltage_domain_lookup'
arch/arm/mach-omap2/omap_twl.c:250: undefined reference to `omap_voltage_domain_lookup'
make: *** [.tmp_vmlinux1] Error 1
The error is reported when omap2plus_defconfig built with CONFIG_PM disabled
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Thara Gopinath <thara@ti.com>
Cc: Kevin Hilman <khilman@deeprootsystems.com>
Cc: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
Fix below build warnings
CC arch/arm/mach-omap2/common.o
CC arch/arm/mach-omap2/gpio.o
In file included from arch/arm/plat-omap/include/plat/omap_hwmod.h:38,
from arch/arm/mach-omap2/gpio.c:25:
arch/arm/plat-omap/include/plat/voltage.h: In function 'omap_voltage_register_pmic':
arch/arm/plat-omap/include/plat/voltage.h:137: warning: no return statement in function returning non-void
CC arch/arm/mach-omap2/dma.o
In file included from arch/arm/plat-omap/include/plat/omap_hwmod.h:38,
from arch/arm/mach-omap2/dma.c:32:
arch/arm/plat-omap/include/plat/voltage.h: In function 'omap_voltage_register_pmic':
arch/arm/plat-omap/include/plat/voltage.h:137: warning: no return statement in function returning non-void
CC arch/arm/mach-omap2/wd_timer.o
In file included from arch/arm/plat-omap/include/plat/omap_hwmod.h:38,
from arch/arm/mach-omap2/wd_timer.c:15:
arch/arm/plat-omap/include/plat/voltage.h: In function 'omap_voltage_register_pmic':
arch/arm/plat-omap/include/plat/voltage.h:137: warning: no return statement in function returning non-void
CC arch/arm/mach-omap2/prm44xx.o
CC arch/arm/mach-omap2/omap_hwmod.o
In file included from arch/arm/plat-omap/include/plat/omap_hwmod.h:38,
from arch/arm/mach-omap2/omap_hwmod.c:145:
arch/arm/plat-omap/include/plat/voltage.h: In function 'omap_voltage_register_pmic':
arch/arm/plat-omap/include/plat/voltage.h:137: warning: no return statement in function returning non-void
CC arch/arm/mach-omap2/omap_hwmod_common_data.o
In file included from arch/arm/plat-omap/include/plat/omap_hwmod.h:38,
from arch/arm/mach-omap2/omap_hwmod_common_data.c:20:
arch/arm/plat-omap/include/plat/voltage.h: In function 'omap_voltage_register_pmic':
arch/arm/plat-omap/include/plat/voltage.h:137: warning: no return statement in function returning non-void
The error is reported when omap2plus_defconfig built with CONFIG_PM disabled
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Thara Gopinath <thara@ti.com>
Cc: Kevin Hilman <khilman@deeprootsystems.com>
Cc: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
struct clockdomain member clktrctrl_mask is available for only for OMAP2
and OMAP3 architectures. Technially it is also used only for these archs
but this breaks the build with custom OMAP4 configuration.
CC arch/arm/mach-omap2/clockdomain.o
arch/arm/mach-omap2/clockdomain.c: In function '_enable_hwsup':
arch/arm/mach-omap2/clockdomain.c:251: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c:254: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c: In function '_disable_hwsup':
arch/arm/mach-omap2/clockdomain.c:277: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c:280: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c: In function 'omap2_clkdm_sleep':
arch/arm/mach-omap2/clockdomain.c:744: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c: In function 'omap2_clkdm_wakeup':
arch/arm/mach-omap2/clockdomain.c:789: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c: In function 'omap2_clkdm_clk_enable':
arch/arm/mach-omap2/clockdomain.c:922: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c:926: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c: In function 'omap2_clkdm_clk_disable':
arch/arm/mach-omap2/clockdomain.c:994: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
arch/arm/mach-omap2/clockdomain.c:998: error: 'struct clockdomain' has no member named 'clktrctrl_mask'
make[1]: *** [arch/arm/mach-omap2/clockdomain.o] Error 1
make: *** [arch/arm/mach-omap2] Error 2
Fix the build break by dropping the #ifdef as suggested by Paul Walmsley
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Paul Walmsley <paul@pwsan.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
Instead of accessing the irq_desc array directly
we can use irq_to_desc(irq). That will allow us to,
if wanted, select SPARSE_IRQ and irq_descs will be
added to a radix tree, instead of a array.
Signed-off-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
omap_pm_runtime_suspend and omap_pm_runtime_resume are used
as function pointers and does not really need to be exposed
to the world.
Fixes sparse warnings:
arch/arm/mach-omap2/pm_bus.c:23:5: warning: symbol 'omap_pm_runtime_suspend' was not declared. Should it be static?
arch/arm/mach-omap2/pm_bus.c:40:5: warning: symbol 'omap_pm_runtime_resume' was not declared. Should it be static?
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
In case on OMAP2+ we call set_24xx_gpio_triggering() instead of
updating reg and l values. However, at the end of the function we
perform a write:
__raw_writel(l, reg);
So on OMAP2+ we end up writing 0 to the bank->base which is not
correct (typically this points to GPIO_REVISION register).
Fix this by returning immediately after call to
set_24xx_gpio_triggering().
Signed-off-by: Mika Westerberg <ext-mika.1.westerberg@nokia.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
twl_init functions are declared in pm.h and used in pm.c
pm.h header defining the protos need to be included to
ensure that omap_twl.c has consistent function definition.
This fixes sparse warning:
arch/arm/mach-omap2/omap_twl.c:237:12: warning: symbol 'omap4_twl_init' was not declared. Should it be static?
arch/arm/mach-omap2/omap_twl.c:256:12: warning: symbol 'omap3_twl_init' was not declared. Should it be static?
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
The uv_to_vsel, vsel_to_uv functions don't need to be exposed to the
world as they are used as function pointers. make them static.
Fixes sparse warnings:
arch/arm/mach-omap2/omap_twl.c:63:15: warning: symbol 'twl4030_vsel_to_uv' was not declared. Should it be static?
arch/arm/mach-omap2/omap_twl.c:68:4: warning: symbol 'twl4030_uv_to_vsel' was not declared. Should it be static?
arch/arm/mach-omap2/omap_twl.c:73:15: warning: symbol 'twl6030_vsel_to_uv' was not declared. Should it be static?
arch/arm/mach-omap2/omap_twl.c:105:4: warning: symbol 'twl6030_uv_to_vsel' was not declared. Should it be static?
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
omap_enable_smartreflex_on_init is meant to be used by boards
which would like to have SR enabled by default on the platform, while
omap_devinit_smartreflex is used by pm code, the protos are defined
in pm.h. This header should be included to ensure that sr_device
function definitions match the prototypes.
including pm.h fixes the sparse warnings (with CONFIG_OMAP_SMARTREFLEX=y):
arch/arm/mach-omap2/sr_device.c:138:13: warning: symbol 'omap_enable_smartreflex_on_init' was not declared. Should it be static?
arch/arm/mach-omap2/sr_device.c:143:12: warning: symbol 'omap_devinit_smartreflex' was not declared. Should it be static?
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
* 'usb-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb-2.6: (144 commits)
USB: add support for Dream Cheeky DL100B Webmail Notifier (1d34:0004)
USB: serial: ftdi_sio: add support for TIOCSERGETLSR
USB: ehci-mxc: Setup portsc register prior to accessing OTG viewport
USB: atmel_usba_udc: fix freeing irq in usba_udc_remove()
usb: ehci-omap: fix tll channel enable mask
usb: ohci-omap3: fix trivial typo
USB: gadget: ci13xxx: don't assume that PAGE_SIZE is 4096
USB: gadget: ci13xxx: fix complete() callback for no_interrupt rq's
USB: gadget: update ci13xxx to work with g_ether
USB: gadgets: ci13xxx: fix probing of compiled-in gadget drivers
Revert "USB: musb: pm: don't rely fully on clock support"
Revert "USB: musb: blackfin: pm: make it work"
USB: uas: Use GFP_NOIO instead of GFP_KERNEL in I/O submission path
USB: uas: Ensure we only bind to a UAS interface
USB: uas: Rename sense pipe and sense urb to status pipe and status urb
USB: uas: Use kzalloc instead of kmalloc
USB: uas: Fix up the Sense IU
usb: musb: core: kill unneeded #include's
DA8xx: assign name to MUSB IRQ resource
usb: gadget: g_ncm added
...
Manually fix up trivial conflicts in USB Kconfig changes in:
arch/arm/mach-omap2/Kconfig
arch/sh/Kconfig
drivers/usb/Kconfig
drivers/usb/host/ehci-hcd.c
and annoying chip clock data conflicts in:
arch/arm/mach-omap2/clock3xxx_data.c
arch/arm/mach-omap2/clock44xx_data.c
* git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6: (147 commits)
[SCSI] arcmsr: fix write to device check
[SCSI] lpfc: lower stack use in lpfc_fc_frame_check
[SCSI] eliminate an unnecessary local variable from scsi_remove_target()
[SCSI] libiscsi: use bh locking instead of irq with session lock
[SCSI] libiscsi: do not take host lock in queuecommand
[SCSI] be2iscsi: fix null ptr when accessing task hdr
[SCSI] be2iscsi: fix gfp use in alloc_pdu
[SCSI] libiscsi: add more informative failure message during iscsi scsi eh
[SCSI] gdth: Add missing call to gdth_ioctl_free
[SCSI] bfa: remove unused defintions and misc cleanups
[SCSI] bfa: remove inactive functions
[SCSI] bfa: replace bfa_assert with WARN_ON
[SCSI] qla2xxx: Use sg_next to fetch next sg element while walking sg list.
[SCSI] qla2xxx: Fix to avoid recursive lock failure during BSG timeout.
[SCSI] qla2xxx: Remove code to not reset ISP82xx on failure.
[SCSI] qla2xxx: Display mailbox register 4 during 8012 AEN for ISP82XX parts.
[SCSI] qla2xxx: Don't perform a BIG_HAMMER if Get-ID (0x20) mailbox command fails on CNAs.
[SCSI] qla2xxx: Remove redundant module parameter permission bits
[SCSI] qla2xxx: Add sysfs node for displaying board temperature.
[SCSI] qla2xxx: Code cleanup to remove unwanted comments and code.
...
There was a semi-colon missing and it broke the compile.
Signed-off-by: Dan Carpenter <error27@gmail.com>
Cc: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The problem that this patch aims to fix is vfsmount refcounting scalability.
We need to take a reference on the vfsmount for every successful path lookup,
which often go to the same mount point.
The fundamental difficulty is that a "simple" reference count can never be made
scalable, because any time a reference is dropped, we must check whether that
was the last reference. To do that requires communication with all other CPUs
that may have taken a reference count.
We can make refcounts more scalable in a couple of ways, involving keeping
distributed counters, and checking for the global-zero condition less
frequently.
- check the global sum once every interval (this will delay zero detection
for some interval, so it's probably a showstopper for vfsmounts).
- keep a local count and only taking the global sum when local reaches 0 (this
is difficult for vfsmounts, because we can't hold preempt off for the life of
a reference, so a counter would need to be per-thread or tied strongly to a
particular CPU which requires more locking).
- keep a local difference of increments and decrements, which allows us to sum
the total difference and hence find the refcount when summing all CPUs. Then,
keep a single integer "long" refcount for slow and long lasting references,
and only take the global sum of local counters when the long refcount is 0.
This last scheme is what I implemented here. Attached mounts and process root
and working directory references are "long" references, and everything else is
a short reference.
This allows scalable vfsmount references during path walking over mounted
subtrees and unattached (lazy umounted) mounts with processes still running
in them.
This results in one fewer atomic op in the fastpath: mntget is now just a
per-CPU inc, rather than an atomic inc; and mntput just requires a spinlock
and non-atomic decrement in the common case. However code is otherwise bigger
and heavier, so single threaded performance is basically a wash.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Suggested by Andreas, mnt_ prefix is clearer namespace, follows kernel
conventions better, and is easier for tab complete. I introduced these
names so I'll admit they were not good choices.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
The standard memcmp function on a Westmere system shows up hot in
profiles in the `git diff` workload (both parallel and single threaded),
and it is likely due to the costs associated with trapping into
microcode, and little opportunity to improve memory access (dentry
name is not likely to take up more than a cacheline).
So replace it with an open-coded byte comparison. This increases code
size by 8 bytes in the critical __d_lookup_rcu function, but the
speedup is huge, averaging 10 runs of each:
git diff st user sys elapsed CPU
before 1.15 2.57 3.82 97.1
after 1.14 2.35 3.61 96.8
git diff mt user sys elapsed CPU
before 1.27 3.85 1.46 349
after 1.26 3.54 1.43 333
Elapsed time for single threaded git diff at 95.0% confidence:
-0.21 +/- 0.01
-5.45% +/- 0.24%
It's -0.66% +/- 0.06% elapsed time on my Opteron, so rep cmp costs on the
fam10h seem to be relatively smaller, but there is still a win.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
This makes single threaded git diff -1.25% +/- 0.05% elapsed time on my
2s12c24t Westmere system, and -0.86% +/- 0.05% on my 2s8c Barcelona, by
prefetching the important first cacheline of the inode in while we do the
actual name compare and other operations on the dentry.
There was no measurable slowdown in the single file stat case, or the creat
case (where negative dentries would be common).
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Regardless of how much we possibly try to scale dcache, there is likely
always going to be some fundamental contention when adding or removing children
under the same parent. Pseudo filesystems do not seem need to have connected
dentries because by definition they are disconnected.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
dcache_inode_lock can be replaced with per-inode locking. Use existing
inode->i_lock for this. This is slightly non-trivial because we sometimes
need to find the inode from the dentry, which requires d_inode to be
stabilised (either with refcount or d_lock).
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Introduce a type of hlist that can support the use of the lowest bit in the
hlist_head. This will be subsequently used to implement per-bucket bit spinlock
for inode and dentry hashes, and may be useful in other cases such as network
hashes.
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
This simple implementation just checks for no ACLs on the inode, and
if so, then the rcu-walk may proceed, otherwise fail it.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
This simple implementation just checks for no ACLs on the inode, and
if so, then the rcu-walk may proceed, otherwise fail it.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
This simple implementation just checks for no ACLs on the inode, and
if so, then the rcu-walk may proceed, otherwise fail it.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
This simple implementation just checks for no ACLs on the inode, and
if so, then the rcu-walk may proceed, otherwise fail it.
This could easily be extended to put acls under RCU and check them
under seqlock, if need be. But this implementation is enough to show
the rcu-walk aware permissions code for path lookups is working, and
will handle cases where there are no ACLs or ACLs in just the final
element.
This patch implicity converts tmpfs to rcu-aware permission check.
Subsequent patches onvert ext*, xfs, and, btrfs. Each of these uses
acl/permission code in a different way, so convert them all to provide
templates and proof of concept.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Require filesystems be aware of .d_revalidate being called in rcu-walk
mode (nd->flags & LOOKUP_RCU). For now do a simple push down, returning
-ECHILD from all implementations.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Put dentry and inode fields into top of data structure. This allows RCU path
traversal to perform an RCU dentry lookup in a path walk by touching only the
first 56 bytes of the dentry.
We also fit in 8 bytes of inline name in the first 64 bytes, so for short
names, only 64 bytes needs to be touched to perform the lookup. We should
get rid of the hash->prev pointer from the first 64 bytes, and fit 16 bytes
of name in there, which will take care of 81% rather than 32% of the kernel
tree.
inode is also rearranged so that RCU lookup will only touch a single cacheline
in the inode, plus one in the i_ops structure.
This is important for directory component lookups in RCU path walking. In the
kernel source, directory names average is around 6 chars, so this works.
When we reach the last element of the lookup, we need to lock it and take its
refcount which requires another cacheline access.
Align dentry and inode operations structs, so members will be at predictable
offsets and we can group common operations into head of structure.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Reduce some branches and memory accesses in dcache lookup by adding dentry
flags to indicate common d_ops are set, rather than having to check them.
This saves a pointer memory access (dentry->d_op) in common path lookup
situations, and saves another pointer load and branch in cases where we
have d_op but not the particular operation.
Patched with:
git grep -E '[.>]([[:space:]])*d_op([[:space:]])*=' | xargs sed -e 's/\([^\t ]*\)->d_op = \(.*\);/d_set_d_op(\1, \2);/' -e 's/\([^\t ]*\)\.d_op = \(.*\);/d_set_d_op(\&\1, \2);/' -i
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Rather than keep a d_mounted count in the dentry, set a dentry flag instead.
The flag can be cleared by checking the hash table to see if there are any
mounts left, which is not time critical because it is performed at detach time.
The mounted state of a dentry is only used to speculatively take a look in the
mount hash table if it is set -- before following the mount, vfsmount lock is
taken and mount re-checked without races.
This saves 4 bytes on 32-bit, nothing on 64-bit but it does provide a hole I
might use later (and some configs have larger than 32-bit spinlocks which might
make use of the hole).
Autofs4 conversion and changelog by Ian Kent <raven@themaw.net>:
In autofs4, when expring direct (or offset) mounts we need to ensure that we
block user path walks into the autofs mount, which is covered by another mount.
To do this we clear the mounted status so that follows stop before walking into
the mount and are essentially blocked until the expire is completed. The
automount daemon still finds the correct dentry for the umount due to the
follow mount logic in fs/autofs4/root.c:autofs4_follow_link(), which is set as
an inode operation for direct and offset mounts only and is called following
the lookup that stopped at the covered mount.
At the end of the expire the covering mount probably has gone away so the
mounted status need not be restored. But we need to check this and only restore
the mounted status if the expire failed.
XXX: autofs may not work right if we have other mounts go over the top of it?
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Use a seqlock in the fs_struct to enable us to take an atomic copy of the
complete cwd and root paths. Use this in the RCU lookup path to avoid a
thread-shared spinlock in RCU lookup operations.
Multi-threaded apps may now perform path lookups with scalability matching
multi-process apps. Operations such as stat(2) become very scalable for
multi-threaded workload.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Perform common cases of path lookups without any stores or locking in the
ancestor dentry elements. This is called rcu-walk, as opposed to the current
algorithm which is a refcount based walk, or ref-walk.
This results in far fewer atomic operations on every path element,
significantly improving path lookup performance. It also avoids cacheline
bouncing on common dentries, significantly improving scalability.
The overall design is like this:
* LOOKUP_RCU is set in nd->flags, which distinguishes rcu-walk from ref-walk.
* Take the RCU lock for the entire path walk, starting with the acquiring
of the starting path (eg. root/cwd/fd-path). So now dentry refcounts are
not required for dentry persistence.
* synchronize_rcu is called when unregistering a filesystem, so we can
access d_ops and i_ops during rcu-walk.
* Similarly take the vfsmount lock for the entire path walk. So now mnt
refcounts are not required for persistence. Also we are free to perform mount
lookups, and to assume dentry mount points and mount roots are stable up and
down the path.
* Have a per-dentry seqlock to protect the dentry name, parent, and inode,
so we can load this tuple atomically, and also check whether any of its
members have changed.
* Dentry lookups (based on parent, candidate string tuple) recheck the parent
sequence after the child is found in case anything changed in the parent
during the path walk.
* inode is also RCU protected so we can load d_inode and use the inode for
limited things.
* i_mode, i_uid, i_gid can be tested for exec permissions during path walk.
* i_op can be loaded.
When we reach the destination dentry, we lock it, recheck lookup sequence,
and increment its refcount and mountpoint refcount. RCU and vfsmount locks
are dropped. This is termed "dropping rcu-walk". If the dentry refcount does
not match, we can not drop rcu-walk gracefully at the current point in the
lokup, so instead return -ECHILD (for want of a better errno). This signals the
path walking code to re-do the entire lookup with a ref-walk.
Aside from the final dentry, there are other situations that may be encounted
where we cannot continue rcu-walk. In that case, we drop rcu-walk (ie. take
a reference on the last good dentry) and continue with a ref-walk. Again, if
we can drop rcu-walk gracefully, we return -ECHILD and do the whole lookup
using ref-walk. But it is very important that we can continue with ref-walk
for most cases, particularly to avoid the overhead of double lookups, and to
gain the scalability advantages on common path elements (like cwd and root).
The cases where rcu-walk cannot continue are:
* NULL dentry (ie. any uncached path element)
* parent with d_inode->i_op->permission or ACLs
* dentries with d_revalidate
* Following links
In future patches, permission checks and d_revalidate become rcu-walk aware. It
may be possible eventually to make following links rcu-walk aware.
Uncached path elements will always require dropping to ref-walk mode, at the
very least because i_mutex needs to be grabbed, and objects allocated.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Add branch annotations for seqlock read fastpath, and introduce
__read_seqcount_begin and __read_seqcount_end functions, that can avoid the
smp_rmb() if used carefully. These will be used by store-free path walking
algorithm performance is critical and seqlocks are in use.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
Pseudo filesystems that don't put inode on RCU list or reachable by
rcu-walk dentries do not need to RCU free their inodes.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
RCU free the struct inode. This will allow:
- Subsequent store-free path walking patch. The inode must be consulted for
permissions when walking, so an RCU inode reference is a must.
- sb_inode_list_lock to be moved inside i_lock because sb list walkers who want
to take i_lock no longer need to take sb_inode_list_lock to walk the list in
the first place. This will simplify and optimize locking.
- Could remove some nested trylock loops in dcache code
- Could potentially simplify things a bit in VM land. Do not need to take the
page lock to follow page->mapping.
The downsides of this is the performance cost of using RCU. In a simple
creat/unlink microbenchmark, performance drops by about 10% due to inability to
reuse cache-hot slab objects. As iterations increase and RCU freeing starts
kicking over, this increases to about 20%.
In cases where inode lifetimes are longer (ie. many inodes may be allocated
during the average life span of a single inode), a lot of this cache reuse is
not applicable, so the regression caused by this patch is smaller.
The cache-hot regression could largely be avoided by using SLAB_DESTROY_BY_RCU,
however this adds some complexity to list walking and store-free path walking,
so I prefer to implement this at a later date, if it is shown to be a win in
real situations. I haven't found a regression in any non-micro benchmark so I
doubt it will be a problem.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
The tricky locking for disposing of a dentry is duplicated 3 times in the
dcache (dput, pruning a dentry from the LRU, and pruning its ancestors).
Consolidate them all into a single function dentry_kill.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
