Linus Torvalds d6a5c7a1a6 mm: introduce new 'lock_mm_and_find_vma()' page fault helper ... commit c2508ec5a5 upstream. .. and make x86 use it. This basically extracts the existing x86 "find and expand faulting vma" code, but extends it to also take the mmap lock for writing in case we actually do need to expand the vma. We've historically short-circuited that case, and have some rather ugly special logic to serialize the stack segment expansion (since we only hold the mmap lock for reading) that doesn't match the normal VM locking. That slight violation of locking worked well, right up until it didn't: the maple tree code really does want proper locking even for simple extension of an existing vma. So extract the code for "look up the vma of the fault" from x86, fix it up to do the necessary write locking, and make it available as a helper function for other architectures that can use the common helper. Note: I say "common helper", but it really only handles the normal stack-grows-down case. Which is all architectures except for PA-RISC and IA64. So some rare architectures can't use the helper, but if they care they'll just need to open-code this logic. It's also worth pointing out that this code really would like to have an optimistic "mmap_upgrade_trylock()" to make it quicker to go from a read-lock (for the common case) to taking the write lock (for having to extend the vma) in the normal single-threaded situation where there is no other locking activity. But that _is_ all the very uncommon special case, so while it would be nice to have such an operation, it probably doesn't matter in reality. I did put in the skeleton code for such a possible future expansion, even if it only acts as pseudo-documentation for what we're doing. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> [6.1: Ignore CONFIG_PER_VMA_LOCK context] Signed-off-by: Samuel Mendoza-Jonas <samjonas@amazon.com> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>		2023-07-01 13:16:24 +02:00
..
damon	mm/damon/paddr: fix missing folio_put()	2023-03-10 09:34:20 +01:00
kasan	kasan: hw_tags: avoid invalid virt_to_page()	2023-05-11 23:03:39 +09:00
kfence	mm: kfence: fix handling discontiguous page	2023-04-13 16:55:30 +02:00
kmsan	mm: kmsan: handle alloc failures in kmsan_vmap_pages_range_noflush()	2023-04-26 14:28:41 +02:00
backing-dev.c	writeback, cgroup: fix null-ptr-deref write in bdi_split_work_to_wbs	2023-04-26 14:28:39 +02:00
balloon_compaction.c	mm: Convert all PageMovable users to movable_operations	2022-08-02 12:34:03 -04:00
bootmem_info.c	bootmem: remove the vmemmap pages from kmemleak in put_page_bootmem	2022-08-28 14:02:45 -07:00
cma_debug.c	mm/cma_debug: show complete cma name in debugfs directories	2022-09-11 20:25:50 -07:00
cma_sysfs.c
cma.c	Revert "mm/cma.c: remove redundant cma_mutex lock"	2022-05-13 15:11:26 -07:00
cma.h	mm/cma: provide option to opt out from exposing pages on activation failure	2022-03-22 15:57:09 -07:00
compaction.c	Revert "mm/compaction: fix set skip in fast_find_migrateblock"	2023-02-01 08:34:49 +01:00
debug_page_ref.c
debug_vm_pgtable.c	docs: rename Documentation/vm to Documentation/mm	2022-06-27 12:52:53 -07:00
debug.c	mm: remove the vma linked list	2022-09-26 19:46:26 -07:00
dmapool.c
early_ioremap.c	mm/early_ioremap: declare early_memremap_pgprot_adjust()	2022-03-22 15:57:11 -07:00
fadvise.c	riscv: compat: syscall: Add compat_sys_call_table implementation	2022-04-26 13:36:25 -07:00
failslab.c	mm: fix unexpected changes to {failslab|fail_page_alloc}.attr	2022-11-22 18:50:44 -08:00
filemap.c	mm/filemap: fix page end in filemap_get_read_batch	2023-02-22 12:59:49 +01:00
folio-compat.c	mm: remove try_to_free_swap()	2022-10-03 14:02:53 -07:00
frontswap.c	frontswap: don't call ->init if no ops are registered	2022-09-26 12:14:34 -07:00
gup_test.c	mm: rename is_pinnable_page() to is_longterm_pinnable_page()	2022-07-17 17:14:27 -07:00
gup_test.h
gup.c	mm/gup: add folio to list when folio_isolate_lru() succeed	2023-02-22 12:59:54 +01:00
highmem.c	highmem: fix kmap_to_page() for kmap_local_page() addresses	2022-10-12 18:51:51 -07:00
hmm.c	mm/swap: add swp_offset_pfn() to fetch PFN from swap entry	2022-09-26 19:46:05 -07:00
huge_memory.c	mm/huge_memory.c: warn with pr_warn_ratelimited instead of VM_WARN_ON_ONCE_FOLIO	2023-04-26 14:28:41 +02:00
hugetlb_cgroup.c	hugetlb_cgroup: use helper for_each_hstate and hstate_index	2022-09-11 20:25:53 -07:00
hugetlb_vmemmap.c	mm: hugetlb_vmemmap: include missing linux/moduleparam.h	2022-11-08 15:57:23 -08:00
hugetlb_vmemmap.h	mm: hugetlb_vmemmap: improve hugetlb_vmemmap code readability	2022-08-08 18:06:43 -07:00
hugetlb.c	mm/hugetlb: fix uffd wr-protection for CoW optimization path	2023-04-13 16:55:36 +02:00
hwpoison-inject.c	mm/hwpoison: add __init/__exit annotations to module init/exit funcs	2022-10-03 14:03:05 -07:00
init-mm.c	mm: remove rb tree.	2022-09-26 19:46:16 -07:00
internal.h	mm/page_alloc: make boot_nodestats static	2022-10-03 14:03:30 -07:00
interval_tree.c
io-mapping.c
ioremap.c	mm: ioremap: Add ioremap/iounmap_allowed()	2022-06-27 12:22:31 +01:00
Kconfig	mm: introduce new 'lock_mm_and_find_vma()' page fault helper	2023-07-01 13:16:24 +02:00
Kconfig.debug	mm: page_table_check: Make it dependent on EXCLUSIVE_SYSTEM_RAM	2023-06-14 11:15:29 +02:00
khugepaged.c	mm/khugepaged: check again on anon uffd-wp during isolation	2023-04-26 14:28:41 +02:00
kmemleak.c	mm/kmemleak: prevent soft lockup in kmemleak_scan()'s object iteration loops	2022-10-28 13:37:22 -07:00
ksm.c	mm/ksm: fix race with VMA iteration and mm_struct teardown	2023-03-30 12:49:29 +02:00
list_lru.c	mm: kmem: make mem_cgroup_from_obj() vmalloc()-safe	2022-06-16 19:48:31 -07:00
maccess.c	mm: Fix copy_from_user_nofault().	2023-06-28 11:12:17 +02:00
madvise.c	use less confusing names for iov_iter direction initializers	2023-02-09 11:28:04 +01:00
Makefile	mm: memcontrol: drop dead CONFIG_MEMCG_SWAP config symbol	2022-10-03 14:03:36 -07:00
mapping_dirty_helpers.c
memblock.c	Revert "mm: Always release pages to the buddy allocator in memblock_free_late()."	2023-02-22 12:59:50 +01:00
memcontrol.c	mm: memcontrol: deprecate charge moving	2023-03-10 09:34:26 +01:00
memfd.c	memfd: check for non-NULL file_seals in memfd_create() syscall	2023-06-28 11:12:27 +02:00
memory_hotplug.c	mm: add pageblock_aligned() macro	2022-10-03 14:03:04 -07:00
memory-failure.c	mm/hwpoison: convert TTU_IGNORE_HWPOISON to TTU_HWPOISON	2023-03-10 09:34:25 +01:00
memory-tiers.c	memory tier: release the new_memtier in find_create_memory_tier()	2023-03-10 09:34:27 +01:00
memory.c	mm: introduce new 'lock_mm_and_find_vma()' page fault helper	2023-07-01 13:16:24 +02:00
mempolicy.c	mm/mempolicy: correctly update prev when policy is equal on mbind	2023-05-11 23:03:41 +09:00
mempool.c	mm/mempool: use might_alloc()	2022-06-16 19:48:30 -07:00
memremap.c	mm/memremap.c: map FS_DAX device memory as decrypted	2022-11-08 15:57:23 -08:00
memtest.c
migrate_device.c	mm/migrate_device: return number of migrating pages in args->cpages	2022-11-22 18:50:43 -08:00
migrate.c	mm/migrate: fix wrongly apply write bit after mkdirty on sparc64	2023-02-22 12:59:49 +01:00
mincore.c	mm: teach mincore_hugetlb about pte markers	2023-03-22 13:34:03 +01:00
mlock.c	mm/mlock: drop dead code in count_mm_mlocked_page_nr()	2022-09-26 19:46:27 -07:00
mm_init.c	mm: multi-gen LRU: groundwork	2022-09-26 19:46:09 -07:00
mm_slot.h	mm: introduce common struct mm_slot	2022-10-03 14:02:43 -07:00
mmap_lock.c
mmap.c	mm/mmap: Fix error return in do_vmi_align_munmap()	2023-07-01 13:16:22 +02:00
mmu_gather.c	mm/khugepaged: fix GUP-fast interaction by sending IPI	2022-11-30 14:49:42 -08:00
mmu_notifier.c	mm/mmu_notifier.c: fix race in mmu_interval_notifier_remove()	2022-04-21 20:01:10 -07:00
mmzone.c	mm: multi-gen LRU: groundwork	2022-09-26 19:46:09 -07:00
mprotect.c	mm/uffd: fix warning without PTE_MARKER_UFFD_WP compiled in	2022-10-12 15:56:46 -07:00
mremap.c	mm, mremap: fix mremap() expanding for vma's with vm_ops->close()	2023-02-09 11:28:22 +01:00
msync.c	mm/msync: use vma_find() instead of vma linked list	2022-09-26 19:46:25 -07:00
nommu.c	nommu: fix split_vma() map_count error	2023-01-24 07:24:33 +01:00
oom_kill.c	mm: reduce noise in show_mem for lowmem allocations	2022-09-26 19:46:29 -07:00
page_alloc.c	mm/page_alloc: fix potential deadlock on zonelist_update_seq seqlock	2023-04-26 14:28:44 +02:00
page_counter.c	mm: page_counter: remove unneeded atomic ops for low/min	2022-09-11 20:26:01 -07:00
page_ext.c	mm/page_exit: fix kernel doc warning in page_ext_put()	2022-11-22 18:50:41 -08:00
page_idle.c	mm: don't be stuck to rmap lock on reclaim path	2022-05-19 14:08:54 -07:00
page_io.c	use less confusing names for iov_iter direction initializers	2023-02-09 11:28:04 +01:00
page_isolation.c	mm/page_isolation: fix clang deadcode warning	2022-10-28 13:37:22 -07:00
page_owner.c	mm: reuse pageblock_start/end_pfn() macro	2022-10-03 14:03:03 -07:00
page_poison.c
page_reporting.c
page_reporting.h
page_table_check.c	mm: page_table_check: Ensure user pages are not slab pages	2023-06-14 11:15:29 +02:00
page_vma_mapped.c	mm/swap: add swp_offset_pfn() to fetch PFN from swap entry	2022-09-26 19:46:05 -07:00
page-writeback.c	mm: export balance_dirty_pages_ratelimited_flags()	2022-09-26 12:28:07 +02:00
pagewalk.c	- Yu Zhao's Multi-Gen LRU patches are here. They've been under test in	2022-10-10 17:53:04 -07:00
percpu-internal.h	percpu: improve percpu_alloc_percpu event trace	2022-05-13 07:20:18 -07:00
percpu-km.c
percpu-stats.c
percpu-vm.c
percpu.c	mm: percpu: use kmemleak_ignore_phys() instead of kmemleak_free()	2022-07-17 17:14:47 -07:00
pgalloc-track.h
pgtable-generic.c	mm: avoid unnecessary flush on change_huge_pmd()	2022-05-13 07:20:05 -07:00
process_vm_access.c	use less confusing names for iov_iter direction initializers	2023-02-09 11:28:04 +01:00
ptdump.c	mm: pagewalk: Fix race between unmap and page walker	2022-09-03 10:13:13 -07:00
readahead.c	mm: add PSI accounting around ->read_folio and ->readahead calls	2022-09-20 08:24:38 -06:00
rmap.c	mm/hwpoison: convert TTU_IGNORE_HWPOISON to TTU_HWPOISON	2023-03-10 09:34:25 +01:00
rodata_test.c	mm/rodata_test: use PAGE_ALIGNED() helper	2022-10-03 14:03:05 -07:00
secretmem.c	mm/secretmem: remove reduntant return value	2022-10-03 14:03:36 -07:00
shmem.c	mm/shmem: restore SHMEM_HUGE_DENY precedence over MADV_COLLAPSE	2023-01-24 07:24:33 +01:00
shrinker_debug.c	mm: shrinkers: fix deadlock in shrinker debugfs	2023-02-22 12:59:46 +01:00
shuffle.c	mm/shuffle: convert module_param_call to module_param_cb	2022-10-03 14:03:07 -07:00
shuffle.h
slab_common.c	mm, slab: remove duplicate kernel-doc comment for ksize()	2022-11-07 17:11:27 +01:00
slab.c	mm/slab: Fix undefined init_cache_node_node() for NUMA and !SMP	2023-03-30 12:49:23 +02:00
slab.h	- Yu Zhao's Multi-Gen LRU patches are here. They've been under test in	2022-10-10 17:53:04 -07:00
slob.c	Merge branch 'slab/for-6.1/kmalloc_size_roundup' into slab/for-next	2022-09-29 11:30:55 +02:00
slub.c	treewide: use prandom_u32_max() when possible, part 1	2022-10-11 17:42:55 -06:00
sparse-vmemmap.c	mm: hugetlb_vmemmap: move vmemmap code related to HugeTLB to hugetlb_vmemmap.c	2022-08-08 18:06:42 -07:00
sparse.c	mm: memory_hotplug: enumerate all supported section flags	2022-07-03 18:08:49 -07:00
swap_cgroup.c	mm: memcontrol: don't allocate cgroup swap arrays when memcg is disabled	2022-10-03 14:03:36 -07:00
swap_slots.c	mm/swap: convert put_swap_page() to put_swap_folio()	2022-10-03 14:02:46 -07:00
swap_state.c	swap_state: convert free_swap_cache() to use a folio	2022-10-03 14:02:51 -07:00
swap.c	mm: add folio_add_lru_vma()	2022-10-03 14:02:45 -07:00
swap.h	mm: remove lookup_swap_cache()	2022-10-03 14:02:51 -07:00
swapfile.c	mm/swap: fix swap_info_struct race between swapoff and get_swap_pages()	2023-04-13 16:55:36 +02:00
truncate.c	mm: add split_folio()	2022-10-03 14:02:45 -07:00
usercopy.c	mm: Fix copy_from_user_nofault().	2023-06-28 11:12:17 +02:00
userfaultfd.c	mm/shmem: use page_mapping() to detect page cache for uffd continue	2022-11-08 15:57:23 -08:00
util.c	- Yu Zhao's Multi-Gen LRU patches are here. They've been under test in	2022-10-10 17:53:04 -07:00
vmalloc.c	mm: kmsan: handle alloc failures in kmsan_vmap_pages_range_noflush()	2023-04-26 14:28:41 +02:00
vmpressure.c
vmscan.c	mm: do not reclaim private data from pinned page	2023-05-11 23:03:39 +09:00
vmstat.c	- Yu Zhao's Multi-Gen LRU patches are here. They've been under test in	2022-10-10 17:53:04 -07:00
workingset.c	mm: multi-gen LRU: minimal implementation	2022-09-26 19:46:09 -07:00
z3fold.c	mm: Convert all PageMovable users to movable_operations	2022-08-02 12:34:03 -04:00
zbud.c
zpool.c
zsmalloc.c	zsmalloc: zs_destroy_pool: add size_class NULL check	2022-10-20 21:27:21 -07:00
zswap.c	zswap: do not shrink if cgroup may not zswap	2023-06-21 16:00:54 +02:00