mirror of
https://mirrors.bfsu.edu.cn/git/linux.git
synced 2024-12-19 09:04:51 +08:00
1d65b771bc
Simplify shmem and file THP collapse's retract_page_tables(), and relax its locking: to improve its success rate and to lessen impact on others. Instead of its MADV_COLLAPSE case doing set_huge_pmd() at target_addr of target_mm, leave that part of the work to madvise_collapse() calling collapse_pte_mapped_thp() afterwards: just adjust collapse_file()'s result code to arrange for that. That spares retract_page_tables() four arguments; and since it will be successful in retracting all of the page tables expected of it, no need to track and return a result code itself. It needs i_mmap_lock_read(mapping) for traversing the vma interval tree, but it does not need i_mmap_lock_write() for that: page_vma_mapped_walk() allows for pte_offset_map_lock() etc to fail, and uses pmd_lock() for THPs. retract_page_tables() just needs to use those same spinlocks to exclude it briefly, while transitioning pmd from page table to none: so restore its use of pmd_lock() inside of which pte lock is nested. Users of pte_offset_map_lock() etc all now allow for them to fail: so retract_page_tables() now has no use for mmap_write_trylock() or vma_try_start_write(). In common with rmap and page_vma_mapped_walk(), it does not even need the mmap_read_lock(). But those users do expect the page table to remain a good page table, until they unlock and rcu_read_unlock(): so the page table cannot be freed immediately, but rather by the recently added pte_free_defer(). Use the (usually a no-op) pmdp_get_lockless_sync() to send an interrupt when PAE, and pmdp_collapse_flush() did not already do so: to make sure that the start,pmdp_get_lockless(),end sequence in __pte_offset_map() cannot pick up a pmd entry with mismatched pmd_low and pmd_high. retract_page_tables() can be enhanced to replace_page_tables(), which inserts the final huge pmd without mmap lock: going through an invalid state instead of pmd_none() followed by fault. But that enhancement does raise some more questions: leave it until a later release. Link: https://lkml.kernel.org/r/f88970d9-d347-9762-ae6d-da978e8a4df@google.com Signed-off-by: Hugh Dickins <hughd@google.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> Cc: Anshuman Khandual <anshuman.khandual@arm.com> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Christoph Hellwig <hch@infradead.org> Cc: Claudio Imbrenda <imbrenda@linux.ibm.com> Cc: David Hildenbrand <david@redhat.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Huang, Ying <ying.huang@intel.com> Cc: Ira Weiny <ira.weiny@intel.com> Cc: Jann Horn <jannh@google.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Lorenzo Stoakes <lstoakes@gmail.com> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Miaohe Lin <linmiaohe@huawei.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Mike Kravetz <mike.kravetz@oracle.com> Cc: Mike Rapoport (IBM) <rppt@kernel.org> Cc: Minchan Kim <minchan@kernel.org> Cc: Naoya Horiguchi <naoya.horiguchi@nec.com> Cc: Pavel Tatashin <pasha.tatashin@soleen.com> Cc: Peter Xu <peterx@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Qi Zheng <zhengqi.arch@bytedance.com> Cc: Ralph Campbell <rcampbell@nvidia.com> Cc: Russell King <linux@armlinux.org.uk> Cc: SeongJae Park <sj@kernel.org> Cc: Song Liu <song@kernel.org> Cc: Steven Price <steven.price@arm.com> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Vishal Moola (Oracle) <vishal.moola@gmail.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Will Deacon <will@kernel.org> Cc: Yang Shi <shy828301@gmail.com> Cc: Yu Zhao <yuzhao@google.com> Cc: Zack Rusin <zackr@vmware.com> Cc: Zi Yan <ziy@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|---|---|---|
| .. | ||
| damon | ||
| kasan | ||
| kfence | ||
| kmsan | ||
| backing-dev.c | ||
| balloon_compaction.c | ||
| bootmem_info.c | ||
| cma_debug.c | ||
| cma_sysfs.c | ||
| cma.c | ||
| cma.h | ||
| compaction.c | ||
| debug_page_alloc.c | ||
| debug_page_ref.c | ||
| debug_vm_pgtable.c | ||
| debug.c | ||
| dmapool_test.c | ||
| dmapool.c | ||
| early_ioremap.c | ||
| fadvise.c | ||
| fail_page_alloc.c | ||
| failslab.c | ||
| filemap.c | ||
| folio-compat.c | ||
| frontswap.c | ||
| gup_test.c | ||
| gup_test.h | ||
| gup.c | ||
| highmem.c | ||
| hmm.c | ||
| huge_memory.c | ||
| hugetlb_cgroup.c | ||
| hugetlb_vmemmap.c | ||
| hugetlb_vmemmap.h | ||
| hugetlb.c | ||
| hwpoison-inject.c | ||
| init-mm.c | ||
| internal.h | ||
| interval_tree.c | ||
| io-mapping.c | ||
| ioremap.c | ||
| Kconfig | ||
| Kconfig.debug | ||
| khugepaged.c | ||
| kmemleak.c | ||
| ksm.c | ||
| list_lru.c | ||
| maccess.c | ||
| madvise.c | ||
| Makefile | ||
| mapping_dirty_helpers.c | ||
| memblock.c | ||
| memcontrol.c | ||
| memfd.c | ||
| memory_hotplug.c | ||
| memory-failure.c | ||
| memory-tiers.c | ||
| memory.c | ||
| mempolicy.c | ||
| mempool.c | ||
| memremap.c | ||
| memtest.c | ||
| migrate_device.c | ||
| migrate.c | ||
| mincore.c | ||
| mlock.c | ||
| mm_init.c | ||
| mm_slot.h | ||
| mmap_lock.c | ||
| mmap.c | ||
| mmu_gather.c | ||
| mmu_notifier.c | ||
| mmzone.c | ||
| mprotect.c | ||
| mremap.c | ||
| msync.c | ||
| nommu.c | ||
| oom_kill.c | ||
| page_alloc.c | ||
| page_counter.c | ||
| page_ext.c | ||
| page_idle.c | ||
| page_io.c | ||
| page_isolation.c | ||
| page_owner.c | ||
| page_poison.c | ||
| page_reporting.c | ||
| page_reporting.h | ||
| page_table_check.c | ||
| page_vma_mapped.c | ||
| page-writeback.c | ||
| pagewalk.c | ||
| percpu-internal.h | ||
| percpu-km.c | ||
| percpu-stats.c | ||
| percpu-vm.c | ||
| percpu.c | ||
| pgalloc-track.h | ||
| pgtable-generic.c | ||
| process_vm_access.c | ||
| ptdump.c | ||
| readahead.c | ||
| rmap.c | ||
| rodata_test.c | ||
| secretmem.c | ||
| shmem.c | ||
| show_mem.c | ||
| shrinker_debug.c | ||
| shuffle.c | ||
| shuffle.h | ||
| slab_common.c | ||
| slab.c | ||
| slab.h | ||
| slub.c | ||
| sparse-vmemmap.c | ||
| sparse.c | ||
| swap_cgroup.c | ||
| swap_slots.c | ||
| swap_state.c | ||
| swap.c | ||
| swap.h | ||
| swapfile.c | ||
| truncate.c | ||
| usercopy.c | ||
| userfaultfd.c | ||
| util.c | ||
| vmalloc.c | ||
| vmpressure.c | ||
| vmscan.c | ||
| vmstat.c | ||
| workingset.c | ||
| z3fold.c | ||
| zbud.c | ||
| zpool.c | ||
| zsmalloc.c | ||
| zswap.c | ||