mirror of
https://mirrors.bfsu.edu.cn/git/linux.git
synced 2024-09-21 20:22:13 +08:00
1f49c1476d
Patch series "mm/swap: clean up and optimize swap cache index", v6. Currently we use one swap_address_space for every 64M chunk to reduce lock contention, this is like having a set of smaller files inside a swap device. But when doing swap cache look up or insert, we are still using the offset of the whole large swap device. This is OK for correctness, as the offset (key) is unique. But Xarray is specially optimized for small indexes, it creates the redix tree levels lazily to be just enough to fit the largest key stored in one Xarray. So we are wasting tree nodes unnecessarily. For 64M chunk it should only take at most 3 level to contain everything. But if we are using the offset from the whole swap device, the offset (key) value will be way beyond 64M, and so will the tree level. Optimize this by reduce the swap cache search space into 64M scope. Test with `time memhog 128G` inside a 8G memcg using 128G swap (ramdisk with SWP_SYNCHRONOUS_IO dropped, tested 3 times, results are stable. The test result is similar but the improvement is smaller if SWP_SYNCHRONOUS_IO is enabled, as swap out path can never skip swap cache): Before: 6.07user 250.74system 4:17.26elapsed 99%CPU (0avgtext+0avgdata 8373376maxresident)k 0inputs+0outputs (55major+33555018minor)pagefaults 0swaps After (+1.8% faster): 6.08user 246.09system 4:12.58elapsed 99%CPU (0avgtext+0avgdata 8373248maxresident)k 0inputs+0outputs (54major+33555027minor)pagefaults 0swaps Similar result with MySQL and sysbench using swap: Before: 94055.61 qps After (+0.8% faster): 94834.91 qps There is alse a very slight drop of radix tree node slab usage: Before: 303952K After: 302224K For this series: There are multiple places that expect mixed type of pages (page cache or swap cache), eg. migration, huge memory split; There are four helpers for that: - page_index - page_file_offset - folio_index - folio_file_pos To keep the code clean and compatible, this series first cleaned up usage of them. page_file_offset and folio_file_pos are historical helpes that can be simply dropped after clean up. And page_index can be all converted to folio_index or folio->index. Then introduce two new helpers swap_cache_index and swap_dev_pos for swap. Replace swp_offset with swap_cache_index when used to retrieve folio from swap cache, and use swap_dev_pos when needed to retrieve the device position of a swap entry. This way, swap_cache_index can return the optimized value with no compatibility issue. The result is better performance and reduced LOC. Idealy, in the future, we may want to reduce SWAP_ADDRESS_SPACE_SHIFT from 14 to 12: Default Xarray chunk offset is 6, so we have 3 level trees instead of 2 level trees just for 2 extra bits. But swap cache is based on address_space struct, with 4 times more metadata sparsely distributed in memory it waste more cacheline, the performance gain from this series is almost canceled according to my test. So first, just have a cleaner seperation of offsets and smaller search space. This patch (of 10): page_index is only for mixed usage of page cache and swap cache, for pure page cache usage, the caller can just use page->index instead. It can't be a swap cache page here (being part of buffer head), so just drop it. And while we are at it, optimize the code by retrieving the offset of the buffer head within the folio directly using bh_offset, and get rid of the loop and usage of page helpers. Link: https://lkml.kernel.org/r/20240521175854.96038-1-ryncsn@gmail.com Link: https://lkml.kernel.org/r/20240521175854.96038-3-ryncsn@gmail.com Suggested-by: Matthew Wilcox <willy@infradead.org> Signed-off-by: Kairui Song <kasong@tencent.com> Acked-by: Ryusuke Konishi <konishi.ryusuke@gmail.com> Cc: Ryusuke Konishi <konishi.ryusuke@gmail.com> Cc: Anna Schumaker <anna@kernel.org> Cc: Barry Song <v-songbaohua@oppo.com> Cc: Chao Yu <chao@kernel.org> Cc: Chris Li <chrisl@kernel.org> Cc: David Hildenbrand <david@redhat.com> Cc: David Howells <dhowells@redhat.com> Cc: "Huang, Ying" <ying.huang@intel.com> Cc: Hugh Dickins <hughd@google.com> Cc: Ilya Dryomov <idryomov@gmail.com> Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Jeff Layton <jlayton@kernel.org> Cc: Marc Dionne <marc.dionne@auristor.com> Cc: Minchan Kim <minchan@kernel.org> Cc: NeilBrown <neilb@suse.de> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Trond Myklebust <trond.myklebust@hammerspace.com> Cc: Xiubo Li <xiubli@redhat.com> Cc: Yosry Ahmed <yosryahmed@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|---|---|---|
| .. | ||
| 9p | ||
| adfs | ||
| affs | ||
| afs | ||
| autofs | ||
| bcachefs | ||
| befs | ||
| bfs | ||
| btrfs | ||
| cachefiles | ||
| ceph | ||
| coda | ||
| configfs | ||
| cramfs | ||
| crypto | ||
| debugfs | ||
| devpts | ||
| dlm | ||
| ecryptfs | ||
| efivarfs | ||
| efs | ||
| erofs | ||
| exfat | ||
| exportfs | ||
| ext2 | ||
| ext4 | ||
| f2fs | ||
| fat | ||
| freevxfs | ||
| fuse | ||
| gfs2 | ||
| hfs | ||
| hfsplus | ||
| hostfs | ||
| hpfs | ||
| hugetlbfs | ||
| iomap | ||
| isofs | ||
| jbd2 | ||
| jffs2 | ||
| jfs | ||
| kernfs | ||
| lockd | ||
| minix | ||
| netfs | ||
| nfs | ||
| nfs_common | ||
| nfsd | ||
| nilfs2 | ||
| nls | ||
| notify | ||
| ntfs3 | ||
| ocfs2 | ||
| omfs | ||
| openpromfs | ||
| orangefs | ||
| overlayfs | ||
| proc | ||
| pstore | ||
| qnx4 | ||
| qnx6 | ||
| quota | ||
| ramfs | ||
| reiserfs | ||
| romfs | ||
| smb | ||
| squashfs | ||
| sysfs | ||
| sysv | ||
| tracefs | ||
| ubifs | ||
| udf | ||
| ufs | ||
| unicode | ||
| vboxsf | ||
| verity | ||
| xfs | ||
| zonefs | ||
| aio.c | ||
| anon_inodes.c | ||
| attr.c | ||
| backing-file.c | ||
| bad_inode.c | ||
| binfmt_elf_fdpic.c | ||
| binfmt_elf_test.c | ||
| binfmt_elf.c | ||
| binfmt_flat.c | ||
| binfmt_misc.c | ||
| binfmt_script.c | ||
| buffer.c | ||
| char_dev.c | ||
| compat_binfmt_elf.c | ||
| coredump.c | ||
| d_path.c | ||
| dax.c | ||
| dcache.c | ||
| direct-io.c | ||
| drop_caches.c | ||
| eventfd.c | ||
| eventpoll.c | ||
| exec.c | ||
| fcntl.c | ||
| fhandle.c | ||
| file_table.c | ||
| file.c | ||
| filesystems.c | ||
| fs_context.c | ||
| fs_parser.c | ||
| fs_pin.c | ||
| fs_struct.c | ||
| fs_types.c | ||
| fs-writeback.c | ||
| fsopen.c | ||
| init.c | ||
| inode.c | ||
| internal.h | ||
| ioctl.c | ||
| Kconfig | ||
| Kconfig.binfmt | ||
| kernel_read_file.c | ||
| libfs.c | ||
| locks.c | ||
| Makefile | ||
| mbcache.c | ||
| mnt_idmapping.c | ||
| mount.h | ||
| mpage.c | ||
| namei.c | ||
| namespace.c | ||
| nsfs.c | ||
| open.c | ||
| pidfs.c | ||
| pipe.c | ||
| pnode.c | ||
| pnode.h | ||
| posix_acl.c | ||
| proc_namespace.c | ||
| read_write.c | ||
| readdir.c | ||
| remap_range.c | ||
| select.c | ||
| seq_file.c | ||
| signalfd.c | ||
| splice.c | ||
| stack.c | ||
| stat.c | ||
| statfs.c | ||
| super.c | ||
| sync.c | ||
| sysctls.c | ||
| timerfd.c | ||
| userfaultfd.c | ||
| utimes.c | ||
| xattr.c | ||