Jens Axboe 7519134178 io_uring/net: add iovec recycling ... Right now the io_async_msghdr is recycled to avoid the overhead of allocating+freeing it for every request. But the iovec is not included, hence that will be allocated and freed for each transfer regardless. This commit enables recyling of the iovec between io_async_msghdr recycles. This avoids alloc+free for each one if an iovec is used, and on top of that, it extends the cache hot nature of msg to the iovec as well. Also enables KASAN for the iovec entries, so that reuse can be detected even while they are in the cache. The io_async_msghdr also shrinks from 376 -> 288 bytes, an 88 byte saving (or ~23% smaller), as the fast_iovec entry is dropped from 8 entries to a single entry. There's no point keeping a big fast iovec entry, if iovecs aren't being allocated and freed continually. Signed-off-by: Jens Axboe <axboe@kernel.dk>		2024-04-15 08:10:25 -06:00
..
advise.c	io_uring: always go async for unsupported fadvise flags	2023-01-29 15:18:26 -07:00
advise.h
alloc_cache.h	io_uring/alloc_cache: shrink default max entries from 512 to 128	2024-04-15 08:10:25 -06:00
cancel.c	io_uring/cancel: don't default to setting req->work.cancel_seq	2024-02-08 13:27:06 -07:00
cancel.h	io_uring/cancel: don't default to setting req->work.cancel_seq	2024-02-08 13:27:06 -07:00
epoll.c	io_uring: undeprecate epoll_ctl support	2023-05-26 20:22:41 -06:00
epoll.h
fdinfo.c	io_uring: Fix sqpoll utilization check racing with dying sqpoll	2024-03-09 07:27:09 -07:00
fdinfo.h
filetable.c	io_uring: drop any code related to SCM_RIGHTS	2023-12-19 12:36:34 -07:00
filetable.h	io_uring: expand main struct io_kiocb flags to 64-bits	2024-02-08 13:27:03 -07:00
fs.c	io_uring/fs: consider link->flags when getting path for LINKAT	2023-11-20 09:01:42 -07:00
fs.h
futex.c	io_uring/futex: always remove futex entry for cancel all	2024-03-15 15:37:15 -06:00
futex.h	io_uring: add support for vectored futex waits	2023-09-29 02:37:08 -06:00
io_uring.c	io_uring: remove timeout/poll specific cancelations	2024-04-15 08:10:25 -06:00
io_uring.h	io_uring: clean up io_lockdep_assert_cq_locked	2024-04-15 08:10:24 -06:00
io-wq.c	io-wq: fully initialize wqe before calling cpuhp_state_add_instance_nocalls()	2023-10-05 14:11:18 -06:00
io-wq.h	io_uring: break out of iowq iopoll on teardown	2023-09-07 09:02:27 -06:00
kbuf.c	io_uring/kbuf: hold io_buffer_list reference over mmap	2024-04-02 19:03:27 -06:00
kbuf.h	io_uring/kbuf: hold io_buffer_list reference over mmap	2024-04-02 19:03:27 -06:00
Makefile	io-uring: add napi busy poll support	2024-02-09 11:54:19 -07:00
msg_ring.c	io_uring: use io_file_from_index in io_msg_grab_file	2023-06-20 09:36:22 -06:00
msg_ring.h	io_uring: get rid of double locking	2022-12-07 06:47:13 -07:00
napi.c	io_uring/napi: enable even with a timeout of 0	2024-02-15 15:37:28 -07:00
napi.h	io_uring: add register/unregister napi function	2024-02-09 11:54:32 -07:00
net.c	io_uring/net: add iovec recycling	2024-04-15 08:10:25 -06:00
net.h	io_uring/net: add iovec recycling	2024-04-15 08:10:25 -06:00
nop.c
nop.h
notif.c	io_uring/notif: add constant for ubuf_info flags	2023-04-15 14:21:04 -06:00
notif.h	io_uring/notif: add constant for ubuf_info flags	2023-04-15 14:21:04 -06:00
opdef.c	io_uring/net: get rid of ->prep_async() for send side	2024-04-15 08:10:25 -06:00
opdef.h	io_uring/rw: mark readv/writev as vectored in the opcode definition	2023-09-21 12:00:46 -06:00
openclose.c	io_uring: enable audit and restrict cred override for IORING_OP_FIXED_FD_INSTALL	2024-01-23 15:25:14 -07:00
openclose.h	io_uring/openclose: add support for IORING_OP_FIXED_FD_INSTALL	2023-12-12 07:42:57 -07:00
poll.c	io_uring: refactor io_fill_cqe_req_aux	2024-04-15 08:10:24 -06:00
poll.h	io_uring/rw: ensure poll based multishot read retries appropriately	2024-01-28 20:37:11 -07:00
refs.h
register.c	io_uring: add register/unregister napi function	2024-02-09 11:54:32 -07:00
register.h	io_uring/register: move io_uring_register(2) related code to register.c	2023-12-19 08:54:20 -07:00
rsrc.c	io_uring: drop any code related to SCM_RIGHTS	2023-12-19 12:36:34 -07:00
rsrc.h	io_uring: Don't include af_unix.h.	2024-02-12 19:02:11 -07:00
rw.c	io_uring: refactor io_fill_cqe_req_aux	2024-04-15 08:10:24 -06:00
rw.h	io_uring/rw: add separate prep handler for fixed read/write	2023-11-06 07:43:16 -07:00
slist.h	io_uring: silence variable ‘prev’ set but not used warning	2023-03-09 10:10:58 -07:00
splice.c	splice: return type ssize_t from all helpers	2023-12-12 16:19:59 +01:00
splice.h
sqpoll.c	io_uring/sqpoll: early exit thread if task_context wasn't allocated	2024-03-18 20:22:42 -06:00
sqpoll.h	io_uring/sqpoll: statistics of the true utilization of sq threads	2024-03-01 06:28:19 -07:00
statx.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
statx.h
sync.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
sync.h
tctx.c	io_uring: Add io_uring_setup flag to pre-register ring fd and never install it	2023-05-16 08:06:00 -06:00
tctx.h	io_uring: simplify __io_uring_add_tctx_node	2022-10-07 12:25:30 -06:00
timeout.c	io_uring: refactor io_fill_cqe_req_aux	2024-04-15 08:10:24 -06:00
timeout.h	io_uring: remove unused return from io_disarm_next	2022-09-21 13:15:01 -06:00
truncate.c	io_uring: add support for ftruncate	2024-02-09 09:04:39 -07:00
truncate.h	io_uring: add support for ftruncate	2024-02-09 09:04:39 -07:00
uring_cmd.c	io_uring: remove struct io_tw_state::locked	2024-04-15 08:10:24 -06:00
uring_cmd.h	io_uring/cmd: move io_uring_try_cancel_uring_cmd()	2024-04-15 08:10:24 -06:00
waitid.c	io_uring: remove struct io_tw_state::locked	2024-04-15 08:10:24 -06:00
waitid.h	io_uring: add IORING_OP_WAITID support	2023-09-21 12:04:45 -06:00
xattr.c	io_uring: use file_mnt_idmap helper	2024-02-06 19:55:14 -07:00
xattr.h