linux/io_uring
Pavel Begunkov c98c81a4ac io_uring: merge conditional unlock flush helpers
There is no reason not to use __io_cq_unlock_post_flush for intermediate
aux CQE flushing, all ->task_complete should apply there, i.e. if set it
should be the submitter task. Combine them, get rid of of
__io_cq_unlock_post() and rename the left function.

This place was also taking a couple percents of CPU according to
profiles for max throughput net benchmarks due to multishot recv
flooding it with completions.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/bbed60734cbec2e833d9c7bdcf9741aada5d8aab.1687518903.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2023-06-23 08:19:40 -06:00
..
advise.c io_uring: always go async for unsupported fadvise flags 2023-01-29 15:18:26 -07:00
advise.h
alloc_cache.h io_uring/rsrc: consolidate node caching 2023-04-12 12:09:41 -06:00
cancel.c io_uring: use io_file_from_index in __io_sync_cancel 2023-06-20 09:36:22 -06:00
cancel.h
epoll.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
epoll.h
fdinfo.c capability: just use a 'u64' instead of a 'u32[2]' array 2023-03-01 10:01:22 -08:00
fdinfo.h
filetable.c io_uring: add helpers to decode the fixed file file_ptr 2023-06-20 09:36:22 -06:00
filetable.h io_uring: add helpers to decode the fixed file file_ptr 2023-06-20 09:36:22 -06:00
fs.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
fs.h
io_uring.c io_uring: merge conditional unlock flush helpers 2023-06-23 08:19:40 -06:00
io_uring.h io_uring: make io_cq_unlock_post static 2023-06-23 08:19:40 -06:00
io-wq.c io_uring/io-wq: drop outdated comment 2023-04-03 07:16:15 -06:00
io-wq.h
kbuf.c for-6.4/io_uring-2023-04-21 2023-04-26 12:40:31 -07:00
kbuf.h io_uring: add support for user mapped provided buffer ring 2023-04-03 07:14:21 -06:00
Makefile
msg_ring.c io_uring: use io_file_from_index in io_msg_grab_file 2023-06-20 09:36:22 -06:00
msg_ring.h io_uring: get rid of double locking 2022-12-07 06:47:13 -07:00
net.c io_uring: cleanup io_aux_cqe() API 2023-06-07 14:59:22 -06:00
net.h io_uring: Add KASAN support for alloc_caches 2023-04-03 07:16:14 -06:00
nop.c
nop.h
notif.c io_uring/notif: add constant for ubuf_info flags 2023-04-15 14:21:04 -06:00
notif.h io_uring/notif: add constant for ubuf_info flags 2023-04-15 14:21:04 -06:00
opdef.c io_uring: Pass whole sqe to commands 2023-05-04 08:19:05 -06:00
opdef.h io_uring: Split io_issue_def struct 2023-01-29 15:17:41 -07:00
openclose.c io_uring: always go async for unsupported open flags 2023-01-29 15:18:26 -07:00
openclose.h
poll.c io_uring: cleanup io_aux_cqe() API 2023-06-07 14:59:22 -06:00
poll.h io_uring: avoid indirect function calls for the hottest task_work 2023-06-02 08:55:37 -06:00
refs.h
rsrc.c io_uring: add helpers to decode the fixed file file_ptr 2023-06-20 09:36:22 -06:00
rsrc.h io_uring/rsrc: disassociate nodes and rsrc_data 2023-04-18 19:38:26 -06:00
rw.c io_uring: return REQ_F_ flags from io_file_get_flags 2023-06-20 09:36:22 -06:00
rw.h io_uring: avoid indirect function calls for the hottest task_work 2023-06-02 08:55:37 -06:00
slist.h io_uring: silence variable ‘prev’ set but not used warning 2023-03-09 10:10:58 -07:00
splice.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
splice.h
sqpoll.c io_uring/sqpoll: Do not set PF_NO_SETAFFINITY on sqpoll threads 2023-03-15 06:50:59 -06:00
sqpoll.h io_uring: make io_sqpoll_wait_sq return void 2023-01-29 15:17:40 -07:00
statx.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
statx.h
sync.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
sync.h
tctx.c io_uring: Add io_uring_setup flag to pre-register ring fd and never install it 2023-05-16 08:06:00 -06:00
tctx.h io_uring: simplify __io_uring_add_tctx_node 2022-10-07 12:25:30 -06:00
timeout.c io_uring: cleanup io_aux_cqe() API 2023-06-07 14:59:22 -06:00
timeout.h io_uring: remove unused return from io_disarm_next 2022-09-21 13:15:01 -06:00
uring_cmd.c io_uring/cmd: add cmd lazy tw wake helper 2023-05-25 08:54:06 -06:00
uring_cmd.h io_uring: Remove unnecessary BUILD_BUG_ON 2023-05-04 08:19:05 -06:00
xattr.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
xattr.h