Jens Axboe 6ff1407e24 io_uring: ensure local task_work is run on wait timeout ... A previous commit added an earlier break condition here, which is fine if we're using non-local task_work as it'll be run on return to userspace. However, if DEFER_TASKRUN is used, then we could be leaving local task_work that is ready to process in the ctx list until next time that we enter the kernel to wait for events. Move the break condition to _after_ we have run task_work. Cc: stable@vger.kernel.org Fixes: 846072f16e ("io_uring: mimimise io_cqring_wait_schedule") Signed-off-by: Jens Axboe <axboe@kernel.dk>		2024-01-04 12:21:08 -07:00
..
advise.c	io_uring: always go async for unsupported fadvise flags	2023-01-29 15:18:26 -07:00
advise.h
alloc_cache.h	io_uring/rsrc: consolidate node caching	2023-04-12 12:09:41 -06:00
cancel.c	io_uring: use fget/fput consistently	2023-11-28 11:56:29 -07:00
cancel.h	io_uring: add support for futex wake and wait	2023-09-29 02:36:57 -06:00
epoll.c	io_uring: undeprecate epoll_ctl support	2023-05-26 20:22:41 -06:00
epoll.h
fdinfo.c	io_uring/fdinfo: remove need for sqpoll lock for thread/pid retrieval	2023-11-15 06:35:46 -07:00
fdinfo.h
filetable.c	io_uring: drop any code related to SCM_RIGHTS	2023-12-19 12:36:34 -07:00
filetable.h	io_uring: add helpers to decode the fixed file file_ptr	2023-06-20 09:36:22 -06:00
fs.c	io_uring/fs: consider link->flags when getting path for LINKAT	2023-11-20 09:01:42 -07:00
fs.h
futex.c	io_uring: add support for vectored futex waits	2023-09-29 02:37:08 -06:00
futex.h	io_uring: add support for vectored futex waits	2023-09-29 02:37:08 -06:00
io_uring.c	io_uring: ensure local task_work is run on wait timeout	2024-01-04 12:21:08 -07:00
io_uring.h	io_uring/unix: drop usage of io_uring socket	2023-12-19 12:33:50 -07:00
io-wq.c	io-wq: fully initialize wqe before calling cpuhp_state_add_instance_nocalls()	2023-10-05 14:11:18 -06:00
io-wq.h	io_uring: break out of iowq iopoll on teardown	2023-09-07 09:02:27 -06:00
kbuf.c	io_uring/kbuf: add method for returning provided buffer ring head	2023-12-21 09:47:06 -07:00
kbuf.h	io_uring/kbuf: add method for returning provided buffer ring head	2023-12-21 09:47:06 -07:00
Makefile	io_uring/register: move io_uring_register(2) related code to register.c	2023-12-19 08:54:20 -07:00
msg_ring.c	io_uring: use io_file_from_index in io_msg_grab_file	2023-06-20 09:36:22 -06:00
msg_ring.h	io_uring: get rid of double locking	2022-12-07 06:47:13 -07:00
net.c	io_uring/net: ensure socket is marked connected on connect retry	2023-11-03 13:25:50 -06:00
net.h	io_uring: Add KASAN support for alloc_caches	2023-04-03 07:16:14 -06:00
nop.c
nop.h
notif.c	io_uring/notif: add constant for ubuf_info flags	2023-04-15 14:21:04 -06:00
notif.h	io_uring/notif: add constant for ubuf_info flags	2023-04-15 14:21:04 -06:00
opdef.c	io_uring/openclose: add support for IORING_OP_FIXED_FD_INSTALL	2023-12-12 07:42:57 -07:00
opdef.h	io_uring/rw: mark readv/writev as vectored in the opcode definition	2023-09-21 12:00:46 -06:00
openclose.c	io_uring/openclose: add support for IORING_OP_FIXED_FD_INSTALL	2023-12-12 07:42:57 -07:00
openclose.h	io_uring/openclose: add support for IORING_OP_FIXED_FD_INSTALL	2023-12-12 07:42:57 -07:00
poll.c	io_uring/poll: use IOU_F_TWQ_LAZY_WAKE for wakeups	2023-10-19 06:42:29 -06:00
poll.h	io_uring: avoid indirect function calls for the hottest task_work	2023-06-02 08:55:37 -06:00
refs.h
register.c	io_uring/kbuf: add method for returning provided buffer ring head	2023-12-21 09:47:06 -07:00
register.h	io_uring/register: move io_uring_register(2) related code to register.c	2023-12-19 08:54:20 -07:00
rsrc.c	io_uring: drop any code related to SCM_RIGHTS	2023-12-19 12:36:34 -07:00
rsrc.h	io_uring: drop any code related to SCM_RIGHTS	2023-12-19 12:36:34 -07:00
rw.c	io_uring/rw: ensure io->bytes_done is always initialized	2023-12-21 08:49:18 -07:00
rw.h	io_uring/rw: add separate prep handler for fixed read/write	2023-11-06 07:43:16 -07:00
slist.h	io_uring: silence variable ‘prev’ set but not used warning	2023-03-09 10:10:58 -07:00
splice.c	io_uring/splice: use fput() directly	2023-08-10 10:24:25 -06:00
splice.h
sqpoll.c	io_uring/fdinfo: remove need for sqpoll lock for thread/pid retrieval	2023-11-15 06:35:46 -07:00
sqpoll.h	io_uring/sqpoll: fix io-wq affinity when IORING_SETUP_SQPOLL is used	2023-08-16 13:40:28 -06:00
statx.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
statx.h
sync.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
sync.h
tctx.c	io_uring: Add io_uring_setup flag to pre-register ring fd and never install it	2023-05-16 08:06:00 -06:00
tctx.h	io_uring: simplify __io_uring_add_tctx_node	2022-10-07 12:25:30 -06:00
timeout.c	io_uring: never overflow io_aux_cqe	2023-08-11 10:42:57 -06:00
timeout.h	io_uring: remove unused return from io_disarm_next	2022-09-21 13:15:01 -06:00
uring_cmd.c	io_uring/cmd: inline io_uring_cmd_get_task	2023-12-12 07:42:52 -07:00
uring_cmd.h	io_uring: Remove unnecessary BUILD_BUG_ON	2023-05-04 08:19:05 -06:00
waitid.c	io_uring: add IORING_OP_WAITID support	2023-09-21 12:04:45 -06:00
waitid.h	io_uring: add IORING_OP_WAITID support	2023-09-21 12:04:45 -06:00
xattr.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
xattr.h