Andrii Nakryiko 10cdb82aa7 uprobes: turn trace_uprobe's nhit counter to be per-CPU one ... trace_uprobe->nhit counter is not incremented atomically, so its value is questionable in when uprobe is hit on multiple CPUs simultaneously. Also, doing this shared counter increment across many CPUs causes heavy cache line bouncing, limiting uprobe/uretprobe performance scaling with number of CPUs. Solve both problems by making this a per-CPU counter. Link: https://lore.kernel.org/all/20240813203409.3985398-1-andrii@kernel.org/ Reviewed-by: Oleg Nesterov <oleg@redhat.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>		2024-09-25 20:10:38 +09:00
..
rv	rv: Update rv_en(dis)able_monitor doc to match kernel-doc	2024-05-21 19:03:35 -04:00
blktrace.c	blktrace: convert strncpy() to strscpy_pad()	2024-04-25 21:07:08 -07:00
bpf_trace.c	bpf: Change bpf_session_cookie return value to __u64 *	2024-06-21 19:32:36 +02:00
bpf_trace.h	tracing/treewide: Remove second parameter of __assign_str()	2024-05-22 20:14:47 -04:00
error_report-traces.c
fgraph.c	tracing: fgraph: Fix to add new fgraph_ops to array after ftrace_startup_subops()	2024-08-21 13:49:59 -04:00
fprobe.c	fprobe: Fix to allocate entry_data_size buffer with rethook instances	2024-03-01 09:18:24 +09:00
ftrace_internal.h	function_graph: Make fgraph_update_pid_func() a stub for !DYNAMIC_FTRACE	2024-06-10 18:08:23 -04:00
ftrace.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
Kconfig	tracing: Build event generation tests only as modules	2024-06-12 08:43:12 +09:00
kprobe_event_gen_test.c	tracing: Fix wrong return in kprobe_event_gen_test.c	2023-03-19 12:20:48 -04:00
Makefile	tracing/probes: Move finding func-proto API and getting func-param API to trace_btf	2023-08-23 09:39:45 +09:00
pid_list.c	trace/pid_list: Change gfp flags in pid_list_fill_irq()	2024-07-15 15:07:14 -04:00
pid_list.h
power-traces.c
preemptirq_delay_test.c	minmax: make generic MIN() and MAX() macros available everywhere	2024-07-28 15:49:18 -07:00
rethook.c	rethook: honor CONFIG_FTRACE_VALIDATE_RCU_IS_WATCHING in rethook_try_get()	2024-05-01 23:18:48 +09:00
ring_buffer_benchmark.c	ring-buffer: Read and write to ring buffers with custom sub buffer size	2023-12-20 07:54:56 -05:00
ring_buffer.c	ring-buffer: Remove unused function ring_buffer_nr_pages()	2024-08-07 20:26:44 -04:00
rpm-traces.c
synth_event_gen_test.c	tracing / synthetic: Disable events after testing in synth_event_gen_test_init()	2023-12-21 10:04:45 -05:00
trace_benchmark.c	tracing: Improve benchmark test performance by using do_div()	2024-05-13 20:00:57 -04:00
trace_benchmark.h
trace_boot.c	tracing: Allow creating instances with specified system events	2023-12-18 23:14:16 -05:00
trace_branch.c
trace_btf.c	tracing/probes: Fix to search structure fields correctly	2024-02-17 21:25:42 +09:00
trace_btf.h	tracing/probes: Add a function to search a member of a struct/union	2023-08-23 09:40:16 +09:00
trace_clock.c
trace_dynevent.c
trace_dynevent.h
trace_entries.h	tracing: Add back FORTIFY_SOURCE logic to kernel_stack event structure	2023-07-30 18:11:44 -04:00
trace_eprobe.c	tracing/probes: Support $argN in return probe (kprobe and fprobe)	2024-03-07 00:27:34 +09:00
trace_event_perf.c
trace_events_filter_test.h
trace_events_filter.c	tracing: Have trace_event_file have ref counters	2023-11-01 23:44:44 -04:00
trace_events_hist.c	tracing: Have format file honor EVENT_FILE_FL_FREED	2024-08-07 18:12:46 -04:00
trace_events_inject.c	tracing: Have format file honor EVENT_FILE_FL_FREED	2024-08-07 18:12:46 -04:00
trace_events_synth.c	tracing/synthetic: Fix trace_string() return value	2024-02-15 11:40:01 -05:00
trace_events_trigger.c	tracing: Have format file honor EVENT_FILE_FL_FREED	2024-08-07 18:12:46 -04:00
trace_events_user.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
trace_events.c	tracing: Use refcount for trace_event_file reference counter	2024-08-07 18:12:46 -04:00
trace_export.c	tracing: Add back FORTIFY_SOURCE logic to kernel_stack event structure	2023-07-30 18:11:44 -04:00
trace_fprobe.c	tracing/probes: support '%pd' type for print struct dentry's name	2024-05-01 23:18:47 +09:00
trace_functions_graph.c	function_graph: Move graph notrace bit to shadow stack global var	2024-06-04 10:37:44 -04:00
trace_functions.c	ftrace: Hide one more entry in stack trace when ftrace_pid is enabled	2024-06-06 15:22:18 -04:00
trace_hwlat.c	tracing: Remove extra space at the end of hwlat_detector/mode	2023-09-01 21:00:00 -04:00
trace_irqsoff.c	function_graph: Move set_graph_function tests to shadow stack global var	2024-06-04 10:37:35 -04:00
trace_kdb.c
trace_kprobe_selftest.c	tracing: arm64: Avoid missing-prototype warnings	2023-07-12 12:06:04 -04:00
trace_kprobe_selftest.h
trace_kprobe.c	tracing/kprobes: Fix build error when find_module() is not available	2024-07-10 09:47:00 +09:00
trace_mmiotrace.c
trace_nop.c
trace_osnoise.c	tracing/timerlat: Add interface_lock around clearing of kthread in stop_kthread()	2024-09-05 12:01:37 -04:00
trace_output.c	tracing: Remove precision vsnprintf() check from print event	2024-03-06 13:26:26 -05:00
trace_output.h	tracing: Add "fields" option to show raw trace event fields	2023-03-29 06:52:08 -04:00
trace_preemptirq.c
trace_printk.c
trace_probe_kernel.h	tracing/probes: Fix to record 0-length data_loc in fetch_store_string*() if fails	2023-07-14 17:04:58 +09:00
trace_probe_tmpl.h	tracing/probes: Support $argN in return probe (kprobe and fprobe)	2024-03-07 00:27:34 +09:00
trace_probe.c	tracing/probes: fix error check in parse_btf_field()	2024-05-27 21:32:35 +09:00
trace_probe.h	tracing/probes: support '%pd' type for print struct dentry's name	2024-05-01 23:18:47 +09:00
trace_recursion_record.c
trace_sched_switch.c	tracing: Move saved_cmdline code into trace_sched_switch.c	2024-03-17 07:58:53 -04:00
trace_sched_wakeup.c	function_graph: Move set_graph_function tests to shadow stack global var	2024-06-04 10:37:35 -04:00
trace_selftest_dynamic.c
trace_selftest.c	tracing: Fix memory leak in fgraph storage selftest	2024-08-21 15:10:33 -04:00
trace_seq.c	trace_seq: Increase the buffer size to almost two pages	2023-12-18 23:14:16 -05:00
trace_stack.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
trace_stat.c
trace_stat.h
trace_synth.h
trace_syscalls.c	bpf: Change syscall_nr type to int in struct syscall_tp_t	2023-10-13 12:39:36 -07:00
trace_uprobe.c	uprobes: turn trace_uprobe's nhit counter to be per-CPU one	2024-09-25 20:10:38 +09:00
trace.c	tracing: Avoid possible softlockup in tracing_iter_reset()	2024-09-05 10:18:48 -04:00
trace.h	tracing: Have format file honor EVENT_FILE_FL_FREED	2024-08-07 18:12:46 -04:00
tracing_map.c	tracing: Fix overflow in get_free_elt()	2024-08-07 20:23:12 -04:00
tracing_map.h	tracing: Remove unused extern declaration tracing_map_set_field_descr()	2023-07-23 11:08:14 -04:00