linux/drivers/hwtracing/coresight/coresight-priv.h

238 lines
6.8 KiB
C
Raw Normal View History

/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (c) 2011-2012, The Linux Foundation. All rights reserved.
*/
#ifndef _CORESIGHT_PRIV_H
#define _CORESIGHT_PRIV_H
ARM: 8838/1: drivers: amba: Updates to component identification for driver matching. The CoreSight specification (ARM IHI 0029E), updates the ID register requirements for components on an AMBA bus, to cover both traditional ARM Primecell type devices, and newer CoreSight and other components. The Peripheral ID (PID) / Component ID (CID) pair is extended in certain cases to uniquely identify components. CoreSight components related to a single function can share Peripheral ID values, and must be further identified using a Unique Component Identifier (UCI). e.g. the ETM, CTI, PMU and Debug hardware of the A35 all share the same PID. Bits 15:12 of the CID are defined to be the device class. Class 0xF remains for PrimeCell and legacy components. Class 0x9 defines the component as CoreSight (CORESIGHT_CID above) Class 0x0, 0x1, 0xB, 0xE define components that do not have driver support at present. Class 0x2-0x8,0xA and 0xD-0xD are presently reserved. The specification futher defines which classes of device use the standard CID/PID pair, and when additional ID registers are required. This patch introduces the amba_cs_uci_id structure which will be used in all coresight drivers for indentification via the private data pointer in the amba_id structure. Existing drivers that currently use the amba_id->data pointer for private data are updated to use the amba_cs_uci_id->data pointer. Macros and inline functions are added to simplify this code. Signed-off-by: Mike Leach <mike.leach@linaro.org> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Tested-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2019-02-13 21:41:49 +08:00
#include <linux/amba/bus.h>
#include <linux/bitops.h>
#include <linux/io.h>
#include <linux/coresight.h>
#include <linux/pm_runtime.h>
extern struct mutex coresight_mutex;
extern struct device_type coresight_dev_type[];
/*
* Coresight management registers (0xf00-0xfcc)
* 0xfa0 - 0xfa4: Management registers in PFTv1.0
* Trace registers in PFTv1.1
*/
#define CORESIGHT_ITCTRL 0xf00
#define CORESIGHT_CLAIMSET 0xfa0
#define CORESIGHT_CLAIMCLR 0xfa4
#define CORESIGHT_LAR 0xfb0
#define CORESIGHT_LSR 0xfb4
#define CORESIGHT_DEVARCH 0xfbc
#define CORESIGHT_AUTHSTATUS 0xfb8
#define CORESIGHT_DEVID 0xfc8
#define CORESIGHT_DEVTYPE 0xfcc
/*
* Coresight device CLAIM protocol.
* See PSCI - ARM DEN 0022D, Section: 6.8.1 Debug and Trace save and restore.
*/
#define CORESIGHT_CLAIM_SELF_HOSTED BIT(1)
#define TIMEOUT_US 100
#define BMVAL(val, lsb, msb) ((val & GENMASK(msb, lsb)) >> lsb)
#define ETM_MODE_EXCL_KERN BIT(30)
#define ETM_MODE_EXCL_USER BIT(31)
struct cs_pair_attribute {
struct device_attribute attr;
u32 lo_off;
u32 hi_off;
};
struct cs_off_attribute {
struct device_attribute attr;
u32 off;
};
extern ssize_t coresight_simple_show32(struct device *_dev,
struct device_attribute *attr, char *buf);
extern ssize_t coresight_simple_show_pair(struct device *_dev,
struct device_attribute *attr, char *buf);
#define coresight_simple_reg32(name, offset) \
(&((struct cs_off_attribute[]) { \
{ \
__ATTR(name, 0444, coresight_simple_show32, NULL), \
offset \
} \
})[0].attr.attr)
#define coresight_simple_reg64(name, lo_off, hi_off) \
(&((struct cs_pair_attribute[]) { \
{ \
__ATTR(name, 0444, coresight_simple_show_pair, NULL), \
lo_off, hi_off \
} \
})[0].attr.attr)
extern const u32 coresight_barrier_pkt[4];
#define CORESIGHT_BARRIER_PKT_SIZE (sizeof(coresight_barrier_pkt))
enum etm_addr_type {
ETM_ADDR_TYPE_NONE,
ETM_ADDR_TYPE_SINGLE,
ETM_ADDR_TYPE_RANGE,
ETM_ADDR_TYPE_START,
ETM_ADDR_TYPE_STOP,
};
/**
* struct cs_buffer - keep track of a recording session' specifics
* @cur: index of the current buffer
* @nr_pages: max number of pages granted to us
coresight: tmc-etf: Fix NULL ptr dereference in tmc_enable_etf_sink_perf() There was a report of NULL pointer dereference in ETF enable path for perf CS mode with PID monitoring. It is almost 100% reproducible when the process to monitor is something very active such as chrome and with ETF as the sink and not ETR. Currently in a bid to find the pid, the owner is dereferenced via task_pid_nr() call in tmc_enable_etf_sink_perf() and with owner being NULL, we get a NULL pointer dereference. Looking at the ETR and other places in the kernel, ETF and the ETB are the only places trying to dereference the task(owner) in tmc_enable_etf_sink_perf() which is also called from the sched_in path as in the call trace. Owner(task) is NULL even in the case of ETR in tmc_enable_etr_sink_perf(), but since we cache the PID in alloc_buffer() callback and it is done as part of etm_setup_aux() when allocating buffer for ETR sink, we never dereference this NULL pointer and we are safe. So lets do the same thing with ETF and cache the PID to which the cs_buffer belongs in tmc_alloc_etf_buffer() as done for ETR. This will also remove the unnecessary function calls(task_pid_nr()) since we are caching the PID. Easily reproducible running below: perf record -e cs_etm/@tmc_etf0/ -N -p <pid> Unable to handle kernel NULL pointer dereference at virtual address 0000000000000548 Mem abort info: ESR = 0x96000006 EC = 0x25: DABT (current EL), IL = 32 bits SET = 0, FnV = 0 EA = 0, S1PTW = 0 Data abort info: ISV = 0, ISS = 0x00000006 CM = 0, WnR = 0 <snip>... Call trace: tmc_enable_etf_sink+0xe4/0x280 coresight_enable_path+0x168/0x1fc etm_event_start+0x8c/0xf8 etm_event_add+0x38/0x54 event_sched_in+0x194/0x2ac group_sched_in+0x54/0x12c flexible_sched_in+0xd8/0x120 visit_groups_merge+0x100/0x16c ctx_flexible_sched_in+0x50/0x74 ctx_sched_in+0xa4/0xa8 perf_event_sched_in+0x60/0x6c perf_event_context_sched_in+0x98/0xe0 __perf_event_task_sched_in+0x5c/0xd8 finish_task_switch+0x184/0x1cc schedule_tail+0x20/0xec ret_from_fork+0x4/0x18 Fixes: 880af782c6e8 ("coresight: tmc-etf: Add support for CPU-wide trace scenarios") Cc: stable@vger.kernel.org Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org> Link: https://lore.kernel.org/r/20201127175256.1092685-10-mathieu.poirier@linaro.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-11-28 01:52:50 +08:00
* @pid: PID this cs_buffer belongs to
* @offset: offset within the current buffer
* @data_size: how much we collected in this run
* @snapshot: is this run in snapshot mode
* @data_pages: a handle the ring buffer
*/
struct cs_buffers {
unsigned int cur;
unsigned int nr_pages;
coresight: tmc-etf: Fix NULL ptr dereference in tmc_enable_etf_sink_perf() There was a report of NULL pointer dereference in ETF enable path for perf CS mode with PID monitoring. It is almost 100% reproducible when the process to monitor is something very active such as chrome and with ETF as the sink and not ETR. Currently in a bid to find the pid, the owner is dereferenced via task_pid_nr() call in tmc_enable_etf_sink_perf() and with owner being NULL, we get a NULL pointer dereference. Looking at the ETR and other places in the kernel, ETF and the ETB are the only places trying to dereference the task(owner) in tmc_enable_etf_sink_perf() which is also called from the sched_in path as in the call trace. Owner(task) is NULL even in the case of ETR in tmc_enable_etr_sink_perf(), but since we cache the PID in alloc_buffer() callback and it is done as part of etm_setup_aux() when allocating buffer for ETR sink, we never dereference this NULL pointer and we are safe. So lets do the same thing with ETF and cache the PID to which the cs_buffer belongs in tmc_alloc_etf_buffer() as done for ETR. This will also remove the unnecessary function calls(task_pid_nr()) since we are caching the PID. Easily reproducible running below: perf record -e cs_etm/@tmc_etf0/ -N -p <pid> Unable to handle kernel NULL pointer dereference at virtual address 0000000000000548 Mem abort info: ESR = 0x96000006 EC = 0x25: DABT (current EL), IL = 32 bits SET = 0, FnV = 0 EA = 0, S1PTW = 0 Data abort info: ISV = 0, ISS = 0x00000006 CM = 0, WnR = 0 <snip>... Call trace: tmc_enable_etf_sink+0xe4/0x280 coresight_enable_path+0x168/0x1fc etm_event_start+0x8c/0xf8 etm_event_add+0x38/0x54 event_sched_in+0x194/0x2ac group_sched_in+0x54/0x12c flexible_sched_in+0xd8/0x120 visit_groups_merge+0x100/0x16c ctx_flexible_sched_in+0x50/0x74 ctx_sched_in+0xa4/0xa8 perf_event_sched_in+0x60/0x6c perf_event_context_sched_in+0x98/0xe0 __perf_event_task_sched_in+0x5c/0xd8 finish_task_switch+0x184/0x1cc schedule_tail+0x20/0xec ret_from_fork+0x4/0x18 Fixes: 880af782c6e8 ("coresight: tmc-etf: Add support for CPU-wide trace scenarios") Cc: stable@vger.kernel.org Signed-off-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org> Link: https://lore.kernel.org/r/20201127175256.1092685-10-mathieu.poirier@linaro.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-11-28 01:52:50 +08:00
pid_t pid;
unsigned long offset;
local_t data_size;
bool snapshot;
void **data_pages;
};
static inline void coresight_insert_barrier_packet(void *buf)
{
if (buf)
memcpy(buf, coresight_barrier_pkt, CORESIGHT_BARRIER_PKT_SIZE);
}
static inline void CS_LOCK(void __iomem *addr)
{
do {
/* Wait for things to settle */
mb();
writel_relaxed(0x0, addr + CORESIGHT_LAR);
} while (0);
}
static inline void CS_UNLOCK(void __iomem *addr)
{
do {
writel_relaxed(CORESIGHT_UNLOCK, addr + CORESIGHT_LAR);
/* Make sure everyone has seen this */
mb();
} while (0);
}
void coresight_disable_path(struct list_head *path);
int coresight_enable_path(struct list_head *path, enum cs_mode mode,
void *sink_data);
struct coresight_device *coresight_get_sink(struct list_head *path);
struct coresight_device *coresight_get_sink_by_id(u32 id);
struct coresight_device *
coresight_find_default_sink(struct coresight_device *csdev);
struct list_head *coresight_build_path(struct coresight_device *csdev,
struct coresight_device *sink);
void coresight_release_path(struct list_head *path);
int coresight_add_sysfs_link(struct coresight_sysfs_link *info);
void coresight_remove_sysfs_link(struct coresight_sysfs_link *info);
int coresight_create_conns_sysfs_group(struct coresight_device *csdev);
void coresight_remove_conns_sysfs_group(struct coresight_device *csdev);
int coresight_make_links(struct coresight_device *orig,
struct coresight_connection *conn,
struct coresight_device *target);
void coresight_remove_links(struct coresight_device *orig,
struct coresight_connection *conn);
#if IS_ENABLED(CONFIG_CORESIGHT_SOURCE_ETM3X)
extern int etm_readl_cp14(u32 off, unsigned int *val);
extern int etm_writel_cp14(u32 off, u32 val);
#else
static inline int etm_readl_cp14(u32 off, unsigned int *val) { return 0; }
static inline int etm_writel_cp14(u32 off, u32 val) { return 0; }
#endif
struct cti_assoc_op {
void (*add)(struct coresight_device *csdev);
void (*remove)(struct coresight_device *csdev);
};
extern void coresight_set_cti_ops(const struct cti_assoc_op *cti_op);
extern void coresight_remove_cti_ops(void);
ARM: 8838/1: drivers: amba: Updates to component identification for driver matching. The CoreSight specification (ARM IHI 0029E), updates the ID register requirements for components on an AMBA bus, to cover both traditional ARM Primecell type devices, and newer CoreSight and other components. The Peripheral ID (PID) / Component ID (CID) pair is extended in certain cases to uniquely identify components. CoreSight components related to a single function can share Peripheral ID values, and must be further identified using a Unique Component Identifier (UCI). e.g. the ETM, CTI, PMU and Debug hardware of the A35 all share the same PID. Bits 15:12 of the CID are defined to be the device class. Class 0xF remains for PrimeCell and legacy components. Class 0x9 defines the component as CoreSight (CORESIGHT_CID above) Class 0x0, 0x1, 0xB, 0xE define components that do not have driver support at present. Class 0x2-0x8,0xA and 0xD-0xD are presently reserved. The specification futher defines which classes of device use the standard CID/PID pair, and when additional ID registers are required. This patch introduces the amba_cs_uci_id structure which will be used in all coresight drivers for indentification via the private data pointer in the amba_id structure. Existing drivers that currently use the amba_id->data pointer for private data are updated to use the amba_cs_uci_id->data pointer. Macros and inline functions are added to simplify this code. Signed-off-by: Mike Leach <mike.leach@linaro.org> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Tested-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2019-02-13 21:41:49 +08:00
/*
* Macros and inline functions to handle CoreSight UCI data and driver
* private data in AMBA ID table entries, and extract data values.
*/
/* coresight AMBA ID, no UCI, no driver data: id table entry */
#define CS_AMBA_ID(pid) \
{ \
.id = pid, \
.mask = 0x000fffff, \
}
/* coresight AMBA ID, UCI with driver data only: id table entry. */
#define CS_AMBA_ID_DATA(pid, dval) \
{ \
.id = pid, \
.mask = 0x000fffff, \
.data = (void *)&(struct amba_cs_uci_id) \
{ \
.data = (void *)dval, \
} \
}
/* coresight AMBA ID, full UCI structure: id table entry. */
#define __CS_AMBA_UCI_ID(pid, m, uci_ptr) \
{ \
.id = pid, \
.mask = m, \
.data = (void *)uci_ptr \
}
#define CS_AMBA_UCI_ID(pid, uci) __CS_AMBA_UCI_ID(pid, 0x000fffff, uci)
/*
* PIDR2[JEDEC], BIT(3) must be 1 (Read As One) to indicate that rest of the
* PIDR1, PIDR2 DES_* fields follow JEDEC encoding for the designer. Use that
* as a match value for blanket matching all devices in the given CoreSight
* device type and architecture.
*/
#define PIDR2_JEDEC BIT(3)
#define PID_PIDR2_JEDEC (PIDR2_JEDEC << 16)
/*
* Match all PIDs in a given CoreSight device type and architecture, defined
* by the uci.
*/
#define CS_AMBA_MATCH_ALL_UCI(uci) \
__CS_AMBA_UCI_ID(PID_PIDR2_JEDEC, PID_PIDR2_JEDEC, uci)
ARM: 8838/1: drivers: amba: Updates to component identification for driver matching. The CoreSight specification (ARM IHI 0029E), updates the ID register requirements for components on an AMBA bus, to cover both traditional ARM Primecell type devices, and newer CoreSight and other components. The Peripheral ID (PID) / Component ID (CID) pair is extended in certain cases to uniquely identify components. CoreSight components related to a single function can share Peripheral ID values, and must be further identified using a Unique Component Identifier (UCI). e.g. the ETM, CTI, PMU and Debug hardware of the A35 all share the same PID. Bits 15:12 of the CID are defined to be the device class. Class 0xF remains for PrimeCell and legacy components. Class 0x9 defines the component as CoreSight (CORESIGHT_CID above) Class 0x0, 0x1, 0xB, 0xE define components that do not have driver support at present. Class 0x2-0x8,0xA and 0xD-0xD are presently reserved. The specification futher defines which classes of device use the standard CID/PID pair, and when additional ID registers are required. This patch introduces the amba_cs_uci_id structure which will be used in all coresight drivers for indentification via the private data pointer in the amba_id structure. Existing drivers that currently use the amba_id->data pointer for private data are updated to use the amba_cs_uci_id->data pointer. Macros and inline functions are added to simplify this code. Signed-off-by: Mike Leach <mike.leach@linaro.org> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Tested-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2019-02-13 21:41:49 +08:00
/* extract the data value from a UCI structure given amba_id pointer. */
static inline void *coresight_get_uci_data(const struct amba_id *id)
{
struct amba_cs_uci_id *uci_id = id->data;
if (!uci_id)
return NULL;
return uci_id->data;
ARM: 8838/1: drivers: amba: Updates to component identification for driver matching. The CoreSight specification (ARM IHI 0029E), updates the ID register requirements for components on an AMBA bus, to cover both traditional ARM Primecell type devices, and newer CoreSight and other components. The Peripheral ID (PID) / Component ID (CID) pair is extended in certain cases to uniquely identify components. CoreSight components related to a single function can share Peripheral ID values, and must be further identified using a Unique Component Identifier (UCI). e.g. the ETM, CTI, PMU and Debug hardware of the A35 all share the same PID. Bits 15:12 of the CID are defined to be the device class. Class 0xF remains for PrimeCell and legacy components. Class 0x9 defines the component as CoreSight (CORESIGHT_CID above) Class 0x0, 0x1, 0xB, 0xE define components that do not have driver support at present. Class 0x2-0x8,0xA and 0xD-0xD are presently reserved. The specification futher defines which classes of device use the standard CID/PID pair, and when additional ID registers are required. This patch introduces the amba_cs_uci_id structure which will be used in all coresight drivers for indentification via the private data pointer in the amba_id structure. Existing drivers that currently use the amba_id->data pointer for private data are updated to use the amba_cs_uci_id->data pointer. Macros and inline functions are added to simplify this code. Signed-off-by: Mike Leach <mike.leach@linaro.org> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Tested-by: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2019-02-13 21:41:49 +08:00
}
void coresight_release_platform_data(struct coresight_device *csdev,
struct device *dev,
struct coresight_platform_data *pdata);
struct coresight_device *
coresight_find_csdev_by_fwnode(struct fwnode_handle *r_fwnode);
void coresight_add_helper(struct coresight_device *csdev,
struct coresight_device *helper);
coresight: core: Add support for dedicated percpu sinks Add support for dedicated sinks that are bound to individual CPUs. (e.g, TRBE). To allow quicker access to the sink for a given CPU bound source, keep a percpu array of the sink devices. Also, add support for building a path to the CPU local sink from the ETM. This adds a new percpu sink type CORESIGHT_DEV_SUBTYPE_SINK_PERCPU_SYSMEM. This new sink type is exclusively available and can only work with percpu source type device CORESIGHT_DEV_SUBTYPE_SOURCE_PROC. This defines a percpu structure that accommodates a single coresight_device which can be used to store an initialized instance from a sink driver. As these sinks are exclusively linked and dependent on corresponding percpu sources devices, they should also be the default sink device during a perf session. Outwards device connections are scanned while establishing paths between a source and a sink device. But such connections are not present for certain percpu source and sink devices which are exclusively linked and dependent. Build the path directly and skip connection scanning for such devices. Cc: Mathieu Poirier <mathieu.poirier@linaro.org> Cc: Mike Leach <mike.leach@linaro.org> Cc: Suzuki K Poulose <suzuki.poulose@arm.com> Tested-by: Suzuki K Poulose <suzuki.poulose@arm.com> Reviewed-by: Mike Leach <mike.leach@linaro.org> Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> [Moved the set/get percpu sink APIs from TRBE patch to here Fixed build break on arm32] Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com> Link: https://lore.kernel.org/r/20210405164307.1720226-17-suzuki.poulose@arm.com Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
2021-04-06 00:43:03 +08:00
void coresight_set_percpu_sink(int cpu, struct coresight_device *csdev);
struct coresight_device *coresight_get_percpu_sink(int cpu);
void coresight_disable_source(struct coresight_device *csdev, void *data);
coresight: core: Add support for dedicated percpu sinks Add support for dedicated sinks that are bound to individual CPUs. (e.g, TRBE). To allow quicker access to the sink for a given CPU bound source, keep a percpu array of the sink devices. Also, add support for building a path to the CPU local sink from the ETM. This adds a new percpu sink type CORESIGHT_DEV_SUBTYPE_SINK_PERCPU_SYSMEM. This new sink type is exclusively available and can only work with percpu source type device CORESIGHT_DEV_SUBTYPE_SOURCE_PROC. This defines a percpu structure that accommodates a single coresight_device which can be used to store an initialized instance from a sink driver. As these sinks are exclusively linked and dependent on corresponding percpu sources devices, they should also be the default sink device during a perf session. Outwards device connections are scanned while establishing paths between a source and a sink device. But such connections are not present for certain percpu source and sink devices which are exclusively linked and dependent. Build the path directly and skip connection scanning for such devices. Cc: Mathieu Poirier <mathieu.poirier@linaro.org> Cc: Mike Leach <mike.leach@linaro.org> Cc: Suzuki K Poulose <suzuki.poulose@arm.com> Tested-by: Suzuki K Poulose <suzuki.poulose@arm.com> Reviewed-by: Mike Leach <mike.leach@linaro.org> Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> [Moved the set/get percpu sink APIs from TRBE patch to here Fixed build break on arm32] Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com> Link: https://lore.kernel.org/r/20210405164307.1720226-17-suzuki.poulose@arm.com Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
2021-04-06 00:43:03 +08:00
#endif