* fix: annotate `pstats.FunctionProfile.ncalls` as `str`
This change aligns the type annotation of `pstats.FunctionProfile.ncalls` with its runtime type.
add:
* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums
`_simple_enum` takes a normal class and converts it into an enum:
@simple_enum(Enum)
class Color:
RED = 1
GREEN = 2
BLUE = 3
`_old_convert_` works much like` _convert_` does, using the original logic:
# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
enum.IntEnum, 'AddressFamily', 'socket',
lambda C: C.isupper() and C.startswith('AF_'),
source=_socket,
)
`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:
# in the REPL or the same module as Color
class CheckedColor(Enum):
RED = 1
GREEN = 2
BLUE = 3
_test_simple_enum(CheckedColor, Color)
_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)
Any important differences will raise a TypeError
add:
_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:
@simple_enum(Enum)
class Color:
RED = 1
GREEN = 2
BLUE = 3
_old_convert_ works much like _convert_ does, using the original logic:
# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
enum.IntEnum, 'AddressFamily', 'socket',
lambda C: C.isupper() and C.startswith('AF_'),
source=_socket,
)
test_simple_enum takes a traditional enum and a simple enum and
compares the two:
# in the REPL or the same module as Color
class CheckedColor(Enum):
RED = 1
GREEN = 2
BLUE = 3
_test_simple_enum(CheckedColor, Color)
_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)
Any important differences will raise a TypeError
pstats is really useful or profiling and printing the output of the execution of some block of code, but I've found on multiple occasions when I'd like to access this output directly in an easily usable dictionary on which I can further analyze or manipulate.
The proposal is to add a function called get_profile_dict inside of pstats that'll automatically return this data the data in an easily accessible dict.
The output of the following script:
```
import cProfile, pstats
import pprint
from pstats import func_std_string, f8
def fib(n):
if n == 0:
return 0
if n == 1:
return 1
return fib(n-1) + fib(n-2)
pr = cProfile.Profile()
pr.enable()
fib(5)
pr.create_stats()
ps = pstats.Stats(pr).sort_stats('tottime', 'cumtime')
def get_profile_dict(self, keys_filter=None):
"""
Returns a dict where the key is a function name and the value is a dict
with the following keys:
- ncalls
- tottime
- percall_tottime
- cumtime
- percall_cumtime
- file_name
- line_number
keys_filter can be optionally set to limit the key-value pairs in the
retrieved dict.
"""
pstats_dict = {}
func_list = self.fcn_list[:] if self.fcn_list else list(self.stats.keys())
if not func_list:
return pstats_dict
pstats_dict["total_tt"] = float(f8(self.total_tt))
for func in func_list:
cc, nc, tt, ct, callers = self.stats[func]
file, line, func_name = func
ncalls = str(nc) if nc == cc else (str(nc) + '/' + str(cc))
tottime = float(f8(tt))
percall_tottime = -1 if nc == 0 else float(f8(tt/nc))
cumtime = float(f8(ct))
percall_cumtime = -1 if cc == 0 else float(f8(ct/cc))
func_dict = {
"ncalls": ncalls,
"tottime": tottime, # time spent in this function alone
"percall_tottime": percall_tottime,
"cumtime": cumtime, # time spent in the function plus all functions that this function called,
"percall_cumtime": percall_cumtime,
"file_name": file,
"line_number": line
}
func_dict_filtered = func_dict if not keys_filter else { key: func_dict[key] for key in keys_filter }
pstats_dict[func_name] = func_dict_filtered
return pstats_dict
pp = pprint.PrettyPrinter(depth=6)
pp.pprint(get_profile_dict(ps))
```
will produce:
```
{"<method 'disable' of '_lsprof.Profiler' objects>": {'cumtime': 0.0,
'file_name': '~',
'line_number': 0,
'ncalls': '1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'create_stats': {'cumtime': 0.0,
'file_name': '/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/cProfile.py',
'line_number': 50,
'ncalls': '1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'fib': {'cumtime': 0.0,
'file_name': 'get_profile_dict.py',
'line_number': 5,
'ncalls': '15/1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'total_tt': 0.0}
```
As an example, this can be used to generate a stacked column chart using various visualization tools which will assist in easily identifying program bottlenecks.
https://bugs.python.org/issue37958
Automerge-Triggered-By: @gpshead
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r86580 | senthil.kumaran | 2010-11-21 01:02:50 +0800 (Sun, 21 Nov 2010) | 3 lines
Fix issue10377 - Output from pstats - it is just secs (i.e, wallclock time) and not CPU time.
........
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r83531 | georg.brandl | 2010-08-02 19:24:49 +0200 (Mo, 02 Aug 2010) | 1 line
#7372: fix regression in pstats: a previous fix to handle cProfile data in add_callers broke handling of profile data.
........
svn+ssh://svn.python.org/python/branches/py3k
........
r83385 | georg.brandl | 2010-08-01 08:42:45 +0200 (So, 01 Aug 2010) | 1 line
#8773: mailbox.py does not need to be executable.
........
r83386 | georg.brandl | 2010-08-01 08:44:46 +0200 (So, 01 Aug 2010) | 1 line
#8768: name test method properly so that it gets executed.
........
r83387 | georg.brandl | 2010-08-01 08:53:28 +0200 (So, 01 Aug 2010) | 1 line
#8735: better explain semantics of *values* argument for parse().
........
r83388 | georg.brandl | 2010-08-01 09:48:43 +0200 (So, 01 Aug 2010) | 1 line
#7395: fix traceback in do_add() when no stats are loaded. Apply same fix for do_sort() and do_reverse().
........
r83389 | georg.brandl | 2010-08-01 09:57:47 +0200 (So, 01 Aug 2010) | 1 line
Small improvements to pstats browser: do not crash on reading invalid file, and actually do a reload when executing "read" as intended.
........
r83391 | georg.brandl | 2010-08-01 10:10:08 +0200 (So, 01 Aug 2010) | 1 line
Add another news entry.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r79603 | andrew.kuchling | 2010-04-02 12:59:16 -0400 (Fri, 02 Apr 2010) | 1 line
#4440: modernize a use of filter(), making it compatible with 3.x
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r60143 | georg.brandl | 2008-01-20 15:50:05 +0100 (Sun, 20 Jan 2008) | 3 lines
Switch mmap from old Py_FindMethod to new PyObject_GenericGetAttr attribute access.
Fixes#1087735.
........
r60145 | georg.brandl | 2008-01-20 20:40:58 +0100 (Sun, 20 Jan 2008) | 2 lines
Add blurb about executable scripts on Windows. #760657.
........
r60146 | georg.brandl | 2008-01-20 20:48:40 +0100 (Sun, 20 Jan 2008) | 2 lines
#1219903: fix tp_richcompare docs.
........
r60147 | georg.brandl | 2008-01-20 22:10:08 +0100 (Sun, 20 Jan 2008) | 2 lines
Fix markup.
........
r60148 | gregory.p.smith | 2008-01-21 08:11:11 +0100 (Mon, 21 Jan 2008) | 14 lines
Provide a sanity check during PyThreadState_DeleteCurrent() and
PyThreadState_Delete() to avoid an infinite loop when the tstate list
is messed up and has somehow becomes circular and does not contain the
current thread.
I don't know how this happens but it does, *very* rarely. On more than
one hardware platform. I have not been able to reproduce it manually.
Attaching to a process where its happening: it has always been in an
infinite loop over a single element tstate list that is not the tstate
we're looking to delete. It has been in t_bootstrap()'s call to
PyThreadState_DeleteCurrent() as a pthread is exiting.
........
r60149 | georg.brandl | 2008-01-21 11:24:59 +0100 (Mon, 21 Jan 2008) | 2 lines
#1269: fix a bug in pstats.add_callers() and add a unit test file for pstats.
........
