2002-04-28 12:11:46 +08:00
|
|
|
/* The PyMem_ family: low-level memory allocation interfaces.
|
|
|
|
|
See objimpl.h for the PyObject_ memory family.
|
|
|
|
|
*/
|
2000-08-01 06:19:30 +08:00
|
|
|
|
|
|
|
|
#ifndef Py_PYMEM_H
|
|
|
|
|
#define Py_PYMEM_H
|
|
|
|
|
|
|
|
|
|
#include "pyport.h"
|
|
|
|
|
|
|
|
|
|
#ifdef __cplusplus
|
|
|
|
|
extern "C" {
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
/* BEWARE:
|
|
|
|
|
|
2002-04-28 12:11:46 +08:00
|
|
|
Each interface exports both functions and macros. Extension modules should
|
|
|
|
|
use the functions, to ensure binary compatibility across Python versions.
|
|
|
|
|
Because the Python implementation is free to change internal details, and
|
|
|
|
|
the macros may (or may not) expose details for speed, if you do use the
|
|
|
|
|
macros you must recompile your extensions with each Python release.
|
|
|
|
|
|
|
|
|
|
Never mix calls to PyMem_ with calls to the platform malloc/realloc/
|
|
|
|
|
calloc/free. For example, on Windows different DLLs may end up using
|
|
|
|
|
different heaps, and if you use PyMem_Malloc you'll get the memory from the
|
|
|
|
|
heap used by the Python DLL; it could be a disaster if you free()'ed that
|
|
|
|
|
directly in your own extension. Using PyMem_Free instead ensures Python
|
|
|
|
|
can return the memory to the proper heap. As another example, in
|
|
|
|
|
PYMALLOC_DEBUG mode, Python wraps all calls to all PyMem_ and PyObject_
|
|
|
|
|
memory functions in special debugging wrappers that add additional
|
|
|
|
|
debugging info to dynamic memory blocks. The system routines have no idea
|
|
|
|
|
what to do with that stuff, and the Python wrappers have no idea what to do
|
|
|
|
|
with raw blocks obtained directly by the system routines then.
|
|
|
|
|
*/
|
2000-08-01 06:19:30 +08:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Raw memory interface
|
|
|
|
|
* ====================
|
|
|
|
|
*/
|
|
|
|
|
|
2002-04-28 12:11:46 +08:00
|
|
|
/* Functions
|
2000-08-01 06:19:30 +08:00
|
|
|
|
2002-04-28 12:11:46 +08:00
|
|
|
Functions supplying platform-independent semantics for malloc/realloc/
|
|
|
|
|
free. These functions make sure that allocating 0 bytes returns a distinct
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
non-NULL pointer (whenever possible -- if we're flat out of memory, NULL
|
|
|
|
|
may be returned), even if the platform malloc and realloc don't.
|
|
|
|
|
Returned pointers must be checked for NULL explicitly. No action is
|
2002-04-28 12:11:46 +08:00
|
|
|
performed on failure (no exception is set, no warning is printed, etc).
|
|
|
|
|
*/
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
|
2002-08-12 15:21:58 +08:00
|
|
|
PyAPI_FUNC(void *) PyMem_Malloc(size_t);
|
|
|
|
|
PyAPI_FUNC(void *) PyMem_Realloc(void *, size_t);
|
|
|
|
|
PyAPI_FUNC(void) PyMem_Free(void *);
|
2000-08-01 06:19:30 +08:00
|
|
|
|
|
|
|
|
/* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
|
|
|
|
|
no longer supported. They used to call PyErr_NoMemory() on failure. */
|
|
|
|
|
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
/* Macros. */
|
2002-04-22 10:33:27 +08:00
|
|
|
#ifdef PYMALLOC_DEBUG
|
|
|
|
|
/* Redirect all memory operations to Python's debugging allocator. */
|
|
|
|
|
#define PyMem_MALLOC PyObject_MALLOC
|
|
|
|
|
#define PyMem_REALLOC PyObject_REALLOC
|
|
|
|
|
|
|
|
|
|
#else /* ! PYMALLOC_DEBUG */
|
|
|
|
|
|
2002-11-23 17:13:40 +08:00
|
|
|
/* PyMem_MALLOC(0) means malloc(1). Some systems would return NULL
|
|
|
|
|
for malloc(0), which would be treated as an error. Some platforms
|
|
|
|
|
would return a pointer with no memory behind it, which would break
|
|
|
|
|
pymalloc. To solve these problems, allocate an extra byte. */
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
#define PyMem_MALLOC(n) malloc((n) ? (n) : 1)
|
|
|
|
|
#define PyMem_REALLOC(p, n) realloc((p), (n) ? (n) : 1)
|
|
|
|
|
|
2002-04-22 10:33:27 +08:00
|
|
|
#endif /* PYMALLOC_DEBUG */
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
|
2002-04-28 12:11:46 +08:00
|
|
|
/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
|
|
|
|
|
PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory"
|
|
|
|
|
functions have to be redirected to the object deallocator. */
|
|
|
|
|
#define PyMem_FREE PyObject_FREE
|
|
|
|
|
|
2000-08-01 06:19:30 +08:00
|
|
|
/*
|
|
|
|
|
* Type-oriented memory interface
|
|
|
|
|
* ==============================
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
*
|
|
|
|
|
* These are carried along for historical reasons. There's rarely a good
|
2002-04-28 12:11:46 +08:00
|
|
|
* reason to use them anymore (you can just as easily do the multiply and
|
|
|
|
|
* cast yourself).
|
2000-08-01 06:19:30 +08:00
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#define PyMem_New(type, n) \
|
|
|
|
|
( (type *) PyMem_Malloc((n) * sizeof(type)) )
|
|
|
|
|
#define PyMem_NEW(type, n) \
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
( (type *) PyMem_MALLOC((n) * sizeof(type)) )
|
2002-04-28 12:11:46 +08:00
|
|
|
|
|
|
|
|
#define PyMem_Resize(p, type, n) \
|
|
|
|
|
( (p) = (type *) PyMem_Realloc((p), (n) * sizeof(type)) )
|
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h: _PyMem_EXTRA is gone.
pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c: PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack(): So sue me. On two lines, this called
PyMem_RESIZE to grow a "const" area. It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE. It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness. IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway. If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 15:22:56 +08:00
|
|
|
#define PyMem_RESIZE(p, type, n) \
|
|
|
|
|
( (p) = (type *) PyMem_REALLOC((p), (n) * sizeof(type)) )
|
|
|
|
|
|
2002-04-28 12:11:46 +08:00
|
|
|
/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
|
|
|
|
|
PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory"
|
|
|
|
|
functions have to be redirected to the object deallocator. */
|
|
|
|
|
#define PyMem_Del PyObject_Free
|
|
|
|
|
#define PyMem_DEL PyObject_FREE
|
Give Python a debug-mode pymalloc, much as sketched on Python-Dev.
When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug
allocator. This can be done independent of build type (release or debug).
A debug build automatically defines PYMALLOC_DEBUG when pymalloc is
enabled. It's a detected error to define PYMALLOC_DEBUG when pymalloc
isn't enabled.
Two debugging entry points defined only under PYMALLOC_DEBUG:
+ _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb)
to sanity-check a memory block obtained from pymalloc. It sprays
info to stderr (see next) and dies via Py_FatalError if the block is
detectably damaged.
+ _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info
about a debug memory block to stderr.
A tiny start at implementing "API family" checks isn't good for
anything yet.
_PyMalloc_DebugRealloc() has been optimized to do little when the new
size is <= old size. However, if the new size is larger, it really
can't call the underlying realloc() routine without either violating its
contract, or knowing something non-trivial about how the underlying
realloc() works. A memcpy is always done in this case.
This was a disaster for (and only) one of the std tests: test_bufio
creates single text file lines up to a million characters long. On
Windows, fileobject.c's get_line() uses the horridly funky
getline_via_fgets(), which keeps growing and growing a string object
hoping to find a newline. It grew the string object 1000 bytes each
time, so for a million-character string it took approximately forever
(I gave up after a few minutes).
So, also:
fileobject.c, getline_via_fgets(): When a single line is outrageously
long, grow the string object at a mildly exponential rate, instead of
just 1000 bytes at a time.
That's enough so that a debug-build test_bufio finishes in about 5 seconds
on my Win98SE box. I'm curious to try this on Win2K, because it has very
different memory behavior than Win9X, and test_bufio always took a factor
of 10 longer to complete on Win2K. It *could* be that the endless
reallocs were simply killing it on Win2K even in the release build.
2002-03-23 18:03:50 +08:00
|
|
|
|
2000-08-01 06:19:30 +08:00
|
|
|
#ifdef __cplusplus
|
|
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
#endif /* !Py_PYMEM_H */
|
