throw_exception_internal will access opline+1, which is not always
defined at the current opline of the generator. To avoid this
decrement the opline before throwing (so the throw occurs at the
YIELD opcode instead of one after it).
Also remove obsolete code for copying the closure op_array. It
should no longer be possible for the closure object to be destroyed
while the generator is still live.
Squashed commit of the following:
commit 1e41295097
Author: Dmitry Stogov <dmitry@zend.com>
Date: Sat Jan 31 07:28:58 2015 +0300
Generalize HashTableIterator API to allows its usage without involvement of HashTable.nInternalPonter
commit 5406f21b11
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 18:08:43 2015 +0300
Reduced alghorithms complexity
commit b37f1d58d2
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 18:08:30 2015 +0300
Fixed test name
commit fb2d079645
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 18:08:05 2015 +0300
API cleanup
commit 08302c0d6d
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 14:20:46 2015 +0300
Make array_splice() to preserve foreach hash position
commit cc4b7be41e
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 12:24:31 2015 +0300
Make internal function, operation on array passed by reference, to preserve foreach hash position
commit 5aa9712b0a
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 09:49:35 2015 +0300
Implement consistent behavior for foreach by value over plain object
commit 4c5b385ff5
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 07:56:37 2015 +0300
More careful iterators update.
commit 721fc9e80d
Author: Dmitry Stogov <dmitry@zend.com>
Date: Thu Jan 29 21:43:28 2015 +0300
Added new test
commit 15a23b1218
Author: Dmitry Stogov <dmitry@zend.com>
Date: Thu Jan 29 21:05:02 2015 +0300
Reimplement iteration magic with HashTableIterators (see https://wiki.php.net/rfc/php7_foreach#implementation_details)
commit 10a3260b1f
Author: Dmitry Stogov <dmitry@zend.com>
Date: Thu Jan 29 21:04:44 2015 +0300
New test
commit eef80c5837
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 16:52:21 2015 +0300
Fixed foreach by reference iteration over constant array
commit 61e7391873
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 14:59:54 2015 +0300
Fixed temporary variable re-allocation pass
commit 92e90c09f0
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 12:44:57 2015 +0300
Fixed operand destruction in case of exceptions in iterator
commit dd2a36a207
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 10:02:34 2015 +0300
Use GET_OP1_ZVAL_PTR_DEREF() (IS_TMP_VAR and IS_CONST can't be IS_REFERENCE)
commit 4638f7b914
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 07:43:28 2015 +0300
Change "foreach" statement behavior (this is just a PoC yet)
- "foreach by value" don't relay on internal array/object pointer and doesnt perform array duplication. It just locks it incrementing reference counter. If the original array is modified by some code, the copy on write is performed and "foreach" still work with the old copy.
- it makes no difference if array given to "foreach by value" is reference itself
- "foreach by reference" still use internal array/object pointer and should work similar to PHP-5. (This id not completely implemented)
* master: (48 commits)
change locale - looks like not everybody has sl_SI
Fix bug #66921 - Wrong argument type hint for function intltz_from_date_time_zone
fix format
Fix bug #67052 (NumberFormatter::parse() resets LC_NUMERIC setting)
Make sure the generator script also creates a newline at the end of file
Add newline at end of file to prevent compilation warning
Fix handling of session user module custom handlers.
Reference bug report instead of github issue in NEWS file
add more exts for Travis
Update NEWS
Fix phpdbg.1 man page installation when build != src directory
BFN for bug #67551 (php://input temp file will be located in sys_temp_dir instead of upload_tmp_dir)
reorder
restore API compatibility
finish
refactor php_stream_temp_create{,_ex} and use it for the php://input stream
refactor _php_stream_fopen_{temporary_,tmp}file()
fix length overflow of HTTP_RAW_POST_DATA
Update NEWS
Fixed bug #67215 (php-cgi work with opcache, may be segmentation fault happen)
...
Conflicts:
ext/opcache/zend_accelerator_util_funcs.c
ext/session/mod_user.c
ext/spl/spl_array.c
ext/spl/spl_dllist.c
ext/standard/file.c
ext/standard/streamsfuncs.c
ext/standard/string.c
main/streams/memory.c
I'm not exactly sure whether this is the right way to fix it. The
question is whether Generator::throw() on a newborn generator (i.e.
a generator that is not yet at yield expression) should first advance to
the first yield and throw the exception there or whether it should
instead throw the exception in the caller's context.
The old behavior was to throw it at the start of the function (i.e.
the very first opcode), which causes issues like the one in #65764.
Effectively it's impossible to properly handle the exceptions in this
case.
For now I choose the variant where the generator advances to the
first yield before throwing, as that's consistent with how all other
methods on the Generator object currently behave. This does not
necessarily match the behavior in other languages, e.g. Python would throw
the exception in the caller's context. But then our send() method already
has this kind of deviation, so it stays internally consistent at least.
All code dealing with unfinished execution cleanup is now in a separate
function (previously most of it was run even when execution was properly
finished.
Furthermore some code dealing with unclean shutdowns has been removed,
which is no longer necessary, because we no longer try to clean up in
this case.
This fixes bugs #65035 and #65161. In one of the bugs the issue is
that function_state.arguments is NULL, but the arg count is pushed
to the stack and the code tries to free it. In the other bug the
stack of the generator is freed twice, once in generator_close and
later during shutdown.
It's rather hard (if at all possible) to do a proper stack cleanup
on an unclean shutdown, so I'm just disabling it in this case.
This also reverses the destruction order of the pushed arguments to
align with how it is done everywhere else.
I'm not exactly sure whether this is the right way to fix it, but it
seems to work fine.
Rule of thumb: Always implement the object clone handler rather
than the object storage clone handler. Actually I think we should
drop the latter. It's nearly never usable.
If a generator is destroyed in a finally block it will resume the generator to run that finally
block before freeing the generator. This was done in the object storage free handler.
Running user code in the free handler isn't safe though because the free handlers may be run
during request shutdown, already after several key components have been shut down.
This is avoided by doing the finally handling in the dtor handler. These handlers are run at the
start of the shutdown sequence.
Generator::throw($exception) throws an exception into the generator. The
exception is thrown at the current point of suspension within the generator.
It basically behaves as if the current yield statement were replaced with
a throw statement and the generator subsequently resumed.
If zend_generator_close is called from within zend_generator_resume (e.g.
due to a return statement) then all the EGs will still be using the values
from the generator. That's why the stack frame has to be the last thing
that is dtored, otherwise some other dtor that is using
EG(current_execute_data) might access the already freed memory segment.
This was the case with the closure dtor.
The fix is to move the dtors for key and value to the start of the handler.
This way the stack frame is the last thing that is freed.
When the return value of yield wasn't used it was leaked.
This is fixed by using a TMP_VAR return value instead of VAR. TMP_VARs are
automatically freed when they aren't used.
EG(arg_types_stack) is now also backed up when generators are used. This
allows the use of yield in nested method calls.
This commit adds two new functions to the zend_ptr_stack API:
zend_ptr_stack_push_from_memory
zend_ptr_stack_pop_into_memory
both taking the following arguments:
zend_ptr_stack *stack, int count, void **pointers
If you try to traverse an already closed generator an exception will now be
thrown.
Furthermore this changes the error for traversing a by-val generator by-ref
from an E_ERROR to an Exception.
* Trying to resume a generator while it is already running now throws a
fatal error.
* Trying to use yield in finally while the generator is being force-closed
(by GC) throws a fatal error.
* Rewinding after the first yield now throws an Exception
It looks like you have to implement the Iterator interface *before*
assigning get_iterator. Otherwise the structure for user iterators isn't
correctly zeroed out.
Additionaly I'm setting class_entry->iterator_funcs.funcs now. Not sure if
this is strictly necessary, but better safe than sorry ;)
If a generator threw an exception and was iterated using foreach (i.e. not
manually) an infinite loop was triggered. The reason was that the exception
was not properly rethrown using zend_throw_exception_internal.
Generators are now automatically detected by the presence of a `yield`
expression in their body.
This removes the ZEND_SUSPEND_AND_RETURN_GENERATOR opcode. Instead
additional checks for ZEND_ACC_GENERATOR are added to the fcall_common
helper and zend_call_function.
This also adds a new function zend_generator_create_zval, which handles
the actual creation of the generator zval from an op array.
I feel like I should deglobalize the zend_create_execute_data_from_op_array
code a bit. It currently changes EG(current_execute_data) and
EG(opline_ptr) which is somewhat confusing (given the name).
Previously the zval* of the generator was passed into the VM by misusing
EG(return_value_ptr_ptr). Now the zend_generator* itself is directly passed
in. This saves us from always having to pass the zval* around everywhere.
To make the generator function show up in backtraces one has to insert an
additional execute_data into the chain, as prev_execute_data->function_state
is used to determine the called function.
Adding the additional stack frame is also required for func_get_args(), as
the arguments are fetched from there too. The arguments have to be copied
in order to keep them around. Due to the way they are saved doing so is
quite ugly, so I added another function zend_copy_arguments to zend_execute.c
which handles this.
The current situation is still not perfect, as the generator function itself
does not appear in the stack trace. This makes sense in some way, but it
would probably be more helpful if it would show up (with the bound arguments)
after the $generator->xyz() call. This could be misleading too though as the
function is not *really* called there.
Generators can now be cloned. I'm pretty sure that my current code does not
yet cover all the edge cases of cloning the execution context, so there are
probably a few bugs in there :)
During function calls arguments are pushed onto the stack. Now these are
backed up on yield and restored on resume. This requires memcpy'ing them,
but there doesn't seem to be any better way to do it.
Also this fixes the issue with exceptions thrown during function calls.
When no key is explicitely yielded PHP will used auto-incrementing keys
as a fallback. They behave the same as with arrays, i.e. the key is the
successor of the largest previously used integer key.
Keys are yielded using the
yield $key => $value
syntax. Currently this is implemented as a statement only and not as an
expression, because conflicts arise considering nesting and use in arrays:
yield yield $a => $b;
// could be either
yield (yield $a) => $b;
// or
yield (yield $a => $b);
Once I find some way to resolve these conflicts this should be available
as an expression too.
Also the key yielding code is rather copy-and-past-y for the value yielding
code, so that should be factored out.
Yield now is an expression and the return value is the value passed to
$generator->send(). By default (i.e. if ->next() is called) the value is
NULL.
Unlike in Python ->send() can be run without priming the generator with a
->next() call first.
To keep things clean two new functions are introduced:
zend_clean_and_cache_symbol_table(HashTable *symbol_table)
zend_free_compiled_variables(zval ***CVs, int num)
If the generator is closed before it has finished running, it may happen
that some FREE or SWITCH_FREE opcodes haven't been executed and memory is
leaked.
This fixes it by walking the brk_cont_array and manually freeing the
variables.
