$he stack reallocation is done in two steps (allocation + free) because
the correction of the pointers pointing into the stack must be done
while both addresses (the old stack and the new one) are valid. In ISO
C, any pointer use after the pointer has been deallocated is undefined
behavior. The compiler option '-fsanitize=pointer-subtract' (plus what
it needs to work) complained about the old implementation.
More uses of macros 'likely'/'unlikely' (renamed to
'l_likely'/'l_unlikely'), both in range (extended to the
libraries) and in scope (extended to hooks, stack growth).
When, inside a coroutine, a C function with to-be-closed slots return,
the corresponding metamethods can yield. ('__close' metamethods called
through 'lua_closeslot' still cannot yield, as there is no continuation
to go when resuming.)
To-be-closed variables are linked in their own list, embedded into the
stack elements. (Due to alignment, this information does not change
the size of the stack elements in most architectures.) This new list
does not produce garbage and avoids memory errors when creating tbc
variables.
- 'L->top' is set once in 'luaD_hook', instead of being set in
'luaD_hookcall' and 'rethook';
- resume discard arguments when returning after an yield inside a hook
(arguments may interfere with the coroutine stack);
- yield inside a hook asserts it has no arguments.
The return hook should be called only after closing variables (which
are still part of the function). C functions were calling the hook
before the metamethods.
Completes commit b07fc10e91. '__close' metamethods can yield even
when they are being called due to an error. '__close' metamethods from
C functions are still not allowed to yield.
Initial implementation to allow yields inside '__close' metamethods.
This current version still does not allow a '__close' metamethod
to yield when called due to an error. '__close' metamethods from
C functions also are not allowed to yield.
When available, use the calling code to find a suitable name for what
was being called; this is particularly useful for errors of non-callable
metamethods. This commit also improved the debug information for
order metamethods.
When a stack grows, its extra area can be in use, and it becomes part
of the common area. So, the extra area must be kept correct all the
times. (Bug introduced by commit 5aa36e894f5.)
The stack size is derived from 'stack_last', when needed. Moreover,
the handling of stack sizes is more consistent, always excluding the
extra space except when allocating/deallocating the array.
Shrink a stack only when the final stack size can be at most 2/3 the
previous size with half of its entries empty. This commit also
improves the clarity of 'luaD_growstack'.
The previous stackless implementations marked all 'luaV_execute'
invocations as fresh. However, re-entering 'luaV_execute' when
resuming a coroutine should not be a fresh invocation. (It works
because 'unroll' called 'luaV_execute' for each call entry, but
it was slower than letting 'luaV_execute' finish all non-fresh
invocations.)
A "with stack" implementation gains too little in performance to be
worth all the noise from C-stack overflows.
This commit is almost a sketch, to test performance. There are several
pending stuff:
- review control of C-stack overflow and error messages;
- what to do with setcstacklimit;
- review comments;
- review unroll of Lua calls.
In luaD_callnoyield, when there is a possible stack overflow, it
zeros the number of CallInfos to force a check when calling the
function. However, if the "function" is not a function, the code will
raise an error before checking the stack. Then, the error handling calls
luaD_callnoyield again and nCcalls is decremented again, crossing the
stack redzone without raising an error. (This loop can only happens
once, because the error handler must be a function. But once is enough
to cross the redzone.)
The field 'L->oldpc' is not always updated when control returns to a
function; an invalid value can seg. fault when computing 'changedline'.
(One example is an error in a finalizer; control can return to
'luaV_execute' without executing 'luaD_poscall'.) Instead of trying to
fix all possible corner cases, it seems safer to be resilient to invalid
values for 'oldpc'. Valid but wrong values at most cause an extra call
to a line hook.
When entering a coroutine, the computation of nCcalls added 'from->nci'
to correct for preallocated CallInfos, but 'nci' includes also the
Callinfos already used.
Macro 'checkstackGC' was doing a GC step after resizing the stack;
the GC could shrink the stack and undo the resize. Moreover, macro
'checkstackp' also does a GC step, which could remove the preallocated
CallInfo when calling a function. (Its name has been changed to
'checkstackGCp' to emphasize that it calls the GC.)
Back to how it was, a coroutine does not unwind its stack in case of
errors (and therefore do not close its to-be-closed variables). This
allows the stack to be examined after the error. The program can
use 'coroutine.kill' to close the variables.
The function created by 'coroutine.wrap', however, closes the
coroutine's variables in case of errors, as it is impossible to examine
the stack any way.
The limit LUAI_MAXCCALLS was renamed LUAI_MAXCSTACK, which better
represents its meaning. Moreover, its definition was moved to
'luaconf.h', given its importance now that Lua does not use
a "stackless" implementation.
To-be-closed variables must contain objects with '__toclose'
metamethods (or nil). Functions were removed for several reasons:
* Functions interact badly with sandboxes. If a sandbox raises
an error to interrupt a script, a to-be-closed function still
can hijack control and continue running arbitrary sandboxed code.
* Functions interact badly with coroutines. If a coroutine yields
and is never resumed again, its to-be-closed functions will never
run. To-be-closed objects, on the other hand, will still be closed,
provided they have appropriate finalizers.
* If you really need a function, it is easy to create a dummy
object to run that function in its '__toclose' metamethod.
This comit also adds closing of variables in case of panic.
* unification of the 'nny' and 'nCcalls' counters;
* external C functions ('lua_CFunction') count more "slots" in
the C stack (to allow for their possible use of buffers)
* added a new test script specific for C-stack overflows. (Most
of those tests were already present, but concentrating them
in a single script easies the task of checking whether
'LUAI_MAXCCALLS' is adequate in a system.)
New functions to reset/kill a thread/coroutine, mainly (only?) to
close any pending to-be-closed variable. ('lua_resetthread' also
allows a thread to be reused...)
- in 'luaB_tonumber', do not need to "checkany" when argument
is a number.
- in 'lua_resume', the call to 'luaD_rawrunprotected' cannot return
a status equal to -1.
(Long time without testing with '-DHARDSTACKTESTS'...)
With the introduction of to-be-closed variables, calls to 'luaF_close'
can move the stack, but some call sites where keeping pointers to the
stack without correcting them.
Start of the implementation of "scoped variables" or "to be closed"
variables, local variables whose '__close' (or themselves) are called
when they go out of scope. This commit implements the syntax, the
opcode, and the creation of the corresponding upvalue, but it still
does not call the finalizations when the variable goes out of scope
(the most important part).
Currently, the syntax is 'local scoped name = exp', but that will
probably change.
Resuming a coroutine uses more C stack than other operations (such as
function calls or recursive syntax). So, to avoid stack overflow
in recursive coroutine invocations, either LUAI_MAXCCALLS must be
too small or a coroutine invocation must "pay" a higher price.
New constant LUAL_COROCSTK ("COROutine C STaK") defines how much
is this price.
Roberto Ierusalimschy