Tarantool fiber behavior with fiber.yield() and fiber.testcancel() - lua

Question

I ran into an unexpected behavior while building Tarantool app based on fibers.
Simple reproducer of my code looks like this:
local log = require('log')
local fiber = require('fiber')
box.cfg{}
local func = function()
for i = 1, 100000 do
if pcall(fiber.testcancel) ~= true then
return 1
end
fiber.yield()
end
return 0
end
local wrapfunc = function()
local ok, resp = pcall(func)
log.info(ok)
log.info(resp)
end
for _ = 1, 100 do
local myfiber = fiber.create(wrapfunc)
fiber.sleep(0.02)
fiber.kill(myfiber)
end
and it prints to log false, fiber is cancelled. Moreover, if I use the following func:
local func = function()
for i = 1, 100000 do
if pcall(fiber.testcancel) ~= true then
return 1
end
pcall(fiber.yield)
end
return 0
end
it prints to log true, 1, and if I use
local func = function()
for i = 1, 100000 do
if pcall(fiber.testcancel) ~= true then
return 1
end
if pcall(fiber.yield) ~= true then
return 2
end
end
return 0
end
it prints to log true, 2.
I expected that after yielding from running myfiber, if control returns to the external fiber and it calls fiber.kill(myfiber), the next time control returns to the cancelled myfiber we will be in the end of cycle iteration and on the next iteration code will successfully return 1. However, the work of func ends with throwing error fiber is cancelled, not with return. So how the real life cycle of yielding fiber works?

Answer 1

Actually, there is no unexpected behaviour here. I believe it mostly documentation issue. Let me explain. I've simplified your example a bit:
#!/usr/bin/env tarantool
local fiber = require('fiber')
local f1 = function() fiber.yield() end
local f2 = function() pcall(fiber.yield) end
local func = function(fn)
fn()
if not pcall(fiber.testcancel) then
return 'fiber.testcancel() failed'
end
end
local fiber1 = fiber.create(function() print(pcall(func, f1)) end)
fiber.kill(fiber1)
local fiber2 = fiber.create(function() print(pcall(func, f2)) end)
fiber.kill(fiber2)
The output would be:
false fiber is cancelled
true fiber.testcancel() failed
When you call fiber.kill, fiber.yield() or fiber.sleep() just raises an error, so your fiber is not able to reach fiber.testcancel and just dies. When you do pcall(fiber.yield), you basically suppress this error and proceed. Then fiber.testcancel checks its fiber status and reraise an exception. But this is a silly example.
Now, with bigger chunks of code, when lots of function invocations involved, you usually want to catch those errors during yield, do some finalisation work and call fiber.testcancel() to promote error upwards (imagine multiple checks of this kind in different parts of big stacktrace). I believe it is the basic use case, fiber.testcancel was introduced for, besides discussions if its design is usable or not.
P.s. And yes, occasional exceptions of such yield calls are not documented. At least I could not find anything in the fiber page