~technomancy/fennel#178: 
REPL locals-saving does not work correctly with var

This file:

(macro incf [var-name] `(do (set ,var-name (+ ,var-name 1)) ,var-name))
(var x 10)
(print (incf x))
(print x)
(incf x)
(print x)

if ran expr by expr in REPL on 1118d8e:

Welcome to Fennel 1.3.1-dev on PUC Lua 5.4!
Use ,help to see available commands.
Try installing readline via luarocks for a better repl experience.
>> (macro incf [var-name] `(do (set ,var-name (+ ,var-name 1)) ,var-name))
nil
>> (var x 10)
nil
>> (print (incf x))
11

>> (print x)
10

>> (incf x)
11
>> (print x)
11

produces 11, 10, 11 messages.

If ran from file it produces 11, 11, 12:

$ fennel ~/bug.fnl
11
11
12

The compiled output looks legit, must be something with REPL-locals?

local x = 10
local function _1_(...)
  x = (x + 1)
  return x
end
print(_1_(...))
print(x)
do
  x = (x + 1)
end
return print(x)
Status
REPORTED
Submitter
~andreyorst
Assigned to
No-one
Submitted
11 months ago
Updated
10 months ago
Labels
bug

~technomancy 10 months ago

A look at the compilation output for the repl version makes it clear what's going on:

>> (print (incf x))
local x = ___replLocals___['x']
local function _1_(...)
  x = (x + 1)
  return x
end
___replLocals___['x'] = x
return print(_1_(...))

The mechanism for saving off the locals gets injected right before the last line, because the last line is the return value, and so saving off the locals after the return value would change the value of the chunk. The assumption is that the last line itself doesn't perform any side-effects which would affect the value of the local being saved. In this case, that assumption is incorrect.

This seems like an inherent limitation of the approach we've chosen to work around this shortcoming of Lua. Not sure I can think of a way around it.

~jaawerth 10 months ago

~technomancy I too thought it was because the function was invoked after saving the replLocals, but it turns out to be a bit more complicated than that.

I went through the same steps you did getting it to print the spliced code, and one big difference is that if it were merely that, then the result of (print (incf v)) wouldn't continue to increase every time you run it. But if you simply this (you don't need the macro or anything, you can just write a function):

(var v 1)
(fn incv []
  (set v (+ v 1))
  v)
(incv)
(incv)
(incv)
v

You'll get the output

2
3
4
1

It took me a minute to figure out where it was being updated and retained, but I finally worked it out: the original local still lives in incv as an upvalue!

The function (generated or not) is creating a closure. Each time the REPL re-introduces saved locals when executing code, declaring a new local v. This value is the one set from and saved to ___replLocals___, distinct from the upvalue v that incv is updating and returning.

This may be tricky to correct - to account for any functions holding REPL locals as upvalues, you'd have to keep track of any such functions and use debug.setupvalue on them on each new REPL loop, which feels like a bit much. It'll likely take some thought to work out the cleanest fix.

#fixing the out-of-order locals saving

There may be edge cases I'm not considering, but I think that one can be worked around by emitting something like

(function (...) 
  <code to save locals>
  return ...
end)(return_statement_with_return_removed)

~jaawerth 10 months ago

One solution coming to mind is to add a :fn plugin hook for the REPL that uses debug.getupvalue to get the names of every upvalue that's also a local, and every time local referredLocal = ___replLocals___['referredLocal'] is injeced on a REPL loop, also inject the necessary debug.setupvalue call to update it for that function.

It would take a fair bit of doing, I think, so it still may be better to rethink how we currently save locals.

Register here or Log in to comment, or comment via email.