#52 Illegal semicolons can be emitted at the beginning of functions 2 days ago

Comment by ~technomancy on ~technomancy/fennel

This is fixed and released in 0.9.1; I think the fix should do the trick. I've tested it across all of the codebases I have in https://git.sr.ht/~technomancy/fennel-compendium and haven't found any trouble.

REPORTED RESOLVED FIXED

#52 Illegal semicolons can be emitted at the beginning of functions 3 days ago

Comment by ~technomancy on ~technomancy/fennel

I pushed a fix for this in 163ab94 that checks to see if it's the first element in the chunk, but I'm not 100% confident of it. I'll test it across a few other codebases but if we find trouble we can bring back the do/end solution.

#51 An equivalent of package.searchers for compile time 8 days ago

Comment by ~technomancy on ~technomancy/fennel

Applied in 761c5c6

REPORTED RESOLVED FIXED

#51 An equivalent of package.searchers for compile time 9 days ago

Comment by ~technomancy on ~technomancy/fennel

This is addressed by a patch here: https://lists.sr.ht/~technomancy/fennel/patches/21778

#51 An equivalent of package.searchers for compile time 15 days ago

enhancement added by ~technomancy on ~technomancy/fennel

#51 An equivalent of package.searchers for compile time 15 days ago

Comment by ~technomancy on ~technomancy/fennel

A little more detail about how it currently works:

The entry point is the require-macros special. Any import-macros calls expand to this. In require-macros it calls compiler-env-domodule; this function takes a module name and returns the module itself. But it looks up the module in a way that is not extensible: by searching fennel.path and using io.open to check for existence of each possible filename on the path.

We need to replace this with an extensible model, and one that is based on package.searchers would fit the bill. We can introduce a new fennel.module-searchers function which defaults to containing one function that works the current way but allows people to add their own searcher functions. Just like with normal modules, custom searcher functions would take a module name as an argument and (if it's able to find the module) returns a function which, when called, would return the macro module.

https://www.lua.org/manual/5.4/manual.html#pdf-package.searchers

So for instance, a love2d-based searcher would know how to check inside the game archive file for a macro module that is bundled with the rest of the source rather than having to look directly on disk.

This might actually solve #40 now that I think about it.

#50 luacheck warnings 15 days ago

Comment by ~technomancy on ~technomancy/fennel

Yeah, this is one of the most complex parts of the compiler.

While it would be nice to clean up the output here I think it's not going to happen without significant changes that are difficult to justify just for a minor cosmetic improvement. I think we can close this out for now.

#50 luacheck warnings 25 days ago

Comment by ~technomancy on ~technomancy/fennel

Do you think this can be seen as an opportunity to make compiler more strict? e.g. if you want to define a recursive function with let you must give it a name

Yes, I think this is probably a good thing to do before we release 1.0. There is some possibility of incompatibility, but it should be easy to fix when it comes up, and the recursion to an internal name is just a lot clearer. Maybe we can even get this in for 0.9.0. Would you like to take a shot at it?

#50 luacheck warnings 26 days ago

Comment by ~technomancy on ~technomancy/fennel

This is very unexpected, transforming return value into function call just because a set of parentheses goes on the next line.

Non-lisps, man. They're wild; what more can I say? =)

Yeah, in this case emitting a bunch of semicolons every time something is called seems like a better alternative to the do end.

Cool; I'll make that change.

But it seems that you can simply throw away the initialization part:

local slength = nil
local
function _0_(_241)
  return #_241
end
slength = _0_

Is exactly the same as:

local function _0_(_241)
  return
#_241
end
local slength = _0_

Well, now that I think it about it, it comes down to the question of whether the slength identifier needs to be in scope for the body of the function. The first above has it, but the second does not, so they can't be considered equivalent to each other.

On the other hand, I'm not convinced that putting slength in scope for the body of the function actually IS correct. It seems like if you wanted to write a recursive function, binding it using let is the wrong way to do it.

I guess a better example would be this:

(let [myfun (fn f [x] (if (= x 0) x (f (- x 1))))]
  (myfun 12))

In this case the "internal name" f is different from the let-bound name of myfun. This is why functions never compile to "bare values" and are always local-bound; because they can have an internal name, and it's simpler to implement if you assume they always have a name, and just gensym one if there's not one provided. There's just one compilation path.

I think it would be better to use the local-bound functions specifically in cases that an internal name is present, and compile to a raw value if not, skipping the gensym altogether.

#50 luacheck warnings 26 days ago

Comment by ~technomancy on ~technomancy/fennel

Static checkers exist to provide more idiomatic code with less ways to break it in an unexpected way. I think it would be great if generated Lua code would not cause any warnings from such checker.

I agree; the current output of the compiler is not as readable as it could be. As far as I know, we don't have any quirks in the output which affect the correctness of the code, but some of the constructs emitted sure do look strange.

And if we can make the compiler output "luacheck-clean" then it might allow us to use luacheck to spot bugs in Fennel code. However, most of the things that luacheck looks for are already spotted by Fennel by default, such as unknown global use; in fact, it was a goal early on with Fennel to make the default settings of the compiler work in a way that would make luacheck unnecessary. =) We actually ran luacheck on Fennel's own compiler implementation up until the point of self-hosting.

(icollect [i v (ipairs [1 2 3])] (do (io.stderr:write i v) (+ v 1)))

do do end (io.stderr):write(i, v) 0 = (v + 1) end

Here, the empty do end block only occurs when we're calling method, so if you replace io.stderr:write with print the do end is gone. A potential bug?

The reasoning behind this is clearer with the following example:

(#1)
(io.stderr:write "hey")
nil

Compiles to:

local function _0_()
  return 1
end
_0_()
do end (io.stderr):write("hey")
return nil

In this case if the do/end were omitted, you would end up with the equivalent of the following:

(function() return 1 end)
(io.stderr):write("hey")

Which is equivalent to 1(io.stderr), which gives a "cannot call a number as a function" error. The do/end serves as a separator in order to disambiguate the parens into being used for grouping purposes rather than as a function call.

However, I think that a semicolon,though very rarely used in Lua written by humans, would be clearer here. It makes it more obvious that it's intended to be used as a separator.

The overshadowing part for instance makes code a lot less readable, for example here fennelcheck '(let [slength #(length $)] (slength :a))':

Generated Lua:
local slength = nil
local function _0_(_241)
  return #_241
end
slength = _0_
return slength("a")

I agree this isn't ideal compiler output. I would love to get this part cleaned up.

However, the root problem here is not overshadowing; the problem is that the compiler will never emit a function as a "bare" value; it will always bind the function to a gensym'd local first, and then take that function and set the existing local to the gensym'd name.

This is maybe my longest-running annoyance in the compiler, (it has been bugging me since 2018: https://technomancy.us/186) but I'm a little concerned that since it's such a core assumption in the compiler it might be difficult to fix.

Executing this snippet will result in another warning fennelcheck '(each [_ (ipairs [:a :b :c])] (match :a (nil) nil))':

Generated Lua:
for _ in ipairs({"a", "b", "c"}) do
  local _0_0 = "a"
  if (_0_0 == nil) then
  end
end
return nil
Checking stdin                                    1 warning

stdin:3:20: empty if branch

Total: 1 warning / 0 errors in 1 file ```

Here, Fennel compiler outsmarts the programmer and just does not include the code into if branch at all, because it is a statement that can't be returned. Not sure if any bugs can crawl in here, but still seems that branch in this case can be dropped completely?

If it were a statement, then there would be a possibility of a bug, for sure. But it's an expression which is guaranteed to have no side-effects, so it's 100% safe to drop. As soon as you replace the nil with any code which could have side-effects, you'll see that it will be included in the compiler output.

I would say that this is a case in which the luacheck warning is valid and useful. It helps you spot cases where you have values that look like they are meaningful but will in fact never be used. The output looks weird, but only because the input is weird! I wouldn't object to omitting the entire if, but I don't think it's worth bothering about personally.