~jaawerth


#208 Wrong error line when using `include` a month ago

bug added by ~jaawerth on ~technomancy/fennel

#208 Wrong error line when using `include` a month ago

Comment by ~jaawerth on ~technomancy/fennel

Yes, include wreaking havoc on module filenames is definitely an issue - glad you filed it, we've sort of known about this class of issues for a while, and it was an oversight not to document them in a todo.

mod.fnl:

{:bork (fn [] (error :oops))}

entry.fnl:

(local f (include :mod))
(print "Testing...")
(f.bork)

Results in

Testing...
error in error handling

With AOT, it just gives the expected incorrect stacktrace; run live, that particularly unhelpful error is likely due to a debug module invocation ala traceback, since it's known to happen at the Lua/C boundary. I'm fairly certain this all comes down to being due to the include'd modules being inlined as loader functions when set on package.preload, which loses filename information.

#Most likely fix, IMO

When I originally re-implemented the very early version of include to respect module semantics via package.preload, I actually used load/loadcode with the embedded lua as a string to generate the inlined module loader. At the time, bakpakin objected to this approach because it negatively impacted readability of the emitted lua output. However, I think this is worth revisiting, since switching back to load allows embedding the original file name: I was already planning to do for --correlate (which include breaks entirely), but it's also worth discussing everywhere: running live would also be a no-brainer, since the readability of the Lua output matters even less there.

Granted, we may also be able to achieve this by doing some more funky stacktrace manipulation, but I'm somewhat skeptical - I think this approach would not only also improve the stack traces in code AOT-compiled with --correlate, but also be cleaner in general than trying to work around it by manipulating the sourcemap data.

#216 improve plugin loading options, allow lua plugins a month ago

next-release added by ~jaawerth on ~technomancy/fennel

#216 improve plugin loading options, allow lua plugins a month ago

enhancement added by ~jaawerth on ~technomancy/fennel

#216 improve plugin loading options, allow lua plugins a month ago

Ticket created by ~jaawerth on ~technomancy/fennel

Right now, loading plugins has a few barriers that make it difficult to easily integrate them into, say, the REPL without

For the CLI, it would be useful to accept lua plugins either with the --plugin option (by extension) or via an extra --lua-plugin option.

The main use-case here is to leverage AOT compilation either for plugins written in a different fennel version or, more likely, to precompile a plugin with --require-as-include to bundle in any library component it relies upon.

#Better plugin loading from API?

Additionally, loading plugins by way of the Fennel API (instead of CLI) currently requires manually reproducing the code in launcher.fnl, as in the usecase example below.

#Usecase example

I have a WIP pretty-macrodebug plugin to replace the old macro hack implementation in fnlfmt. It works, but depends on fnlfmt.fnl being somewhere on fennel.path. It would be a lot more useful in tooling plugins that incorporate a library to be able to simply drop a single file either into a project, or into e.g. ${XDG_CONFIG_HOME/fennel for easy use from fennelrc.

Currently, using the plugin requires a fair amount of boilerplate:

(local (*opts* {: view &as fennel}) ...)
(local {:format str/fmt :gmatch str/gmatch :gsub str/gsub :sub str/sub} string)
(local HOME (os.getenv :HOME))
(local dir-sep (str/sub package.config 1 1))

(λ warn [msg-str ...]
  (if (= 0 (select :# ...))
    (io.stderr:write (.. msg-str "\n"))
    (io.stderr:write (str/fmt (.. msg-str "\n") ...)))
  nil)
 
(fn pat/escape [str]
  (str/gsub str "([#$%^%%&*()%][-])" "%%%1") )

(fn tbl/copy [from to]
  (collect [k v (pairs (or from {})) &into (or to {})] k v))

(fn str/split [str sep]
  (icollect [part (str/gmatch str (.. "[^" (pat/escape sep) "]+"))]
    part))

(fn load-plugin [plugin-file ?opts ...]
  (let [opts (tbl/copy (or ?opts {})
                       {:env :_COMPILER :compiler-env _G :useMetadata true})
        (ok plugin) (pcall fnl.dofile plugin-file opts ...) ]
    (if ok
        (table.insert *opts*.plugins plugin)
        (warn (.. "Failed to load plugin '%s': %s" plugin-file
                  (or (and plugin (tostring plugin)) ""))))
    ok))

;; load REPL plugins
(let [fnlfmt-path (table.concat [HOME "dev" "fnl-libs" "fnlfmt"] dir-sep)]
  ;; pretty-macrodebug needs fnlfmt on package.path
  (set fnl.path (.. fnl.path ";" fnlfmt-path dir-sep "?.fnl"))
  (load-plugin (.. fnlfmt-path dir-sep "macrodebug-plugin.fnl") {} ...))

#214 macrodebug should expand macros w/ scope at invocation point a month ago

Comment by ~jaawerth on ~technomancy/fennel

For clarity, it's this change to the tests that seems to still pass without changing macrodebug to use get-scope:

diff --git a/test/macro.fnl b/test/macro.fnl
index 983c3d5..514d9f3 100644
--- a/test/macro.fnl
+++ b/test/macro.fnl
@@ -127,9 +127,13 @@
   (let [eval-normalize #(-> (pick-values 1 (fennel.eval $1 $2))
                             (: :gsub "table: 0x[0-9a-f]+" "#<TABLE>")
                             (: :gsub "\n%s*" " "))
-        code "(macrodebug (when (= 1 1) (let [x :X] {: x})) true)"
-        expected "(if (= 1 1) (do (let [x \"X\"] {:x x})))"]
-    (t.= (eval-normalize code) expected)))
+        cases [["(macrodebug (when (= 1 1) (let [x :X] {: x})) true)"
+                "(if (= 1 1) (do (let [x \"X\"] {:x x})))"]
+               ["(macro reflect-parent [] (next (. (get-scope) :parent :parent :macros)))
+                 (do (do (macrodebug (reflect-parent) true)))"
+                "\"reflect-parent\""]]]
+    (each [_ [code expected] (ipairs cases)]
+      (t.= expected (eval-normalize code)))))
 
 ;; many of these are copied wholesale from test-match, pending implementation,
 ;; if match is implemented via case then it should be reasonable to remove much

#214 macrodebug should expand macros w/ scope at invocation point a month ago

bug removed by ~jaawerth on ~technomancy/fennel

#214 macrodebug should expand macros w/ scope at invocation point a month ago

Comment by ~jaawerth on ~technomancy/fennel

Tests on a fix suggest I may be misremembering the difference between _SCOPE and (get-scope).. I remember (get-scope) being created distinctly for a reason, though. Will look into this further before I close it.

#214 macrodebug should expand macros w/ scope at invocation point a month ago

bug added by ~jaawerth on ~technomancy/fennel

#214 macrodebug should expand macros w/ scope at invocation point a month ago

Ticket created by ~jaawerth on ~technomancy/fennel

Macrodebug is currently implemented as

(fn macrodebug* [form return?]
  "Print the resulting form after performing macroexpansion.
With a second argument, returns expanded form as a string instead of printing."
  (let [handle (if return? `do `print)]
    `(,handle ,(view (macroexpand form _SCOPE)))))

The _SCOPE variable it's passing to macroexpand is the scope in which the macro itself was defined, but it should be (get-scope), the scope at the point the macro is invoked.

This likely hasn't come up because most of what depends on correct scope in a macro relates to when it comes time to compile to lua - gensym for hygiene and the like. Still, it should be fixed.