Ticket created by ~rktjmp on ~technomancy/fennel
The test suite seems to depend on state from previous tests to pass sometimes.
Run against 85449ab (1.3.0-dev), lua 5.4.2
fex:
make test
-> all passing- Disable all suites in test/init.lua aside from
failures
make test
-> fails to run- Re-enable
core
(socore
+failures
)make test
-> all passingOr a tighter focus:
- Disable all suites but
core
&&failures
intest/init.lua
.- Disable all tests in those suites except
test/core#test-nest
&&test/failures#test-suggestions
make test
-> 2/2 tests passing- Disable
test/core#test-nest
make test
-> test failsPreviously:
The compiler plugin used in
failures#test-macro-traces
would be retained in subsequent tests until one of therepl
tests seemed to unintentionally remove it, see https://github.com/bakpakin/Fennel/pull/427#issuecomment-1138286136(The compiler plugin test currently includes a work-around via an internal guard to only intentionally fail once.)
Ticket created by ~rktjmp on ~technomancy/fennel
#(do #(values $...))Compile error: use $... in hashfn
Non vargs are ok:
#(do #(values $1))local function _1_() local function _2_(_2410) return _2410 end return _2_ end return _1_
Ticket created by ~rktjmp on ~technomancy/fennel
(macro wont-output-body [] ;; let also fails to output (local body [`(do (+ 1 1) (values 1)) `(do (+ 2 2) (values 2))]) `(fn [] ,body (values :val))) (macro will-output-body [] (local body [`(do (+ 1 1) (values 1)) `(do (+ 2 2) (values 2))]) `(fn [] nil ;; can be any expression ,body ;; (do ,body) also works (values :val))) (comment the following WONT output body statements) (wont-output-body) (comment the following WILL output body statements) (will-output-body) (macro just-body [] (local body [`(do (+ 1 1) (values 1)) `(do (+ 2 2) (values 2))]) body) (comment otherwise this normally works) (just-body)--[[ the following WONT output body statements ]] local function _1_() return "val" end --[[ the following WILL output body statements ]] local function _2_() local _3_ do do local _ = (1 + 1) end _3_ = 1 end local function _4_() do local _ = (2 + 2) end return 2 end do local _ = {_3_, _4_()} end return "val" end --[[ otherwise this normally works ]] local _5_ do do local _ = (1 + 1) end _5_ = 1 end local function _6_(...) do local _ = (2 + 2) end return 2 end return {_5_, _6_(...)}
Comment by ~rktjmp on ~technomancy/fennel
Probably something you've already thought about, but in my own macro I have used something similar to Elixir's Ecto library which lets you explicitly "pin" a value to an outer value in queries with the
^
symbol.In
match
it might look something like(local var 10) (match x ^var true _ false)Using an in-scope symbol without
^
is an compile error, using^
on a symbol that is not in scope is also an error.Obviously this is a breaking change, not really appropriate (2.0? I think the opt-in behaviour is a lot clearer.) but maybe something to think about - opt in on the symbol level instead of all or nothing.
Adding an opt-out syntax wouldn't be breaking but not as useful, you still have the accidental-match occurrences which is what you're trying to avoid.
Comment by ~rktjmp on ~technomancy/fennel
Should the implementation be more complex than passing an option around? Just a POC.
match
's default behaviour is impacting a macro for me, so I guess I am selfishly +1'ing this feature.I think the name is a bit awkward, and I think explaining the differences is probably awkward too. Probably a good extension to have available when it really is needed though when locals or patterns cant be renamed.
How would it look if it were an option to
match
instead? After the match value "reads" better to me and is sort of inline with*collect
options.(match [a b] &scoped ;; &nolocals? [1 2] 3) (match &less [a b] [1 2] 3)Anyway here's a patch, probably not the patch. Do you have an implementation in mind? Instead of
(or opts (})
'ing everywhere I just elected to(?. opts key)
where needed.diff --git a/src/fennel/macros.fnl b/src/fennel/macros.fnl index f2b7980..12d9c99 100644 --- a/src/fennel/macros.fnl +++ b/src/fennel/macros.fnl @@ -390,18 +390,18 @@ Example: ;;; Pattern matching -(fn match-values [vals pattern unifications match-pattern] +(fn match-values [vals pattern unifications match-pattern opts] (let [condition `(and) bindings []] (each [i pat (ipairs pattern)] (let [(subcondition subbindings) (match-pattern [(. vals i)] pat - unifications)] + unifications opts)] (table.insert condition subcondition) (each [_ b (ipairs subbindings)] (table.insert bindings b)))) (values condition bindings))) -(fn match-table [val pattern unifications match-pattern] +(fn match-table [val pattern unifications match-pattern opts] (let [condition `(and (= (_G.type ,val) :table)) bindings []] (each [k pat (pairs pattern)] @@ -409,7 +409,7 @@ Example: (let [rest-pat (. pattern (+ k 1)) rest-val `(select ,k ((or table.unpack _G.unpack) ,val)) subcondition (match-table `(pick-values 1 ,rest-val) - rest-pat unifications match-pattern)] + rest-pat unifications match-pattern opts)] (if (not (sym? rest-pat)) (table.insert condition subcondition)) (assert (= nil (. pattern (+ k 2))) @@ -431,13 +431,13 @@ Example: (not= `& (. pattern (- k 1))))) (let [subval `(. ,val ,k) (subcondition subbindings) (match-pattern [subval] pat - unifications)] + unifications opts)] (table.insert condition subcondition) (each [_ b (ipairs subbindings)] (table.insert bindings b))))) (values condition bindings))) -(fn match-pattern [vals pattern unifications] +(fn match-pattern [vals pattern unifications opts] "Take the AST of values and a single pattern and returns a condition to determine if it matches as well as a list of bindings to introduce for the duration of the body if it does match." @@ -445,10 +445,11 @@ introduce for the duration of the body if it does match." ;; know we're either in a multi-valued clause (in which case we know the # ;; of vals) or we're not, in which case we only care about the first one. (let [[val] vals] - (if (or (and (sym? pattern) ; unification with outer locals (or nil) - (not= "_" (tostring pattern)) ; never unify _ - (or (in-scope? pattern) (= :nil (tostring pattern)))) - (and (multi-sym? pattern) (in-scope? (. (multi-sym? pattern) 1)))) + (if (and (not (?. opts :matchless?)) + (or (and (sym? pattern) ; unification with outer locals (or nil) + (not= "_" (tostring pattern)) ; never unify _ + (or (in-scope? pattern) (= :nil (tostring pattern)))) + (and (multi-sym? pattern) (in-scope? (. (multi-sym? pattern) 1))))) (values `(= ,val ,pattern) []) ;; unify a local we've seen already (and (sym? pattern) (. unifications (tostring pattern))) @@ -462,21 +463,21 @@ introduce for the duration of the body if it does match." ;; guard clause (and (list? pattern) (= (. pattern 2) `?)) (let [(pcondition bindings) (match-pattern vals (. pattern 1) - unifications) + unifications opts) condition `(and ,(unpack pattern 3))] (values `(and ,pcondition (let ,bindings ,condition)) bindings)) ;; multi-valued patterns (represented as lists) (list? pattern) - (match-values vals pattern unifications match-pattern) + (match-values vals pattern unifications match-pattern opts) ;; table patterns (= (type pattern) :table) - (match-table val pattern unifications match-pattern) + (match-table val pattern unifications match-pattern opts) ;; literal value (values `(= ,val ,pattern) [])))) -(fn match-condition [vals clauses] +(fn match-condition [vals clauses opts] "Construct the actual `if` AST for the given match values and clauses." (if (not= 0 (% (length clauses) 2)) ; treat odd final clause as default (table.insert clauses (length clauses) (sym "_"))) @@ -484,7 +485,7 @@ introduce for the duration of the body if it does match." (for [i 1 (length clauses) 2] (let [pattern (. clauses i) body (. clauses (+ i 1)) - (condition bindings) (match-pattern vals pattern {})] + (condition bindings) (match-pattern vals pattern {} opts)] (table.insert out condition) (table.insert out `(let ,bindings ,body)))) @@ -513,6 +514,12 @@ introduce for the duration of the body if it does match." ;; many values as we ever match against in the clauses. (list `let [vals val] (match-condition vals clauses)))) +(fn matchless* [val ...] + ;; identical to match* but for match-condition options + (let [clauses [...] + vals (match-val-syms clauses)] + (list `let [vals val] (match-condition vals clauses {:matchless? true})))) + ;; Construction of old match syntax from new syntax (fn partition-2 [seq] @@ -636,4 +643,5 @@ returned as the value of the entire expression." :macrodebug macrodebug* :import-macros import-macros* :match match-where + :matchless matchless* :match-try match-try*} diff --git a/test/macro.fnl b/test/macro.fnl index 51c193c..7cb47e1 100644 --- a/test/macro.fnl +++ b/test/macro.fnl @@ -239,6 +239,20 @@ (== (match nil _ :yes nil :no) "yes") (== (let [_ :bar] (match :foo _ :should-match :foo :no)) "should-match")) +(fn test-matchless [] + (== (let [a 10 + b 20] + (matchless [1 2] + [x y] [x y a b])) + [1 2 10 20] + nil "matchless without a shadow") + (== (let [a 10 + b 20] + (matchless [1 2] + [a x] [a x b])) + [1 2 20] + nil "matchless with a shadow")) + (fn test-match-try [] (== (match-try [1 2 1] [1 a b] [b a] @@ -330,4 +344,5 @@ : test-disabled-sandbox-searcher : test-expand : test-match-try + : test-matchless : test-literal}
Comment by ~rktjmp on ~technomancy/fennel
You can enable github flavoured markdown, via
-f gfm
, this has a more permissivetext->id
filter and gives you links like:changelog#121--2022-10-15 changelog#new-features changelog#110--2022-04-09 changelog#new-forms-1 changelog#new-features-1
Your versions are nicely anchored, subheadings wont be "nice" as they're duplicated, but will be uniquely postfixed.
The other option is manually tagging an id with
## 1.2.1 / 2022-10-15 {#1-2-1} ### New Features {#1-2-1-new-features}
for
/changelog#1-2-1 /changelog#1-2-1-new-features
Which seems like quite a pain and probably not worth it unless you really want to link to specific features in a release.
Comment by ~rktjmp on ~technomancy/fennel
Fixed in 4e9ab92aefe6dc82c6925b5fdbfb3f5d9ddbdfde
Comment by ~rktjmp on ~technomancy/fennel
Comment by ~rktjmp on ~technomancy/fennel
So looking for a proper way to fix it, I noticed that we set the module name in the module searcher but not the macro searcher.
It's pretty easy to fix that, but I am not 100% on the repercussions.
diff --git a/src/fennel/specials.fnl b/src/fennel/specials.fnl index 5d9fc7f..58301a7 100644 --- a/src/fennel/specials.fnl +++ b/src/fennel/specials.fnl @@ -1099,6 +1099,7 @@ table.insert(package.loaders or package.searchers, fennel.searcher)" (fn fennel-macro-searcher [module-name] (let [opts (doto (utils.copy utils.root.options) + (tset :module-name module-name) (tset :env :_COMPILER) (tset :requireAsInclude false) (tset :allowedGlobals nil))] @@ -1148,12 +1149,19 @@ modules in the compiler environment." "expected each macro to be function" ast) (tset scope.macros k v))) -(fn resolve-module-name [{: filename 2 second} _scope _parent opts] +(fn resolve-module-name [ast _scope _parent opts] ;; Compile module path to resolve real module name. Allows using ;; (.. ... :.foo.bar) expressions and self-contained ;; statement-expressions in `require`, `include`, `require-macros`, ;; and `import-macros`. - (let [filename (or filename (and (utils.table? second) second.filename)) + ;; The first statement in the AST will be `import-macros` which is not + ;; appropriate to give as the filename to the compiler. + ;; We instead check the user-suppied form (right after `import-macros`) and + ;; use that value. If the user only gave a literal string, the filename will + ;; be nil, but in the context of compiling that literal to code, that wont + ;; really matter + (let [{2 second} ast + {: filename} second module-name utils.root.options.module-name modexpr (compiler.compile second opts) modname-chunk (load-code modexpr)]
Ticket created by ~rktjmp on ~technomancy/fennel
I ran into two issues with relative requires today, two cases:
Repros
https://github.com/rktjmp/fennel-macro-2
A macro, requires a module which requires a macro.
In the example (
mac-mod-mac
), there is a macrorsym
which accepts a list of syms and returns them in reverse, as strings. This uses the moduleseq
which has a reverse iteration function. Somewhat contritely, this module employs adec
macro to generate a(- x 1)
expression.So mac->mod->mac require chain.
This functions fine with hard coded require paths but using
...
fails as...
is nil in the context.
A macro-file requires another macro.
In the example (
mac-mac
), there are two macros.complicate
which accepts a sym and generates a local var named "sym-complicated".inc
which generates a(+ 1 x)
expression.The
inc.fnl
file wants to internally, in the compiler scope of the macro file, use thecomplicate
macro.Again, this functionality is fine when using hardcoded paths but fails when using
...
inimport-macros
because the the module is never passed in.Exploration
Some things I have discovered:
Currently fennel extracts the filename of the first form in the AST which is
import-macros
, which is not useful in the context of the passed in expression. We can get the correct filename by looking past it into the expression or string. If an expression is given, the correct filename is attached and we can pull that out, a strings filename will be nil, but since it's a raw string that seems OK.The
module-name
fromutils.root.options.module-name
is always nil, so themodule-name
passed to the lua chunk is alwaysnil
currently.We can construct the modname by fiddling with the filename, OR at least we can extract the more relevant filename from the given AST and pass that into the expression and at least the end user can match on
[nil filename]
and manually construct the relative modname (but that would be pretty annoying).Patch
This is a "makes it work" patch, not sure if it's actually the best way to do it. Perhaps sorting out why
util.root.options.module-name
is the better fix or maybe that's intentional?(fn resolve-module-name [ast _scope _parent opts] ;; Compile module path to resolve real module name. Allows using ;; (.. ... :.foo.bar) expressions and self-contained ;; statement-expressions in `require`, `include`, `require-macros`, ;; and `import-macros`. ++ ;; filename here will be `src/fennel/macros.fnl` as it's pulled from ++ ;; `import-macros`. this is probably largely useless in this context. ++ (local {: filename 2 second} ast) ++ (let [;; filename (or filename (and (utils.table? second) second.filename)) ++ ;; extract filename from actual form from user, not the filename for ++ ;; import-macros which is, not so useful. ++ ;; when given only a string, the filename will be nil, which is probably ++ ;; fine as the string just has to exist for half a moment. ++ ;; this always seems to be false, even though it is a table. ; _ (print (utils.table? second)) ++ {: filename} second ;; module-name is always nil from utils.root.options ;; module-name utils.root.options.module-name ++ module-name (-> (or filename "") ;; or "" for seconds as literal strings with no filenames ++ (string.gsub "%.fnl$" "") ++ (string.gsub "%.lua$" "") ;; needed? ++ (string.gsub "^%./" "") ;; not x-compat ++ (string.gsub "/" ".")) ;; not x-compat ; _ (print :filename filename :module-name module-name) ; _ (print second (fennel.view opts)) ;; _ (require "pl") ; _ (print (pretty (. ast 2))) modexpr (compiler.compile second opts) modname-chunk (load-code modexpr nil filename)] ; (print :resovle-module-name (fennel.view module-name)) (modname-chunk module-name filename)))Sort of related: https://todo.sr.ht/~technomancy/fennel/49