~technomancy 7 months ago

You can already do this in most circumstances:

foo.fnl

(import-macros {: inc} (.. ... :.macros))

In most cases, ... at the top level of a file will evaluate to the name of the current module. So the above will load the macros in foo/macros.fnl. The main limitation is that this only works easily if A) the current file is being loaded as a module (from require and not fennel.dofile) and B) the module you want to load is "under" the current one.

The documentation explains how to do this with require but it should be extended to cover the fact that it works with import-macros as well: https://fennel-lang.org/tutorial#relative-require

~technomancy added docs 7 months ago

~lordmzte 7 months ago

Neither of these conditions apply here. ... is useless here, as it simply evaluates to nil when the fennel file is compiled using fennel -c foo.fnl. Here's a more clear description of the problem:

Let's consider this directory structure:

macros.fnl
some-dir/
| foo.fnl

#macros.fnl

;; example macro
(macro example [] 42)

#foo.fnl

;; Something like this does not currently exist
;(import-macros-path {: example} "../macros.fnl")

;; The `example` macro is used here
(print (example))

The goal here is to compile foo.fnl to a lua file foo.lua. This lua file should contain the following code:

print(42)

This does not currently appear to be possible. I currently do the compilation by invoking fennel -c some-dir/foo.fnl. This does however not allow the use of macros from macros.fnl.

Your proposed solution does not help here, as there is no consistently established "module path" in the compile-time environment in my use-case. My build system does not enforce any sort of consistent location of source files, making it impossible to solve this using module paths. What is needed for this is for a file to simply include macros from another without depending on any form of module structure.

We could also extend the above example by adding another file into some-dir/. This file should be able to use macros.fnl just as well. It will be compiled to another lua file by a seperate invocation of fennel -c.

~technomancy 7 months ago

Your proposed solution does not help here, as there is no consistently established "module path" in the compile-time environment in my use-case.

Right, I see what you mean. I think what you're asking is inherently impossible, because macros are fundamentally imported by module name, and you are trying to use them relatively in a case where there are no modules, so there is nothing for them to be relative with respect to.

Can you explain again the use case for why things need to be relative in this situation? The existing use case for this is for publishing libraries that might get re-rooted when they get included in the codebase, but that's very different from what you have happening here.

Seems like the simplest solution is to just start naming the modules consistently, or stop doing single-file AOT compilation.

~lordmzte 7 months ago

I'm thinking that the import path would be relative to the directory the importing file is in. This would allow re-rooting just fine. My use-case here is admittedly very niche: the program I'm writing here is basically one you can give a directory to, and it will compile lua files in there to LuaJIT bytecode. This is to improve loading times of that lua code by doing the JIT compilation of LuaJIT ahead of time. I've recently added in fennel support to this, where fennel files are converted to lua in-place. This needs to work given any directory, and it should also recurse into subdirectories. Treating the given directory as root will not work, because it might be a directory containing multiple projects (for example, my neovim plugin folder). Given this architecture of being able to plug the build system into any lua or fennel project means that there is no way to have any sort of project-specific configuration that I can rely on. I don't see any good solution to allow shared macros here except my proposed relative-path based import.

~technomancy 7 months ago*

I'm thinking that the import path would be relative to the directory the importing file is in.

In the context of doing AOT on a single file, this would be accomplished most easily by passing in an --add-macro-path argument to the compiler along with the path of the file being compiled.

Changing the default macro path would be a serious backwards-compatibility regression.

Another option would be to somehow populate the ... value during AOT so it behaves the same way that it does during normal compilation, but it would have to accept this as a CLI argument, because I don't think it can be inferred automatically.

~lordmzte 7 months ago

Well, as I stated, I don't really have a module path I could give to the compiler here. I also wouldn't suggest changing existing behavior, but I believe it would be a great addition to have a function that operates on plain file paths instead of module paths of any sort.

~technomancy 7 months ago

Well, as I stated, I don't really have a module path I could give to the compiler here

What I mean is you clearly have some form of automation which invokes the compiler on these files; this script must have the path to the file in question. From that path, the script can calculate a macro-path to pass in to the --add-macro-path argument.

a function that operates on plain file paths instead of module paths of any sort.

The whole system of sharing macros is based on modules. In fact, the whole system of sharing anything across files in a codebase is based on modules. The equivalent for doing this in Lua without modules is dofile but it's considered to be a low-level construct that is normally only invoked for side-effects. There are already three different ways to bring in macros in Fennel; adding a fourth for such an extreme edge case would be very awkward.

~lordmzte 7 months ago

I do not see how my build system could possible calculate any reasonable macro path. Do you have any suggestions? The best I can think of would be adding the directory the script is in, but that's not useful when the macro file is 2 directories higher. As I explained already, there is no root project path.