~andrewcrook

UK


#230 Malformed Carriage Returns in output 4 months ago

Comment by ~andrewcrook on ~technomancy/fennel

with fennel 1.5.0 and Lua 5.1.5 it seem to work without the above (1.5.0 has rewritten the serialize-string function swell) luaJiIT still returns \13 for \r though I think this is because its based on an earlier version of 5.1.

#31 fennel_ls quit with exit code 1 and signal 0 4 months ago

Comment by ~andrewcrook on ~xerool/fennel-ls

[START][2024-07-23 15:09:05] LSP logging initiated [ERROR][2024-07-23 15:09:05] .../vim/lsp/rpc.lua:770 "rpc" "/Users/XXXX/.local/share/nvim/mason/bin/fennel-ls" "stderr" "/Users/XXXX/.local/share/asdf/installs/lua/5.1.5/bin/lua: ...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:86: attempt to index global 'vim' (a nil value)\nstack traceback:\n\t...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:86: in function 'make_configuration_from_template'\n\t...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:95: in function <...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:80>\n\t(tail call): ?\n\t...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:136: in function 'init-state'\n\t...kages/fennel-ls/share/lua/5.1/fennel-ls/handlers.lua:66: in function 'callback'\n\t...kages/fennel-ls/share/lua/5.1/fennel-ls/dispatch.lua:17: in function <...kages/fennel-ls/share/lua/5.1/fennel-ls/dispatch.lua:12>\n\t(tail call): ?\n\t...b/luarocks/rocks-5.1/fennel-ls/0.1.2-2/bin/fennel-ls:33: in function <...b/luarocks/rocks-5.1/fennel-ls/0.1.2-2/bin/fennel-ls:28>\n\t(tail call): ?\n\t(tail call): ?\n\t[C]: ?\n”

#31 fennel_ls quit with exit code 1 and signal 0 4 months ago

Ticket created by ~andrewcrook on ~xerool/fennel-ls

HI I am still having trouble getting fennel-ls working.

versions info

>uname -a
Darwin XXXX-MBP 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:16:51 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T8103 arm64

> sw_vers
ProductName:		macOS
ProductVersion:		14.5
BuildVersion:		23F79
Neovim
NVIM v0.10.0
Build type: Release
LuaJIT 2.1.1713484068

Mason vim plugin installs luarocks package

fennel-ls@0.1.2-2

external Lua and fennel

Lua 5.1.5
Fennel 1.4.2 on PUC Lua 5.1

I get the following error in nvim

Client fennel_ls quit with exit code 1 and signal 0. Check log for errors: /Users/XXXX/.local/state/nvim/lsp.log

The log file /Users/XXXX/.local/state/nvim/lsp.log contains the following

[START][2024-07-23 15:09:05] LSP logging initiated
[ERROR][2024-07-23 15:09:05] .../vim/lsp/rpc.lua:770	"rpc"	"/Users/XXXX/.local/share/nvim/mason/bin/fennel-ls"	"stderr"	"/Users/XXXX/.local/share/asdf/installs/lua/5.1.5/bin/lua: ...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:86: attempt to index global 'vim' (a nil value)\nstack traceback:\n\t...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:86: in function 'make_configuration_from_template'\n\t...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:95: in function <...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:80>\n\t(tail call): ?\n\t...packages/fennel-ls/share/lua/5.1/fennel-ls/state.lua:136: in function 'init-state'\n\t...kages/fennel-ls/share/lua/5.1/fennel-ls/handlers.lua:66: in function 'callback'\n\t...kages/fennel-ls/share/lua/5.1/fennel-ls/dispatch.lua:17: in function <...kages/fennel-ls/share/lua/5.1/fennel-ls/dispatch.lua:12>\n\t(tail call): ?\n\t...b/luarocks/rocks-5.1/fennel-ls/0.1.2-2/bin/fennel-ls:33: in function <...b/luarocks/rocks-5.1/fennel-ls/0.1.2-2/bin/fennel-ls:28>\n\t(tail call): ?\n\t(tail call): ?\n\t[C]: ?\n”

#230 Malformed Carriage Returns in output 4 months ago

Comment by ~andrewcrook on ~technomancy/fennel

Sorry I have been preoccupied

The issue is Lua's string.format(“%q”, str) so this isn't likely to be fixed in older versions of Lua especially 5.1.x It can be fixed in the compiler.fnl by literally converting numerical to char variants I don't have access to fennel tooling at the moment for testing because I am on ARM. Lua 5.1.5 seems to be okay with the code below. I think 5.1.0 tests were failing with the latest version of fennel anyway, I don't know what your policy is on backward compatibility.

;; Allow printing a string to Lua, also keep as 1 line.
(local serialize-subst {"\a" "\\a"
                        "\b" "\\b"
                        "\t" "\\t"
                        "\r" "\\r"
                        "\n" :n
                        "\v" "\\v"
                        "\f" "\\f"})

(local serialize-subst-digits { "\\7" "\\a"
                                "\\8" "\\b"
                                "\\9" "\\t"
                                "\\10" :n
                                "\\11" "\\v"
                                "\\12" "\\f"
                                "\\13" "\\r"
                        })


(fn serialize-string [str]
  (-> (string.format "%q" str)
      (string.gsub "." serialize-subst)
      ;; macos convert from digit form to display correctly
      (string.gsub "\\..?" serialize-subst-digits)
      (string.gsub "[\128-\255]" #(.. "\\" ($:byte)))))

#230 Malformed Carriage Returns in output 5 months ago

Comment by ~andrewcrook on ~technomancy/fennel

Good news! I have managed to fix the issue. I run will tests tomorrow and read the docs to submit fix

#230 Malformed Carriage Returns in output 5 months ago

Comment by ~andrewcrook on ~technomancy/fennel

Hi I have been testing more escaping and byte conversion in fennel repl outside neovim on macOS it all seems to be working as it should apart from the fennel compiler output. I can confirm the fennel compiler works correctly in a Linux ARM VM Also tried different terminals. There are a lot more differences in the built code found using diff between Mac and Linux versions. I cannot really see the issue in the code not that I am experienced navigating and debugging large lisp projects. I tried correcting the differences but it made no difference to the complied code. If these char and byte functions work in lua and fennel in macOS I would have thought there should be no issue with the compiler but obviously its doing something differently somewhere.

#Results from macOS

Welcome to Fennel 1.4.2 on PUC Lua 5.4!
Use ,help to see available commands.


(ins)>> (local cbyte {"\r" 13})
nil
(ins)>> (print (. cbyte "\r"))
13

(ins)>> (local bchar {13 "\\r"})
nil
(ins)>> (print (. bchar 13))
\r

(ins)>> (print (..">>>>>>"  (string.char 13) "<<<<"))
<<<<>>

(ins)>> (print (..">>>>>>"  "<<<<"))
>>>>>><<<<

(print (..">>>>>>"  "\r"  "<<<<"))
<<<<>>

(ins)>> (print (string.byte "\r" ))
13

,compile  (print (..">>>>>>"  "\r"  "<<<<"))
return print((">>>>>>" .. "\13" .. "<<<<"))

#diff fennel on Mac and Fennel on linux

Fennel > diff --color fennel-copy fennel-linux
232c232
<     io.write(table.concat(xs, "\9"))
---
>     io.write(table.concat(xs, "\t"))
2212c2212
<           local tried_paths = table.concat((_3ftried_paths or {}), "\n\9")
---
>           local tried_paths = table.concat((_3ftried_paths or {}), "\n\t")
2214c2214
<             return ("\n\9" .. tried_paths)
---
>             return ("\n\t" .. tried_paths)
2408c2408
<         return assert(f:read("*all")):gsub("[\13\n]*$", "")
---
>         return assert(f:read("*all")):gsub("[\r\n]*$", "")
2579c2579
<   local serialize_subst = {["\11"] = "\\v", ["\12"] = "\\f", ["\7"] = "\\a", ["\8"] = "\\b", ["\9"] = "\\t", ["\n"] = "n"}
---
>   local serialize_subst = {["\a"] = "\\a", ["\b"] = "\\b", ["\f"] = "\\f", ["\n"] = "n", ["\t"] = "\\t", ["\v"] = "\\v"}
3520c3520
<       return string.format("\9[C]: in function '%s'", info.name)
---
>       return string.format("\t[C]: in function '%s'", info.name)
3522c3522
<       return "\9[C]: in ?"
---
>       return "\t[C]: in ?"
3541c3541
<         return string.format("\9%s:%d: in function %s", info.short_src, info.currentline, _400_())
---
>         return string.format("\t%s:%d: in function %s", info.short_src, info.currentline, _400_())
3545c3545
<         return string.format("\9%s:%d: in main chunk", info.short_src, info.currentline)
---
>         return string.format("\t%s:%d: in main chunk", info.short_src, info.currentline)
3713c3713
<       local matcher = string.gmatch((_3fsource .. "\n"), "(.-)(\13?\n)")
---
>       local matcher = string.gmatch((_3fsource .. "\n"), "(.-)(\r?\n)")
3759c3759
<       local _192_ = (error_pinpoint or {"\27[7m", "\27[0m"})
---
>       local _192_ = (error_pinpoint or {"", ""})
4129c4129
<         local formatted = raw:gsub("[\7-\13]", escape_char)
---
>         local formatted = raw:gsub("[\a-\r]", escape_char)
4502c4502
<     for _ in string.gmatch(x, "[%z\1-\127\192-\247]") do
---
>     for _ in string.gmatch(x, "[%z-\192-\247]") do
4793c4793
<     escs = setmetatable({["\""] = "\\\"", ["\11"] = "\\v", ["\12"] = "\\f", ["\13"] = "\\r", ["\7"] = "\\a", ["\8"] = "\\b", ["\9"] = "\\t", ["\\"] = "\\\\", ["\n"] = _88_}, {__index = _90_})
---
>     escs = setmetatable({["\""] = "\\\"", ["\\"] = "\\\\", ["\a"] = "\\a", ["\b"] = "\\b", ["\f"] = "\\f", ["\n"] = _88_, ["\r"] = "\\r", ["\t"] = "\\t", ["\v"] = "\\v"}, {__index = _90_})

#230 Malformed Carriage Returns in output 5 months ago

Comment by ~andrewcrook on ~technomancy/fennel

Lua 5.1  Copyright (C) 1994-2006 Lua.org, PUC-Rio
> print("hello\r\n")
hello

> print("\\r\\n")
\r\n
>
Welcome to Fennel 1.4.2 on PUC Lua 5.1!
>> ,compile (print "\r\n")
\n")rn print(“
>> ,compile (print "\\r\\n")
return print("\\r\\n")
Welcome to Fennel 1.4.2 on PUC Lua 5.4!
Use ,help to see available commands.
Try installing readline via luarocks for a better repl experience.
>> ,compile (print "\r\n")
return print("\13\n”)

#230 Malformed Carriage Returns in output 5 months ago

Comment by ~andrewcrook on ~technomancy/fennel

Thanks I have been compiling and installing lua versions and running fennel through them

  • Lua 5.1 and 5.1.5 have this but there a lot of warnings about deprecations in the macOS SDK so 5.1 has it.
  • Lua 5.2.0, 5.4.0 and 5.4.6 actually convert to ‘’\13’ like luajit but all compile warning free

Could this be fennel?

lua print works as it should via REPL etc

#29 Malformed Carriage R̲eturns in trans-compiled lua source code of the plugin 5 months ago

Comment by ~andrewcrook on ~xerool/fennel-ls

#230 Malformed Carriage Returns in output 5 months ago

Ticket created by ~andrewcrook on ~technomancy/fennel

I wonder if you could please help I am having an issue with carriage returns ’r'. I noticed this issue when installing fennel-ls and my own investigations. Wonder if this is a bug or configuration issue?

> cat test.fnl
(print "\r\n”)
> fennel --compile test.fnl > test.lua
>cat test.lua
\n")rn print(“

under neovim this displays as

return print(“^M\n”)

I have also seen it as this alternate in Neovim when using the tangerine plugin to invoke fennel

return print ("\13\n”)

#System details

  • macOS 14.5
  • Darwin Kernel Version 23.5.0:
  • I use Lua 5.1 outside Neovim because it matches LuaJIT and Neovms embedded version.
>> lua -v
Lua 5.1  Copyright (C) 1994-2006 Lua.org, PUC-Rio


> fennel --version
Fennel 1.4.2 on PUC Lua 5.1
>> luarocks list
...
fennel
   1.4.2-1 (installed)
...

>nvim --version
NVIM v0.10.0
Build type: Release
LuaJIT 2.1.1713484068
Run "nvim -V1 -v" for more info