After a lot of debugging this, I tend to think that there's probably some fundamental issue in how Emacs or Eglot on Windows communicate with programs via stdio. From debugging the output of eglot it seems that every \n that fennel-ls sends is automatically prepended by a \r. Therefore when fennel-ls writes \r\n, the output in the debug is \r\r\n, and whenI change it to \n only, the output is \r\n.

The official specification of the LSP defines that the headers should be separated by just \r\n, and the headers from the content should be separated by \r\n (thus last header is separated from the body by \r\n\r\n). I tried to replicate this, testing various combinations, and even when the output looked exactly as the described in the specification, it still was not working. Eglot seems to never understand that fennel-ls responded to it, and keeps waiting, timing out after 30 seconds. This is happening even though the debug output that I'm reading is the output taken from the Emacs/Eglot streams themselevs, so it does receive the information, just something isn't working.

I also tried including various combinatins of \r\n at the end of the answers from fennel-ls, just in case, because I read somewhere that some LSP implementations were known to require that, even though it wasn't in the spec. This didn't have any effect.

Communicating the other way, as described in #34, apparently, leaves \r out completely from the messages received by fennel-ls, so that's another strangeness. It is fixed by making fennel-ls to expect \n, and not \r\n. Sadly, I couldn't yet find the way to fix the other way around.

An interesting fact is that when I tried launching fennel-ls from the CLI by executing lua fennel-ls.lua in cmd, and manually entered the same header and content as Eglot sends to it, pressing the return button where each \r\n should be, I was able to communicate with fennel-ls reliably. There was no timeout, and no problem, the messages were flowig both directions, we chatted a bit, without any issues.

Also, I tried using fennel-ls in Emacs on Ubuntu, and it worked immediately without any issues. The configuration of Emacs is the same, and is absolutely minimal.

So at this point I'm not sure what else can I do, or if anyone can do anything at all. The issue is probably in some cross-platform specifics of some communication facility in Emacs. Might revisit this later.

So after fixing issues described in #34 and #35, I finally got to the state where eglot and fennel-ls managed to perform the initial exchange. However, at that point the next problem appears, and I'm totally not sure about this one this time.

It goes like this:

#1. fennel-ls receives from eglot, apparently in 3 different messages, as indicated in #35:

Content-Length: 2382

{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"processId":31636,"rootPath":"d:/work/repos/Fennel/","rootUri":"file:///d%3A/work/repos/Fennel","initializationOptions":{},"capabilities":{"workspace":{"applyEdit":true,"executeCommand":{"dynamicRegistration":false},"workspaceEdit":{"documentChanges":true},"didChangeWatchedFiles":{"dynamicRegistration":true},"symbol":{"dynamicRegistration":false},"configuration":true,"workspaceFolders":true},"textDocument":{"synchronization":{"dynamicRegistration":false,"willSave":true,"willSaveWaitUntil":true,"didSave":true},"completion":{"dynamicRegistration":false,"completionItem":{"snippetSupport":false,"deprecatedSupport":true,"resolveSupport":{"properties":["documentation","details","additionalTextEdits"]},"tagSupport":{"valueSet":[1]}},"contextSupport":true},"hover":{"dynamicRegistration":false,"contentFormat":["plaintext"]},"signatureHelp":{"dynamicRegistration":false,"signatureInformation":{"parameterInformation":{"labelOffsetSupport":true},"activeParameterSupport":true}},"references":{"dynamicRegistration":false},"definition":{"dynamicRegistration":false,"linkSupport":true},"declaration":{"dynamicRegistration":false,"linkSupport":true},"implementation":{"dynamicRegistration":false,"linkSupport":true},"typeDefinition":{"dynamicRegistration":false,"linkSupport":true},"documentSymbol":{"dynamicRegistration":false,"hierarchicalDocumentSymbolSupport":true,"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"documentHighlight":{"dynamicRegistration":false},"codeAction":{"dynamicRegistration":false,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}},"isPreferredSupport":true},"formatting":{"dynamicRegistration":false},"rangeFormatting":{"dynamicRegistration":false},"rename":{"dynamicRegistration":false},"inlayHint":{"dynamicRegistration":false},"publishDiagnostics":{"relatedInformation":false,"codeDescriptionSupport":false,"tagSupport":{"valueSet":[1,2]}}},"window":{"workDoneProgress":true},"general":{"positionEncodings":["utf-32","utf-8","utf-16"]},"experimental":{}},"workspaceFolders":[{"uri":"file:///d%3A/work/repos/Fennel","name":"d:/work/repos/Fennel/"}]}}

#2. fennel-ls answers to eglot:

{ ["id"] = 1,["jsonrpc"] = 2.0,["result"] = { ["capabilities"] = { ["completionProvider"] = { ["workDoneProgress"] = false,["resolveProvider"] = false,["triggerCharacters"] = { [1] = (,[2] = [,[3] = {,[4] = .,[5] = :,[6] = ",} ,["completionItem"] = { ["labelDetailsSupport"] = false,} ,} ,["positionEncoding"] = utf-8,["referencesProvider"] = { ["workDoneProgress"] = false,} ,["codeActionProvider"] = { ["workDoneProgress"] = false,} ,["definitionProvider"] = { ["workDoneProgress"] = false,} ,["hoverProvider"] = { ["workDoneProgress"] = false,} ,["renameProvider"] = { ["workDoneProgress"] = false,} ,["textDocumentSync"] = { ["openClose"] = true,["change"] = 2,} ,} ,["serverInfo"] = { ["version"] = 0.1.0,["name"] = fennel-ls,} ,} ,} 

#3. What follows then in the eglot events window:

[internal] Sat Sep 28 15:50:38 2024:
(:message "Connection state changed" :change "killed\n")
----------b---y---e---b---y---e----------

#4. Then there's a 30 seconds pause, nothing is going on. But asking Emacs about eglot servers with M-x eglot-server-menu says "no servers", so it's indeed already dead.

#5. fennel-ls main loop executes exactly 30 times, in quick succession, as if reading 30 nils. I'm able to track this because I've added io.stderr:write() with an incremental counter around the read and write calls in the loop. Writing to stderr because writing to stdout or just print()-ing doesn't produce output in the eglot window.

#6. On exactly the 30th time the read of fennel-ls starts, but this time it doesn't proceed beyond it, and this is what's printed:

[stderr] 
[stderr] nil
[stderr] nil
[stderr] Process EGLOT (Fennel/(fennel-mode)) stderr finished

I'm totally at a loss here. The eglot server seems to immediately terminate after the handhake, when the "byebye" is written. But why?

On the other hand, if returning nil should not be something tha ever happens, then why is there is an explicit nil that can be returned from https://git.sr.ht/~xerool/fennel-ls/tree/main/item/src/fennel-ls/json-rpc.fnl#L18, which causes a cascade of nill returns..

There is a condition under which json-rpc.read returns nil. As far as I've been able to understand when debugging the communication between eglot and fennel-ls, eglot initially sends 3 messages:

• Content length
• An empty string
• The json content itself.

The problem occurs with the point 2 of that list, when json-rpc.read returns nil because the msg at https://git.sr.ht/~xerool/fennel-ls/tree/main/item/src/fennel-ls.fnl#L37 gets immediately sent to dispatch.handle, and if it was nil, it breaks stuff there.

So I had to wrap it into a when, to prevent the LS from crashing internally, and for it to be able to survive on to actually get to process the json content that comes next.

At this point, I 'm not sure what's going on. Obviously, fennel-ls should not have this problem, and the problem that I described in the previous issue about cutting the content size string too short.

I think it might be some OS-specific or version-specific issue on my side, but I would have no idea since it's the first time that I'm dealing with eglot and fennel-ls.

Does this situation with the nil look off?

Holy crap, I can't even edit my post. Sorry for the typos.

I found that the upper limit of substringing at https://git.sr.ht/~xerool/fennel-ls/tree/main/item/src/fennel-ls/json-rpc.fnl#L23 cuts off one character of the content size number, so that, for example this:

Content-Length: 2382

yields number 238, and not 2382, and the subsequent communication gets ruined.

In the original code cited below the upper cut is done at -2 characters, which, as far as I understand, implies cutting of 2 characters, even though the comment says about trimming off the \r.

I have replaced this with a regexed trim, just to be on the safe side, using this instead of the original code:

v (string.sub header-line (+ sep 2))
v (string.gsub v "[ \t]+%f[\r\n%z]" "")

and it resolved the problem, so that the LS started actually seting the real message that was coming.

Not sure if it's somehow related to the environment, I'm on Windows 10, Emacs 29.4, trying out fennel-ls via M-x eglot.