~l3kn/org-fc#43: 
org-fc/awk/utils.awk not working on windows

I type org-fc-review and select all. Instead of cards following error appears:

org-fc/awk/utils.awk:18: fatal: 3 is invalid as number of arguments for strftime

gawk -Wversion :

gawk -Wversion
GNU Awk 5.3.0, API 4.0, (GNU MPFR 4.0.2, GNU MP 6.1.2)
Copyright (C) 1989, 1991-2023 Free Software Foundation.

GNU AWK and other dependencies (find, xargs) were installed in the following way

scoop install unxutils
scoop install gawk
scoop reset gawk

scoop is modern package manager for windows — https://scoop.sh

NB: Scoop would also recommend to install uutils-coreutils instead of unmaintained unxutils, but uutils-coreutils package doesn't include findutils like find or grep yet. Maybe I will report this as issue in the scoop repo soon.

Status
REPORTED
Submitter
~artsi0m
Assigned to
No-one
Submitted
5 months ago
Updated
15 days ago
Labels
No labels applied.

~artsi0m 5 months ago

Might be permissions issue, here is errors from `Messages buffer

Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/03
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/06
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/07
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/08
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/09
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/10
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/11
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2023/12
Access denied - C:/USERS/ARTSI0M/APPDATA/ROAMING/HOWM/2024/01
File not found - -TYPE
File not found - F
File not found - -NAME
File not found - -NOT
File not found - -NAME
Access denied - .GIT
File not found - -PRINT0
gawk: c:/Users/artsi0m/AppData/Roaming/.emacs.d/src/org-fc/awk/utils.awk:18: fatal: 3 is invalid as number of arguments for strftime

~l3kn 5 months ago

Regarding the AWK issue, the documentations lists the function as having 1 to 3 arguments: https://www.gnu.org/software/gawk/manual/html_node/Time-Functions.html#index-strftime_0028_0029-function-_0028gawk_0029

I'd like to remove the date calculations in AWK at some point and pass timestamps from Emacs so the end of a day and the different time-spans can be configured but that will take some time.

The messages look very odd, like one of the commands is interpreting the flags it's getting as files. I suspect it's the find command.

You can try running (shell-command-to-string "find --version") to check if the correct find is being used.

~artsi0m 5 months ago

(shell-command-to-string "find --version") will return: : FIND: Parameter format not correct

And I guess that it is windows internal find, because: (shell-command-to-string "where find") will return:

: C:\Windows\System32\find.exe
: C:\Users\artsi0m\scoop\shims\find.exe

~l3kn 5 months ago

As a hotfix, you can try overwriting the current code with something like this:

(defun org-fc-awk--find (paths)
  (format
   "C:\Users\artsi0m\scoop\shims\find.exe -L %s -type f -name \"*.org\" -not -name \".*\" -print0"
   (mapconcat
    (lambda (path) (shell-quote-argument (expand-file-name path)))
    paths " ")))

I'm not sure how a more general solution could look like.

~artsi0m 5 months ago

I would try it soon, but just rebooted into OpenBSD for friday night.

Sorry, that I didn't figured this out myself and for the late response.

~artsi0m 5 months ago

I tried and I've got a cryptic error:

*** Read error ***  Non-hex character used for Unicode escape: s (115)

~l3kn 5 months ago

Ah, I just copied your path into the command but we'll need to use "\" instead of "" in the path. In the current form, it would expect the first "\Us" to be some code for unicode character.

Probably something like this:

(defun org-fc-awk--find (paths)
  (format
   "C:\\Users\\artsi0m\\scoop\\shims\\find.exe -L %s -type f -name \"*.org\" -not -name \".*\" -print0"
   (mapconcat
    (lambda (path) (shell-quote-argument (expand-file-name path)))
    paths " ")))

~artsi0m 5 months ago

Yeah, thanks, that changed error messages. I guess using unxutils wasn't really a good idea after all.

if: Org-fc shell error: C:\Users\artsi0m\scoop\apps\unxutils\current\usr\local\wbin\find.exe: invalid predicate `-L'
gawk: c:/Users/artsi0m/AppData/Roaming/.emacs.d/src/org-fc/awk/utils.awk:18: fatal: 3 is invalid as number of arguments for strftime

~l3kn 5 months ago

If I remember correctly, there are people using org-fc on Windows with MYSYS.

Alternatively if you're not using symlinks, you can remove the "- L" from the find command. As for gawk and srftime, if you want something that works right now, you can just remove the , 1) in the two places it's used in utils.awk and index.awk. Depending on your timezone, that will mess the review statistics in the dashboard but there should be no other effect on org-fc.

Removing strftime from the awk code would have some other benefits but I'm not sure when I'll find the time to look into that.

~l3kn 5 months ago

Where did you get your version of unxutils and when was it last updated? https://en.wikipedia.org/wiki/UnxUtils says it hasn't been updated in 10 years.

~artsi0m 5 months ago

Where did you get your version of unxutils and when was it last updated? https://en.wikipedia.org/wiki/UnxUtils says it hasn't been updated in 10 years.

From scoop, https://scoop.sh As I said it before, it showed me notification about the fact, that I use very old package.

~artsi0m 5 months ago

If I remember correctly, there are people using org-fc on Windows with MYSYS.

I may try to use emacs from wsl. I remember launching graphical session of emacs from wsl once.

~artsi0m 5 months ago*

I tried to use emacs with org-fc from wsl and it was successful. I was using OpenSUSE-Tumbleweed and following guide this two1 guides2.

~artsi0m 4 months ago*

I tried to use wsl, but failed. I am thinking about executing find and xargs as powershell commands.

What I get after applying my changed and executing org-fc-review:

if: Org-fc shell error: xargs: invalid option -- 'g'
Try 'xargs --help' for more information.
find: ‘c:/Users/artsi0m/howm/2023/02’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/03’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/06’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/07’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/08’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/09’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/10’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/11’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2023/12’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2024/01’: No such file or directory
find: ‘c:/Users/artsi0m/howm/2024/02’: No such file or directory

~artsi0m 4 months ago

I added workarounds for org-fc, some functions that redefine org-fc functions invoking find and xargs with powershell 7 commandlets.

Now I can run org-fc-dashboard on windows, but it seems to me that org-fc can't find my cards, because org-fc-dashboard displays that I have 0 cards and I get no cards due to review when I ran org-fc-review-all.

Please, can you give me some hints or some other form of help, so I can run org-fc on windows ?

~l3kn 4 months ago

If the review history shows up in the dashboard, we at least know that awk is working, maybe there's some issue with find?

The way you are rewriting org-fc-awk--find the paths argument is never used but if the path you set manually is correct, running (org-fc-awk--find '()) by itself should return some list of files.

Will org-fc-awk--pipe work with your redefined commands? It just inserts | characters between the individual arguments so there's a chance that while your overwrites work individually, they won't work when combined by these |.

~artsi0m 4 months ago*

Running (org-fc-awk--find '()) just returns command by itself. If I execute this command in terminal it would return list of all org files in howm. Pipes are present in pwsh, but they may not work exactly like in unix shell.

Sorry for the late response.

~artsi0m 4 months ago

I think I made it work. Now I can run org-fc-dashboard and it seems to me that org-fc finally can find my cards. I also reviewed all my cards two times on windows.

Here is parts my config where I changed org-fc functions: https://github.com/artsi0m/emacs-organizer/tree/da2ce20a146738aec75592e17148f695e017b7bb?tab=readme-ov-file#try-to-use-busybox-from-scoop

Sometimes error with find and short sexpr would appear in Messages, but I cannot reproduce it now and everything seems to be working.

~artsi0m 4 months ago

The error that I get in Messages sometimes:

(:path "-" :cards (
  )  :title nil)

~l3kn 4 months ago

Nice that you got it to work!

I now have access to a windows computer and once I have set up Emacs, I can try to reproduce these bugs.

Regarding the error message you're seeing, something similar seems to happen in the org-fc-cache-hashes function when it's used with directories that contain no flashcards. In your case, there are probably some of these in your org-fc-directories list.

~artsi0m 3 months ago

I now have access to a windows computer and once I have set up Emacs, I can try to reproduce these bugs.

The most easier way to setup emacs on windows is typing winget install GNU.EMacs

But there is also emacs in scoop (scoop.sh), which i used for installing busybox and other commandline apps, that can be called from terminal. It solves wingets problem with finding program in PATH.

Regarding the error message you're seeing, something similar seems to happen in the org-fc-cache-hashes function when it's used with directories that contain no flashcards. In your case, there are probably some of these in your org-fc-directories list.

Probably, there are some. But I can review cards without this error, when executing org-fc-review in *Messages* buffer instead of opened org buffer with flashcards.

~artsi0m 3 months ago

I am sorry. I subscribed to org mode mailing list and have been told there that by removing escaping I can get vulnerability. Also problems with shell-quote-argument highly likely have been caused by myself setting shell-file-name to pwsh.exe I will remove this setting from my config and go back to having shell-quote-argument in org-fc functions.

~artsi0m 15 days ago

Suddenly it's all stopped working again. Either No cards due to review or *.tsv file not found

Register here or Log in to comment, or comment via email.