~technomancy/fennel#231: 
Imprecise compiled numbers in LuaJIT

Fennel with LuaJIT compiles 141791343654238 into 1.4179134365424e+14 (141791343654240).

This does not happen with PUC Lua 5.3.

Welcome to Fennel 1.4.2 on LuaJIT 2.1.1693350652 Linux/x64!
Use ,help to see available commands.
>> 141791343654238
1.4179134365424e+14
>> (string.format "%0.f" 141791343654238)
"141791343654240"

Welcome to Fennel 1.4.2 on PUC Lua 5.3!
Use ,help to see available commands.
Try installing readline via luarocks for a better repl experience.
>> 141791343654238
141791343654238
>> (string.format "%0.f" 141791343654238)
"141791343654238"
Status
RESOLVED FIXED
Submitter
~adjuvant
Assigned to
No-one
Submitted
10 months ago
Updated
9 months ago
Labels
bug next-release

~adjuvant 10 months ago

See Fennel shows the same problem. In both directions. 141791343654238 => 141791343654240.0

~xerool 10 months ago

print(string.format("%0.f", 141791343654238)) Works correctly on all lua versions, so the inaccuracy is coming from fennel

~technomancy 10 months ago

The inaccuracy is coming from LuaJIT's tostring:

$ luajit
LuaJIT 2.1.0-beta3 -- Copyright (C) 2005-2022 Mike Pall. https://luajit.org/
JIT: ON SSE3 SSE4.1 BMI2 fold cse dce fwd dse narrow loop abc sink fuse
> print(tostring(141791343654238))
1.4179134365424e+14

So I guess we just need to find the right format specifier that will work across integers and floats.

~technomancy added bug 10 months ago

~technomancy added next-release 9 months ago

~technomancy REPORTED FIXED 9 months ago

I think this ought to be fixed in 08d25cb.

~technomancy referenced this from #236 9 months ago

Register here or Log in to comment, or comment via email.