~eschwartz 1 year, 11 months ago

This is not a bug, but undefined behavior: https://git.sr.ht/~lattis/muon/tree/master/item/doc/differences.md#run_command-cwd

~eschwartz 1 year, 11 months ago

The general solution here to ensure that file paths are correctly handled is to pass a files() object to run_command, but in this specific case, lz4 upstream no longer has this issue anyway. No idea when 1.9.5 is scheduled for release.

~wdlkmpx 1 year, 11 months ago · edit

I doubt people write meson projects with muon in mind

This was easy to spot because it literally fails with muon, but this problem may be present in other projects causing muon to misconfigure stuff, and it's hard to tell where the issue is. So unless people start acknowledging muon's existence, it should just follow meson's behavior where possible, unless it's incredibly complex to do that

~wdlkmpx 1 year, 11 months ago · edit

Yeah the latest git revision looks ok, GetLz4LibraryVersion.py is in the root dir and is called from the main meson.build

https://github.com/lz4/lz4/blob/dev/build/meson/meson.build

Meanwhile I use this to patch the file from v1.9.4 sed -i 's%\.\./\.\./\.\.%../..%' meson/meson.build

But as I said, other projects certainly has this issue, and unless eschwartz is there to fix it, hmm the issue will remain forever..

~wdlkmpx 1 year, 11 months ago · edit

It seems that meson's approach is derived from how Makefiles work... With Makefiles every subdirectory = a new make instance with the current directory set to the Makefile dir

automake/conf creates variables like $(top_srcdir) and $(srcdir) = cwd, any command that is invoked operates in the directory where the makefile is located, full paths can be provided, i.e. $(top_srcdir), so in the end that approach makes sense because it has historical, political, religious and philosophical roots.

It might not make much sense for meson/muon since no new process is spawned?... I don't know how cmake works, but for those who have dealt with autotools stuff, it makes perfect sense.

I was looking at the code to see where to insert chdr(), but it will take days before I can understand the most simple things...

~wdlkmpx 1 year, 11 months ago · edit

muon fails if I add -Dcontrib=true

v1.9.4: err /mnt/sda3/lz4-1.9.4/contrib/meson/builddir/meson/contrib/gen_manual/gen_manual: No such file or directory

latest git

# muon setup -Dcontrib=true builddir
detected compiler gcc '5.5.0' (['cc']), linker ld.bfd
configuring 'lz4', version: 1.9.4
/mnt/sda3/gitx/lz4/build/meson/meson/ossfuzz/meson.build:22:21: error file '../../../../../ossfuzz/compress_frame_fuzzer.c' does not exist
 22 |     lz4_source_root / 'ossfuzz/@0@.c'.format(f),
                          ^
/mnt/sda3/gitx/lz4/build/meson/meson/ossfuzz/meson.build:20:9: error in function static_library()
 20 |   lib = static_library(
              ^
/mnt/sda3/gitx/lz4/build/meson/meson/meson.build:125:3: error in function subdir()
125 |   subdir('ossfuzz')
        ^
/mnt/sda3/gitx/lz4/build/meson/meson.build:31:1: error in function subdir()
 31 | subdir('meson')
      ^                          ^
        ^
/mnt/sda3/a_tarballs/lz4/build/meson/meson.build:31:1: error in function subdir()
 31 | subdir('meson')
      ^

The lz4 meson project is missing the actual tests (-Dtests=true), nothing to run or compile

Aside from the GNOME DE, meson projects are not that common yet, or meson-only projects are not that common, so far I have only tested a few projects and I found issues with muon in the more complex projects, this being the most obvious and critical issue, because if a projects runs a command with relative paths, that's the issue... The issue is exacerbated when the meson project is somewhere in a subdirectory and not in the root of the source tree

~lattis 1 year, 11 months ago

This would be very easy to fix in muon. I'd rather meson updated its documentation to state that commands are actually run from the current source directory first. As it is now, I've left it this way to be more in spirit with the documentation.

~wdlkmpx 1 year, 11 months ago · edit

Since this is in the docs, something that I didn't read, it took me a while to figure out what the issue is

Perhaps there should be a special option to compile code with discouraged meson behavior, only if it's easy to implement. I would certainly implement this for myself but this is not easy for me, I don't understand a thing. And I don't think it's a good idea for me to try to understand the whole thing since frustration can lead me to ditch muon

I'm pretty sure this is what really breaks basic meson compatibility in several projects. The higher the compatibility, the more popular a meson compat app is..

~wdlkmpx 1 year, 11 months ago · edit

So apps may have hardcoded paths. how to deal with that? https://github.com/mm2/Little-CMS/blob/master/testbed/testcms2.c#LL640

as you can see here, testcms_srcs = files(... https://github.com/mm2/Little-CMS/blob/master/testbed/meson.build

If I add more files, meson fails: ERROR: No specified compiler can handle file testbed/bad.icc. muon fails to pass the tests as usual

How to tweak test() so that the test suite passes with meson & muon knowing that meson.current_source_dir() differs unless it's the topmost meson.build (not in this case)

test(
  'testcms',
  testcms,
  workdir: meson.current_source_dir(),
  timeout: 600,
)

This doesn't seem to work

test(
  'testcms',
  testcms,
  workdir: join_paths(meson.global_build_root(), 'testbed'),
  timeout: 600,
)

err failed chdir(test): No such file or directory

So far the only solution seems to patch the .c file (a huge patch) to work with muon and muon alone

This project was ported from autotools (it's still there), so the subdirs logic will remain forevah

~eschwartz 1 year, 11 months ago

There should not be anything wrong with the portability of meson.current_source_dir() and in comparison global_build_root definitely doesn't have a testbed subdirectory.

Specifically, it works fine for me with muon.

~wdlkmpx 1 year, 11 months ago · edit

It doesn't work for me, for a test suite to work, make check, ctest, meson test, muon test must return 0... the buildsys tells you if something is wrong. meson works, muon fails

# meson setup builddirx ; samu -C builddirx ; meson test -C builddirx
# echo $?
0

# muon setup builddir ; samu -C builddir ; muon -C builddir test 
# echo $?
1

meson takes longer to finish the tests in this slow machine (1min), but muon does take some time (37secs). It partially works, it's a kind of magic

Now running the tests manually from the topmost dir, the app fails right away (3secs), doesn't seem to replicate what muon does

# builddir/testbed/testcms
File 'crayons.icc' not found
# echo $?
1

Now running the tests manually from the testbed subdir, this is what meson does (1min)

# cd testbed
# ../builddir/testbed/testcms
# echo $?
0

this actually works.

~wdlkmpx 1 year, 11 months ago · edit

So I booted a x86_64 system and ran the static "muon edge small", this actually passes the lcms2 tests, or so I think... There was a power outage so can't test again

This changes everything, perhaps my binary is miscompiled

One things is for sure, I had the same results with muon linked to musl and glibc on a i686 system, so, kernel issues probably. I'll soon find out ...

~lattis 1 year, 11 months ago

Are you correctly bootstrapping muon? (I.e. building it twice). You can refer to the readme for more information.

~wdlkmpx 1 year, 11 months ago · edit

No, my build scripts compile muon directly. There are many choices, system gcc (shared), glibc static, musl static, many cpu arch's's. I already have muon and meson installed (python3), I just replace the system muon everytime I compile a new version

I just compiled a static muon x86_64 and also passes the lcms2 tests, just like the official binary, the same results on 2 distro versions. It still fails to produce a static test_libucontext (no CFLAGS/LDFLAGS).

So this specific case (lcms2 tests) there is an issue with i686 binaries, other than that the results are the same in other projects, so it's voodoo. It doesn't make much sense with musl static i686 binaries, but there might be issues with glibc, I think it was failing to setup some projects and it worked with the musl binary, can't say for sure. Testing many binaries sometimes lead to brain damage, and one can't assess the situation properly

It's my fault of course, I try to find what's wrong with meson projects but I haven't run the muon test suite to make sure muon is compiled correctly, I'll do it from now on unless it always fails..

~eschwartz 1 year, 11 months ago*

So in the broken environment it isn't chdir'ing as instructed to with the workdir argument from meson.build?

~wdlkmpx 1 year, 11 months ago · edit

Not sure if the environment is broken but I just tested the i686 binary on a x86_64 system, this would happen anyway 'cause I always use the same static binaries no matter the distro version/cpu arch, but I'm used to test everything under a real i686 system

It's the i686 kernel that's probably triggering stuff in this specific case, what else can it be?

# muon32 -v -C builddir test -R
dbg executing /usr/bin/samu: '/usr/bin/samu' 'build.ninja'
samu: nothing to do
running tests for project 'Little-CMS'
dbg executing /mnt/sda3/build_tmp/x86_64/lcms2-2.15/builddir/testbed/testcms: 'testbed/testcms'
dbg env: MUON_PATH='muon32' MESON_BUILD_ROOT='/mnt/sda3/build_tmp/x86_64/lcms2-2.15/builddir' MESON_SOURCE_ROOT='/mnt/sda3/build_tmp/x86_64/lcms2-2.15'
.
finished 1 tests, 0 expected fail, 0 fail, 0 skipped

~wdlkmpx 1 year, 11 months ago · edit

The same muon binary on a i686 system, compiling for i686 so that the apps run...

# muon -v -C builddir test -R
dbg executing /usr/bin/samu: '/usr/bin/samu' 'build.ninja'
samu: nothing to do
running tests for project 'Little-CMS'
dbg executing /mnt/sda3/build_tmp/i686/lcms2-2.15/builddir/testbed/testcms: 'testbed/testcms'
dbg env: MUON_PATH='muon' MESON_BUILD_ROOT='/mnt/sda3/build_tmp/i686/lcms2-2.15/builddir' MESON_SOURCE_ROOT='/mnt/sda3/build_tmp/i686/lcms2-2.15'
E
finished 1 tests, 0 expected fail, 1 fail, 0 skipped

It does run the tests, I see PERFORMANCE TESTS table or something, many ...ok lines... but something fails

So if a command fails, there should be a way to fix it (files(), workdir: ...). The meson.build projects must be fixed, unless it's a system bug

This can be closed.

~lattis REPORTED CLOSED 1 year, 11 months ago