#1 Performance test updates 5 hours ago

Comment by ~cannam on ~cannam/bisquay

Added a report of memory (max RSS) as well as elapsed time. An accidental result of this is that the tests that dump significant amounts to stdout now go faster in general, because of the way stdout is processed - this is closer to the way it always was on the Mac. So I guess we have a new timing baseline too.

40022c756a06 tip
Commit date: Tue Oct 19 08:33:40 2021 +0100

                        polyml          mlton_noffi     mlton_release   mem/polyml      mem/release
   waveform             0:00.57         0:00.47         0:00.36         529M            131M
   waveform-max         0:04.13         0:03.27         0:02.71         542M            247M
   cqt-dump             0:10.11         0:08.47         0:03.35         1017M           214M
   spec-dump            0:01.68         0:02.25         0:01.15         505M            122M
   spec-max             0:05.73         0:03.08         0:02.69         1093M           695M
   reassigning          1:42.87         1:02.41         0:28.87         1256M           1106M

I was curious what difference the switch to StrongestFrequencyComponentBlockStreamFn had made (i.e. storing only 80 components per column in the random-access instead of all of them). So I went back to cbb782f98eb3 to check memory usage in the reassigning test:

   reassigning          2:53.31         1:11.09         0:37.37         9152M           7950M

Reporting memory usage for the Poly/ML runs seems to be a tricky thing to do, as it's possible to some extent to trade off memory against runtime. The examples above all used the --minheap 500M parameter - Poly/ML will sometimes abort in early allocation if you don't specify a reasonably chunky minimum heap size - but we can use less memory in most cases with a lower value. For example, 40022c756a06 again with --minheap 200M:

                        polyml          mlton_noffi     mlton_release   mem/polyml      mem/release
   waveform             0:00.64         0:00.47         0:00.37         241M            131M
   waveform-max         0:04.47         0:03.31         0:02.70         439M            247M
   cqt-dump             0:10.07         0:08.59         0:03.38         407M            214M
   spec-dump            0:01.63         0:02.27         0:01.20         202M            122M
   spec-max             0:06.59         0:03.18         0:02.80         921M            695M
   reassigning          1:42.02         1:02.96         0:29.23         1568M           1106M

or with --minheap 100M

                        polyml          mlton_noffi     mlton_release   mem/polyml      mem/release
   waveform             0:00.82         0:00.48         0:00.37         182M            131M
   waveform-max         0:04.62         0:03.32         0:02.76         384M            247M
   cqt-dump             0:10.71         0:08.48         0:03.42         487M            214M
   spec-dump            0:01.62         0:02.29         0:01.17         155M            122M
   spec-max             0:06.78         0:03.19         0:02.75         935M            695M
   reassigning          1:42.06         1:02.90         0:29.23         1162M           1106M

I think I will leave in --minheap 200M, but let's consider the Poly/ML results of interest more to spot really outlandish cases than anything else.

#1 Performance test updates 20 hours ago

Comment by ~cannam on ~cannam/bisquay

After switching to ordering-derived summarisation logic for the reassigning spectrogram:

64eb5e69f5da tip
Commit date: Mon Oct 18 16:53:03 2021 +0100

                        polyml          mlton_noffi     mlton_release
   waveform             0:00.54         0:00.47         0:00.37
   waveform-max         0:04.31         0:03.34         0:02.76
   cqt-dump             0:10.16         0:08.44         0:03.37
   spec-dump            0:01.65         0:02.25         0:01.16
   spec-max             0:05.84         0:03.11         0:02.68
   reassigning          1:40.57         1:02.39         0:28.90

This isn't a performance fix (evidently!) - it's just the necessary part to make StrongestFrequencyComponentBlockStreamFn useful.

#1 Performance test updates 23 hours ago

Comment by ~cannam on ~cannam/bisquay

With the simple-sv-alike build, MLton currently throws an exception in the deepFlatten pass. How useful is that pass? Here's a build with deepFlatten disabled.

541677cd0f6e+ tip
Commit date: Mon Oct 18 13:47:25 2021 +0100

                        polyml          mlton_noffi     mlton_release
   waveform             0:00.59         0:00.52         0:00.36
   waveform-max         0:04.26         0:03.38         0:02.79
   cqt-dump             0:10.32         0:08.51         0:03.54
   spec-dump            0:01.73         0:02.34         0:01.22
   spec-max             0:06.03         0:02.73         0:02.58
   reassigning          1:38.86         1:02.83         0:28.66

So deepFlatten isn't important at all, which is nice to know. I'll leave it enabled in the Bisquay builds but if I can't get a good test case to report the problem in simple-sv-alike I may just disable it there.

#1 Performance test updates 23 hours ago

Comment by ~cannam on ~cannam/bisquay

Switching to using StrongestFrequencyComponentBlockStreamFn in the reassignment test (with n = 80). No further optimisations yet.

541677cd0f6e tip
Commit date: Mon Oct 18 13:47:25 2021 +0100

                        polyml          mlton_noffi     mlton_release
   waveform             0:00.59         0:00.49         0:00.38
   waveform-max         0:04.21         0:03.35         0:02.79
   cqt-dump             0:10.10         0:08.36         0:03.35
   spec-dump            0:01.67         0:02.23         0:01.34
   spec-max             0:06.05         0:02.74         0:02.71
   reassigning          1:39.95         1:03.11         0:28.94

#1 Performance test updates 23 hours ago

Comment by ~cannam on ~cannam/bisquay

Checking cbb782f98eb3 again after an update to MLton, from 2020-something to 20210831:

cbb782f98eb3
Commit date: Fri Oct 15 16:42:17 2021 +0100

                        polyml          mlton_noffi     mlton_release
   waveform             0:00.59         0:00.47         0:00.36
   waveform-max         0:04.21         0:03.32         0:02.75
   cqt-dump             0:10.05         0:08.35         0:03.30
   spec-dump            0:01.67         0:02.18         0:01.15
   spec-max             0:05.90         0:02.80         0:02.73
   reassigning          2:55.16         1:10.63         0:36.66

Everything is faster - including Poly/ML, even though that compiler is unchanged. Either routinely updating Linux (to 5.14.12-arch1-1) has made a difference, or it's just that the laptop is not as hot.

#1 Performance test updates 3 days ago

Comment by ~cannam on ~cannam/bisquay

Switching the frequency component record to contain a RealTime.t instead of RealTime.t option:

   reassigning          2:44.06         1:13.60         0:38.99

Does not improve release builds at all - but I quite like the change anyway as (now that we store RealTime rather than offset) we do at least always have something to put there.

Also commenting out some spurious debug output in reassign - the output is never made but there are two decisions per cell about whether to make it or not:

   reassigning          2:45.61         1:13.75         0:36.80

Well, it's reassuring that unprinted debug output is not expensive anyway.

#1 Performance test updates 3 days ago

Comment by ~cannam on ~cannam/bisquay

Much later on in frequency component work, after adding a reassigning test to the perftest program:

cbb782f98eb3 tip
Commit date: Fri Oct 15 16:42:17 2021 +0100

                        polyml          mlton_noffi     mlton_release
   waveform             0:00.59         0:00.50         0:00.38
   waveform-max         0:04.39         0:03.57         0:02.84
   cqt-dump             0:10.83         0:09.09         0:03.56
   spec-dump            0:01.78         0:02.37         0:01.20
   spec-max             0:06.24         0:03.26         0:02.89
   reassigning          2:58.72         1:14.94         0:38.36

All of the tests that use peak-towers have got a bit slower recently - possibly as a result of functorising out the matrix type for random access and peak tower.

The reassigning test uses a fairly arbitrary set of parameters and lacks almost all forms of optimisation. It calculates spectrograms for the same sequence of regions as spec-max. I don't think it can ever be as fast as spec-max, because the reassigning spectrogram needs far more overlap and has to do at least twice as many ffts, but there is room to get much closer than it is now.

#1 Doesn't do the right thing when remote access fails 14 days ago

Comment by ~cannam on ~cannam/repoint

Fixed in commit:6085f4a60296 and release 1.3.

REPORTED RESOLVED FIXED

#1 Doesn't do the right thing when remote access fails a month ago

Ticket created by ~cannam on ~cannam/repoint

A remote access failure during update (authentication failure, certificate failure, host down, etc) doesn't prevent Repoint from querying and rewriting the lock file based on the status of the local repo at the end of the update procedure.

This has two nasty consequences:

• If the repo hasn't been created locally at all yet, then the revision id written into the lock file will be that of the parent repo (if it uses the same version control system) or an invalid id - the result is super-confusing
• If the repo exists locally but was out of date, the lock file will be rewritten with the outdated revision id, producing wrong results for subsequent install/status operations and causing potential merge conflict/confusion

Both are obviously wrong - review.

#1 Performance test updates 2 months ago

Comment by ~cannam on ~cannam/bisquay

While working on frequency component stuff:

dbdd4ecb9833 tip
Commit date: Thu Aug 12 15:38:20 2021 +0100

                        polyml          mlton_noffi     mlton_release
   waveform             0:00.59         0:00.47         0:00.36
   waveform-max         0:04.24         0:03.44         0:02.74
   cqt-dump             0:10.76         0:08.88         0:03.42
   spec-dump            0:01.78         0:02.38         0:01.22
   spec-max             0:06.15         0:03.15         0:02.61

When did spec-max become so much faster? I'd better investigate.

(Investigates)

OK, it's rev 11d5a8ca2e13, which I'm afraid is just summarised as "Subrepo update". It updated bsq-matrix from 4dd32ca26b97 to 2c6f154345fc ("Docs"), bsq-randomaccess from fe12ddcf74d4 to d7c27deb3c08 ("Max is more sensible I think"), and bsq-cqt from cd8af54fa8ec to 6a4683bbcadb ("Add fill option to (real) CQ spectrogram").

And, indeed, reverting the change in bsq-randomaccess:

-                                          summary = Summarise.RMS,
+                                          summary = Summarise.MAX,

restores the previous slower runtime. So there we are. But MAX is definitely the better behaviour, so I guess this becomes the new baseline.