Unbounded memory usage and non-termination with certain tempo maps

(Formerly Bitbucket issue 25, filed by Wes Weber)

With certain combinations of input audio + time map, the Rubber Band command-line tool fails to complete, getting into a loop in which more and more memory is allocated. This happens at the end of audio after the 99% mark is printed for the Processing phase.

Wes posted some test cases, with which the problem is still reproducible (rubberband --timemap ./time_map.txt --time 1.0 input.wav output.wav). Example stack traces when interrupted during this loop:

Thread 3 (Thread 0x7ffff6c0d640 (LWP 4466)):
# 0  RubberBand::RingBuffer<float>::write<float> (this=0x7fffc18d4a30, source=0x7ffff6c0cca4, n=1) at src/base/RingBuffer.h:470
# 1  0x00005555555609dc in RubberBand::RingBuffer<float>::resized (this=0x7fffc18d49e0, newSize=323652) at src/base/RingBuffer.h:252
# 2  0x000055555555d408 in RubberBand::RubberBandStretcher::Impl::processChunkForChannel (this=0x5555555bbfd0, c=1, phaseIncrement=165784, shiftIncrement=512, phaseReset=false) at src/StretcherProcess.cpp:524
# 3  0x000055555555ca1e in RubberBand::RubberBandStretcher::Impl::processChunks (this=0x5555555bbfd0, c=1, any=@0x7ffff6c0ce0e: true, last=@0x7ffff6c0ce0f: false) at src/StretcherProcess.cpp:322
# 4  0x000055555555be3c in RubberBand::RubberBandStretcher::Impl::ProcessThread::run (this=0x5555556a5840) at src/StretcherProcess.cpp:79
# 5  0x0000555555579459 in RubberBand::Thread::staticRun (arg=0x5555556a5840) at src/system/Thread.cpp:367
# 6  0x00007ffff7b2c3e9 in start_thread () from /usr/lib/libpthread.so.0
# 7  0x00007ffff771b293 in clone () from /usr/lib/libc.so.6

Thread 2 (Thread 0x7ffff740e640 (LWP 4465)):
# 0  0x00005555555608d3 in RubberBand::allocate<float> (count=324677) at src/system/Allocators.h:145
# 1  0x00005555555617d1 in RubberBand::RingBuffer<float>::RingBuffer (this=0x7fffcd8d4ad0, n=324676) at src/base/RingBuffer.h:204
# 2  0x0000555555560987 in RubberBand::RingBuffer<float>::resized (this=0x7fffcd8d4a80, newSize=324676) at src/base/RingBuffer.h:245
# 3  0x000055555555d408 in RubberBand::RubberBandStretcher::Impl::processChunkForChannel (this=0x5555555bbfd0, c=0, phaseIncrement=166808, shiftIncrement=512, phaseReset=false) at src/StretcherProcess.cpp:524
# 4  0x000055555555ca1e in RubberBand::RubberBandStretcher::Impl::processChunks (this=0x5555555bbfd0, c=0, any=@0x7ffff740de0e: true, last=@0x7ffff740de0f: false) at src/StretcherProcess.cpp:322
# 5  0x000055555555be3c in RubberBand::RubberBandStretcher::Impl::ProcessThread::run (this=0x55555569b0e0) at src/StretcherProcess.cpp:79
# 6  0x0000555555579459 in RubberBand::Thread::staticRun (arg=0x55555569b0e0) at src/system/Thread.cpp:367
# 7  0x00007ffff7b2c3e9 in start_thread () from /usr/lib/libpthread.so.0
# 8  0x00007ffff771b293 in clone () from /usr/lib/libc.so.6

--Type <RET> for more, q to quit, c to continue without paging--
Thread 1 (Thread 0x7ffff74f3740 (LWP 4461)):
# 0  0x00007ffff7b329c8 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
# 1  0x000055555557969a in RubberBand::Condition::wait (this=0x5555555bc0b8, us=500) at src/system/Thread.cpp:539
# 2  0x00005555555811cc in RubberBand::RubberBandStretcher::Impl::process (this=0x5555555bbfd0, input=0x5555555f0030, samples=1024, final=false) at src/StretcherImpl.cpp:1326
# 3  0x000055555555b918 in RubberBand::RubberBandStretcher::process (this=0x7fffffffde58, input=0x5555555f0030, samples=1024, final=false) at src/RubberBandStretcher.cpp:147
# 4  0x000055555558d92a in main (argc=7, argv=0x7fffffffe398) at main/main.cpp:540
Assigned to
8 months ago
8 months ago
No labels applied.

~cannam 8 months ago

This should be fixed now in commit:b5f64f41c499.

The problem resulted from the handling of very extreme ratios at the end of the timemap. In the case above, there is a timemap specified that (overall) reduces the audio duration from about 11 million to about 9 million samples - but the --time 1.0 option is also specified, and this means that Rubber Band will attempt to produce a file with unchanged duration, i.e. the original 11 million samples. So at the end of the area defined by the timemap, there is a period spanning effectively nothing at the input side, that has to somehow map to over 2 million samples at the output.

The code can deal with this, and in fact the loop it had entered was not an infinite one - given enough time and memory it would have completed the job eventually. But it required reallocating one of the internal buffers to a great duration, and unfortunately it was doing this only a few samples at a time, as it kept adding each new frame of output and finding it still hadn't got enough. It was this constant reallocation that was the problem, especially as the old buffers weren't being reaped until after the whole sequence had completed.

The obvious fix is to do as container implementations everywhere do, and reallocate to a multiple of the original size (e.g. twice the length) each time instead of just adding a bit. This reduces the overall time taken from "essentially infinite" to "essentially unnoticeable", which, while not nothing, is I think acceptable for such an extreme case.

We obviously also need to clarify the use of the --timemap and --time options together, as the documentation in the command-line tool usage output is unhelpful. The original submitter of this report had been surprised to find they needed to use --time or related arguments at all when --timemap was provided. The reason for it is that the timemap is not required (or expected) to span the whole of the file - you can provide a timemap with just one or two key points in it and they don't have to include the start or end of the file - and providing the global option as well is the simplest way of telling Rubber Band where to place the end of the audio.

~breakfastquay REPORTED FIXED 8 months ago