Audio strips crackle when animating the volume #49764

New Issue

Nik Prodanov · 2016-10-17T19:36:53+02:00

Nik Prodanov commented

2016-10-17 19:36:53 +02:00

When the Volume value of an Audio Strip is animated inside the video sequencer, you can hear "crackling" stepping artifacts no matter what the input file format is. This may be related to the audio buffer as changing the size in User Preferences will affect the character and severity of the crackle. I'm attaching an example .blend file with a sound strip that has its' volume animated by a sine-wave to make the problem more audible.

crackletest.zip

When the Volume value of an Audio Strip is animated inside the video sequencer, you can hear "crackling" stepping artifacts no matter what the input file format is. This may be related to the audio buffer as changing the size in User Preferences will affect the character and severity of the crackle. I'm attaching an example .blend file with a sound strip that has its' volume animated by a sine-wave to make the problem more audible. [crackletest.zip](https://archive.blender.org/developer/F380068/crackletest.zip)

Nik Prodanov commented

2016-10-17 19:36:53 +02:00

Changed status to: 'Open'

Nik Prodanov commented

2016-10-17 19:36:53 +02:00

Added subscriber: @NikolayProdanov

hudson barkley commented

2016-10-17 19:46:14 +02:00

Added subscriber: @snuq

hudson barkley commented

2016-10-17 19:46:14 +02:00

I noticed that when setting the buffer very high, the volume is not changed mid-buffer (it adjusts the entire buffer's volume to the keyframe value at the beginning of the buffer). My guess is the lack of interpolation between buffers is causing a sudden volume change that is causing the pops.

Olly Funkster commented

2016-10-18 10:29:24 +02:00

Added subscriber: @Funkster-3

Olly Funkster commented

2016-10-18 10:29:24 +02:00

Am I right in thinking that this is only a problem in preview playback? I've noticed it recently due to cranking the buffer size while CPU was high, but rendered output was fine.

Nik Prodanov commented

2016-10-18 11:20:42 +02:00

Have you tried doing a mixdown? I'm getting the same problem in the rendered result too. It's also not as noticeable with some sounds.

Olly Funkster commented

2016-10-18 19:06:18 +02:00

Have just tried your sample. What you are hearing is the audio level being changed with each blender frame - it does not evaluate f-curves at the audio sample rate, only at the video frame rate, so if you are changing levels that quickly you will definitely hear zipper noise.

If I get a minute (or, more likely, several hours, so don't hold your breath) I'll see if I can make it do audio strip f-curves at the sample rate, but you can bet that nobody would want the CPU time penalty.

Suspect this falls under the category of "not a bug according to the current design of Blender's audio system".

Have just tried your sample. What you are hearing is the audio level being changed with each blender frame - it does not evaluate f-curves at the audio sample rate, only at the video frame rate, so if you are changing levels that quickly you will definitely hear zipper noise. If I get a minute (or, more likely, several hours, so don't hold your breath) I'll see if I can make it do audio strip f-curves at the sample rate, but you can bet that nobody would want the CPU time penalty. Suspect this falls under the category of "not a bug according to the current design of Blender's audio system".

Nik Prodanov commented

2016-10-18 19:29:37 +02:00

This isn't really about F-Curves, i used one in my example to illustrate the issue better. I originally noticed the zipper noise when trying to do fade-ins and fade-outs for audio strips in the VSE, which is a common use case. The only way to do them is to animate the Volume value with a couple of keyframes and a bezier curve.

What about using a high sample rate only during Mixdown and Rendering? Those processes are CPU-intensive anyway, and the only cases where smooth fades really matter.

This isn't really about F-Curves, i used one in my example to illustrate the issue better. I originally noticed the zipper noise when trying to do fade-ins and fade-outs for audio strips in the VSE, which is a common use case. The only way to do them is to animate the Volume value with a couple of keyframes and a bezier curve. What about using a high sample rate only during Mixdown and Rendering? Those processes are CPU-intensive anyway, and the only cases where smooth fades really matter.

Olly Funkster commented

2016-10-18 21:15:01 +02:00

f-curves / keyframes / modifiers / all the same evaluation system, at the video frame rate.

Having had a quick squizz there's absolutely no way I'm going to try to make that whole system run at anything other than video frame rate as it uses the frame number for all its maths.

However, I may be able to do linear interpolation between those calculated values during audio mixdown only as you suggest, which won't be perfect but would be a lot less noisy than the current system.

I'll keep you posted.

f-curves / keyframes / modifiers / all the same evaluation system, at the video frame rate. Having had a quick squizz there's absolutely no way I'm going to try to make that whole system run at anything other than video frame rate as it uses the frame number for all its maths. However, I may be able to do linear interpolation between those calculated values during audio mixdown only as you suggest, which won't be perfect but would be a lot less noisy than the current system. I'll keep you posted.

Nik Prodanov commented

2016-10-18 21:23:45 +02:00

Nice, thanks :)

Aaron Carlisle commented

2016-10-18 21:27:03 +02:00

Added subscribers: @neXyon, @Blendify

Aaron Carlisle commented

2016-10-18 21:27:03 +02:00

I believe this is a known limitation. @neXyon ?

hudson barkley commented

2016-10-19 00:06:40 +02:00

Thank you for looking into this, Olly! Interpolation between frames sounds great, and I doubt anyone would notice unless they are dealing with ridiculously low framerates for some reason.

Would it be too cpu intensive to implement this interpolation for live playback just for the audio buffer chunks, not on a sample-by-sample basis? I understand if it is, just curious if that would be more reasonable.

Thank you for looking into this, Olly! Interpolation between frames sounds great, and I doubt anyone would notice unless they are dealing with ridiculously low framerates for some reason. Would it be too cpu intensive to implement this interpolation for live playback just for the audio buffer chunks, not on a sample-by-sample basis? I understand if it is, just curious if that would be more reasonable.

Olly Funkster commented

2016-10-19 12:03:50 +02:00

Proof of concept done... I had to touch more stuff than I thought, as it turns out the animation of volume isn't done where I thought it was (anyone know why there are two layers of volume animation? The m_volume.read in AUD_SequenceReader.cpp line 186 does nothing, always gives volume of 1.0!).

Here's a rendering of the test piece:
crackletest_linearinterp.flac

I'll make a paste of the diff shortly, but I've probably broken some other stuff so I won't submit it for code review yet.

Proof of concept done... I had to touch more stuff than I thought, as it turns out the animation of volume isn't done where I thought it was (anyone know why there are two layers of volume animation? The m_volume.read in AUD_SequenceReader.cpp line 186 does nothing, always gives volume of 1.0!). Here's a rendering of the test piece: [crackletest_linearinterp.flac](https://archive.blender.org/developer/F381285/crackletest_linearinterp.flac) I'll make a paste of the diff shortly, but I've probably broken some other stuff so I won't submit it for code review yet.

Joerg Mueller commented

2016-10-19 12:39:35 +02:00

I'm currently a bit unsure if this is a limitation or an actual bug @Blendify. Reading so far, the solution with the interpolation for audio buffer chunks sounds like what should be implemented right now, but I could be wrong and need to check the code better. Afair there is even spline interpolation in the code. @Funkster-3 can you just post the diff of what you did and then we can work together on a proper solution? Thanks!

Olly Funkster commented

2016-10-19 12:49:03 +02:00

Certainly, here you go:

https://developer.blender.org/P418

I freely admit that it is poorly documented and likely to break other stuff!

The interpolation for the value in between frames was already there, I have just added getting the value for the time that will be at the end of the current buffer length, and the interpolating between those values while going through that buffer.

Certainly, here you go: https://developer.blender.org/P418 I freely admit that it is poorly documented and likely to break other stuff! The interpolation for the value in between frames was already there, I have just added getting the value for the time that will be at the end of the current buffer length, and the interpolating between those values while going through that buffer.

Sergey Sharybin commented

2016-10-21 10:31:35 +02:00

Added subscriber: @Sergey

Joerg Mueller was assigned by Sergey Sharybin

2016-10-21 10:31:35 +02:00

Sergey Sharybin commented

2016-10-21 10:31:35 +02:00

@neXyon, assigning to you. You're the only one who can solve and investigate this issue anyway :)

Joerg Mueller commented

2016-10-21 16:19:34 +02:00

Thanks @Sergey, I was already on it, so here are the results of my investigation:

The crackling is caused for two different reasons in the attached test file.

The amplitude of the f-curve is above 1, the sinus goes between 0 and 2. The problem here is that for a volume > 1 you get clipping and clipping can cause crackling. So if you change the amplitude to 0.5 and the mean to 0.5 as well, you get volume to change between 0 and 1 and no clipping anymore.
The audaspace animation system and blender animation system need to be synchronized and this is unforunately not happening (fully) automatically. Open a properties window and hit the "Update Animation Cache" button under Scene -> Audio or call the operator directly using space.

After doing those two steps, the crackling should be gone, at least for me it is. I am talking to @Sergey in the chat right now to have the user use that button less, but it's difficult and a known regression. The whole existence of the button is based on the fact that the Blender animation system was never designed with audio in mind.

Please let me know if this solves it, or still get crackling.

@Funkster-3 thank you for your help, but unfortunately you just reimplemented stuff that was already there.

Thanks @Sergey, I was already on it, so here are the results of my investigation: The crackling is caused for two different reasons in the attached test file. 1. The amplitude of the f-curve is above 1, the sinus goes between 0 and 2. The problem here is that for a volume > 1 you get clipping and clipping can cause crackling. So if you change the amplitude to 0.5 and the mean to 0.5 as well, you get volume to change between 0 and 1 and no clipping anymore. 2. The audaspace animation system and blender animation system need to be synchronized and this is unforunately not happening (fully) automatically. Open a properties window and hit the "Update Animation Cache" button under Scene -> Audio or call the operator directly using space. After doing those two steps, the crackling should be gone, at least for me it is. I am talking to @Sergey in the chat right now to have the user use that button less, but it's difficult and a known regression. The whole existence of the button is based on the fact that the Blender animation system was never designed with audio in mind. Please let me know if this solves it, or still get crackling. @Funkster-3 thank you for your help, but unfortunately you just reimplemented stuff that was already there.

Olly Funkster commented

2016-10-21 16:38:33 +02:00

FWIW, there is no clipping happening in the attached test file. I noticed the volume > 1.0 thing too, but the wind sample is quiet enough that the multiplied output never reaches 0dBFS. In my rendered example the sine still reaches 2.0.

I clicked your button (in the released 2.78) and the crackling is reduced but not gone.

FWIW, there is no clipping happening in the attached test file. I noticed the volume > 1.0 thing too, but the wind sample is quiet enough that the multiplied output never reaches 0dBFS. In my rendered example the sine still reaches 2.0. I clicked your button (in the released 2.78) and the crackling is reduced but not gone.

Olly Funkster commented

2016-10-21 16:56:26 +02:00

Bit more info... having clicked the same button in my patched build, it now no longer crackles in preview playback.

So there are two sources of crackles:

wrong (out of date?) volume values being passed to the mixer in preview playback if the button has not been pressed (which doesn't affect rendered output... presumably the animation cache is rebuild first)
lack of per-audio-sample evaluation of the volume, leading to zipper noise at the frequency of the buffer length (which my patch does a plausible job of masking).

Bit more info... having clicked the same button in my patched build, it now no longer crackles in preview playback. So there are two sources of crackles: 1) wrong (out of date?) volume values being passed to the mixer in preview playback if the button has not been pressed (which doesn't affect rendered output... presumably the animation cache is rebuild first) 2) lack of per-audio-sample evaluation of the volume, leading to zipper noise at the frequency of the buffer length (which my patch does a plausible job of masking).

blender-admin commented

2016-10-22 13:38:41 +02:00

This issue was referenced by d5ee031f76

This issue was referenced by d5ee031f76137339e3af0fdadb706fbb8db4c961

Joerg Mueller commented

2016-10-22 13:42:16 +02:00

Changed status from 'Open' to: 'Resolved'

Joerg Mueller closed this issue

2016-10-22 13:42:16 +02:00

Joerg Mueller commented

2016-10-22 13:43:00 +02:00

@Funkster-3: you're right, it's not completely gone. I implemented a fix for this which is a bit simpler and less intrusive in the library. Thanks for your help again!

Olly Funkster commented

2016-10-23 00:09:24 +02:00

Heh, sneaky. Animated volume will be late by the audio buffer length, but I doubt anyone will notice unless they're using very long buffers. Might get a click at the very start of the render if the first animated value is supposed to be 0, as it'll fade down from 1.0 to 0 over the first buffer.

Joerg Mueller commented

2016-10-24 09:12:29 +02:00

Yeah, I could set the starting value to 0, then every strip gets an automatic fadein at the beginning, but not when seeking (this includes Blender jumping back to the beginning of the animation when it reaches the end), so in any case the solution won't be perfect. :-/ A nice solution that I have in mind requires quite big changes in the library, that are planned for audaspace 2, that I haven't started coding yet.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Audio strips crackle when animating the volume #49764