VSE: Any video with 854x480 dimensions have a glitch on its left edge in both preview and render output #94237

New Issue

Sami Salt · 2021-12-19T00:02:25+01:00

Sami Salt commented

2021-12-19 00:02:25 +01:00

System Information
Operating system: Linux-5.15.5-artix1-1-x86_64-with-glibc2.33
Graphics card: AMD Radeon RX 460 Graphics (POLARIS11, DRM 3.42.0, 5.15.5-artix1-1, LLVM 13.0.0)

Blender Version
Broken: 3.1.0 Alpha, branch: master, commit date: 2021-12-17 23:55, hash: 214a56ce8c, type: release
Worked: 2.93.7, branch: master, commit date: 2021-12-14 16:35, hash: a5b7f9dc90, type: release

Short description of error
Any 854x480 mp4 video added to the sequencer has a glitch on its left edge both in the preview and the render output. Converting the video format to mkv or avi didn't fix the problem. However, changing the dimensions of the video to either one of 1920x1080, 1280x720, 480x360, 352x240, or 1708x860 all fixed the problem. It only seems to happen in 854x480 dimensions so far.

Exact steps for others to reproduce the error
Open the project.blend file with Blender 3.0.0 or newer and you will immediately see the glitch on the left edge of the video preview. It can also be observed while playing the video. The artifact will also be visible after rendering. With a new project, just drag and drop a 480p video onto the sequencer.

system-info.txt

project_and_video.zip

project.blend

The project looks for the following video in the same folder.

glitch_480p.mp4

**System Information** Operating system: Linux-5.15.5-artix1-1-x86_64-with-glibc2.33 Graphics card: AMD Radeon RX 460 Graphics (POLARIS11, DRM 3.42.0, 5.15.5-artix1-1, LLVM 13.0.0) **Blender Version** Broken: 3.1.0 Alpha, branch: master, commit date: 2021-12-17 23:55, hash: 214a56ce8c67, type: release Worked: 2.93.7, branch: master, commit date: 2021-12-14 16:35, hash: a5b7f9dc9011, type: release **Short description of error** Any 854x480 mp4 video added to the sequencer has a glitch on its left edge both in the preview and the render output. Converting the video format to mkv or avi didn't fix the problem. However, changing the dimensions of the video to either one of 1920x1080, 1280x720, 480x360, 352x240, or 1708x860 all fixed the problem. It only seems to happen in 854x480 dimensions so far. **Exact steps for others to reproduce the error** Open the project.blend file with Blender 3.0.0 or newer and you will immediately see the glitch on the left edge of the video preview. It can also be observed while playing the video. The artifact will also be visible after rendering. With a new project, just drag and drop a 480p video onto the sequencer. ![glitch.png](https://archive.blender.org/developer/F12762171/glitch.png) [system-info.txt](https://archive.blender.org/developer/F12762184/system-info.txt) [project_and_video.zip](https://archive.blender.org/developer/F12762180/project_and_video.zip) [project.blend](https://archive.blender.org/developer/F12762181/project.blend) The project looks for the following video in the same folder. [glitch_480p.mp4](https://archive.blender.org/developer/F12762182/glitch_480p.mp4)

Sami Salt commented

2021-12-19 00:02:25 +01:00

Added subscriber: @mss1451

blender-admin commented

2021-12-19 00:02:25 +01:00

#94676 was marked as duplicate of this issue

blender-admin commented

2021-12-19 00:02:25 +01:00

#94910 was marked as duplicate of this issue

Pratik Borhade commented

2021-12-21 12:00:32 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Pratik Borhade commented

2021-12-21 12:00:36 +01:00

Added subscriber: @PratikPB2123

Richard Antalik commented

2021-12-22 04:55:03 +01:00

Added subscriber: @iss

Richard Antalik commented

2021-12-22 04:55:03 +01:00

I can see this broken between 2.93 release and 0a83f32d79 will have to bisect what caused this

I can see this broken between 2.93 release and 0a83f32d79 will have to bisect what caused this

Germano Cavalcante commented

2022-01-18 16:29:20 +01:00

Added subscribers: @wannabe.animated, @mano-wii

Michael commented

2022-01-20 16:25:16 +01:00

Added subscriber: @michael64

Michael commented

2022-01-20 16:25:16 +01:00

I have an idea what causes this. In anim_movie.c we find "Scale and flip image over Y axis in one go, using negative strides".
Unfortunately this cool scale&flip combo was never officially supported and here it breaks.
To verify this on any platform first apply D13881 then temporarily replace "# if defined(x86_64) || defined(_M_X64)" with "if 0"
to use the correct scale&manual flip.
For me this example is then fixed in that the animation looks correct.

Maybe this unsound scale&flip combo can be salvaged given certain alignment that can be checked at runtime but it was never a correct method to begin with.
Also the manual flipping can be sped up by various means this is only to show to root cause of this error.

Adding @brecht as a subscriber as well.

I have an idea what causes this. In anim_movie.c we find "Scale and flip image over Y axis in one go, using negative strides". Unfortunately this cool scale&flip combo was never officially supported and here it breaks. To verify this on any platform first apply [D13881](https://archive.blender.org/developer/D13881) then temporarily replace "# if defined(__x86_64__) || defined(_M_X64)" with "if 0" to use the correct scale&manual flip. For me this example is then fixed in that the animation looks correct. Maybe this unsound scale&flip combo can be salvaged given certain alignment that can be checked at runtime but it was never a correct method to begin with. Also the manual flipping can be sped up by various means this is only to show to root cause of this error. Adding @brecht as a subscriber as well.

Michael commented

2022-01-20 16:28:25 +01:00

Added subscriber: @brecht

Michael commented

2022-01-21 11:59:06 +01:00

I still cannot say for sure if the scale&flip combo should work anywhere and is just broken on arm64 neon but D13888 might be able to keep the performance at least for x64.

I still cannot say for sure if the scale&flip combo should work anywhere and is just broken on arm64 neon but [D13888](https://archive.blender.org/developer/D13888) might be able to keep the performance at least for x64.

Richard Antalik commented

2022-01-21 19:22:18 +01:00

@michael64 Just out of curiosity, this seems to be caused by updating ffmpeg library, so it's something that has changed in their code, is that correct? I haven't checked that. and I don't quite understand how this flipping magic works exactly.

Also in D13888 I was thinking whether it would be possible to create oversized buffer and use IMB_rect_crop? it would be bit less hacky and still should be pretty fast. Not sure if this is function I have in mind exactly though.

@michael64 Just out of curiosity, this seems to be caused by updating ffmpeg library, so it's something that has changed in their code, is that correct? I haven't checked that. and I don't quite understand how this flipping magic works exactly. Also in [D13888](https://archive.blender.org/developer/D13888) I was thinking whether it would be possible to create oversized buffer and use `IMB_rect_crop`? it would be bit less hacky and still should be pretty fast. Not sure if this is function I have in mind exactly though.

Michael commented

2022-01-21 20:56:00 +01:00

@iss, from what I could find out by reading filter_design.txt and writing_filters.txt you are supposed to treat frames delicately through APIs like av_frame_make_writable, ff_get_video_buffer and similar.
The flip logic here from vf_vflip.c was last changed in 2016 and I assume it is rock stable on any platform if and only if accompanied by the correct access methods.

The libavfilter methods do the vflip trick perfectly but to use the same method correctly requires a little effort.
To begin with at the moment libavfilter is included with the Blender deps but disabled (in ffmpeg.cmake). I enabled it to try this out and basically squashed the filtergraph setup from the ffmpeg example filtering_video.c into anim_movie.c and could use filter definitions like "vflip", "vflip,hflip" just as on the ffmpeg command line. There were other problems but this could actually work.

At the moment the general flip code is slower than it needs to be.
In the needs-alignment case for example, the image is manually flipped in place and then copied into the final buffer. So each byte is effectively written three times.
I am not sure if you'd like to ponder engaging libavfilter properly but in summary anim_movie.c tries to apply too much unsound guesswork about alignment and private data structures inside ffmpeg for very little gain.

As to your question about IMB_rect_crop when using libavfilter I think it might be possile to set up a filter chain so that ffmpeg directly writes into the buffers of ImBuf without even one manual copy. ffmpeg might already have some SIMD shifting magic that maybe gets you close maximum speed even without an oversized buffer.
Also ffmpeg already knows how to CUDA while memcpy and memmove do not. With libavfilter you get the threading for free as well I think (look up slices in the docs).

@iss, from what I could find out by reading filter_design.txt and writing_filters.txt you are supposed to treat frames delicately through APIs like av_frame_make_writable, ff_get_video_buffer and similar. The flip logic here from vf_vflip.c was last changed in 2016 and I assume it is rock stable on any platform if and only if accompanied by the correct access methods. The libavfilter methods do the vflip trick perfectly but to use the same method correctly requires a little effort. To begin with at the moment libavfilter is included with the Blender deps but disabled (in ffmpeg.cmake). I enabled it to try this out and basically squashed the filtergraph setup from the ffmpeg example filtering_video.c into anim_movie.c and could use filter definitions like "vflip", "vflip,hflip" just as on the ffmpeg command line. There were other problems but this could actually work. At the moment the general flip code is slower than it needs to be. In the needs-alignment case for example, the image is manually flipped in place and then copied into the final buffer. So each byte is effectively written three times. I am not sure if you'd like to ponder engaging libavfilter properly but in summary anim_movie.c tries to apply too much unsound guesswork about alignment and private data structures inside ffmpeg for very little gain. As to your question about IMB_rect_crop when using libavfilter I think it might be possile to set up a filter chain so that ffmpeg directly writes into the buffers of ImBuf without even one manual copy. ffmpeg might already have some SIMD shifting magic that maybe gets you close maximum speed even without an oversized buffer. Also ffmpeg already knows how to CUDA while memcpy and memmove do not. With libavfilter you get the threading for free as well I think (look up slices in the docs).

Richard Antalik commented

2022-01-21 23:11:39 +01:00

That sounds interesting, I would definitely agree to use provided filters instead of custom specialized code. If this gives performance benefits, that's even better. Will read more about filters, so far I was mostly involved only in decoding/encoding process.

I understand these patches you provided are rather experimental right? Just asking in terms of review process, even though I saw Brecht has looked at these. We can perhaps go with approach in D13881 to fix the issue sooner and explore filters after that separately. It sounds more practical too.

That sounds interesting, I would definitely agree to use provided filters instead of custom specialized code. If this gives performance benefits, that's even better. Will read more about filters, so far I was mostly involved only in decoding/encoding process. I understand these patches you provided are rather experimental right? Just asking in terms of review process, even though I saw Brecht has looked at these. We can perhaps go with approach in [D13881](https://archive.blender.org/developer/D13881) to fix the issue sooner and explore filters after that separately. It sounds more practical too.

Michael commented

2022-01-24 10:04:28 +01:00

The patch D13908 corrects this problem on all OSes. Regarding the performance impact when playing 8k videos there was no noticable difference for me at all. Real speedups would need to come from hwaccel anyway and maybe directly using YUV textures which is what ffplay is already using.

The patch [D13908](https://archive.blender.org/developer/D13908) corrects this problem on all OSes. Regarding the performance impact when playing 8k videos there was no noticable difference for me at all. Real speedups would need to come from hwaccel anyway and maybe directly using YUV textures which is what ffplay is already using.

Richard Antalik commented

2022-01-24 15:57:41 +01:00

Regarding to hwaccel, I don't think this would be easy task. For sequencer, most operations are done on CPU and using RGB buffer. I think long term, working on GPU could give much better performance though.