Previously, the Blender video renderer did not have support for
encoding video to AV1 (not to be confused with the container AVI).
The proposed solution is to leverage the existing FFMpeg renderer
to encode to AV1.
Note that avcodec_find_encoder(AV_CODEC_ID_AV1) usually returns
"libaom-av1" which is the "reference implementation" for AV1 encoding
(the default for FFMpeg, and is slow). "libsvtav1" is faster and
preferred so there is extra handling when fetching the AV1 codec for
encoding such that "libsvtav1" is used when possible.
This commit should only affect the options available for video
rendering, which includes the additional AV1 codec to choose from, and
setting "-crf".
Also note that the current release of FFMpeg for ArchLinux does not
support "-crf" for "libsvtav1", but the equivalent option "-qp" is
supported and used as a fallback when "libsvtav1" is used (as
mentioned here: https://trac.ffmpeg.org/wiki/Encode/AV1#SVT-AV1 ).
(Actually, both "-crf" and "-qp" is specified with the same value in
the code. When a release of FFMpeg obtains support for "-crf" for
"libsvtav1" is released, the code shouldn't be needed to change.)
The usage of the AV1 codec should be very similar to the usage of the
H264 codec, but is limited to the "mp4" and "mkv" containers.
This patch pertains to the "VFX & Video" module, as its main purpose
is to supplement the Video Sequencer tool with the additional AV1
codec for encoded video output.
Differential Revision: https://developer.blender.org/D14920
Reviewed By: sergey , ISS, zeddb
Use const pointers to ImageSaveOptions and ImageFormatData for API
parameters where appropriate.
Differential Revision: https://developer.blender.org/D15400
In ffmpeg 5.0, several variables were made const to try to prevent bad API usage.
Removed some dead code that wasn't used anymore as well.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D14063
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Since 0ea0ccc4ff, `AV_PIX_FMT_YUV444P` pixel format was used for
lossless renders, which did override `AV_PIX_FMT_YUVA420P` format when
"RGBA" output is chosen. VP9 encoder doesn't seem to support
`AV_PIX_FMT_YUVA444P` pixel format, so use `AV_PIX_FMT_YUVA420P` for
lossless RGBA ouput instead.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D13947
Audio PTS was reset for each new file. This caused misalignment of video
and audio streams. In Blender, these files can't be loaded, other
players will fail to align audio and video.
Since timestamps are reset intentionally, reset also video stream
timestamps.
There were other bugs:
After timestamp was reset for audio, write_audio_frames started
encoding from timeline start until target frame, so each split video
had more audio than it should.
Also audio for last frame before splitting was written into new file.
Differential Revision: https://developer.blender.org/D13280
Before we didn't encode the audio up until the current frame.
This lead to us not encoding the last video frame of audio.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11918
We didn't flush audio after encoding finished which lead to audio
packets being lost.
In addition to this the audio timestamps were wrong because we
incremented the current audio time before using it.
Reviewed By: Richard Antalik
Differential Revision: http://developer.blender.org/D11916
This change was prompted by D6408 which moves thumbnail extraction into
a shared function that happens use these endian defines but only links
blenlib.
There is no need for these defines to be associated with globals
so move into their own header.
While encoder parameters for lossless encoding are set correctly,
output is not lossless due to pixel format being set to
`AV_PIX_FMT_YUV420P` which is inherently lossy due to chroma subsampling.
This was reported in T61569 and was merged to T57397, but there were
2 bugs - one for encoding and one for decoding.
Set pixel format to `AV_PIX_FMT_YUV444P` when rendering lossless H264
files. This format isn't available in `codec->pix_fmts[0]` and it looks,
that it has to be hard-coded.
Reviewed By: sergey
Differential Revision: D11458
The issue was two fold. We didn't properly:
1. Initialize the codec default values which would lead to VLC
complaining because of garbage/wrong codec settings.
2.Calculate the time base for the video. FFmpeg would happily accept
this but VLC seems to assume the time base value is at least somewhat
correct and couldn't properly display the frames as the internal time
base was huge. We are talking about 90k ticks (tbn) for one second of
video!
This patch initializes all codecs to use their default values and fixes
the time base calculation so it follows the guidelines from ffmpeg.
Reviewed By: Sergey, Richard Antalik
Differential Revision: http://developer.blender.org/D11426
Before the FFmpeg commit: github.com/FFmpeg/FFmpeg/commit/1c0885334dda9ee8652e60c586fa2e3674056586
FFmpeg would use deprecated variables to calculate the video fps.
We don't use these deprecated variables anymore, so ensure that the
duration is correct in ffmpeg versions without this fix.
Reviewed By: Sergey, Richard Antalik
Differential Revision: http://developer.blender.org/D11417
There need to be more cleanup for ffmpeg 4.5 (ffmpeg master branch).
However this now compiles on ffmpeg 4.4 without and deprication
warnings.
Reviewed By: Sergey, Richard Antalik
Differential Revision: http://developer.blender.org/D10338
We were not assigning the amount of sound channels to the output frames.
Newer ffmpeg releases has sanity checks in place and doesn't fall back
to two channels anymore.
Generalize threading settings in proxy building and use them for encoding
and decoding in general. Check codec capabilities, prefer FF_THREAD_FRAME
threading over FF_THREAD_SLICE and automatic thread count over setting it
explicitly.
ffmpeg-codecs man page suggests that threads option is global and used by
codecs, that supports this option. Form some tests I have done, it seems that
`av_dict_set_int(&codec_opts, "threads", BLI_system_thread_count(), 0)`
has same effect as
```
pCodecCtx->thread_count = BLI_system_thread_count();
pCodecCtx->thread_type = FF_THREAD_FRAME;
```
Looking at `ff_frame_thread_encoder_init()` code, these cases are not
equivalent. It is probably safer to leave threading setup on libavcodec than
setting up each codec threading individually.
From what I have read all over the internet, frame multithreading should be
faster than slice multithreading. Slice multithreading is mainly used for low
latency streaming.
When running Blender with --debug-ffmpeg it complains about
`pCodecCtx->thread_count = BLI_system_thread_count()` that using thread count
above 16 is not recommended. Using too many threads can negatively affect image
quality, but I am not sure if this is the case for decoding as well - see
https://streaminglearningcenter.com/blogs/ffmpeg-command-threads-how-it-affects-quality-and-performance.html
This is fine for proxies but may be undesirable for final renders.
Number of threads is limited by image size, because of size of motion vectors,
so if it is possible let libavcodec determine optimal thread count.
Performance difference:
Proxy building: None
Playback speed: 2x better on 1920x1080 sample h264 file
Scrubbing: Hard to quantify, but it's much more responsive
Rendering speed: None on 1920x1080 sample h264 file, there is improvement with codecs that do support FF_THREAD_FRAME for encoding like MPNG
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D10791
Allow use all system threads for frame encoding/decoding. This is very
straightforward: the value of zero basically disables threading.
Change threading policy to slice when decoding frames. The reason for
this is because decoding happens frame-by-frame, so inter-frame threading
policy will not bring any speedup.
The change for threading policy to slice is less obvious and is based on
benchmark of the demo files from T78986. This gives best performance so
far.
Rendering the following file went down from 190sec down to 160sec.
https://storage.googleapis.com/institute-storage/vse_simplified_example.zip
This change makes both reading and writing faster. The animation render
is just easiest to get actual time metrics.
Differential Revision: https://developer.blender.org/D8627
Field time_base of video stream must be set for some containers,
otherwise avformat_write_header() will set it to default values.
Rendered file in such case won't be played at desired frame rate.
See init_muxer() in mux.c in ffpmeg sources.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D9213
We define Lossless as CRF 0 (which is usually the best quality and is
working fine with other codecs afaict), but since WebM only allows for
CRF values between 2-32 and actually has a dedicated "lossless" mode, I
suggest using that (it produces large files though, so double-checking
would be welcome).
https://trac.ffmpeg.org/wiki/Encode/VP9#LosslessVP9
Maniphest Tasks: T74443
Differential Revision: https://developer.blender.org/D7800
FFmpeg expects Blender to feed it pixels in the output pixel format. If
the output pixel format is different than Blender's RGBA, a conversion
is needed (via FFmpeg's `sws_scale()` function). There were a few issues
with this conversion (and surrounding code) that are fixed in this
commit:
- When conversion was necessary a temporary buffer was allocated and
deallocated for every frame. This is now allocated once and re-used.
- Copying data to the buffer was done byte-for-byte. On little-endian
machines it is now done line-by-line using `memcpy` for a little speedup.
- The decision whether pixel format conversion is necessary is now
correctly done based on the pixel format Blender is actually using.
- The pixel format of the buffer sent to FFmpeg is no longer hard-coded
incorrectly to a fixed pixel format, but uses the actual output pixel
format. This is fixes T53058 properly, making RGB QTRLE export possible.
- I added some comments to make it clear which pixel format is referred
to (either Blender's internal format or the FFmpeg output format).
Combined these improvements not only correct a bug (T53058) but also
results in approximately 5% speed improvement (tested with a 117-frame
shot from Spring, loaded as PNGs in the VSE, encoding to h.264 with
preset 'realtime').
Reviewed By: brecht, sergey
Differential Revision: https://developer.blender.org/D5174
This fixes an incompatibility with Visual Studio 2019 introduced in
631d5026c7. It is likely caused by using
`# ifdef` inside the use of the `ELEM()` macro.
This commit adds support for the WebM container. Previously we only
supported the WebM/VP9 video codec, but still required that it was
stored in a Matroska, MP4, or other compatible container format.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5156
The VP9 video codec supports writing alpha values; now this is available
in Blender too.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5156
Simplified `BKE_ffmpeg_alpha_channel_is_supported()` to use `ELEM()`
instead of a row consecutive `if`-statements.
No functional changes.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5156
The root cause seems to be an assumption in
[generate_video_frame()](https://developer.blender.org/diffusion/B/browse/master/source/blender/blenkernel/intern/writeffmpeg.c)
that we're always using 4 bytes per pixel. This is not true when using
QTRLE in RGB mode, because that uses the RGB24 pixel format (so 3 bytes
per pixel). Just updating the `linesize` property doesn't fix it though,
but just creates a crash somewhere else.
This at least fixes the crash by always forcing RGBA to be written, even
when the user selects RGB.
ffmpeg defines some of the math constants if they are not
found before including any of its headers, this lead to
a build warnings about M_E, M_LN2 and M_SQRT1_2 being
redefined once BLI_math_base.h gets included.
- MPEG4/DivX has a maximum value of 65535 for the timebase denominator.
- MPEG1 and 2 have a list of supported frame rate ratios. These use
ratios like 24000/1001 and need those exact numbers.
This fixes an issue introduced in c5b1e7cd4e
where the correct ratio was passed to FFmpeg, but not with the identical
numbers FFmpeg has in a lookup table.
FFmpeg uses a fraction of integers to indicate the frame rate, whereas
Blender uses `int / float`. When a custom frame rate is used with
non-integer base, the FPS and Base settings were multiplied with 100000
before passing to FFmpeg as `int`. This could overflow when a high
enough FPS setting was used, which is the case when importing a video of
almost-but-not-quite-integer frame rate into the VSE. The overflow
caused FFmpeg to return an error "The encoder timebase is not set",
which is rather cryptic for users.
The new solution is to take the max int and divide that by the frame
rate, and use that ratio to pass to FFmpeg. This won't overflow, and
thus allows exporting arbitrary frame rates.
This also makes `IDP_CopyProperty` the "opposite"
of `IDP_FreeProperty`, which is what I'd expect.
Two refactoring steps:
* rename IDP_FreeProperty to IDP_FreePropertyContent
* new IDP_FreeProperty function that actually frees the property
Reviewers: brecht
Differential Revision: https://developer.blender.org/D4872
