VSE: Delay conversion of color space until it is needed #119501

New Issue

Sergey Sharybin · 2024-03-15T10:58:56+01:00

Sergey Sharybin commented

2024-03-15 10:58:56 +01:00

The design of VSE is such that it performs color operations, effects, and such in a special color space, denoted by Sequencer color space in the scene's Color Management settings. This gives more natural control for color graders.

The way it is currently implemented in the sequencer is that all image buffers which go through the VSE are converted to the sequencer color space, even if no color grading, or cross-fade is used. It should be possible to do the conversion on-demand, when an image is accessed for modifier stack, or effect, or requires other color related processing.

From a quick test of rendering 250 Full-HD EXR frames into a movie file avoiding the extra color space conversion gives speedup of about ~23% (rendering such sequence went down from 9 to 7 seconds on M2 Ultra). tested by grabbing the Old Factory footage, and converting it to a sequence of EXR images (not sure it is crucial to have exactly this sequence, is just something I have on my drive for all sorts of tests).

It is not uncommon during production to have an edit which has opaque EXR images which then assembled into a version of final movie, relying on the View Transform to give decent look to the film. Having 23% speedup will help speeding up iteration over the movie edits.

The design of VSE is such that it performs color operations, effects, and such in a special color space, denoted by Sequencer color space in the scene's Color Management settings. This gives more natural control for color graders. The way it is currently implemented in the sequencer is that all image buffers which go through the VSE are converted to the sequencer color space, even if no color grading, or cross-fade is used. It should be possible to do the conversion on-demand, when an image is accessed for modifier stack, or effect, or requires other color related processing. From a quick test of rendering 250 Full-HD EXR frames into a movie file avoiding the extra color space conversion gives speedup of about ~23% (rendering such sequence went down from 9 to 7 seconds on M2 Ultra). tested by grabbing the [Old Factory](https://studio.blender.org/training/track-match-blend/56040f9b044a2a00ad6c660d/) footage, and converting it to a sequence of EXR images (not sure it is crucial to have exactly this sequence, is just something I have on my drive for all sorts of tests). It is not uncommon during production to have an edit which has opaque EXR images which then assembled into a version of final movie, relying on the View Transform to give decent look to the film. Having 23% speedup will help speeding up iteration over the movie edits.

Sergey Sharybin added the

Type

To Do

label 2024-03-15 10:58:56 +01:00

Sergey Sharybin added this to the Video Sequencer project 2024-03-15 10:58:58 +01:00

Aras Pranckevicius commented

2024-03-15 12:35:30 +01:00

Just so that I understand what is going on:

Is is that today this would happen:
0. exr files are stored in linear space,

sequencer loads them, converts from float linear -> float srgb (assuming vse colorspace is default "srgb") space,
nothing interesting is done with the image, it is "just" displayed in a single image strip
for rendering the final result out, float srgb image is first converted back to linear, and then converted into byte srgb.

And your proposed savings would be like:

sequencer loads exr file, keeps it in float linear,
for rendering the final result out, it already has float linear, and converts into byte srgb.

Just so that I understand what is going on: Is is that today this would happen: 0. exr files are stored in linear space, 1. sequencer loads them, converts from float linear -> float srgb (assuming vse colorspace is default "srgb") space, 2. nothing interesting is done with the image, it is "just" displayed in a single image strip 3. for rendering the final result out, float srgb image is first converted back to linear, and then converted into byte srgb. And your proposed savings would be like: 1. sequencer loads exr file, keeps it in float linear, 2. for rendering the final result out, it already has float linear, and converts into byte srgb.

Sergey Sharybin commented

2024-03-15 14:26:00 +01:00

@aras_p Roughly, yes. At least I think it is a very good summary of the higher-level overview of what's going on. I should probably have left some pointers in the code which could be interesting:

SEQ_render_imbuf_from_sequencer_space
seq_imbuf_to_sequencer_space
seq_imbuf_assign_spaces

Making those no-op is what gave speedup in the example i've shown in the description.

@aras_p Roughly, yes. At least I think it is a very good summary of the higher-level overview of what's going on. I should probably have left some pointers in the code which could be interesting: - `SEQ_render_imbuf_from_sequencer_space` - `seq_imbuf_to_sequencer_space` - `seq_imbuf_assign_spaces` Making those no-op is what gave speedup in the example i've shown in the description.

Richard Antalik commented

2024-03-15 15:20:25 +01:00

I think this would make sense. Personally, I would implement this by moving responsibility of colorspace conversion to place, where some processing is being done. Most functions that try to do introspection whether something will be done with strip are not very nice to read.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

VSE: Delay conversion of color space until it is needed #119501