Realtime Compositor: Implement Vector Blur node #116977

Merged

Omar Emara merged 8 commits from OmarEmaraDev/blender:compositor-motion-blur into main

2024-01-12 12:12:11 +01:00

Omar Emara commented

2024-01-10 11:02:20 +01:00

Member

This patch implements the Vector Blur node for the Realtime Compositor.
The implementation is a direct and mostly identical port of the EEVEE
motion blur implementation with the necessary adjustments to make it
work with the compositor.

The exposed parameters in the node does not match those exposed in
EEVEE, so only the parameters shared between both are currently
implemented. In the future, we should make a decision to either unify
both, or just consider them independent implementations, with the
possibility of sharing the full or part of the code.

Further, it would also make sense to port the implementation to the CPU
compositor, since the new implementation is higher in quality while also
being faster.

This patch implements the Vector Blur node for the Realtime Compositor. The implementation is a direct and mostly identical port of the EEVEE motion blur implementation with the necessary adjustments to make it work with the compositor. The exposed parameters in the node does not match those exposed in EEVEE, so only the parameters shared between both are currently implemented. In the future, we should make a decision to either unify both, or just consider them independent implementations, with the possibility of sharing the full or part of the code. Further, it would also make sense to port the implementation to the CPU compositor, since the new implementation is higher in quality while also being faster.

Omar Emara added the

labels 2024-01-10 11:02:20 +01:00

Omar Emara added 4 commits 2024-01-10 11:02:30 +01:00

aed86fc218 Realtime Compositor: Implement Vector Blur node

0788304eae Port EEVEE implementation

8f7c6b6c3e Port max tile velocity dilation

18826301bb Document dilation step

Omar Emara requested review from Sergey Sharybin 2024-01-10 11:02:55 +01:00

Omar Emara commented

2024-01-10 11:09:14 +01:00

Author

Member

Ground truth (Cycles):

GPU Compositor (This patch):

CPU Compositor:

Ground truth (Cycles): ![20240110-120617.png](/attachments/88b9492b-14ec-45aa-b696-3b15fa4b6264) GPU Compositor (This patch): ![20240110-120522.png](/attachments/77d098a8-0cb5-4323-81aa-2ad9f7e0ce02) CPU Compositor: ![20240110-120511.png](/attachments/de915817-3b28-4120-9855-b82186844232)

20240110-120617.png

503 KiB

20240110-120511.png

519 KiB

20240110-120522.png

501 KiB

Omar Emara changed title from ~~Realtime Compositor: Implement Motion Blur node~~ to Realtime Compositor: Implement Vector Blur node

2024-01-10 13:03:57 +01:00

Sergey Sharybin reviewed 2024-01-10 17:01:22 +01:00

Sergey Sharybin left a comment

Owner

On a code side it all looks fine, and in theory it does sound very appealing to align implementation with Eevee.

In practice I have a bigger struggle trying to figure out which of the algorithms (the proposed GPU compositor vs. the existing CPU compositor) gives better results. I've attached demo file.

With the old CPU implementation there are those vertical artifacts around node, eye, and ear, which do not go away with the increased number of samples.

With the Eevee based implementation there seems to be some sort of "leaking" of a brighter checkerboard tile to the top of the head, and the bottom of the ear, which do not seem fully convincing from a gold-standard of motion blur point of view.

I am curious what Clément's suggestion here will be. @fclem are people generally satisfied with the Eevee motion blur and it is fine for us to follow its implementation?

@OmarEmaraDev Another thing I've noticed is that "scale" of blur is different: if i render motion blur is Eevee without compositor setup, and compare it to the GPU compositor with vector blur which uses vectors provided by Eevee the results are quite different. Not sure if that is expected?

On a code side it all looks fine, and in theory it does sound very appealing to align implementation with Eevee. In practice I have a bigger struggle trying to figure out which of the algorithms (the proposed GPU compositor vs. the existing CPU compositor) gives better results. I've attached demo file. With the old CPU implementation there are those vertical artifacts around node, eye, and ear, which do not go away with the increased number of samples. With the Eevee based implementation there seems to be some sort of "leaking" of a brighter checkerboard tile to the top of the head, and the bottom of the ear, which do not seem fully convincing from a gold-standard of motion blur point of view. I am curious what Clément's suggestion here will be. @fclem are people generally satisfied with the Eevee motion blur and it is fine for us to follow its implementation? @OmarEmaraDev Another thing I've noticed is that "scale" of blur is different: if i render motion blur is Eevee without compositor setup, and compare it to the GPU compositor with vector blur which uses vectors provided by Eevee the results are quite different. Not sure if that is expected?

vector_blur.blend

947 KiB

source/blender/compositor/realtime_compositor/shaders/compositor_motion_blur.glsl Outdated

						
				@ -0,0 +1,203 @@

				/* SPDX-FileCopyrightText: 2023 Blender Authors

Sergey Sharybin commented

2024-01-10 16:27:14 +01:00

Owner

It is 2024 :)

It is 2024 :)

source/blender/compositor/realtime_compositor/shaders/compositor_motion_blur.glsl Outdated

						
				@ -0,0 +31,4 @@

				vec2 depth_compare(float center_depth, float sample_depth)

				{

				  vec2 depth_scale = vec2(depth_scale, -depth_scale);

Sergey Sharybin commented

2024-01-10 15:43:43 +01:00

Owner

This is not correct C, and in metal it gives an error since this is an equivalent of

vec2 depth_scale;
depth_scale = vec2(depth_scale, -depth_scale);

I guess the global constant is to be defined as g_depth_scale, and the expression here to become vec2 depth_scale = vec2(g_depth_scale, -g_depth_scale);

This is not correct C, and in metal it gives an error since this is an equivalent of ``` vec2 depth_scale; depth_scale = vec2(depth_scale, -depth_scale); ``` I guess the global constant is to be defined as `g_depth_scale`, and the expression here to become `vec2 depth_scale = vec2(g_depth_scale, -g_depth_scale);`

Clément Foucault commented

2024-01-10 22:35:40 +01:00

Member

@Sergey I don't really know what people think of it. I only know about its limitation.

The main issue is the amount of noise in a tile (tiles being an internal speedup structure of the algorithm) where there is conflicting motion.
The other issue is the background separation is a bit tricky to use but it provides good smearing of foreground over background.
And finally, the tile structure can become visible with very huge motion values.

Here is the reference paper with comparisons of worse case scenarios:
https://diglib.eg.org/handle/10.2312/hpg.20141093.051-060

@Sergey I don't really know what people think of it. I only know about its limitation. The main issue is the amount of noise in a tile (tiles being an internal speedup structure of the algorithm) where there is conflicting motion. The other issue is the background separation is a bit tricky to use but it provides good smearing of foreground over background. And finally, the tile structure can become visible with very huge motion values. Here is the reference paper with comparisons of worse case scenarios: https://diglib.eg.org/handle/10.2312/hpg.20141093.051-060

Omar Emara added 2 commits 2024-01-11 09:46:51 +01:00

7d2c94cde3 Merge branch 'main' into compositor-motion-blur

81245f5eef Address review

Omar Emara commented

2024-01-11 10:10:57 +01:00

Author

Member

@Sergey I am looking into the blur scale. But just to make sure you know, the vector pass is not affected by the shutter speed specified in the motion blur settings, so you need to match them. It defaults to 0.5, so I thought you might have left it as such when testing.

@Sergey I am looking into the blur scale. But just to make sure you know, the vector pass is not affected by the shutter speed specified in the motion blur settings, so you need to match them. It defaults to 0.5, so I thought you might have left it as such when testing.

Sergey Sharybin commented

2024-01-11 10:29:37 +01:00

Owner

Yeah, probably should have gone in more details. I wasn't sure which frames Eevee uses for the vector pass calculation, and couldn't have matched results at any intuitive shutter speed, so left it at default value.

If the previous and next frame are used for the motion pass, then shutter speed should be 2, but then Eevee's render time blur is stronger. The shutter curve is left at constant 1. The closest results i got were at shutter of 0.68.

Yeah, probably should have gone in more details. I wasn't sure which frames Eevee uses for the vector pass calculation, and couldn't have matched results at any intuitive shutter speed, so left it at default value. If the previous and next frame are used for the motion pass, then shutter speed should be 2, but then Eevee's render time blur is stronger. The shutter curve is left at constant 1. The closest results i got were at shutter of 0.68.

Omar Emara added 1 commit 2024-01-11 16:17:03 +01:00

038873383e Use shutter speed without adjustments but change its default

Omar Emara commented

2024-01-11 16:27:42 +01:00

Author

Member

@Sergey After a long debugging session, it turned out that it is not a scale issue after all.

If motion blur is disabled, EEVEE evaluates the scene at frame - 1 and frame + 1, while if motion blur is enabled, EEVEE evaluates the scene according to the motion blur options, so assuming a "Center On Frame" position and a shutter of 1, EEVEE will evaluate the scene at frame - 0.5 and frame + 0.5, most likely producing completely different velocity vectors, which is the case in our test scene.

So it is impossible to replicate the EEVEE result in most cases since the velocity vectors themselves will be different, and the user can try adjusting the node shutter to try to match the render, hence my last commit.

@Sergey After a long debugging session, it turned out that it is not a scale issue after all. If motion blur is disabled, EEVEE evaluates the scene at `frame - 1` and `frame + 1`, while if motion blur is enabled, EEVEE evaluates the scene according to the motion blur options, so assuming a "Center On Frame" position and a shutter of 1, EEVEE will evaluate the scene at `frame - 0.5` and `frame + 0.5`, most likely producing completely different velocity vectors, which is the case in our test scene. So it is impossible to replicate the EEVEE result in most cases since the velocity vectors themselves will be different, and the user can try adjusting the node shutter to try to match the render, hence my last commit.

Sergey Sharybin commented

2024-01-11 16:48:22 +01:00

Owner

Ah, it escaped my mind that the Blur setting in the node defines shutter as well. So tested things again.

The default of 0.5 is probably fine, but the scaling node_storage(bnode()).fac / 2.0f actually seems required. If that part is restored, then Eevee render time motion blur with shutter set to 2 seems to match the compositor result with Blur set to 2.

So i propose keep Blur default at 0.5, but restore the float get_shutter_speed().

Ah, it escaped my mind that the Blur setting in the node defines shutter as well. So tested things again. The default of 0.5 is probably fine, but the scaling `node_storage(bnode()).fac / 2.0f` actually seems required. If that part is restored, then Eevee render time motion blur with shutter set to 2 seems to match the compositor result with Blur set to 2. So i propose keep Blur default at 0.5, but restore the `float get_shutter_speed()`.

vector_blur.blend

946 KiB

Omar Emara added 1 commit 2024-01-12 09:23:36 +01:00

3f65cf1512 Change default shutter to 0.25

Omar Emara commented

2024-01-12 09:30:39 +01:00

Author

Member

@Sergey My other proposal is to change the default to 0.25 without any internal scaling. This was chosen as the closest possible value to the default settings in the motion blur settings. The default is Center On Frame with shutter set to 0.5, which means frames will be evaluated at +-0.25. For simple linear animations, the motion blur will match exactly, while for non linear animations, it will approximate the result, and the user can change the shutter further to adjust the motion blur amount.

@Sergey My other proposal is to change the default to 0.25 without any internal scaling. This was chosen as the closest possible value to the default settings in the motion blur settings. The default is Center On Frame with shutter set to 0.5, which means frames will be evaluated at +-0.25. For simple linear animations, the motion blur will match exactly, while for non linear animations, it will approximate the result, and the user can change the shutter further to adjust the motion blur amount.

Sergey Sharybin approved these changes 2024-01-12 10:21:48 +01:00

Sergey Sharybin left a comment

Owner

I guess any decision could be seen as somewhat arbitrary.
Matching default Eevee motion blur without extra internal scaling does sound good to me. Lets just move on :)

I guess any decision could be seen as somewhat arbitrary. Matching default Eevee motion blur without extra internal scaling does sound good to me. Lets just move on :)

Omar Emara merged commit e84dc990b1 into main

2024-01-12 12:12:11 +01:00

Omar Emara referenced this issue from a commit

2024-01-12 12:12:12 +01:00

Realtime Compositor: Implement Vector Blur node

Omar Emara deleted branch compositor-motion-blur

2024-01-12 12:12:13 +01:00

Kazashi Yoshioka referenced this issue from a commit

2024-01-12 19:10:54 +01:00

Realtime Compositor: Implement Vector Blur node

Arye Ramaty referenced this issue from a commit

2024-01-21 15:37:40 +01:00

Realtime Compositor: Implement Vector Blur node

Sign in to join this conversation.

No reviewers

No Label

Animation & Rigging

Asset Browser Project Overview

Automated Testing

Blender Asset Bundle

Dependency Graph

Development Management

EEVEE & Viewport

Images & Movies

Motion Tracking

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Render Pipeline

Sculpt, Paint & Texture

Video Sequencer

Virtual Reality

Blender 2.8 Project

Milestone 1: Basic, Local Asset Browser

Good First Issue

Animation & Rigging

Development Management

EEVEE & Viewport

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Sculpt, Paint & Texture

Needs Info from Developers

Needs Information from User

No Milestone

No project

No Assignees

3 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#116977

No description provided.