Compositor: Re-write Pixelate node for CPU compositor #117223

Sergey Sharybin · 2024-01-17T12:14:17+01:00

Sergey Sharybin commented

2024-01-17 12:14:17 +01:00

The old implementation was a simple rounding operation and was not
implemented for full-frame compositor.

The issue with the old implementation is that it will not give
satisfactory results for images with high frequency details,
including cases when is used for a preview on Cycles render with
low number of samples. Additionally, when applied on animated
footage it produces very noisy result.

The new algorithm uses an explicit pixel size setting, which allows
the node to be used on its own, without need to have scale-down and
scale-up nodes. It also uses neighbour averaging, which produces
better looking result during animation and noisy input images.

The old tiled compositor setup will render without changes with
this change. This commit does not include modifications in the GPU
compositor implementation.

Ref #88150

The old implementation was a simple rounding operation and was not implemented for full-frame compositor. The issue with the old implementation is that it will not give satisfactory results for images with high frequency details, including cases when is used for a preview on Cycles render with low number of samples. Additionally, when applied on animated footage it produces very noisy result. The new algorithm uses an explicit pixel size setting, which allows the node to be used on its own, without need to have scale-down and scale-up nodes. It also uses neighbour averaging, which produces better looking result during animation and noisy input images. The old tiled compositor setup will render without changes with this change. This commit does not include modifications in the GPU compositor implementation. Ref #88150

Sergey Sharybin added 1 commit 2024-01-17 12:14:25 +01:00

dd5848b56c Compositor: Re-write Pixelate node for CPU compositor

The old implementation was a simple rounding operation and was not
implemented for full-frame compositor.

The issue with the old implementation is that it will not give
satisfactory results for images with high frequency details,
including cases when is used for a preview on Cycles render with
low number of samples. Additionally, when applied on animated
footage it produces very noisy result.

The new algorithm uses an explicit pixel size setting, which allows
the node to be used on its own, without need to have scale-down and
scale-up nodes. It also uses neighbour averaging, which produces
better looking result during animation and noisy input images.

The old tiled compositor setup will render without changes with
this change. This commit does not include modifications in the GPU
compositor implementation.

Sergey Sharybin commented

2024-01-17 12:14:56 +01:00

The .blend files and render results of the old and the new algorithms for comparison.

comp_pixelate_old.blend

956 KiB

comp_pixelate_old.mp4

925 KiB

comp_pixelate_new.blend

951 KiB

comp_pixelate_new.mp4

949 KiB

Sergey Sharybin commented

2024-01-17 12:29:31 +01:00

One thing to mention here, is that there Manuel had implementation of the node for the full-frame compositor as a patch (available in the archive). It has an interesting feature of supporting variable pixel size, and independent X and Y sizes. However, it has the same issues as the tile-based pixelate when it comes to animation and high-frequency details because it only does one input sample per output sample.

One thing to mention here, is that there Manuel had implementation of the node for the full-frame compositor as a patch (available in the [archive](https://archive.blender.org/developer/D11801)). It has an interesting feature of supporting variable pixel size, and independent X and Y sizes. However, it has the same issues as the tile-based pixelate when it comes to animation and high-frequency details because it only does one input sample per output sample.

Sergey Sharybin added this to the Compositing project 2024-01-17 12:29:56 +01:00

Sergey Sharybin requested review from Omar Emara 2024-01-17 12:31:58 +01:00

Omar Emara commented

2024-01-17 12:41:16 +01:00

I was thinking that maybe we make the pixel size a float. The integer part of the float is done similar to this patch, while the fractional part will be done with rounded sampling of the result of integer pixelation. But this is a good start for simplicity.

Performance-wise, would it make sense to do successive 2x2 average downsampling until we reach the desired pixel size. I suspect this will be better for larger pixel sizes.

I was thinking that maybe we make the pixel size a float. The integer part of the float is done similar to this patch, while the fractional part will be done with rounded sampling of the result of integer pixelation. But this is a good start for simplicity. Performance-wise, would it make sense to do successive 2x2 average downsampling until we reach the desired pixel size. I suspect this will be better for larger pixel sizes.

Sergey Sharybin added 1 commit 2024-01-17 12:43:18 +01:00

ce2fa90c7f Fix crash when Pixelate is applied on Depth pass

The node tried to be smart w.r.t the socket type.

While with the previous simple pass-through of rounded coordinate
did not require any special code for such flexibility, now it will
be more tricky to support it.

There are likely to be other sources of slowdown and memory usage,
and the node is likely to be used on a color anyway, so choose simpler
way of dealing with the data type.

Sergey Sharybin commented

2024-01-17 12:48:14 +01:00

Performance-wise, would it make sense to do successive 2x2 average downsampling until we reach the desired pixel size. I suspect this will be better for larger pixel sizes.

I am not sure how to implement pixel sizes which are not power of two with such approach. It also feels that it will make it more difficult to support variable size in the future.

> Performance-wise, would it make sense to do successive 2x2 average downsampling until we reach the desired pixel size. I suspect this will be better for larger pixel sizes. I am not sure how to implement pixel sizes which are not power of two with such approach. It also feels that it will make it more difficult to support variable size in the future.

Omar Emara approved these changes 2024-01-17 17:02:12 +01:00

Omar Emara left a comment

Just this small change, otherwise looks good.

I will do the GPU implementation and look into optimizing this a bit.

Just this small change, otherwise looks good. I will do the GPU implementation and look into optimizing this a bit.

source/blender/makesrna/intern/rna_nodetree.cc Outdated

						
				@ -8308,0 +8310,4 @@

				  PropertyRNA *prop;

				  prop = RNA_def_property(srna, "pixel_size", PROP_INT, PROP_NONE);

				  RNA_def_property_int_sdna(prop, nullptr, "custom1");