EEVEE: Cryptomatte #81058

New Issue

Jeroen Bakker · 2020-09-22T16:41:56+02:00

Jeroen Bakker commented

2020-09-22 16:41:56 +02:00

Research how to get Cryptomatte inside EEVEE

Cycles

This section explains the current implementation in Cycles.

Cryptomatte Layers
Cycles supports 3 different kind of cryptomatte layers:

Material
Object
Asset (in Blender this maps to a collection)

The names of the render passes are Crypto[Object/Material/Asset]{99}

Weights and Hashes

The bsdf transparency is used as weight.
For hashes the name of the object, material or asset is hashed using util_murmur_hash3 and converted to a float using util_hash_to_float.

Coverage

When using Cycles CPU the cryptomatte result is sorted and flattened coverage.cpp. The flattening makes sure that samples that points to the same hash only occupying a single slot. The sorting process sorts the items based on its weight. This could be an optimization so highest coverage are first in the list.

EEVEE

In EEVEE transparency is done as a pre-pass this updates the depth buffer to the objects we want to get the data from. This allows to output a single float per sample/layer as we don't need the weight. Every number of samples we can download the texture to CPU RAM and update the Coverage (number of occurances / number of samples). After the final sample has completed we can store the hashes with the highest coverage in the RenderResult.

Pros:

No need to do material evaluations a simple depth test would be sufficient.
Needs up to 3 floats per object+material combination
We could use gl_Layer in the geometry shader to store the result in a 3d texture
Texture depth is (number of render samples * number of layers). When render samples are large, we could download the texture to CPU and start processing it in the background. A different texture will be bound to the GPU so the rendering can continue.
Shading groups only need to be setup once and can be reused between the different samples.
Feature complete (and compatible) with Cycles.
Cons:
Not possible to support Viewport previews of the cryptomatte render pass as it needs CPU processing.
Rendering overhead as downloading the textures floods the PCI bus. This can slow down final rendering.
Blend Alpha isn't supported.

NOTE: Eventually we didn't use a 3d texture due to the OpenGL Pipeline performance penalty. It was faster to do it in a 2d texture and download the texture for every sample.

Additional

There was a request to access the asset, material and object cryptomatte hash from bpy. When implementing this project it can be one of the first patches. BKE will have a method to find the asset/object and material cryptomatte hash (BKE_cryptomatte). RNA read only properties are added to the object/material to calculate the hashes for the specific object/material.

Research how to get Cryptomatte inside EEVEE # Cycles This section explains the current implementation in Cycles. **Cryptomatte Layers** Cycles supports 3 different kind of cryptomatte layers: * Material * Object * Asset (in Blender this maps to a collection) The names of the render passes are `Crypto[Object/Material/Asset]{99}` **Weights and Hashes** The bsdf transparency is used as weight. For hashes the name of the object, material or asset is hashed using `util_murmur_hash3` and converted to a float using `util_hash_to_float`. **Coverage** When using Cycles CPU the cryptomatte result is sorted and flattened `coverage.cpp`. The flattening makes sure that samples that points to the same hash only occupying a single slot. The sorting process sorts the items based on its weight. This could be an optimization so highest coverage are first in the list. # EEVEE In EEVEE transparency is done as a pre-pass this updates the depth buffer to the objects we want to get the data from. This allows to output a single float per sample/layer as we don't need the weight. Every number of samples we can download the texture to CPU RAM and update the Coverage (`number of occurances / number of samples`). After the final sample has completed we can store the hashes with the highest coverage in the RenderResult. Pros: * No need to do material evaluations a simple depth test would be sufficient. * Needs up to 3 floats per object+material combination * We could use `gl_Layer` in the geometry shader to store the result in a 3d texture **Texture depth is (number of render samples * number of layers).** When render samples are large, we could download the texture to CPU and start processing it in the background. A different texture will be bound to the GPU so the rendering can continue. * Shading groups only need to be setup once and can be reused between the different samples. * Feature complete (and compatible) with Cycles. Cons: * Not possible to support Viewport previews of the cryptomatte render pass as it needs CPU processing. * Rendering overhead as downloading the textures floods the PCI bus. This can slow down final rendering. * Blend Alpha isn't supported. NOTE: Eventually we didn't use a 3d texture due to the OpenGL Pipeline performance penalty. It was faster to do it in a 2d texture and download the texture for every sample. # Additional There was a request to access the asset, material and object cryptomatte hash from `bpy`. When implementing this project it can be one of the first patches. BKE will have a method to find the asset/object and material cryptomatte hash (`BKE_cryptomatte`). RNA read only properties are added to the object/material to calculate the hashes for the specific object/material.

Jeroen Bakker commented

2020-09-22 16:41:56 +02:00

Added subscriber: @Jeroen-Bakker