WIP: Cycles: Importance Resampling #121023

Weizhen Huang · 2024-04-24T15:19:10+02:00

Weizhen Huang commented

2024-04-24 15:19:10 +02:00

Implement Reservoir-based Spatial and Temporal Importance Resampling, aka ReSTIR.
This is heavily WIP and inefficient, currently only implemented initial and spatial resampling for direct illumination, works on CPU with surface lit by area light, sphere light and mesh light.

Direct Light Only	Before (5 or 4 spp)	After (1 spp)
7.4 s, with light tree
~4.5s, no light tree

TODO:

Multiple spatial sampling iterations
Improve spatial sampling MIS weight to remove fireflies
Temporal resampling (ReSTIR)
Global Illumination (ReSTIR PT)
Shift mapping (different sampling method based on roughness and light types)
Respect MIS setting of light
Reduce reservoir pass storage
- Random replay from camera
- Compress data
Increase speed
- Move BSDF sampling outside of NEE
- Check visibility only when necessary
- Use light tiles to reduce cache thrash
Init from bake (?)
Properly handle tiled rendering
Volume
Transparent surfaces (properly check visibility)
Subsurface
Curve and point cloud
Improve sphere light with soft falloff
Spot light
Background light
Distant light
Support GPU
Add back MNEE (or can we replace MNEE?)
Check compatibility with path guiding

FIX:

Bias in spatial resampling
- due to wrong formulation of the visibility term
Denoiser not working with spatial resampling
Random crashes with NaN (?)

Debug options:

Implement Reservoir-based Spatial and Temporal Importance Resampling, aka **ReSTIR**. This is heavily WIP and inefficient, currently only implemented initial and spatial resampling for direct illumination, works on CPU with surface lit by area light, sphere light and mesh light. | Direct Light Only | Before (5 or 4 spp) | After (1 spp) | | -------- | -------- | -------- | | 7.4 s, with light tree | ![before_7.41s_5spp.png](/attachments/66f1562d-01f2-4bec-afbe-8f9a361f29e1) | ![after_7.49s_1spp.png](/attachments/5c14bd79-c589-4939-ac6f-4038d6de7e93) | |~4.5s, no light tree|![before_4.84s_4spp_no_tree.png](/attachments/482ddbed-6d2a-412b-8d3f-84cc9a690ea3)|![after_4.35s_1spp_no_tree.png](/attachments/b866029f-b39e-472a-abd3-1bb3bd1a773f)| **TODO:** - [ ] Multiple spatial sampling iterations - [x] Improve spatial sampling MIS weight to remove fireflies - [ ] Temporal resampling ([ReSTIR](https://research.nvidia.com/sites/default/files/pubs/2020-07_Spatiotemporal-reservoir-resampling/ReSTIR.pdf)) - [ ] Global Illumination ([ReSTIR PT](https://research.nvidia.com/publication/2022-07_generalized-resampled-importance-sampling-foundations-restir)) - [ ] Shift mapping (different sampling method based on roughness and light types) - [ ] Respect MIS setting of light - [ ] Reduce reservoir pass storage - [x] Random replay from camera - [ ] Compress data - [ ] Increase speed - [ ] Move BSDF sampling outside of NEE - [ ] Check visibility only when necessary - [ ] Use light tiles to reduce cache thrash - [ ] Init from bake (?) - [ ] Properly handle tiled rendering - [ ] Volume - [ ] Transparent surfaces (properly check visibility) - [ ] Subsurface - [ ] Curve and point cloud - [ ] Improve sphere light with soft falloff - [ ] Spot light - [ ] Background light - [ ] Distant light - [ ] Support GPU - [ ] Add back MNEE (or can we replace MNEE?) - [ ] Check compatibility with path guiding **FIX:** - [ ] Bias in spatial resampling - [ ] due to wrong formulation of the visibility term - [x] Denoiser not working with spatial resampling - [ ] Random crashes with NaN (?) **Debug options:** ![Screenshot 2024-04-24 at 15.10.56.png](/attachments/53d175e8-54a7-4e4f-a990-f8f2ea07afd4)

before_4.84s_4spp_no_tree.png

3.4 MiB

after_4.35s_1spp_no_tree.png

2.5 MiB

before_7.41s_5spp.png

2.6 MiB

after_7.49s_1spp.png

4.4 MiB

Screenshot 2024-04-24 at 15.10.56.png

239 KiB

👍 9 👀 6 ❤️ 21 🎉 8 🚀 8

Weizhen Huang added 36 commits 2024-04-24 15:19:21 +02:00

c52d1d1cbc Simple RIS for direct illumination

Take several light and BSDF sample candidates, pick one of them based on
the (unoccluded) contribution.
Quite messy. No MNEE, no support for distant/background light in BSDF
sampling, has bias when lamps overlap.
Less noise in direct illumination, but seems more noisy in indirect illumination.

685442ed2b Dummy ReSTIR kernel, only calls `film_write_direct_light()`

09c60cf233 Refactor: add `Reservoir` struct

b1bf68bec9 Explicitly check for visibility

only works for surface

cfc67fa04b Schedule one sample each time, dummy spatial resampling function

d123e1d266 Recover Area Light Sample from uv

8c27266b0b Write ShaderData to render buffer and render later

Is redundant but makes spatial resampling easier
Currently require many storage, but can be improved later.
Only support area light for now.

b29c11e972 Refactor: shuffle some code around in Cycles light

987b78f7c1 Also support point light

348b49bd5d Cleanup: wrap block of code into functions

a4a5ad5696 Cleanup: wrap block of codes into functions

5b9b396898 Cleanup: split `restir.h` and `reservoir.h`

b3e1252e93 Refactor: write to reservoir only when it is direct illumination

ee75ef9f2a Fix: point light `t` and `P` not written

def87bdcf6 Read reservoir in a second pass

e5b32cb5f6 Fix point light zero radius `ls->t` not written

753b133f8a Fix function parameter difference in declaration and definition

b6ec3af65f Remove ReSTIR from megakernel

bd560cae39 Refactor: move 1/pdf to weight instead of a multiplicator of radiance

cb1ac4e501 Cleanup: comments, change function parameters, split reservoir and sd

9fd31af1fc Refactor: make restir an init function

4aeebe5655 Add spatial resampling

Always check 8 neighbors.
Currently doesn't work well with point light with soft falloff, because
uv can only recover `ls->D`, and the visibility between neighboring
pixels vary too much.

883c6fdf73 Cycles: Implement blue-noise dithered sampling

110b247b0a Fix crash without light tree

does not support emissive triangles

9dab3bec79 Add option to use ReSTIR

412f7c1215 Add more ReSTIR options

the ones not implemented yet are grayed out

f93cd67e5f Add options to check sample visibility

6b8f18efe1 Fix division by zero

9e37bc6cd2 Fix biase in spatial resampling

461cab6829 Put BSDF samples in reservoir when spatial resampling is enabled

16fdd010ed Fix checking visibility when ReSTIR is not used

4392ddb30f Refactor: use int to represent initial and spatial resampling

because we are hitting the socket number limit.

54b1d11e66 Refactor: use sample sequences in reservoir sampling

cf2eb5685e Randomly sample neighbors instead of deterministically to reduce artifact

daa1abe00e Support mesh light

5637440df4 Add ray differentials

Weizhen Huang added this to the Render & Cycles project 2024-04-24 15:19:26 +02:00

Weizhen Huang added 2 commits 2024-04-25 18:41:53 +02:00

89f53ec2c4 Add ReSTIR profiling

0b83af2c4d Fix spatial sampling bias

Weizhen Huang force-pushed restir from 0b83af2c4d to 0023d17cb7

2024-04-26 16:10:14 +02:00

Compare

Weizhen Huang added 5 commits 2024-04-29 14:33:25 +02:00

079d87b46c Fix not using initial resampling not used when it should

Initial resampling should be used if spatial resampling is used and
initial visibility is checked

73ae85f710 Copy direction instead of position for area light with zero spread

and point light with soft falloff

73fd2dd703 Split reservoir and surface data passes

6ad2099f93 Reduce reservoir storage by setting up ray from the camera

f72682d289 Write surface data pass with other data passes

Weizhen Huang added 3 commits 2024-04-30 15:53:42 +02:00

f0e4388db3 Fix unintialized field in triangle light

095f6ca7e2 Add utility function to evaluate radiance

shared for initial and spatial resampling

ec6bd6ca21 Use rendering equation in area formulation instead of solid angle

Weizhen Huang force-pushed restir from ec6bd6ca21 to cb15fa32dc

2024-04-30 18:31:41 +02:00

Compare

Weizhen Huang added 5 commits 2024-05-02 13:12:34 +02:00

cf5e14211b Cleanup: improve precision of point light sampling

634dca62d4 Add back `weight > 0.0f` check due to floating point precision issue

92ad7d6186 Replace `reduce_add()` with `luminance()` to reduce color noise

a572e79cf0 Refactor: add more restir utility functions

3968b522ac Cleanup: improve function comment

Weizhen Huang force-pushed restir from 3968b522ac to c3bc745a6b

2024-05-02 15:17:03 +02:00

Compare

Weizhen Huang added 3 commits 2024-05-06 12:04:51 +02:00

2113e6982e Use `ls->pdf > 0` as an indicator of successful sample

5f86d5ea5a Refactor: renaming, reduce minimal sample numbers

add utility function to compute render pixel index
neighbor numbers can be zero for debugging purpose

1be2376176 Add pairwise MIS, reduce noise

This pull request is marked as a work in progress.

This branch is out-of-date with the base branch

View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u restir:weizhen-restir

git checkout weizhen-restir

Sign in to join this conversation.

No reviewers

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

WIP: Cycles: Importance Resampling #121023

Checkout