Add Cycles render tests for texture nodes #6

Closed

Alaska wants to merge 16 commits from Alaska:add-texture-tests into main

pull from: Alaska:add-texture-tests

merge into: blender:main

blender:main

blender:new-volume-scatter-phases

blender:add-fassociative-math-test

blender:test-object-conversion

blender:zero-weight-coat

blender:overlapping-volume-anisotropy

blender:ffmpeg_yuv_precision

blender:blender-v4.3-release

blender:add-texture-tests

blender:volume-color-channel

blender:metallic-light-tree

blender:blender-v4.2-release

blender:fix-image-colorspace

blender:grease-pencil-billie-render-test

blender:blender-v3.6-release

blender:blender-v4.1-release

blender:blender-v3.3-release

When changing the target branch, be careful to rebase the branch in your fork to match. See documentation.

Conversation 9 Commits 16 Files Changed 76 +228

Alaska commented 2024-07-15 18:16:11 +02:00

Member

Copy Link

Ref blender/blender#123012

Ref blender/blender#123012

❤️ 1

Alaska added 1 commit 2024-07-15 18:16:13 +02:00

Add files for testing texture nodes da07f0f94e

Weizhen Huang commented 2024-07-15 18:46:09 +02:00

Member

Copy Link

I would suggest to improve the object names to represent the features they are testing. For example, in white_noise_xxx.blend, you seem to be testing different dimensions of the noise texture, then you can name the objects to Cube 4D, etc.

BTW, when you were making the tests, did you browse through the (closed) bug reports and see if there is anything worth adding? I'm asking because I plan to check the bug reports, but maybe you have already done that.

I would suggest to improve the object names to represent the features they are testing. For example, in `white_noise_xxx.blend`, you seem to be testing different dimensions of the noise texture, then you can name the objects to `Cube 4D`, etc. BTW, when you were making the tests, did you browse through the (closed) bug reports and see if there is anything worth adding? I'm asking because I plan to check the bug reports, but maybe you have already done that.

Weizhen Huang requested review from Weizhen Huang 2024-07-15 18:46:20 +02:00

Alaska commented 2024-07-16 02:53:39 +02:00

Author

Member

Copy Link

I would suggest to improve the object names to represent the features they are testing.

I'll work on this today.

did you browse through the (closed) bug reports and see if there is anything worth adding?

I didn't look through the bug list. But I'll add a test for blender/blender#119797 today (It's the only texture node bug I can remember in recent history).

> I would suggest to improve the object names to represent the features they are testing. I'll work on this today. > did you browse through the (closed) bug reports and see if there is anything worth adding? I didn't look through the bug list. But I'll add a test for blender/blender#119797 today (It's the only texture node bug I can remember in recent history).

Alaska added 1 commit 2024-07-16 06:33:46 +02:00

Update names of objects in tests and add new tests ... 10e00fa7ce

- Added test for "no mapping white nosie"
- Added a test for a precision issue in the musgrave texture

Alaska commented 2024-07-16 07:09:36 +02:00

Author

Member

Copy Link

I've added names to basically every object to help describe what it's testing. Along with that:

• I've added a test for 119797 to the background of musgrave_multifractal_36.blend (This issue was present in the musgrave node, and appeared in the noise texture node once they merged.
• I've added a test for white noise without any mapping applied to it in white_noise_*.blend.
  • Most textures use some sort of co-ordinate system when nothing is connected to the vector input. While the white noise seems to just use 0,0,0. This is different from basically every other node, so I added a test for it.

---

I've been generating test images on Windows with a x86 CPU in Blender 4.3 with builds from the build bot, and I've been comparing it to OptiX in that same build. Some tests are currently failing on OptiX (see attached image).

• The white nosie tests are extremely sensitive to changes in anything. I could quantize the coordinates attached to the vector input of the white noise texture to reduce position based sensitivity. Or should I do something else, like remove the 1 SPP test? Or black list white noise tests on certain platforms?
  • Note: ARM CPU on macOS renders a few pixels differently in the white noise 1 SPP test for me, and fails the test. 256 SPP is within the noise threadhold on ARM macOS and doesn't fail.
• The musgrave and noise tests are failing due to fine details that render subtly different (Differnet sampling positions?). Should I reduce the detail to avoid this issue?
  • The noise test is also failing on ARM macOS with similar differences.

---

I've looked into bug reports from the last year (The oldest report I looked at was from Janurary 20th, 2023) to see if I can find other tests to add.

• blender/blender#116545 and blender/blender#109442 (Faultly loading of certain image file types from disk, primarily due to OIIO)
  • Maybe a seperate set of image texture load tests should be made for this? I'll skip it for this pull request.
• I found some other issues, I haven't added them yet.
  • blender/blender#109254 (Voronoi output is incorrectly clamped)
  • blender/blender#109253 (Incorrect Voronoi output when smoothness is 0)
  • blender/blender#109945 (Brick texture renders incorrectly with certain inputs)
    • Would you like these bug report tests to be added to /render/reports/ or to /render/texture/

I've added names to basically every object to help describe what it's testing. Along with that: - I've added a test for `119797` to the background of `musgrave_multifractal_36.blend` (This issue was present in the musgrave node, and appeared in the noise texture node once they merged. - I've added a test for white noise without any mapping applied to it in `white_noise_*.blend`. - Most textures use some sort of co-ordinate system when nothing is connected to the vector input. While the white noise seems to just use 0,0,0. This is different from basically every other node, so I added a test for it. --- I've been generating test images on Windows with a x86 CPU in Blender 4.3 with builds from the build bot, and I've been comparing it to OptiX in that same build. Some tests are currently failing on OptiX (see attached image). - The white nosie tests are extremely sensitive to changes in anything. I could quantize the coordinates attached to the vector input of the white noise texture to reduce position based sensitivity. Or should I do something else, like remove the 1 SPP test? Or black list white noise tests on certain platforms? - Note: ARM CPU on macOS renders a few pixels differently in the white noise 1 SPP test for me, and fails the test. 256 SPP is within the noise threadhold on ARM macOS and doesn't fail. - The musgrave and noise tests are failing due to fine details that render subtly different (Differnet sampling positions?). Should I reduce the detail to avoid this issue? - The noise test is also failing on ARM macOS with similar differences. --- I've looked into bug reports from the last year (The oldest report I looked at was from Janurary 20th, 2023) to see if I can find other tests to add. - blender/blender#116545 and blender/blender#109442 (Faultly loading of certain image file types from disk, primarily due to OIIO) - Maybe a seperate set of image texture load tests should be made for this? I'll skip it for this pull request. - I found some other issues, I haven't added them yet. - blender/blender#109254 (Voronoi output is incorrectly clamped) - blender/blender#109253 (Incorrect Voronoi output when smoothness is 0) - blender/blender#109945 (Brick texture renders incorrectly with certain inputs) - Would you like these bug report tests to be added to `/render/reports/` or to `/render/texture/`

Failing tests.jpg

255 KiB

Weizhen Huang referenced this pull request from blender/blender 2024-07-16 13:56:07 +02:00

Cycles: Improve the coverage of automated regression tests #123012

Weizhen Huang commented 2024-07-16 17:00:49 +02:00

Member

Copy Link

The white nosie tests are extremely sensitive to changes in anything

I would expect that property from a hash function. We can insert a snap node before white noise, just as in the manual.
I created a pattern with white noise just for fun.

Would you like these bug report tests to be added to /render/reports/ or to /render/texture/

I would prefer /render/texture. render/reports/ is when we don't know how to catergorize the test and it's not clear at all.

blender/blender#109253 (Brick texture renders incorrectly with certain inputs)

This link is wrong.

A few more points:

• checker: there is a typo in Odd Scale - Fac.
• Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero.
• I'm not sure about the IES texture, don't we already have the same in render/light?

> The white nosie tests are extremely sensitive to changes in anything I would expect that property from a hash function. We can insert a snap node before white noise, just as in the manual. I created a pattern with white noise just for fun. > Would you like these bug report tests to be added to /render/reports/ or to /render/texture/ I would prefer `/render/texture`. `render/reports/` is when we don't know how to catergorize the test and it's not clear at all. > blender/blender#109253 (Brick texture renders incorrectly with certain inputs) This link is wrong. A few more points: * checker: there is a typo in Odd Scale - Fac. * Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero. * I'm not sure about the IES texture, don't we already have the same in `render/light`?

white_noise_1spp.blend

996 KiB

Alaska commented 2024-07-16 17:14:32 +02:00

Author

Member

Copy Link

This link is wrong.

I've fixed it. Thanks for spotting.

• checker: there is a typo in Odd Scale - Fac.

I will fix that and add some tests for the reports I mentioned above.

• Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero.

I made most of the Voronoi texture tests in Blender 3.6 which didn't have lacunarity or detail. And when they're loaded in newer versions of Blender, their detail is set to 0.
So I added some extra tests created in Blender 4.2 (voronoi_*_fractal) that does test lacunarity with a detail greater than 0.

• I'm not sure about the IES texture, don't we already have the same in render/light?

It does test a new case, IES on a mesh light, but I'm not sure how useful that is.
Note: In my testing, the bahviour of a IES texture on a mesh light changed between Blender 3.6 and 4.3. So if the behaviour on a mesh light is important, then we should test it. But we can probably just add that to render/light

I created a pattern with white noise just for fun.

It's a nice pattern.

> This link is wrong. I've fixed it. Thanks for spotting. > * checker: there is a typo in Odd Scale - Fac. I will fix that and add some tests for the reports I mentioned above. > * Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero. I made most of the Voronoi texture tests in Blender 3.6 which didn't have lacunarity or detail. And when they're loaded in newer versions of Blender, their detail is set to 0. So I added some extra tests created in Blender 4.2 (`voronoi_*_fractal`) that does test lacunarity with a detail greater than 0. > * I'm not sure about the IES texture, don't we already have the same in `render/light`? It does test a new case, IES on a mesh light, but I'm not sure how useful that is. Note: In my testing, the bahviour of a IES texture on a mesh light changed between Blender 3.6 and 4.3. So if the behaviour on a mesh light is important, then we should test it. But we can probably just add that to `render/light` > I created a pattern with white noise just for fun. It's a nice pattern.

👍 1

Weizhen Huang commented 2024-07-16 17:20:06 +02:00

Member

Copy Link

It does test a new case, IES on a mesh light

Ok, I haven't used it myself so not sure. We can leave it for now and see what other people think. If it's needed I would prefer to put it in render/light.

> It does test a new case, IES on a mesh light Ok, I haven't used it myself so not sure. We can leave it for now and see what other people think. If it's needed I would prefer to put it in `render/light`.

Alaska added 8 commits 2024-07-16 20:25:30 +02:00

Fix typo in checker test 7d1c4206fb

Remove IES texture test 0247b57156

Add test for 109253 to brick texture test 5ac30341e1

Add test for 109253 (Borken smoothness 0 OSL render) to voronoi_smooth_f1 86b500af54

Add test for 109254 (Voronoi clamped to 8) to voronoi_f1.blend 6f4c214395

Quantize inputs for white noise to fix failing GPU tests 301ca0663c

Adjust parameters of musgrave_fbm test to avoid failing GPU tests 74640a3d8c

Adjust parameters of noise_lacunarity to reduce GPU noise fails 3bb3cc74ee

Alaska commented 2024-07-16 20:28:28 +02:00

Author

Member

Copy Link

The issue in 109253 only occured with OSL on. I haven't enabled OSL in the file I added the test to, so the bug report isn't currently tested. But that should be resolved with blender/blender!124601. Let me know if you would like this test to be seperated out to a different file with OSL enabled.

I haven't added the mesh IES test to the /render/light/ suite. Would you like that to happen as part of this pull request? Or prefer that be updated seperately when iterating on/updating other tests?

I also believe Cycles supports the loading of three different IES types. We should probably update the IES tests to test all three. I'll make a note on blender/blender#123012

The issue in 109253 only occured with OSL on. I haven't enabled OSL in the file I added the test to, so the bug report isn't currently tested. But that should be resolved with blender/blender!124601. Let me know if you would like this test to be seperated out to a different file with OSL enabled. I haven't added the mesh IES test to the `/render/light/` suite. Would you like that to happen as part of this pull request? Or prefer that be updated seperately when iterating on/updating other tests? I also believe Cycles supports the loading of three different IES types. We should probably update the IES tests to test all three. I'll make a note on blender/blender#123012

Alaska added 1 commit 2024-07-17 07:26:31 +02:00

Improve white noise 256spp test as a convergence test by adjusting coordinate quantization settings ae299b903e

Alaska changed title from WIP: Add Cycles render tests for texture nodes to Add Cycles render tests for texture nodes 2024-07-30 10:54:36 +02:00

Weizhen Huang requested review from Sergey Sharybin 2024-08-07 16:27:30 +02:00

Weizhen Huang requested review from Lukas Stockner 2024-08-07 16:27:37 +02:00

Alaska added 4 commits 2024-08-09 10:30:20 +02:00

Use group nodes improve consistent testing ... 28580b88a5

If there are two materials, and the nodes are the same, but the outputs are different between them
then use group nodes to keep the nodes in sync.

In the process of making this change, it was found that some materials weren't kept in sync

I also adjusted the anisotropy in the Garbor noise test. 12598 talks about a rotational bias
in garbor noise anisotropy, so increasing the ansiotropy should make it more visible in the
test file, and thus make it easier to fail when the issue is fixed.

Merge branch 'main' into add-texture-tests 077febcc65

Remove IES texture test reference image as it no longer exists 0a65274747

Reduce the sensitivity of noise_lacunarity test 51ec986df1

Alaska commented 2024-08-14 17:21:59 +02:00

Author

Member

Copy Link

Note to self: Retest and potentially update test file for Garbor due to:
ce89d7949c
075bdbcd63

Edit: Done

~~Note to self: Retest and potentially update test file for Garbor due to: https://projects.blender.org/blender/blender/commit/ce89d7949c63b0aa12244f9eab6f0dce4237c272 https://projects.blender.org/blender/blender/commit/075bdbcd63a75df3e270c3b0f81f5e1e9a61e83c~~ Edit: Done

Alaska added 1 commit 2024-08-15 08:49:30 +02:00

Update Garbor test after recent Garbor noise changes ... e6aee94183

Increase the frequency of the low frequency test from 0.02 to 0.3
blender/blender@ce89d7949c
adjusted the appearance and thus I needed to increase the value
to make it visible.

Add more tests for 2D Garbor for anisotropy, triggered by:
blender/blender@075bdbcd63

Alaska commented 2024-10-11 05:17:00 +02:00

Author

Member

Copy Link

I am closing this PR in favour of #16

All the test files and images are the same (And I verified they still pass in latest main). But by having the branch be in blender/blender-test-data, it makes it easier to test with the build bot.

I am closing this PR in favour of https://projects.blender.org/blender/blender-test-data/pulls/16 All the test files and images are the same (And I verified they still pass in latest main). But by having the branch be in `blender/blender-test-data`, it makes it easier to test with the build bot.

Alaska closed this pull request 2024-10-11 05:17:01 +02:00

Alaska deleted branch add-texture-tests 2024-10-11 05:17:35 +02:00

Alaska referenced this pull request 2024-10-11 05:19:38 +02:00

Cycles: Add render tests for texture nodes #16

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

Reviewers

No reviewers

Weizhen Huang

Sergey Sharybin

Lukas Stockner

Labels

Clear labels

No items

No Label

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

2 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender-test-data#6

No description provided.