blender/blender-test-data
0
0
Fork 5
Code Pull Requests 6 Activity

Add Cycles render tests for texture nodes #6

Open
Alaska wants to merge 16 commits from Alaska/blender-test-data:add-texture-tests into main
pull from: Alaska/blender-test-data:add-texture-tests
merge into: blender:main

When changing the target branch, be careful to rebase the branch in your fork to match. See documentation.
Conversation 8 Commits 16 Files Changed 76 +228
Alaska commented 2024-07-15 18:16:11 +02:00
Member
Copy Link

Ref blender/blender#123012

Ref blender/blender#123012
❤️ 1
Alaska added 1 commit 2024-07-15 18:16:13 +02:00
Weizhen Huang commented 2024-07-15 18:46:09 +02:00
Member
Copy Link

I would suggest to improve the object names to represent the features they are testing. For example, in white_noise_xxx.blend, you seem to be testing different dimensions of the noise texture, then you can name the objects to Cube 4D, etc.

BTW, when you were making the tests, did you browse through the (closed) bug reports and see if there is anything worth adding? I'm asking because I plan to check the bug reports, but maybe you have already done that.

I would suggest to improve the object names to represent the features they are testing. For example, in `white_noise_xxx.blend`, you seem to be testing different dimensions of the noise texture, then you can name the objects to `Cube 4D`, etc. BTW, when you were making the tests, did you browse through the (closed) bug reports and see if there is anything worth adding? I'm asking because I plan to check the bug reports, but maybe you have already done that.
Weizhen Huang requested review from Weizhen Huang 2024-07-15 18:46:20 +02:00
Alaska commented 2024-07-16 02:53:39 +02:00
Author
Member
Copy Link

I would suggest to improve the object names to represent the features they are testing.

I'll work on this today.

did you browse through the (closed) bug reports and see if there is anything worth adding?

I didn't look through the bug list. But I'll add a test for blender/blender#119797 today (It's the only texture node bug I can remember in recent history).

> I would suggest to improve the object names to represent the features they are testing. I'll work on this today. > did you browse through the (closed) bug reports and see if there is anything worth adding? I didn't look through the bug list. But I'll add a test for blender/blender#119797 today (It's the only texture node bug I can remember in recent history).
Alaska added 1 commit 2024-07-16 06:33:46 +02:00 
- Added test for "no mapping white nosie"
- Added a test for a precision issue in the musgrave texture
Alaska commented 2024-07-16 07:09:36 +02:00
Author
Member
Copy Link

I've added names to basically every object to help describe what it's testing. Along with that:

  • I've added a test for 119797 to the background of musgrave_multifractal_36.blend (This issue was present in the musgrave node, and appeared in the noise texture node once they merged.
  • I've added a test for white noise without any mapping applied to it in white_noise_*.blend.
    • Most textures use some sort of co-ordinate system when nothing is connected to the vector input. While the white noise seems to just use 0,0,0. This is different from basically every other node, so I added a test for it.

I've been generating test images on Windows with a x86 CPU in Blender 4.3 with builds from the build bot, and I've been comparing it to OptiX in that same build. Some tests are currently failing on OptiX (see attached image).

  • The white nosie tests are extremely sensitive to changes in anything. I could quantize the coordinates attached to the vector input of the white noise texture to reduce position based sensitivity. Or should I do something else, like remove the 1 SPP test? Or black list white noise tests on certain platforms?
    • Note: ARM CPU on macOS renders a few pixels differently in the white noise 1 SPP test for me, and fails the test. 256 SPP is within the noise threadhold on ARM macOS and doesn't fail.
  • The musgrave and noise tests are failing due to fine details that render subtly different (Differnet sampling positions?). Should I reduce the detail to avoid this issue?
    • The noise test is also failing on ARM macOS with similar differences.

I've looked into bug reports from the last year (The oldest report I looked at was from Janurary 20th, 2023) to see if I can find other tests to add.

  • blender/blender#116545 and blender/blender#109442 (Faultly loading of certain image file types from disk, primarily due to OIIO)
    • Maybe a seperate set of image texture load tests should be made for this? I'll skip it for this pull request.
  • I found some other issues, I haven't added them yet.
I've added names to basically every object to help describe what it's testing. Along with that: - I've added a test for `119797` to the background of `musgrave_multifractal_36.blend` (This issue was present in the musgrave node, and appeared in the noise texture node once they merged. - I've added a test for white noise without any mapping applied to it in `white_noise_*.blend`. - Most textures use some sort of co-ordinate system when nothing is connected to the vector input. While the white noise seems to just use 0,0,0. This is different from basically every other node, so I added a test for it. --- I've been generating test images on Windows with a x86 CPU in Blender 4.3 with builds from the build bot, and I've been comparing it to OptiX in that same build. Some tests are currently failing on OptiX (see attached image). - The white nosie tests are extremely sensitive to changes in anything. I could quantize the coordinates attached to the vector input of the white noise texture to reduce position based sensitivity. Or should I do something else, like remove the 1 SPP test? Or black list white noise tests on certain platforms? - Note: ARM CPU on macOS renders a few pixels differently in the white noise 1 SPP test for me, and fails the test. 256 SPP is within the noise threadhold on ARM macOS and doesn't fail. - The musgrave and noise tests are failing due to fine details that render subtly different (Differnet sampling positions?). Should I reduce the detail to avoid this issue? - The noise test is also failing on ARM macOS with similar differences. --- I've looked into bug reports from the last year (The oldest report I looked at was from Janurary 20th, 2023) to see if I can find other tests to add. - blender/blender#116545 and blender/blender#109442 (Faultly loading of certain image file types from disk, primarily due to OIIO) - Maybe a seperate set of image texture load tests should be made for this? I'll skip it for this pull request. - I found some other issues, I haven't added them yet. - blender/blender#109254 (Voronoi output is incorrectly clamped) - blender/blender#109253 (Incorrect Voronoi output when smoothness is 0) - blender/blender#109945 (Brick texture renders incorrectly with certain inputs) - Would you like these bug report tests to be added to `/render/reports/` or to `/render/texture/`
Failing tests.jpg
255 KiB
Failing tests.jpg
Weizhen Huang commented 2024-07-16 17:00:49 +02:00
Member
Copy Link

The white nosie tests are extremely sensitive to changes in anything

I would expect that property from a hash function. We can insert a snap node before white noise, just as in the manual.
I created a pattern with white noise just for fun.

Would you like these bug report tests to be added to /render/reports/ or to /render/texture/

I would prefer /render/texture. render/reports/ is when we don't know how to catergorize the test and it's not clear at all.

blender/blender#109253 (Brick texture renders incorrectly with certain inputs)

This link is wrong.

A few more points:

  • checker: there is a typo in Odd Scale - Fac.
  • Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero.
  • I'm not sure about the IES texture, don't we already have the same in render/light?
> The white nosie tests are extremely sensitive to changes in anything I would expect that property from a hash function. We can insert a snap node before white noise, just as in the manual. I created a pattern with white noise just for fun. > Would you like these bug report tests to be added to /render/reports/ or to /render/texture/ I would prefer `/render/texture`. `render/reports/` is when we don't know how to catergorize the test and it's not clear at all. > blender/blender#109253 (Brick texture renders incorrectly with certain inputs) This link is wrong. A few more points: * checker: there is a typo in Odd Scale - Fac. * Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero. * I'm not sure about the IES texture, don't we already have the same in `render/light`?
white_noise_1spp.blend
996 KiB
Alaska commented 2024-07-16 17:14:32 +02:00
Author
Member
Copy Link

This link is wrong.

I've fixed it. Thanks for spotting.

  • checker: there is a typo in Odd Scale - Fac.

I will fix that and add some tests for the reports I mentioned above.

  • Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero.

I made most of the Voronoi texture tests in Blender 3.6 which didn't have lacunarity or detail. And when they're loaded in newer versions of Blender, their detail is set to 0.
So I added some extra tests created in Blender 4.2 (voronoi_*_fractal) that does test lacunarity with a detail greater than 0.

  • I'm not sure about the IES texture, don't we already have the same in render/light?

It does test a new case, IES on a mesh light, but I'm not sure how useful that is.
Note: In my testing, the bahviour of a IES texture on a mesh light changed between Blender 3.6 and 4.3. So if the behaviour on a mesh light is important, then we should test it. But we can probably just add that to render/light

I created a pattern with white noise just for fun.

It's a nice pattern.

> This link is wrong. I've fixed it. Thanks for spotting. > * checker: there is a typo in Odd Scale - Fac. I will fix that and add some tests for the reports I mentioned above. > * Did you test lacunarity for Voronoi texture? Seems that the Detail is always zero. I made most of the Voronoi texture tests in Blender 3.6 which didn't have lacunarity or detail. And when they're loaded in newer versions of Blender, their detail is set to 0. So I added some extra tests created in Blender 4.2 (`voronoi_*_fractal`) that does test lacunarity with a detail greater than 0. > * I'm not sure about the IES texture, don't we already have the same in `render/light`? It does test a new case, IES on a mesh light, but I'm not sure how useful that is. Note: In my testing, the bahviour of a IES texture on a mesh light changed between Blender 3.6 and 4.3. So if the behaviour on a mesh light is important, then we should test it. But we can probably just add that to `render/light` > I created a pattern with white noise just for fun. It's a nice pattern.
👍 1
Weizhen Huang commented 2024-07-16 17:20:06 +02:00
Member
Copy Link

It does test a new case, IES on a mesh light

Ok, I haven't used it myself so not sure. We can leave it for now and see what other people think. If it's needed I would prefer to put it in render/light.

> It does test a new case, IES on a mesh light Ok, I haven't used it myself so not sure. We can leave it for now and see what other people think. If it's needed I would prefer to put it in `render/light`.
Alaska added 8 commits 2024-07-16 20:25:30 +02:00
Alaska commented 2024-07-16 20:28:28 +02:00
Author
Member
Copy Link

The issue in 109253 only occured with OSL on. I haven't enabled OSL in the file I added the test to, so the bug report isn't currently tested. But that should be resolved with blender/blender!124601. Let me know if you would like this test to be seperated out to a different file with OSL enabled.

I haven't added the mesh IES test to the /render/light/ suite. Would you like that to happen as part of this pull request? Or prefer that be updated seperately when iterating on/updating other tests?

I also believe Cycles supports the loading of three different IES types. We should probably update the IES tests to test all three. I'll make a note on blender/blender#123012

The issue in 109253 only occured with OSL on. I haven't enabled OSL in the file I added the test to, so the bug report isn't currently tested. But that should be resolved with blender/blender!124601. Let me know if you would like this test to be seperated out to a different file with OSL enabled. I haven't added the mesh IES test to the `/render/light/` suite. Would you like that to happen as part of this pull request? Or prefer that be updated seperately when iterating on/updating other tests? I also believe Cycles supports the loading of three different IES types. We should probably update the IES tests to test all three. I'll make a note on blender/blender#123012
Alaska added 1 commit 2024-07-17 07:26:31 +02:00
Alaska changed title from WIP: Add Cycles render tests for texture nodes to Add Cycles render tests for texture nodes 2024-07-30 10:54:36 +02:00
Weizhen Huang requested review from Sergey Sharybin 2024-08-07 16:27:30 +02:00
Weizhen Huang requested review from Lukas Stockner 2024-08-07 16:27:37 +02:00
Alaska added 4 commits 2024-08-09 10:30:20 +02:00 
If there are two materials, and the nodes are the same, but the outputs are different between them
then use group nodes to keep the nodes in sync.

In the process of making this change, it was found that some materials weren't kept in sync

I also adjusted the anisotropy in the Garbor noise test. 12598 talks about a rotational bias
in garbor noise anisotropy, so increasing the ansiotropy should make it more visible in the
test file, and thus make it easier to fail when the issue is fixed.
Alaska commented 2024-08-14 17:21:59 +02:00
Author
Member
Copy Link

Note to self: Retest and potentially update test file for Garbor due to:
ce89d7949c
075bdbcd63

Edit: Done

~~Note to self: Retest and potentially update test file for Garbor due to: https://projects.blender.org/blender/blender/commit/ce89d7949c63b0aa12244f9eab6f0dce4237c272 https://projects.blender.org/blender/blender/commit/075bdbcd63a75df3e270c3b0f81f5e1e9a61e83c~~ Edit: Done
Alaska added 1 commit 2024-08-15 08:49:30 +02:00 
Increase the frequency of the low frequency test from 0.02 to 0.3
blender/blender@ce89d7949c
adjusted the appearance and thus I needed to increase the value
to make it visible.

Add more tests for 2D Garbor for anisotropy, triggered by:
blender/blender@075bdbcd63
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u add-texture-tests:Alaska-add-texture-tests
git checkout Alaska-add-texture-tests
Sign in to join this conversation.
Reviewers
No reviewers
Weizhen Huang
Sergey Sharybin
Lukas Stockner
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender-test-data#6
No description provided.
Delete Branch "Alaska/blender-test-data:add-texture-tests"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?