Viewport/Eevee regression tests improvements #99045

New Issue

Jeroen Bakker · 2022-06-21T11:47:29+02:00

Jeroen Bakker commented

2022-06-21 11:47:29 +02:00

When compiling blender with WITH_OPENGL_RENDER_TESTS=On the cycles render tests are also run for Eevee and workbench. But their are some issues.

Eevee specific features and options (bloom, transparencies, reflections/refractions) are not covered in the test suite.
The reference images for eevee are rendered using NVIDIA and due to other texture filtering are not usable for AMD.
No tests to measure viewport performance.

Improvements
This task will describe improvements we want to implement in the test suite specific for viewport and eevee.

Use platform specific reference images when the reference images differs between platforms. This allows the tests to be run with success on more platforms, but doesn't add to much of an overhead updating the reference images.
** We could reduce the amount of platform specific images by using a different bias.
Per test we should be able to point to a platform specific reference image. When not configured the common reference image will be used.
We should be able to compare reference images between multiple platforms. (similar to how we currently do for Cycles/Eevee).
Add animation viewport tests that outputs a FPS over 10 seconds of running time (after all shaders have been compiled). The results including some system info should be stored in a file. Eventually we want those to be run on the builtbot and plot a graph over time (accessible via a web).
Q: should we add option to create eevee-specific tests (scene name starts with eevee-) or also run for cycles?

Platforms

NVIDIA latest closed drivers (should be able to run on linux and windows with same results)
AMD latest closed source drivers (should be able to run on linux and windows with same results)
AMD open source drivers (linux only?)
Intel GPU (open source drivers/latest intel Windows/Linux) Unclear if we should make more deviations. otherwise we should stick to Windows and use the latest Intel drivers.
MacOS we might want to stick to Mac mini (intel and m1) for now.

When compiling blender with `WITH_OPENGL_RENDER_TESTS=On` the cycles render tests are also run for Eevee and workbench. But their are some issues. * Eevee specific features and options (bloom, transparencies, reflections/refractions) are not covered in the test suite. * The reference images for eevee are rendered using NVIDIA and due to other texture filtering are not usable for AMD. * No tests to measure viewport performance. **Improvements** This task will describe improvements we want to implement in the test suite specific for viewport and eevee. * Use platform specific reference images when the reference images differs between platforms. This allows the tests to be run with success on more platforms, but doesn't add to much of an overhead updating the reference images. ** We could reduce the amount of platform specific images by using a different bias. * Per test we should be able to point to a platform specific reference image. When not configured the common reference image will be used. * We should be able to compare reference images between multiple platforms. (similar to how we currently do for Cycles/Eevee). * Add animation viewport tests that outputs a FPS over 10 seconds of running time (after all shaders have been compiled). The results including some system info should be stored in a file. Eventually we want those to be run on the builtbot and plot a graph over time (accessible via a web). * Q: should we add option to create eevee-specific tests (scene name starts with `eevee-`) or also run for cycles? **Platforms** * NVIDIA latest closed drivers (should be able to run on linux and windows with same results) * AMD latest closed source drivers (should be able to run on linux and windows with same results) * AMD open source drivers (linux only?) * Intel GPU (open source drivers/latest intel Windows/Linux) Unclear if we should make more deviations. otherwise we should stick to Windows and use the latest Intel drivers. * MacOS we might want to stick to Mac mini (intel and m1) for now.

Jeroen Bakker self-assigned this 2022-06-21 11:47:29 +02:00

Jeroen Bakker commented

2022-06-21 11:47:29 +02:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker commented

2022-06-21 11:55:12 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Viewport/Eevee regression tests improvements #99045