Cycles regression testing for GPU devices #82193

New Issue

Brecht Van Lommel · 2020-10-28T14:55:31+01:00

Brecht Van Lommel commented

2020-10-28 14:55:31 +01:00

Background

The Cycles automated regression tests so far only ran on the CPU.

With some environment flags it was possible to execute a script that runs them on other devices, but this is cumbersome. The goal of this task is to add a CMake option to enable running the regression tests on GPU devices in addition to CPU devices.

The reference images are currently only valid for CPU rendering. Keeping differeing reference images updated will have to wait until we have CI infrastructure for testing GPU rendering. For now we can do a few things:

Do nothing and manually check when tests fail
Increase difference thresholds for tests that only differ a little
Disable failing tests for GPU testing

Status

CYCLES_TEST_DEVICES option to specify devices to test
Blacklist failing CUDA tests
Blacklist failing OPTIX tests
Verify HIP tests are working
Enable GPU testing on the buildbot

**Background** The Cycles automated regression tests so far only ran on the CPU. With some environment flags it was possible to execute a script that runs them on other devices, but this is cumbersome. The goal of this task is to add a CMake option to enable running the regression tests on GPU devices in addition to CPU devices. The reference images are currently only valid for CPU rendering. Keeping differeing reference images updated will have to wait until we have CI infrastructure for testing GPU rendering. For now we can do a few things: * Do nothing and manually check when tests fail * Increase difference thresholds for tests that only differ a little * Disable failing tests for GPU testing **Status** - [x] CYCLES_TEST_DEVICES option to specify devices to test - [x] Blacklist failing CUDA tests - [x] Blacklist failing OPTIX tests - [ ] Verify HIP tests are working - [ ] Enable GPU testing on the buildbot

Brecht Van Lommel commented

2020-10-28 14:55:31 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Brecht Van Lommel commented

2020-10-28 14:55:31 +01:00

Added subscriber: @brecht

Brecht Van Lommel self-assigned this 2020-10-28 14:55:56 +01:00

Jeroen Bakker commented

2020-10-28 15:20:53 +01:00

Added subscriber: @Jeroen-Bakker

blender-admin commented

2020-10-28 16:19:03 +01:00

This issue was referenced by dd391d38f6

This issue was referenced by dd391d38f6ef2165bc76a1e69da52e1bd1208e53

blender-admin commented

2020-10-28 19:55:41 +01:00

This issue was referenced by f75b09e7e6

This issue was referenced by f75b09e7e6b3ed9345993455ecab8a6d1690179f

blender-admin commented

2020-11-27 16:40:16 +01:00

This issue was referenced by 2a430a670c

This issue was referenced by 2a430a670cc833033f2f140b017b2004003d49a2

Brecht Van Lommel commented

2020-11-30 16:04:12 +01:00

There is now an initial implementation of this, with a CYCLES_TEST_DEVICES cmake variable that you can CUDA, OPTIX, OPENCL add to.

CUDA and OptiX tests are now passing after blacklisting a bunch of tests that show differences. Most of them are known limitations, a few would be worth investigating as potential bugs. The time for running the tests can be quite long compared to CPU currently, it's likely possible to optimize this with bigger tiles. For OptiX the hair test takes multiple minutes, that's likely a bug somewhere.

For OpenCL, this is not very usable at all, as the kernel is recompiled many times. Completing the tests likely takes hours because of that, I have no finished it. With my AMD card some of the tests are also causing BSoD, so I have not completed running them.

There is now an initial implementation of this, with a `CYCLES_TEST_DEVICES` cmake variable that you can `CUDA`, `OPTIX`, `OPENCL` add to. CUDA and OptiX tests are now passing after blacklisting a bunch of tests that show differences. Most of them are known limitations, a few would be worth investigating as potential bugs. The time for running the tests can be quite long compared to CPU currently, it's likely possible to optimize this with bigger tiles. For OptiX the hair test takes multiple minutes, that's likely a bug somewhere. For OpenCL, this is not very usable at all, as the kernel is recompiled many times. Completing the tests likely takes hours because of that, I have no finished it. With my AMD card some of the tests are also causing BSoD, so I have not completed running them.

Patrick Mours commented

2021-01-06 14:34:51 +01:00

Added subscriber: @pmoursnv

Brecht Van Lommel removed their assignment 2021-10-04 12:10:29 +02:00

Brecht Van Lommel added this to the Render & Cycles project 2023-02-07 19:08:06 +01:00

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Cycles regression testing for GPU devices #82193