Commit Graph

21 Commits

Author SHA1 Message Date
006e905957 Tests: support running benchmark when Blender prints unrelated text
I noticed that sometimes the geometry nodes benchmark would not work
if there were some left-over debug prints in the code. The reason it did
not work was because the prints were mixed with the test output print.
I also tried using explicit flusing and `atexit` for printing the test output,
but that didn't solve it. Just printing additional newlines to better separate
the test output from other printed output helped though.
2023-01-06 13:04:49 +01:00
cc12f3e0ba Cleanup: fix various typos
Contributed by luzpaz.

Differential Revision: https://developer.blender.org/D15588
2022-08-03 19:28:05 +02:00
3035235def Cleanup: run autopep8 on tests/ 2022-04-19 15:25:55 +10:00
c434782e3a File headers: SPDX License migration
Use a shorter/simpler license convention, stops the header taking so
much space.

Follow the SPDX license specification: https://spdx.org/licenses

- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile

While most of the source tree has been included

- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
  use different header conventions.

doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.

See P2788 for the script that automated these edits.

Reviewed By: brecht, mont29, sergey

Ref D14069
2022-02-11 09:14:36 +11:00
8ae6995e98 Tests: show more precise numbers in benchmark charts 2022-01-14 15:17:22 +01:00
7d5ef64bfb Cleanup: fix typos in comments and docs
Contributed by luzpaz.

Differential Revision: https://developer.blender.org/D13264
2021-11-19 12:46:49 +01:00
71131b4969 Tests: disable audio system for performance tests 2021-11-12 20:03:46 +01:00
7c25399576 tests/benchmarks: Fix operation on windows
The test script did not work on windows
since it had some trouble importing the
api module on the blender side of things.

turning the file path to the module into
a raw string literal sidesteps the
backslash issue in the path.

Differential Revision: https://developer.blender.org/D13163

Reviewed by: brecht
2021-11-09 09:58:48 -07:00
c516659b5e Cleanup: use static sets 2021-11-08 14:52:08 +11:00
335f40ebfa Tests: include device type in benchmark graph labels 2021-10-06 14:21:26 +02:00
cc653c9b02 Fix potential render tests error with invalid utf-8 characters
In general should not happen, but better to report the actual error instead
of the Python test code failing.
2021-09-28 20:50:38 +02:00
42215d7cb8 Tests: more graceful handling of keyboard interrupting benchmarks
Leave current test result unchanged and stop executing immediately,
so it can be continued.
2021-09-10 16:48:26 +02:00
5bfc3a3421 Fix error running benchmark script with environment variables for builds
Ref D12434
2021-09-08 19:58:27 +02:00
6fc94d1848 Tests: updates for performance benchmarking
* Make "run" command (re-)run all tests, add "update" command to only
  run queued and outdated tests equivalent to the old "run" command.
* Support specifying environment variables for revisions, to easily
  compare multiple parameter values.
* Better sorting of revisions in graph.
2021-09-08 16:40:58 +02:00
214e4aac97 Fix Python error in ./benchmark init after recent changes 2021-08-19 17:45:37 +02:00
22b03e1c68 Fix Python error in benchmark executable detection after recent changes 2021-07-26 18:04:40 +02:00
320f34af86 Tests: continue running benchmarks if some tests fail to build or run
Convenient when testing many revisions where some might be broken.
2021-07-22 16:35:00 +02:00
19b597c55d Tests: improve finding of Blender executables in benchmarking
* Allow specifying a folder and automatically setting the proper executable
  name depending on the operating system
* Use executables from configs for listing devices instead of a blender
  command being available
2021-07-22 16:34:57 +02:00
37a5ff4a84 Tests: support graphing peak memory in Cycles performance tests
The general graphing mechanism will create one graph for each output
variable. So it's not limited to time and memory, but that is what the
Cycles tests now output.
2021-07-14 16:12:10 +02:00
432bfbf7a3 Cleanup: pep8 2021-07-06 12:05:27 +10:00
dc3f46d96b Tests: performance testing framework
These are scripts for benchmarking Blender features on real-world .blend
files. They were originally written for benchmarking Cycles performance, and
were made generic so they can be used for more Blender features.

The benchmarks can be run locally by developers. But the plan is to also run
these as part of continuous integration to track performance over time.

Currently there are tests for Cycles rendering and .blend file loading.

Documentation:
https://wiki.blender.org/wiki/Tools/Tests/Performance

Main features:
* User created configurations to quickly run, re-run and analyze a selected
  subset of tests.
* Supports both benchmarking with existing builds, and automatic building of
  specified git commits, tags and branches.
* Generate HTML page with bar and line graphs from test results.
* Controlled using simple command line tool.
* For writing tests, convenient abstraction to run a Python function in Blender
  with arguments and return value.

Ref T74730

Differential Revision: https://developer.blender.org/D11662
2021-07-05 12:32:32 +02:00