The old text was suggesting to run `BLENDER_TEST_UPDATE=1 ctest` for
failed tests. Now it's more clear that this is for the regeneration of
reference (ground truth) images, and that it will not touch passing test
cases.
It now also mentions to commit the new reference images to SVN, driving
the point home that this is for updating those, and not for making
failing tests succeed in general.
Over-the-shoulder reviewed by: @sergey
It can be assumed that all scripts comply with basic pep8 formatting
regarding white-space, indentation etc.
Also remove note in best practices page & update `tests/python/pep8.py`.
If we want to exclude some scripts from make format,
this can be done by adding them to `ignore_files` in:
source/tools/utils_maintenance/autopep8_format_paths.py
Or using `# nopep8` for to ignore for individual lines.
Ref T98554
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Runs tests based on blend files with minimum python interaction.
Developed as part of GSoC 2021 - Regression Testing of Geometry Nodes.
Earlier, tests were built from scratch by adding a modifier/operation
from the Python API.
Now, tests can also be created inside blender and are compared using
Python script.
Features: Automatically adding expected object if it doesn't exist.
This patch adds tests for the following Geometry Nodes category:
* Curves
* Geometry
* Mesh
* Points
The implemented UML diagram for refactoring of mesh test framework.
{F10225906}
Technical Changes:
SpecMeshTest: It adds the modifier/operation based on the Spec provided.
BlendFileTest: It applies already existing modifier/operation from the blend file.
Test folders hierarchy with tests. This folder should be extracted to `lib\tests\modeling`
{F10240651}
Note: The `geometry_nodes` folder might lie under another `geometry_nodes` folder while extracting, please double check. Use the inner-most one.
The hierarchy should be:
-`lib\tests\modeling\geometry_nodes\mesh`
-`lib\tests\modeling\geometry_nodes\points`
and so on.
* From `ctest` the tests should be run as `ctest -R geo_node -C [Configuration]` on Windows.
* Each single test can be run with its entire name e..g `ctest -R geo_node_geometry_join_geometry`.(just an example). Run `ctest -N -R geo_node` to see all tests.
* From blender, the tests can be run `blender -b path\to\blend\file --python path\to\geo_node_test.py`
Reviewed By: zazizizou, JacquesLucke
Differential Revision: https://developer.blender.org/D11611
MeshTest now compares selection between evaluated mesh and expected mesh. This way, we can test more operators
such as `faces_select_linked_flat`
Note: selection comparison intentionally does not happen in BKE_mesh_cmp() on C side but rather on Python side, because
selection is independent of mesh generation.
Reviewed By: calra, mont29
Differential Revision: https://developer.blender.org/D10279
Print number of total tests with each test to show how many tests have been executed and how many are left.
Example: `Running test 27/36: PlaneFaceSplitByEdges...`
Reviewed By: calra, mont29
Differential Revision: https://developer.blender.org/D10278
Tests are not identified with indexes, so no need to maintain comments with indexes anymore
Reviewed By: calra, mont29
Differential Revision: https://developer.blender.org/D10277
This revision contains the following changes-
- Updated the existing testing framework for Modifiers for Regression
Testing.
- Tests for Physics modifiers and remaining Generate and Deform modifiers are added.
- The existing `ModifierSpec` is updated with backward compatibility to support Physics Modifiers.
- Now there is support for frame number and giving nested parameters for attributes.
- Some Deform modifiers required Object Operators, e.g. "Bind" in Mesh Deform, so a new class was added to support that functionality.
- A separate class for holding Particles System, they are tested by converting all the particles to mesh and joining it to the mesh they were added.
- Updated the format to add tests for Bevel, Boolean and Operators as
well.
Reviewed By: zazizizou, mont29, campbellbarton
Differential Revision: https://developer.blender.org/D8507
Blacklist a bunch of tests on the GPU, which are known to currently have
differences with CPU. These can be enabled as they are fixed or the test
is modified to give compatible results when there are known limitations.
OSL tests were also moved to their own directory since those are not
supported on the GPU.
Ref T82193
CYCLES_TEST_DEVICES is a list of devices (CPU, CUDA, OPTIX, OPENCL). It is set
to CPU only by default.
Test output is now writen to build/tests/cycles/<device>, and the HTML report
has separate report pages for the different devices, with option to compare
between CPU and GPU renders.
Various GPU tests are still failing due to CPU/GPU differences, these are to be
fixed or blacklisted still.
Ref T82193
This adds a new `--debug-exit-on-error` flag. When it is set, Blender
will abort with a non-zero exit code when there are internal errors.
Currently, "internal errors" includes memory leaks detected by
guardedalloc and error/fatal log entries in clog.
The new flag is passed to Blender in various places where automated
tests are run. Furthermore, the `--debug-memory` flag is used in tests,
because that makes the verbose output more useful, when dealing
with memory leaks.
Reviewers: brecht, sergey
Differential Revision: https://developer.blender.org/D8665
This uses the same framework as automated modifier tests. It adds a physics
modifier, bakes and compares vertex coordinates on the end frame.
Differential Revision: https://developer.blender.org/D7017
Submitting on behalf of Jesse Y (deadpin).
In test harness for modifier testing, now run mesh validation
on output mesh. Also, fix printing so it interleaves properly.
Patch from Habib Gahbiche (zazizizou) moves the "run operator and
compare mesh to a golden" paradigm used in bevel and boolean tests
into a general framework that separates the test specs from the
blend files. Then adds some other operator and modifier tests using
the new framework. Diff D5357.id20724.diff was applied.
New .blend files, modifiers.blend and operators.blend are needed
in the tests/modeling svn directory; those were separately committed.
There are deeper issues than just updating the regression test .blend file
and the solution is dragging for far too long.
Considering this a known broken feature, which will either be fixed next week
or completely removed from the interface for the coming release.
Blender startup time and shader compilation is a big factor when running
hundreds of tests, so now all renders in the same ctest run in the same
process.
This was previously reverted due to skipping other tests when one test
crashed. Now if a test crashes, Blender is re-run with the remaining
tests so we get results from them still.
Blender startup time and shader compilation is a big factor when running
hundreds of tests, so now all renders in the same ctest run in the same
process. If a test crashes, the remaining tests in the same category will
be marked as skipped.
Benchmarked on a quad core with ctest -j8.
cycles: 118.1s -> 94.3s
eevee: 66.2s -> 29.2s
workbench: 31.7s -> 8.6s
