Tests: Add tests for image format saving and loading

This adds saving and loading tests for our supported image formats.

**Saving - bf_imbuf_save.py**
There are 2 template images which are loaded anew for each file save
attempt.  One is an 8-bit RGBA image and the other 32-bit. This is
required as many formats use a variety of factors to determine which of
`ibuf->rect` or `ibuf->rectfloat` to use for processing.  The templates
are constructed to have alpha transparency as well as values > 1 (or
clamped to 1 for the case of the 8-bit template).

Test flow:
 - Load in an appropriate template image
 - Save it to the desired format with the desired set of options
 - Compare against the reference image

Notes:
 - 98 references are used totaling ~3.6MB
 - 10-12 second test runtime
 - Templates can be reconstructed with the create-templates.blend file

**Loading - bf_imbuf_load.py**
Test flow:
 - Load in each of the reference images
 - Save them back out as .exr
 - Save additional metadata to a secondary file (alpha mode, colorspace etc)
 - Compare the saved out .exr with another set of reference .exrs
 - Compare the saved out file metadata with set of reference metadata

Notes:
 - 98 exr references are used totaling ~10MB
 - 10-12 second test runtime as well

A HTML report is not implemented. The diff output organization is very
similar to the other tests so it should be somewhat easy to do in the
future if we want.

The standard set of environment variables are implemented for both:
BLENDER_TEST_UPDATE, BLENDER_VERBOSE, and BLENDER_TEST_COLOR

Pull Request #104442
This commit is contained in:
2023-02-20 19:04:34 -08:00
parent ff3fd5f1ce
commit 7699c7407d
6 changed files with 634 additions and 40 deletions

View File

@@ -0,0 +1,92 @@
# SPDX-License-Identifier: GPL-2.0-or-later
import os
import pathlib
import shutil
import subprocess
import unittest
from .colored_print import (print_message, use_message_colors)
class AbstractImBufTest(unittest.TestCase):
@classmethod
def init(cls, args):
cls.test_dir = pathlib.Path(args.test_dir)
cls.reference_dir = pathlib.Path(args.test_dir).joinpath("reference")
cls.reference_load_dir = pathlib.Path(args.test_dir).joinpath("reference_load")
cls.output_dir = pathlib.Path(args.output_dir)
cls.diff_dir = pathlib.Path(args.output_dir).joinpath("diff")
cls.idiff = pathlib.Path(args.idiff)
cls.optional_formats = args.optional_formats
os.makedirs(cls.diff_dir, exist_ok=True)
cls.errors = 0
cls.fail_threshold = 0.001
cls.fail_percent = 1
cls.verbose = os.environ.get("BLENDER_VERBOSE") is not None
cls.update = os.getenv('BLENDER_TEST_UPDATE') is not None
if os.environ.get("BLENDER_TEST_COLOR") is not None:
use_message_colors()
def setUp(self):
self.errors = 0
print_message("")
def tearDown(self):
if self.errors > 0:
self.fail("{} errors encountered" . format(self.errors))
def skip_if_format_missing(self, format):
if self.optional_formats.find(format) < 0:
self.skipTest("format not available")
def call_idiff(self, ref_path, out_path):
ref_filepath = str(ref_path)
out_filepath = str(out_path)
out_name = out_path.name
if os.path.exists(ref_filepath):
# Diff images test with threshold.
command = (
str(self.idiff),
"-fail", str(self.fail_threshold),
"-failpercent", str(self.fail_percent),
ref_filepath,
out_filepath,
)
try:
subprocess.check_output(command)
failed = False
except subprocess.CalledProcessError as e:
if self.verbose:
print_message(e.output.decode("utf-8", 'ignore'))
failed = e.returncode != 1
else:
if not self.update:
return False
failed = True
if failed and self.update:
# Update reference image if requested.
shutil.copy(out_filepath, ref_filepath)
failed = False
# Generate diff image.
diff_img = str(self.diff_dir.joinpath(out_name + ".diff.png"))
command = (
str(self.idiff),
"-o", diff_img,
"-abs", "-scale", "16",
ref_filepath,
out_filepath
)
try:
subprocess.check_output(command)
except subprocess.CalledProcessError as e:
if self.verbose:
print_message(e.output.decode("utf-8", 'ignore'))
return not failed