This patch reimplements the image partial updates. Biggest design motivation for the redesign
is that currently GPUTextures must be owned by the image. This reduces flexibility and adds
complexity to a single component especially when we want to have different structures.
The new design is not limited to GPUTextures and can also be used by reducing overhead in image
operations like scaling. Or partial image updating in Cycles.
The usecase in hand is that we want to support virtual images in the image editor so we can
work with images that don't fit in a single GPUTexture.
Using `BKE_image_partial_update_mark_region` or `BKE_image_partial_update_mark_full_update`
a part of an image can be marked as dirty. These regions are stored per ImageTile (UDIM).
When a part of the code wants to receive partial changes it needs to construct a `PartialUpdateUser`
by calling `BKE_image_partial_update_create`. As long as this instance is kept alive the changes can
be received.
When a user wants to update its own data it will call `BKE_image_partial_update_collect_changes`
This will collect the changes since the last time the user called this function. When the partial changes
are available the partial change can be read by calling `BKE_image_partial_update_get_next_change`
It can happen that the introduced mechanism doesn't have the data anymore to construct the
changes since the last time a PartialUpdateUser requested it. In this case it will get a request
to perform a full update.
Maniphest Tasks: T92613
Differential Revision: https://developer.blender.org/D13238
This patch implements the vector types (i.e:`float2`) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the `blender::math` namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
####Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others
we currently don't have (uintX, intX). All these variations were
asking for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector
functions should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the `BLI_(float|double|mpq)(2|3|4).hh` is a
bit of a let down. Most clases are incomplete, out of sync with each
others with different codestyles, and some functions that should be
static are not (i.e: `float3::reflect()`).
####Upsides:
- Still support `.x, .y, .z, .w` for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types
and can be restricted to certain types. Also template specialization
let us define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance
is the same.
####Downsides:
- Might impact debugability. Though I would arge that the bugs are
rarelly caused by the vector class itself (since the operations are
quite trivial) but by the type conversions.
- Might impact compile time. I did not saw a significant impact since
the usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length.
For instance, one can't call `len_squared_v3v3` in
`math::length_squared()` and call it a day.
- Type cast does not work with the template version of the `math::`
vector functions. Meaning you need to manually cast `float *` and
`(float *)[3]` to `float3` for the function calls.
i.e: `math::distance_squared(float3(nearest.co), positions[i]);`
- Some parts might loose in readability:
`float3::dot(v1.normalized(), v2.normalized())`
becoming
`math::dot(math::normalize(v1), math::normalize(v2))`
But I propose, when appropriate, to use
`using namespace blender::math;` on function local or file scope to
increase readability.
`dot(normalize(v1), normalize(v2))`
####Consideration:
- Include back `.length()` method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement. It felt
like too much for what we need and would be difficult to extend / modify
to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches `delaunay_2d.cc` and the intersection code. I would like
to know @howardt opinion on the matter.
- The `noexcept` on the copy constructor of `mpq(2|3)` is being removed.
But according to @JacquesLucke it is not a real problem for now.
I would like to give a huge thanks to @JacquesLucke who helped during this
and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: https://developer.blender.org/D13791
This patch implements the vector types (i.e:float2) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the blender::math namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others we
currently don't have (uintX, intX). All these variations were asking
for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector functions
should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the BLI_(float|double|mpq)(2|3|4).hh is a bit of a
let down. Most clases are incomplete, out of sync with each others with
different codestyles, and some functions that should be static are not
(i.e: float3::reflect()).
Upsides:
- Still support .x, .y, .z, .w for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types and
can be restricted to certain types. Also template specialization let us
define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance is
the same.
Downsides:
- Might impact debugability. Though I would arge that the bugs are rarelly
caused by the vector class itself (since the operations are quite trivial)
but by the type conversions.
- Might impact compile time. I did not saw a significant impact since the
usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length. For
instance, one can't call len_squared_v3v3 in math::length_squared() and
call it a day.
- Type cast does not work with the template version of the math:: vector
functions. Meaning you need to manually cast float * and (float *)[3] to
float3 for the function calls.
i.e: math::distance_squared(float3(nearest.co), positions[i]);
- Some parts might loose in readability:
float3::dot(v1.normalized(), v2.normalized())
becoming
math::dot(math::normalize(v1), math::normalize(v2))
But I propose, when appropriate, to use
using namespace blender::math; on function local or file scope to
increase readability. dot(normalize(v1), normalize(v2))
Consideration:
- Include back .length() method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement.
It felt like too much for what we need and would be difficult to
extend / modify to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches delaunay_2d.cc and the intersection code. I would like to
know @Howard Trickey (howardt) opinion on the matter.
- The noexcept on the copy constructor of mpq(2|3) is being removed.
But according to @Jacques Lucke (JacquesLucke) it is not a real problem
for now.
I would like to give a huge thanks to @Jacques Lucke (JacquesLucke) who
helped during this and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: http://developer.blender.org/D13791
Reduction of the number of uses of the define BLF_DRAW_STR_DUMMY_MAX
by using actual sizes of static character arrays.
See D13793 for more details.
Differential Revision: https://developer.blender.org/D13793
Reviewed by Campbell Barton
Assert that only the file name component is passed in
since special handling for UDIM should only be applied to the file name.
Also remove an unnecessary NULL check on the filename argument.
This implements the design detailed in T92696 to support virtual
filenames for UDIM textures. Currently, the following 2 substitution
tokens are supported:
| Token | Meaning |
| ----- | ---- |
| <UDIM> | 1001 + u-tile + v-tile * 10 |
| <UVTILE> | Equivalent to u<u-tile + 1>_v<v-tile + 1> |
Example for u-tile of 3 and v-tile of 1:
filename.<UDIM>_ver0023.png --> filename.1014_ver0023.png
filename.<UVTILE>_ver0023.png --> filename.u4_v2_ver0023.png
For image loading, the existing workflow is unchanged. A user can select
one or more image files, belonging to one or more UDIM tile sets, and
have Blender load them all as it does today. Now the <UVTILE> format is
"guessed" just as the <UDIM> format was guessed before.
If guessing fails, the user can simply go into the Image Editor and type
the proper substitution in the filename. Once typing is complete,
Blender will reload the files and correctly fill the tiles. This
workflow is new as attempting to fix the guessing in current versions
did not really work, and the user was often stuck with a confusing
situation.
For image saving, the existing workflow is changed slightly. Currently,
when saving, a user has to be sure to type the filename of the first
tile (e.g. filename.1001.png) to save the entire UDIM set. The number
could differ if they start at a different tile etc. This is confusing.
Now, the user should type a filename containing the appropriate
substitution token. By default Blender will fill in a default name using
the <UDIM> token but the user is free to save out images using <UVTILE>
if they wish.
Differential Revision: https://developer.blender.org/D13057
In the past that worked because the `GPUMaterial` referenced the
`ImageUser` from the image node. However, that design was incompatible
with the recent node tree update refactor (rB7e712b2d6a0d257d272e).
Also, in general it is a bad idea to have references between data that is
owned by two different data blocks.
This incompatibility was resolved by copying the image user from the node
to the `GPUMaterial` (rB28df0107d4a8). Unfortunately, eevee depended
on this reference, because the image user on the node was update when the
frame changed. Because the image user was copied, the image user in the
`GPUMaterial` did not receive the frame update anymore.
This frame update is added back by this commit. The main change is that
the image user iterator now also iterates over image users in `GPUMaterial`s
on material and world data blocks. An issue is that these materials don't
exist on the original data blocks and that caused the check in
`build_animation_images` in the depsgraph to give the wrong answer.
Therefore the check is extended.
Right now the check is not optimal, because it results in more depsgraph
nodes than are necessary. This can be improved when it becomes cheaper
to check if a node tree contains any references to a video texture.
The node tree update refactor mentioned before makes it much easier
to construct this kind of run-time data from the bottom up, instead of
scanning the entire node tree recursively every time some information
is needed.
Goals of this refactor:
* More unified approach to updating everything that needs to be updated
after a change in a node tree.
* The updates should happen in the correct order and quadratic or worse
algorithms should be avoided.
* Improve detection of changes to the output to avoid tagging the depsgraph
when it's not necessary.
* Move towards a more declarative style of defining nodes by having a
more centralized update procedure.
The refactor consists of two main parts:
* Node tree tagging and update refactor.
* Generally, when changes are done to a node tree, it is tagged dirty
until a global update function is called that updates everything in
the correct order.
* The tagging is more fine-grained compared to before, to allow for more
precise depsgraph update tagging.
* Depsgraph changes.
* The shading specific depsgraph node for node trees as been removed.
* Instead, there is a new `NTREE_OUTPUT` depsgrap node, which is only
tagged when the output of the node tree changed (e.g. the Group Output
or Material Output node).
* The copy-on-write relation from node trees to the data block they are
embedded in is now non-flushing. This avoids e.g. triggering a material
update after the shader node tree changed in unrelated ways. Instead
the material has a flushing relation to the new `NTREE_OUTPUT` node now.
* The depsgraph no longer reports data block changes through to cycles
through `Depsgraph.updates` when only the node tree changed in ways
that do not affect the output.
Avoiding unnecessary updates seems to work well for geometry nodes and cycles.
The situation is a bit worse when there are drivers on the node tree, but that
could potentially be improved separately in the future.
Avoiding updates in eevee and the compositor is more tricky, but also less urgent.
* Eevee updates are triggered by calling `DRW_notify_view_update` in
`ED_render_view3d_update` indirectly from `DEG_editors_update`.
* Compositor updates are triggered by `ED_node_composite_job` in `node_area_refresh`.
This is triggered by calling `ED_area_tag_refresh` in `node_area_listener`.
Removing updates always has the risk of breaking some dependency that no
one was aware of. It's not unlikely that this will happen here as well. Adding
back missing updates should be quite a bit easier than getting rid of
unnecessary updates though.
Differential Revision: https://developer.blender.org/D13246
In almost all cases there is no difference between `G.relbase_valid`
and checking `G.main->filepath` isn't an empty string.
In many places a non-empty string is already being used instead of
`G.relbase_valid`.
The only situation where this was needed was when saving from
`wm_file_write` where they temporarily became out of sync.
This has been replaced by adding a new member to `BlendFileWriteParams`
to account for saving an unsaved file for the first time.
Reviewed By: brecht
Ref D13564
Use "filepath" which is the current convention for naming full paths.
- Main use "name" which isn't obviously a file path.
- BlendFileData & FileGlobal used "filename" which is often
used for the name component of a path (without the directory).
- Added space below non doc-string comments to make it clear
these aren't comments for the symbols directly below them.
- Use doxy sections for some headers.
- Minor improvements to doc-strings.
Ref T92709
The main goal of this refactor is to make BPath module use `IDTypeInfo`,
and move each ID-specific part of the `foreach_path` looper into their
own IDTypeInfo struct, using a new `foreach_path` callback.
Additionally, following improvements/cleanups are included:
* Attempt to get better, more consistent namings.
** In particular, move from `path_visitor` to more standard `foreach_path`.
* Update and extend documentation.
** API doc was moved to header, according to recent discussions on this
topic.
* Remove `BKE_bpath_relocate_visitor` from API, this is specific
callback that belongs in `lib_id.c` user code.
NOTE: This commit is expected to be 100% non-behavioral-change. This
implies that several potential further changes were only noted as
comments (like using a more generic solution for
`lib_id_library_local_paths`, addressing inconsistencies like path of
packed libraries always being skipped, regardless of the
`BKE_BPATH_FOREACH_PATH_SKIP_PACKED` `eBPathForeachFlag` flag value,
etc.).
NOTE: basic unittests were added to master already in
rBdcc500e5a265093bc9cc.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D13381
This was crashing using the clone tool without a clone image assigned.
Caused by {rB9111ea78acf4}.
Since above commit, `BKE_image_acquire_ibuf` was using `ima->runtime`
without checking for NULL first.
Since callers are not required to check for this, just return early
here.
note: there is still a memory leak using the clone tool without a clone
image assigned (but this was also the case before said commit and needs
to be investigated separately).
Maniphest Tasks: T93380
Differential Revision: https://developer.blender.org/D13377
Two issues addressed here:
I) `asset_type_info` is sub-data, not a callback. Therefore, move it
before the callbacks in the `IDTypeInfo` struct.
II) More important, initialize this new attribute in *ALL* `IDTypeInfo`
instances. No member of this struct should ever be left implicitely
uninitilazed, ever.
Aftermath of rBa84f1c02d251.
This was broken by rB0c3b215e7d5456878b155d13440864f49ad1f230.
The caching of loaded exr files needed some special treatment.
Differential Revision: https://developer.blender.org/D13313
Previously, `ImageTile->ok` and `ImageUser->ok` were used to indicate
whether an image failed to load. There were three possible values
which (probably) had the following meanings:
* `0`: There was an error while loading the image. Don't try to load again.
* `1`: Default value. Try to load the image.
* `2`: The image was loaded successfully.
This image-wide flag did not make sense unfortunately, because loading
may work for some frames of an image sequence but not for others.
Remember than an image data block can also contain a movie.
The purpose of the `->ok` flag was to serve as an optimization to avoid
trying to load a file over and over again when there is an error (e.g. the
file does not exist or is invalid). To get the optimization back, the patch
is changing `MovieCache` so that it can also cache failed load attempts.
As a consequence, `ibuf` is allowed to be `NULL` in a few more places.
I added the appropriate null checks.
This also solves issues when image sequences are used with the
Image Texture node in Geometry nodes (also see D12827).
Differential Revision: https://developer.blender.org/D12957
Allows to avoid a global lock being held while reading files from disk,
solving performance issues when Cycles needs to read a lot of packed
images.
Simple test file F11597666
Differential Revision: https://developer.blender.org/D13032
This commit adds to ID struct a new optional 'weak reference' to a
linked ID (in the form of a blend file library path and full ID name).
This can then be used on next append to try to find a matching local ID
instead of re-making the linked data local again.
Ref. T90545
NOTE: ID re-use will be disabled for regular append for the time being
(3.0 release), and only used for assets. Therefore, this commit should
not change anything user-wise.
Differential Revision: https://developer.blender.org/D12545
Improve texture painting/uv editing performance when limited scale is active.
Cause of the slow down is that the image editor draws the image in maximum resolution,
but the 3d viewport uses the limited scale. The variation reuses the same GPU texture
and needed to be uploaded/scaled twice to the GPU.
This patch will adds texture slots that can hold the scaled down and the maximum
resolution image. This would allow better cache hits and reuse of existing caches.
Maximum resolution textures are reused for limited scale when they fit to reduce memory
and CPU footprint.
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D12388
Several pure runtime data in this ID type were not properly cleared by
write/read processes.
Note that the initial undo step (the one leading back to initial read
file state) is still forcing re-load of image, for some reasons.
Common investigation together with Jeroen Bakker (@jbakker), thanks. See
also D12242.
This was not really useful, and added estra useless steps in case and ID
should not actually be written.
Further more, it prevented clearing the usercount on write, which can be
cause a false positive 'chanhged' detection in undo/redo case.
Removes the artificial requirement that UDIM tile sets start at 1001.
Blender was already capable of handling sparse tile sets (non-contiguous
tiles) so the restriction around starting at 1001 was unnecessary in
general.
This required fixing a few UDIM-related python bugs around manually
updating the `tile_number` field on images as well. See the differential
for details. No script changes are necessary but they will now work,
correctly, in many more cases.
Differential Revision: https://developer.blender.org/D11859
This patch fixes many minor spelling mistakes, all in comments or
console output. Mostly contractions like can't, won't, don't, its/it's,
etc.
Differential Revision: https://developer.blender.org/D11663
Reviewed by Harley Acheson
After looking into task isolation issues with Sergey, we couldn't find the
reason behind the deadlocks that we are getting in T87938 and a Sprite Fright
file involving motion blur renders.
There is no apparent place where we adding or waiting on tasks in a task group
from different isolation regions, which is what is known to cause problems. Yet
it still hangs. Either we do not understand some limitation of TBB isolation,
or there is a bug in TBB, but we could not figure it out.
Instead the idea is to use isolation only where we know we need it: when
holding a mutex lock and then doing some multithreaded operation within that
locked region. Three places where we do this now:
* Generated images
* Cached BVH tree building
* OpenVDB lazy grid loading
Compared to the more automatic approach previously used, there is the downside
that it is easy to miss places where we need isolation. Yet doing it more
automatically is also causing unexpected issue and bugs that we found no
solution for, so this seems better.
Patch implemented by Sergey and me.
Differential Revision: https://developer.blender.org/D11603
With Constrain to Image Bounds selected, UVs will be constrained to the
correct/closest UDIM if the image is tiled.
UVs will be constrained to the 0-1 UV space if the image is not tiled.
This will override the present behavior of always constraining selected
UVs to the 0-1 UV space (UDIM 1001).
Reviewed By: campbellbarton
Ref D11202
This concerns currently only collections (`master_collection` of scenes)
and root node trees. It removes the matching type-specific helpers
(`BKE_collection_master_scene_search` and `BKE_node_tree_find_owner_ID`).
No functional change expected here.
NOTE: Current implementation of `owner_get` is far from optimal, we
could probably do it better, see {T69169}.
NOTE: While it could also have it, shapekeys IDTypeInfo was left out of
this change for now. Mainly because it sould not be used currently, and
we ultimately want to demote shape keys from ID status anyway.
Currently this is needed to properly tag PointCache's for info update
(fixes an issue reported in T82503).
Suspect we may need this in more cases in the future though, RNA
assign/update processes are not always 100% enough to deal with
complicated corner cases.
