This allows fast access to various arrays in the Python API.
Most notably, `image.pixels` can be accessed much more efficiently now.
**Benchmark**
Below are the results of a benchmark that compares different ways to
set/get all pixel values. I do the tests on 2048x2048 rgba images.
The benchmark tests the following dimensions:
- Byte vs. float per color channel
- Python list vs. numpy array containing floats
- `foreach_set` (new) vs. `image.pixels = ...` (old)
```
Pixel amount: 2048 * 2048 = 4.194.304
Byte buffer size: 16.8 mb
Float buffer size: 67.1 mb
Set pixel colors:
byte - new - list: 271 ms
byte - new - buffer: 29 ms
byte - old - list: 350 ms
byte - old - buffer: 2900 ms
float - new - list: 249 ms
float - new - buffer: 8 ms
float - old - list: 330 ms
float - old - buffer: 2880 ms
Get pixel colors:
byte - list: 128 ms
byte - buffer: 9 ms
float - list: 125 ms
float - buffer: 8 ms
```
**Observations**
The best set and get speed can be achieved with buffers and a float image,
at the cost of higher memory consumption. Furthermore, using buffers when
using `pixels = ...` is incredibly slow, because it is not optimized.
Optimizing this is possible, but might not be trivial (there were multiple
attempts afaik).
Float images are faster due to overhead introduced by the api for byte images.
If I profiled it correctly, a lot of time is spend in the `[0, 1] -> {0, ..., 255}`
conversion. The functions doing that conversion is `unit_float_to_uchar_clamp`.
While I have an idea on how it can be optimized, I do not know if it can be done
without changing its functionality slightly. Performance wise the best solution
would be to not do this conversion at all and accept byte input from the api
user directly, but that seems to be a more involved task as well.
Differential Revision: https://developer.blender.org/D7053
Reviewers: JacquesLucke, mont29
'Private' can be a rather confusing term, especially when considering
its meaning in programming languages.
So now root node trees and master collections are 'embedded' IDs
instead.
While it might be handy to have type-less functionality which is
similar to how C++ math is implemented it can not be easily achieved
with just preprocessor in a way which does not have side-effects on
wrong usage.
There macros where often used on a non-trivial expression, and there
was at least one usage where it was causing an actual side effect/bug
on Windows (see change around square_f(sh[index++]) in studiolight.c).
For such cases it is handy to have a function which is guaranteed to
have zero side-effects. The motivation behind actually removing the
macros is that there is already a way to do similar calculation. Also,
not having such macros is a way to guarantee that its usage is not
changed in a way which have side-effects and that it's not used as an
inspiration for cases where it should not be used.
Differential Revision: https://developer.blender.org/D7051
The old convention was easy to confuse with ScrArea.
Part of https://developer.blender.org/T74432.
This is mostly a batch rename with some manual fixing. Only single word
variable names are changed, no prefixed/suffixed names.
Brecht van Lommel and Campbell Barton both gave me a green light for
this convention change.
Also ran clan clang format on affected files.
This moves the `alembic`, `avi`, `collada`, and `usd` modules into a common
`io` directory.
This also cleans up some `#include "../../{somedir}/{somefile}.h"` by
adding `../../io/{somedir}` to `CMakeLists.txt` and then just using
`#include "{somefile}.h"`.
No functional changes.
The docstring was likely copy-pasted, and the return type was not adjusted.
Since the `bpy.msgbus.subscribe_rna()` returns nothing, I just removed the
return type documentation.
Motivation the functions get 3 different kind of flag parameters (ImBuf,
DrawType, OffscreenRendering) the naming of the flags were not clear,
leading to mistakes and unnecessary time spend debugging.
This was not set anywhere, code relies on MLOOPUV_VERTSEL everywhere.
BMLoopUV.select_edge was never returning True and wasnt updating uv
select state when set manually.
Afaict, there were no official Addons using this from python.
Resolves T65836
Maniphest Tasks: T65836
Differential Revision: https://developer.blender.org/D6772
- Use 'int' for counters instead of short.
- Use 'bool' instead of a counter when only a change is being detected.
- Use typed enum for keying set flags.
- Include in comments when a negate error code may be returned.
- Split 'verify_fcurve' into two functions:
ED_action_fcurve_ensure which adds the f-curve if needed.
ED_action_fcurve_find which returns NULL when not found.
Callers of ED_action_fcurve_find had unused 'group'
argument which has been removed.
- Rename verify_adt_action to ED_id_action_ensure
It had an argument to add data which was always true,
remove this instead of splitting in into a separate function.
- Use 'BKE_object_defgroup' prefix for object functions.
- Rename 'defvert_verify_index' to 'defvert_ensure_index'
since this adds the group if it isn't found.
The OpenXR-SDK contains utilities for using the OpenXR standard
(https://www.khronos.org/openxr/). Namely C-headers and a so called
"loader" to manage runtime linking to OpenXR platforms ("runtimes")
installed on the user's system.
The WITH_XR_OPENXR build option is disabled by default for now, as there
is no code using it yet. On macOS it will remain disabled for now, it's
untested and there's no OpenXR runtime in sight for it.
Some points on the OpenXR-SDK dependency:
* The repository is located at
https://github.com/KhronosGroup/OpenXR-SDK (Apache 2).
* Notes on updating the dependency:
https://wiki.blender.org/wiki/Source/OpenXR_SDK_Dependency
* It contains a bunch of generated files, for which the sources are in a
separate repository
(https://github.com/KhronosGroup/OpenXR-SDK-Source).
* We could use that other repo by default, but I'd rather go with the
simpler solution and allow people to opt in if they want advanced dev
features.
* We currently use the OpenXR loader lib from it and the headers.
* To use the injected OpenXR API-layers from the SDK (e.g. API
validation layers), the SDK needs to be compiled from this other
repository.
The extra "XR_" prefix in the build option is to avoid mix-ups of OpenXR
with OpenEXR.
Most of this comes from the 2019 GSoC project, "Core Support of Virtual
Reality Headsets through OpenXR"
(https://wiki.blender.org/wiki/User:Severin/GSoC-2019/).
Differential Revision: https://developer.blender.org/D6188
Reviewed by: Campbell Barton, Sergey Sharybin, Bastien Montagne, Ray
Molenkamp
Color management currently happens after the python handlers. This means
that all common drawing operations needs to happen in SRS. The
PyGPUOffscreen requested that the color management was applied, that
resulted into applying colormanagement twice.
This patch makes sure that PyGPUOffscreen.draw_view3d renders in SRS.
There are more code paths that needs to be checked that the correct
space is being requested.
The 'random' unit tests and some examples from the new boolean code
triggered asserts and crashes. This fixes those.
There is a new flag in the input that optionally disables a pass
over input to snap segment edges to other segments.
The introduction of python instancing for ID data in 2.80 completely
broke the 'smart & efficient' hack of using a same py object as key for
initial quick check, since rebuilding the RNAPointer in the
BPy_StructRNA would actually affect the py instance of the first ID used
to generate that py object...
TL;DR: No need for this complex and unclear optimization anymore, since
we do not actually rebuild a whole py object anymore every time we call
`pyrna_id_CreatePyObject()` from a same ID pointer.
Using a struct here allows to change given parameters to the callbacks
without having to edit all callbacks functions, which is always noisy
and time consuming.
Pixar recently released USD 20.02 [1]. I think it's important for people
to be able to figure out which version of the USD library is used in
Blender.
[1] https://github.com/PixarAnimationStudios/USD/releases/tag/v20.02
This commit exposes the USD library information via `bpy.app.usd`, and
includes that info in the `system-info.txt` saved via Help → Save System
Info.
Reviewed by: brecht
Differential Revision: https://developer.blender.org/D6724
This 'fixes' T68554: 'API mathutils.geometry.tessellate_polygon returns
bad results sometimes' by documenting the limitations of the current
implementation.
I've also added a unit test for the function, so that any change in this
behaviour will get noticed.
No functional changes.
This is a more correct fix to the issue Brecht was fixing in D6600.
While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).
Top-level idea: leave it to CMake to keep track of dependency graph.
The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.
It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.
For example, this order will likely fail:
libbf_blenfont.a libfreetype6.a libbf_blenfont.a
This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.
General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.
The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.
Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.
The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:
- Have every library prefixed as "optimized" or "debug" if
separation is needed (non-prefixed libraries will be considered
"generic").
- Loop through libraries passed to function and do simple parsing
which will look for "optimized" and "debug" words and specify
following library to corresponding category.
This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.
Tested the following configurations on Linux, macOS and Windows:
- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer
NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.
Differential Revision: https://developer.blender.org/D6642
This avoids the problem where Blender doesn't start because
the PYTHONPATH points to an incompatible Python version,
see T72807.
Previously we chose to assume people who set the PYTHONPATH know what
they're doing, however users may have set this for non Blender projects.
So it's not obvious that this is the cause of Blender not to launch
on their system.
To use Python's environment vars, pass the argument:
--python-use-system-env
Note that this only impacts Python run-time environment variables
documented in `python --help`, Access from `os.environ` remains.
A collection of smaller changes that are required in the /blender/source files. A lot of them are also due to variable renaming.
Reviewed By: sergey
Maniphest Tasks: T59995
Differential Revision: https://developer.blender.org/D3855
This commit introduces the first version of an exporter to Pixar's
Universal Scene Description (USD) format.
Reviewed By: sergey, LazyDodo
Differential Revision: https://developer.blender.org/D6287
- The USD libraries are built by `make deps`, but not yet built by
install_deps.sh.
- Only experimental support for instancing; by default all duplicated
objects are made real in the USD file. This is fine for exporting a
linked-in posed character, not so much for thousands of pebbles etc.
- The way materials and UV coordinates and Normals are exported is going
to change soon.
- This patch contains LazyDodo's fixes for building on Windows in D5359.
== Meshes ==
USD seems to support neither per-material nor per-face-group
double-sidedness, so we just use the flag from the first non-empty
material slot. If there is no material we default to double-sidedness.
Each UV map is stored on the mesh in a separate primvar. Materials can
refer to these UV maps, but this is not yet exported by Blender. The
primvar name is the same as the UV Map name. This is to allow the
standard name "st" for texture coordinates by naming the UV Map as such,
without having to guess which UV Map is the "standard" one.
Face-varying mesh normals are written to USD. When the mesh has custom
loop normals those are written. Otherwise the poly flag `ME_SMOOTH` is
inspected to determine the normals.
The UV maps and mesh normals take up a significant amount of space, so
exporting them is optional. They're still enabled by default, though.
For comparison: a shot of Spring (03_035_A) is 1.2 GiB when exported
with UVs and normals, and 262 MiB without. We probably have room for
optimisation of written UVs and normals.
The mesh subdivision scheme isn't using the default value 'Catmull
Clark', but uses 'None', indicating we're exporting a polygonal mesh.
This is necessary for USD to understand our normals; otherwise the mesh
is always rendered smooth. In the future we may want to expose this
choice of subdivision scheme to the user, or auto-detect it when we
actually support exporting pre-subdivision meshes.
A possible optimisation could be to inspect whether all polygons are
smooth or flat, and mark the USD mesh as such. This can be added when
needed.
== Animation ==
Mesh and transform animation are now written when passing
`animation=True` to the export operator. There is no inspection of
whether an object is actually animated or not; USD can handle
deduplication of static values for us.
The administration of which timecode to use for the export is left to
the file-format-specific concrete subclasses of
`AbstractHierarchyIterator`; the abstract iterator itself doesn't know
anything about the passage of time. This will allow subclasses for the
frame-based USD format and time-based Alembic format.
== Support for simple preview materials ==
Very simple versions of the materials are now exported, using only the
viewport diffuse RGB, metallic, and roughness.
When there are multiple materials, the mesh faces are stored as geometry
subset and each material is assigned to the appropriate subset. If there
is only one material this is skipped.
The first material if any) is always applied to the mesh itself
(regardless of the existence of geometry subsets), because the Hydra
viewport doesn't support materials on subsets. See
https://github.com/PixarAnimationStudios/USD/issues/542 for more info.
Note that the geometry subsets are not yet time-sampled, so it may break
when an animated mesh changes topology.
Materials are exported as a flat list under a top-level '/_materials'
namespace. This inhibits instancing of the objects using those
materials, so this is subject to change.
== Hair ==
Only the parent strands are exported, and only with a constant colour.
No UV coordinates, no information about the normals.
== Camera ==
Only perspective cameras are supported for now.
== Particles ==
Particles are only written when they are alive, which means that they
are always visible (there is currently no code that deals with marking
them as invisible outside their lifespan).
Particle-system-instanced objects are exported by suffixing the object
name with the particle's persistent ID, giving each particle XForm a
unique name.
== Instancing/referencing ==
This exporter has experimental support for instancing/referencing.
Dupli-object meshes are now written to USD as references to the original
mesh. This is still very limited in correctness, as there are issues
referencing to materials from a referenced mesh.
I am still committing this, as it gives us a place to start when
continuing the quest for proper instancing in USD.
== Lights ==
USD does not directly support spot lights, so those aren't exported yet.
It's possible to add this in the future via the UsdLuxShapingAPI. The
units used for the light intensity are also still a bit of a mystery.
== Fluid vertex velocities ==
Currently only fluid simulations (not meshes in general) have explicit
vertex velocities. This is the most important case for exporting
velocities, though, as the baked mesh changes topology all the time, and
thus computing the velocities at import time in a post-processing step
is hard.
== The Building Process ==
- USD is built as monolithic library, instead of 25 smaller libraries.
We were linking all of them as 'whole archive' anyway, so this doesn't
affect the final file size. It does, however, make life easier with
respect to linking order, and handling upstream changes.
- The JSON files required by USD are installed into datafiles/usd; they
are required on every platform. Set the `PXR_PATH_DEBUG` to any value
to have the USD library print the paths it uses to find those files.
- USD is patched so that it finds the aforementioned JSON files in a path
that we pass to it from Blender.
- USD is patched to have a `PXR_BUILD_USD_TOOLS` CMake option to disable
building the tools in its `bin` directory. This is sent as a pull
request at https://github.com/PixarAnimationStudios/USD/pull/1048
Logic for registering and checking properties of registrable classes was
broken, allowing to ignore some errors.
Recent fix rBeb798de101a `broke` the result of the
pyapi_idprop_datablock test, because previously that test would fail
(i.e. suceed, as it is an 'expected to break test') for a reason it was
not designed to check.
This is the problem with that kind of tests - you cannot really check
that they are failing on the expected reason(s)...
This was an old bug which could be caused by saving after separating.
Changes from 79b703bb63 made this fail reliably.
Update shape key indices when they may be used again later.
