This moves the `alembic`, `avi`, `collada`, and `usd` modules into a common
`io` directory.
This also cleans up some `#include "../../{somedir}/{somefile}.h"` by
adding `../../io/{somedir}` to `CMakeLists.txt` and then just using
`#include "{somefile}.h"`.
No functional changes.
Importers/exporters should show the sidebar by default (if they have
any settings). For some reason this was missing for the Alembic and
Collada exporters.
- "Only Export Selected Objects" → "Selection Only", as this is the
standard used in other exporters.
- "Evaluation Mode" → "Use Settings for", as this is clearer for users
who aren't as familiar with the technical details of the depsgraph.
- "Use Instancing" → "Instancing", and moved the checkbox to the end of
the option list into its own box "Experimental". This looks a bit
cleaner than just plonking "EXPERIMENTAL" in the checkbox label.
The way the USD exporter currently works, it is not possible to export
invisible objects. As such, the 'Visible Objects Only' option was
confusing.
Exporting invisible objects means obtaining invisible evaluated objects
from the depsgraph, which is not something that's currently implemented.
Once that's done, we can reintroduce this option.
There are options for the exporter that are relevant for each export, for
example the choice between Viewport or Render settings, or whether to
export the current frame or an animation. It's better to have the options
panel opened by default.
This commit introduces the first version of an exporter to Pixar's
Universal Scene Description (USD) format.
Reviewed By: sergey, LazyDodo
Differential Revision: https://developer.blender.org/D6287
- The USD libraries are built by `make deps`, but not yet built by
install_deps.sh.
- Only experimental support for instancing; by default all duplicated
objects are made real in the USD file. This is fine for exporting a
linked-in posed character, not so much for thousands of pebbles etc.
- The way materials and UV coordinates and Normals are exported is going
to change soon.
- This patch contains LazyDodo's fixes for building on Windows in D5359.
== Meshes ==
USD seems to support neither per-material nor per-face-group
double-sidedness, so we just use the flag from the first non-empty
material slot. If there is no material we default to double-sidedness.
Each UV map is stored on the mesh in a separate primvar. Materials can
refer to these UV maps, but this is not yet exported by Blender. The
primvar name is the same as the UV Map name. This is to allow the
standard name "st" for texture coordinates by naming the UV Map as such,
without having to guess which UV Map is the "standard" one.
Face-varying mesh normals are written to USD. When the mesh has custom
loop normals those are written. Otherwise the poly flag `ME_SMOOTH` is
inspected to determine the normals.
The UV maps and mesh normals take up a significant amount of space, so
exporting them is optional. They're still enabled by default, though.
For comparison: a shot of Spring (03_035_A) is 1.2 GiB when exported
with UVs and normals, and 262 MiB without. We probably have room for
optimisation of written UVs and normals.
The mesh subdivision scheme isn't using the default value 'Catmull
Clark', but uses 'None', indicating we're exporting a polygonal mesh.
This is necessary for USD to understand our normals; otherwise the mesh
is always rendered smooth. In the future we may want to expose this
choice of subdivision scheme to the user, or auto-detect it when we
actually support exporting pre-subdivision meshes.
A possible optimisation could be to inspect whether all polygons are
smooth or flat, and mark the USD mesh as such. This can be added when
needed.
== Animation ==
Mesh and transform animation are now written when passing
`animation=True` to the export operator. There is no inspection of
whether an object is actually animated or not; USD can handle
deduplication of static values for us.
The administration of which timecode to use for the export is left to
the file-format-specific concrete subclasses of
`AbstractHierarchyIterator`; the abstract iterator itself doesn't know
anything about the passage of time. This will allow subclasses for the
frame-based USD format and time-based Alembic format.
== Support for simple preview materials ==
Very simple versions of the materials are now exported, using only the
viewport diffuse RGB, metallic, and roughness.
When there are multiple materials, the mesh faces are stored as geometry
subset and each material is assigned to the appropriate subset. If there
is only one material this is skipped.
The first material if any) is always applied to the mesh itself
(regardless of the existence of geometry subsets), because the Hydra
viewport doesn't support materials on subsets. See
https://github.com/PixarAnimationStudios/USD/issues/542 for more info.
Note that the geometry subsets are not yet time-sampled, so it may break
when an animated mesh changes topology.
Materials are exported as a flat list under a top-level '/_materials'
namespace. This inhibits instancing of the objects using those
materials, so this is subject to change.
== Hair ==
Only the parent strands are exported, and only with a constant colour.
No UV coordinates, no information about the normals.
== Camera ==
Only perspective cameras are supported for now.
== Particles ==
Particles are only written when they are alive, which means that they
are always visible (there is currently no code that deals with marking
them as invisible outside their lifespan).
Particle-system-instanced objects are exported by suffixing the object
name with the particle's persistent ID, giving each particle XForm a
unique name.
== Instancing/referencing ==
This exporter has experimental support for instancing/referencing.
Dupli-object meshes are now written to USD as references to the original
mesh. This is still very limited in correctness, as there are issues
referencing to materials from a referenced mesh.
I am still committing this, as it gives us a place to start when
continuing the quest for proper instancing in USD.
== Lights ==
USD does not directly support spot lights, so those aren't exported yet.
It's possible to add this in the future via the UsdLuxShapingAPI. The
units used for the light intensity are also still a bit of a mystery.
== Fluid vertex velocities ==
Currently only fluid simulations (not meshes in general) have explicit
vertex velocities. This is the most important case for exporting
velocities, though, as the baked mesh changes topology all the time, and
thus computing the velocities at import time in a post-processing step
is hard.
== The Building Process ==
- USD is built as monolithic library, instead of 25 smaller libraries.
We were linking all of them as 'whole archive' anyway, so this doesn't
affect the final file size. It does, however, make life easier with
respect to linking order, and handling upstream changes.
- The JSON files required by USD are installed into datafiles/usd; they
are required on every platform. Set the `PXR_PATH_DEBUG` to any value
to have the USD library print the paths it uses to find those files.
- USD is patched so that it finds the aforementioned JSON files in a path
that we pass to it from Blender.
- USD is patched to have a `PXR_BUILD_USD_TOOLS` CMake option to disable
building the tools in its `bin` directory. This is sent as a pull
request at https://github.com/PixarAnimationStudios/USD/pull/1048
There are no more 'layers' in Blender. I chose 'Visible Objects' rather
than 'Visible Collections' to be consistent with the other
'{Renderable,Selected} Objects Only' options.
No functional changes.
This change ensures that operators which needs access to evaluated data
first makes sure there is a dependency graph.
Other accesses to the dependency graph made it more explicit about
whether they just need a valid dependency graph pointer or whether they
expect the graph to be already evaluated.
This replaces OPTYPE_USE_EVAL_DATA which is now removed.
Some general rules about usage of accessors:
- Drawing is expected to happen from a fully evaluated dependency graph.
There is now a function to access it, which will in the future control
that dependency graph is actually evaluated.
This check is not yet done because there are some things to be taken
care about first: for example, post-update hooks might leave scene in
a state where something is still tagged for update.
- All operators which needs to access evaluated state must use
CTX_data_ensure_evaluated_depsgraph().
This function replaces OPTYPE_USE_EVAL_DATA.
The call is generally to be done in the very beginning of the
operator, prior other logic (unless this is some comprehensive
operator which might or might not need access to an evaluated state).
This call is never to be used from a loop.
If some utility function requires evaluated state of dependency graph
the graph is to be passed as an explicit argument. This way it is
clear that no evaluation happens in a loop or something like this.
- All cases which needs to know dependency graph pointer, but which
doesn't want to actually evaluate it can use old-style function
CTX_data_depsgraph_pointer(), assuming that underlying code will
ensure dependency graph is evaluated prior to accessing it.
- The new functions are replacing OPTYPE_USE_EVAL_DATA, so now it is
explicit and local about where dependency graph is being ensured.
This commit also contains some fixes of wrong usage of evaluation
functions on original objects. Ideally should be split out, but in
reality with all the APIs being renamed is quite tricky.
Fixes T67454: Blender crash on rapid undo and select
Speculation here is that sometimes undo and selection operators are
sometimes handled in the same event loop iteration, which leaves
non-evaluated dependency graph.
Fixes T67973: Crash on Fix Deforms operator
Fixes T67902: Crash when undo a loop cut
Reviewers: brecht
Reviewed By: brecht
Subscribers: lichtwerk
Maniphest Tasks: T67454
Differential Revision: https://developer.blender.org/D5343
The collada exporter allows to export transforms either as Matrix
or as a decomposition of Translation, Rotation aand Scale.
The decomposition option was falsely named "TransRotLoc".
I renamed it to the much more descriptive word "Decomposed".
The tooltip already contains sufficient information, so there
is no need to change that.
When exporting an object we can choose the transformation type 'Matrix'
or 'trans/rot/scale' When exporting an animation we have the same choice
regarding the used transformation type.
However we must make sure that animations and objects use the same
transformation type within one colleda export. The user interface is
now reworked such that the correct settings are always guaranteed.
I also reworked the tool tips
Added new feature: Collada: global axis rotation upon export (UI)
The new feature allows to specify the target rest coordinate system upon export.
This allows for example to export a character that is in Blender orientation (Y forward)
to match the Secondlife orientation where (-X forward)
- Refactor:Added new utility methods to collada_utils
Made BCMatrix class more powerfull
moved Blender related structures into new BlenderContext class
added class wrapper to encapsulate ExportSettings structure
Added blender context getters to ExportSettings
added access methods to BlenderContext into ExportSettings class
Moved class BCMatrix into BlenderContext
moved utility functions from collada_util into BlenderContext
replace own function for parenting by a call to ED_object_parent_set()
- Cleanup: removed obsolete parameters from methods
renamed parameters for better understanding
cleanup whitespace and indentation
removed obsolete comments
The goal is to prevent assignment of temporary or evaluated meshes
to objects from the main database.
Majority of the change is actually related on passing reports around.
On a positive side there are more error prints which can become more
visible to scripters.
There are still possible further improvements in the related areas.
For example, disable user counting for evaluated ID datablocks when
assignment happens. But can also happen later on as a separate
improvement.
Reviewers: brecht, campbellbarton, mont29
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D4884
* The cache file datablock is now evaluated as part of the dependency graph,
creating/freeing the Alembic file handle matching the current frame.
Modifiers and constraints depend on this evaluation.
* Cache file handles and readers now only exist on COW datablocks, never the
original ones.
* Object data paths are flushed back to the original for the user interface.
* The cache file keeps a list of all readers associated with its handle, and
automatically frees them when the handle is freed. This kind of sharing of
data across datablocks is weak but we have no better mechanism for it.
Fix T62720: Alembic sequences not working and crashing
Differential Revision: https://developer.blender.org/D4774
No functional change, this adds LIB definition and args to cmake files.
Without this it's difficult to migrate away from 'BLENDER_SORTED_LIBS'
since there are many platforms/configurations that could break when
changing linking order.
Manually add and enable WITHOUT_SORTED_LIBS to try building
without sorted libs (currently fails since all variables are empty).
This check will eventually be removed.
See T46725.
BF-admins agree to remove header information that isn't useful,
to reduce noise.
- BEGIN/END license blocks
Developers should add non license comments as separate comment blocks.
No need for separator text.
- Contributors
This is often invalid, outdated or misleading
especially when splitting files.
It's more useful to git-blame to find out who has developed the code.
See P901 for script to perform these edits.
It's now possible to export curves and NURBS as mesh data to Alembic.
This allows artists to do any crazy thing on curves and export the
visual result to Alembic for interoperability with other software (or
caching for later use, etc.). It's an often-requested feature.
This works around T60503 and the fixes export part of T51311.
Note that exporting zero-width curves is currently not supported, as
exporting a faceless mesh (e.g. just edges and vertices) is not
supported by the mesh writer at all.
To test, create a curve with thickness (for example extruded), export to
Alembic and check the 'Curves to Mesh' checkbox in the export options.
Reviewers: sergey
Differential Revision: https://developer.blender.org/D4213
The Collada exporter suppresses the export of flat animation curves
to optimize the animation (in fact to make the exported file smaller).
But sometimes it is important to also have the flat curves exported
because they may be needed to define an initial transformation to
a fixed location - like translating the weapon from the ground floor
to the back of the model in the report.
I added a new option "all keyed curves" which is disabled by default
but when enabled it also exports flat curves.
feedback is very welcome
This is similar to what physics baking is doing: invoking the operator
runs a background job, whereas executing blocks. This makes Python
scripts calling the Alembic import/export operators more predictable.
For backwardward compatibility with existing Python code the
`as_background_job` parameter still exists, which overrides the
behaviour chosen by INVOKE/EXECUTE.
Reviewers: brecht
Reviewed by: brecht
Differential revision: https://developer.blender.org/D4137/new/
There were at least three copies of those:
- OB_RECALC* family of flags, which are rudiment of an old
dependency graph system.
- PSYS_RECALC* which were used by old dependency graph system
as a separate set since the graph itself did not handle
particle systems.
- DEG_TAG_* which was used to tag IDs.
Now there is a single set, which defines what can be tagged
and queried for an update. It also has some aggregate flags
to make queries simpler.
Lets once and for all solve the madness of those flags, stick
to a single set, which will not overlap with anything or require
any extra conversion.
Technically, shouldn't be measurable user difference, but some
of the agregate flags for few dependency graph components did
change.
Fixes T58632: Particle don't update rotation settings
Most important changes are in the Animation exporter and Animation Importer.
There is still some cleaning up to be done. But the Exporter/Importer basically
work within Blender 2.8
Some details:
User Interface:
The interface has been reorganized to look more like the FBX interface.
New options in user interface:
* keep_keyframes:
When sampling the distance between 2 keyframes is defined by
the sampling rate. Furthermore the keyframes defined in the
FCurves are not exported. However when this option is enabled
then also the defined keyframes will be added to the exported fcurves
* keep_smooth_curves:
When sampling we do not use FCurves. So we also have no Curve handles
for smooth exporting. However when this option is enabled, Blender
does its best to recreate the handles for export. This is a very
experimental feature and it is know to break when:
- the exported animated objects have parent inverse matrices
different from the unit matrix
- The exported objects have negative scaling
There may be many other situations when this feature breaks.
This needs to be further tested. It may be removed later or replaced
by something less wonky.
BlenderContext:
is a new class that contains the bridge to Blender. It contains
pointers to the current export/import context plus derived values
of Depsgraph, Scene, Main
Reporting:
I reorganized the output on the Blender Console to become more
informative and more readable
Preservation of Item names:
name attributes are now encoded with XML entities. This makes
sure that i can export/import names exactly defined in the tool.
This affects material names, bone names and object names.
Hierarchy export:
* Object and Bone Hierarchies are now exported correctly
by taking the Blender parent/child hierarchy into account
* Export also not selected intermediate objects
Problem:
When we export an Object Hierarchy, then we must export
all elements of the hierarchy to maintain the transforms. This
is especially important when exporting animated objects, because the
animation curves are exported as relative curves based on the
parent-child hierarchy. If an intermediate animated object is missing
then the exported animation breaks.
Solution:
If the "Selected" Optioon is enabled, then take care
to also export all objects which are not selected and hidden,
but which are parents of selected objects.
Node Based Material Importer (wip):
Added basic support for Materials with diffuse color and
diffuse textures. More properties (opacity, emission) need
changes in the used shader.
Note: Materials are all constructed by using the principled BSDF shader.
Animation Exporter:
* Massive optimization of the Animation Bake tool (Animation Sampler).
Instead of sampling each fcurve separately, i now sample all
exported fcurves simultaneously. So i avoid many (many!)
scene updates during animation export.
* Add support for Continuous Acceleration (Fcurve handles)
This allows us to create smoother FCurves during importing Collada
Animation curves. Possibly this should become an option ionstead of
a fixed import feature.
* Add support for sampling curves (to bake animations)
* The animation sampler now can be used for any animation curve.
Before the sampler only looked at curves which are supported by
Standard Collada 1.4. However the Collada exporter currently
ignores all animation curves which are not covered by the 1.4.1
Collada Standards. There is still some room for improvements
here (work in progres)
Known issues:
* Some exports do currently not work reliably, among those
are the camera animations, material animations and light animations
those animations will be added back next (work in progres)
* Exporting animation curves with keyframes (and tangents)
sometimes results in odd curves (when parent inverse matrix is involved)
This needs to be checked in more depth (probably it can not be solved).
* Export of "all animations in scene" is disabled because the
Collada Importer can not handle this reliably at the
moment (work in progres).
* Support for Animation Clip export
Added one extra level to the exported animations
such that now all scene animations are enclosed:
<Animation name="id_name(ob)_Action">
<Animation>...</Animation>
...
</Animation>
Animation Importer:
* Import of animations for objects with multiple materials
When importing multiple materials for one object,
the imported material animation curves have all been
assigned to the first material in the object.
Error handling (wip):
The Importer was a bit confused as it sometimes ignored fatal
parsing errors and continued to import. I did my best to
unconfuse it, but i believe that this needs to be tested more.
Refactoring:
update : move generation of effect id names into own function
update : adjust importer/exporter for no longer supported HEMI lights
cleanup: Removed no lopnger existing attribute from the exporter presets
cleanup: Removed not needed Context attribute from DocumentExporter
fix : Avoid duplicate deletion of temporary items
cleanup: fixed indentation and white space issues
update : Make BCAnimation class more self contained
cleanup: Renamed classes, updated comments for better reading
cleanup: Moved static class functions to collada_utils
cleanup: Moved typedefs to more intuitive locations
cleanup: indentation and class method declarations
cleanup: Removed no longer needed methods
update : Moved Classes into separate files
cleanup: Added comments
cleanup: take care of name conventions
... : many more small changes, not helpful to list them all
