with scene audio recursion in mattieu's gooseberry edit in sequencer.
Will probably cause something else to explode but let's just guard
against this in the audio lib for now.
The issue was caused by memory optimization marking particle system to recalc,
and because of the way how particle flags works it was possible that it'll
cause particle's re-distribution. Now this memory optimization will act the
same as loading the file.
Would need to review if this fix is to go to master/cycles_memory branches
but for now it'll be cool to figure out what to do with the farm.
Now file open operator also sets a temporary flag for collapse. This
allows the user to configure the option to always collapse files and the
editor will remember the setting and not reset it back after finishing a
collapsed file open.
This way we store quite reasonable amount of memory (density is stored as RGBA
textures, which ends up in gigabytes of RAM for hires smoke sims).
It's not really pretty implementation and we'll really need to support textures
with 1 and 3 channels instead of doing such packing, but we need to be able to
render some files here.
This happens if all object's smokes are backed to files and rendering is either
happening from command line of with locked interface.
The idea behind this change is to make as much memory available for path tracing
as possible, so after heavy swapping during synchronization period there's loads
of free memory to put important data back to RAM.
Notes:
* This is own coocking, since it seems hard to find papers on simplifying already existing bezier curves,
and we do not really need the 'generic' least-square fitting of bezier in a set of points, here.
* It takes advantage of specificities of FCurves (e.g. only difference that matters here is Y value at same X frame,
no vertical overlapping, etc.).
* It gives reasonably good results, but could most likely be enhanced quite a bit still.
* Only 'hooked' to bake action operator right now (needs more work to add it to graph editor too).
* Ultimately should probably be redone in C. Would keep it in Python until we have a real good
cleanup behavior though, much easier to experiment in later language.
Very dirty feature that will have to be reimplemented when the new asset engine makes it into blender.
This commit tweaks the filebrowser to detect sequences of images that form parts of a movie. The
files need to be in the format <number>filelame.ext or <number>.filename.ext
There is a new option next to hidden file option that makes it so this collapsing takes place.
When collapsing is on, any movie files that have the same filename will be collapsed to the first
detected file in the sequence. Selecting that file will select all files in the sequence.
There is a shortcut in the sequence editor in the Add menu, "Image Sequence" that enables collapsing
by default. Unfortunately, selecting multiple movies will not add multiple sequences to the sequencer,
at least for now, but it should make it quite easier for people to quickly select the whole range of
images for a movie. Importing the image sequence into the sequencer will use placeholders, so any missing images
will display as black.
Using ints and floats would require prefixing the Alembic metadata
keys, which becomes messy and ambiguous. Encoding other data types as
strings can be done on the python side as well.
properties.
This is necessary to distinguish e.g. vector semantics (linear vs. slerp
interpolation for positional/normal vectors). It would be simpler to
do this based purely on types, but the typing of common Alembic
properties is not sufficiently unique. Semantics arguments are enum
values, that can be omitted if the default interpolation should be used.
This features is needed for motion blur. All properties with position
semantics (transform matrices, mesh and curve vertex locations, etc.)
need to be interpolated for fractions of full frames, in order to
reproduce the motion and deformation correctly.
Alternatively we could store subframe time sampling for this subset of
properties. However, this would increase the amount of data stored,
and it might actually show artifacts from subframe interpolation in
Blender which is not visible to animators otherwise and should be
ignored.
archives, instead of Blender frame values.
This is easier to use for interpolation, the frame value is really only
required externally, before and after reading a sample.
This requires some modifications to the existing preset python class.
Serializing curve mappings can not be done with a simple property
assignment.
Also the fact that operator confirm popups drop all layout context
pointers means that we have to store the modifier name in the operator
in order to apply the preset to the correct modifier.
* Add space key to easily start-stop the player.
* Keep old behaviour of stopping when scrubbing.
* Sound scrubbing can be heard now even when scrubbing after pausing.
The previous way of selecting preview ("realtime") and render settings
on the cache library level was complicated and confusing. Now there are
just 2 buttons for baking either the preview or render level.
Technically the cache still works the same way, so a 3rd button could be
added for baking both passes into the same cache, should the need arise.
The preview cache now also uses the preview frame range, which should
help in quick testing of hair simulation settings. Render always uses
the scene frame range, like render buttons do.
For display in the viewport and rendering the cache will now use
whatever data is available in the archive. If the cache contains only
render data, the viewport will also draw full render data. Vice versa,
if the cache contains only preview data, this will also be used for
rendering.
Hair toggles for children and motion now are only used for the viewport
display. This simplifies the settings a lot and removes one potential
button to be overlooked.
cutting.
When using a dupli object from the same group as a target it is
sufficient to calculate the relative transform in the shared duplicator
local space. When using an external object though we have to transform
into world space first.
for child hairs.
The algorithm works by finding the first intersection of each child
hair with the target mesh. The child hairs have to be deformed with
their parents first (this usually happens *after* modifiers).
Support speed of sound based on how fps is different from the movie fps.
For high fps the picture thread can lag behind the sound still, need
some way to framedrop.
Basically it's a clean tool, but also removes a channel if the only
remaining keyframe has the default value only and is not used by
drivers or generative modifiers.
It's supposed to help with performance of the heavy scenes in
gooseberry.
Field test in gooseberry for now, masterification later.
Strand shapekeys will be interpreted as delta keys without a basis,
since the basis would be animated and therefore useless for static keys.
To avoid complexity in the already overburdened shapekey functions and
to skip old legacy code, the strands shapekeys now have their own
functions.
of an animated base.
This is necessary for strand shape keys. The basis of the data comes
from the cached animation or simulation results, i.e. we can't use the
fixed basis key or the result would always become static.
motion state.
This means shape keys work on the goal positions for hair as well as
the simulation results (hair motion).
If the simulation is created before shape keys are applied, the
resulting sim will be mixed with the shape keys, becoming more rigid
with increased shape weights.
OTOH the shape keys can also be applied to the animation result, before
doing hair simulation. In that case the shape will only affect the goal
positions and the bending shape of the hair, but the hair is otherwise
free to move.
gooseberry and alembic branches.
The shapekey code has been modified in both branches, adding particle
shapekeys in gooseberry and cache library shapekeys in alembic. The
two approaches are slightly different, but based on the same idea of
extending the 'from' owner info in Key, so that keys can be used without
a 'from' ID and/or with an index for identification.
This patch unifies the approach in both branches to make it slightly
less messy and avoid merge conflicts.
The shapekey code has been modified in both branches, adding particle
shapekeys in gooseberry and cache library shapekeys in alembic. The
two approaches are slightly different, but based on the same idea of
extending the 'from' owner info in Key, so that keys can be used without
a 'from' ID and/or with an index for identification.
for slices.
Also the time sampling for slice output properties must be passed down
from the output archive, since the input properties use the original
start frame and cycle times.
than relying on values from Scene.
This gives more flexibility when creating archives. A default variant
for opening archives using the scene settings is still available.
range to a new archive.
This feature is useful when cache size becomes to big and unwieldy.
For sending shots to a render farm a large cache can now be split into
smaller parts, which are spliced off and can be sent individually.
Still lots of corner cases to check:
* sound FPS not changing with speed settings of player
* picture and video still not 100% in synch
* Haven't checked what happens when audio is finished.
Some changes were made to make things nicer:
* Scrub now initiates on mouse click instead of mouse motion, makes
things more predictable.
* When scrubbing a movie with sound, don't pause playing after
scrubbing.
team.
There are 3 options here:
1) Keep range (previous behaviour)
2) Seconds - allows a specified offset in seconds around current frame
3) keyframes - zoom to include a number of keyframes around the cursor
Options 2 and 3 have their own properties to tweak the behaviour and all
options can be found in User Preferences->Interface under the 2D
viewports section.
Number 3 will probably need some refinement so commiting here for the
hwoozeberry team to test first.
This reverts commit 1b32ac9779.
The goal spring is not working as intended, but is better than having
no bending force at the roots at all. Needs a better solution (virtual
root vertex).
There are larger bugs still roaming free which need to be solved first.
Goal springs also are inherently unstable because the solver can not
properly estimate their Jacobian.
caches.
The 'find' function was returning a bool value, but this does not
guarantee non-null pointers. Removed the return value to avoid
confusion, now callers should simply check the returned pointers.
The filter group was used by omitting any object instances that were
not in the group. This reduces the cache size, but it also entirely
skips objects.
Now the filter group only prevents writing of the actual object data,
but still allows storing dupli instances and dummy entries in the cache.
Any objects from cached dupli lists that don't have cached data (mesh or
strands) will now use the original object data instead.
The cache bake operator would always create a job in the window manager.
However, when running in background mode via a script, the operator can
not be modal and the script would return right after starting the job.
Then Blender is stopped and the baking job with it.
To prevent this the operator now only runs in a job when executed from
the invoke function (or when 'use_job' is set explicitly).
shape.
Shape keys distinguish between the base shape (shape without any keys)
and the individual reference shapes for each shape key (which define
the delta offsets). For strand shapes, which have to work on top of
animation, the base shape must be the original cache result rather than
a static shape, but the delta for each keyblock is still calculated
wrt. an explicit shape.
group objects when editing shape keys.
The edit data (including CD_SHAPEKEY layers) is in the active object's
space, i.e. the duplicator. The original data however is cached strands
data, which is in the dupli object's space. All geometry data needs to
be transformed properly when entering/exiting edit mode.
This already exists for goal springs. Bending stiffness, however, is
preferable to global goal springs for physical realism. Controlling
stiffness in this way allows using bending forces to also simulate
unconventional hair, such as clumpy strands that are thicker and stiffer
at the base.
Note that hair interaction can also be simulated with this tool in some
way, although eventually better methods may be needed for such effects.
solver.
This happens when using goal springs, which are now defined as linear
springs between actual vertices and virtual goal vertices. The Jacobian
was not accounting for the possibility of zero rest length.
Both the gooseberry and alembic branches contain extensions of the Key
DNA structure, in order to make it work for particles and caches
respectively. This was giving badly resolved conflicts on merge.
function.
This is a feature for display in the viewport and rendering, which has
nothing to do with the core cache reading functionality. In the general
case you'd want the child data to remain unmodified.
Getting a consistent look here is difficult since editors apply their
own scaling. ideally, here we would have a consistent look with constant
font size but this is still to be realized in master and needs some
extra thought.
The parent matrix is used for single-parent children ('simple'), in
which case the pa->num_dmcache has to be used to get the correct
position on subdivided meshes.
This data is not strictly necessary for the strands drawing or rendering
or even hair simulation, because the hair root offset and rotation is
already cached explicitly. However, the strand edit mode needs this
information to correctly apply length constraints.
Strands require a valid DerivedMesh for calculating their root matrix
and surface mapping. The scheduler tasks were created such that strands
would be calculated while the DM task was not finished yet, leading to
missing DM data.
object.
The strand edit mode uses the local space of the active object, which
is the duplicator (and dupli cache owner). The strands data is in local
space of the original particle system object however, so have to
convert. This is a very hackish solution, using the first instance of
the strands data, which only works for single instance data.
Currently strand edit mode tool use the object data 'x mirror' flag.
This flag is only available on mesh objects, but the strand editing
can be used for dupli override caches etc. as well. Eventually strand
editing should get it's own independent flag to avoid this problem.
separate file.
This may seem a bit like overkill, but it helps ensure that no particle
depedencies messes up the strand editing code. The same will be done for
other use cases of the strand editing code in the future.
Now shape keys don't replace the cached base data entirely with the
static Basis shape. Instead the current state as coming from the cache
is interpreted as the variable refkey instead of the Basis key.
The Basis key is in fact redundant, but removing it would require
changing a lot of shared shapekey code, so it's left in for now.
The Key struct in the cache strands shape modifier is in fact "owned"
by the modifier, but linked in the Main blenddata, so it does not have
to be written explicitly.
the source cache and then applies modifiers afterward.
This only performs non-iterative modifiers, which don't require the
passage of time to work (e.g. shrinkwrap).
This includes storage of the shape keys in a `Key` instance in the
modifier data, and accompanying operators for adding, removing, moving,
etc. shape keys, as well as the necessary UI code.
pointer for 'from'.
The Key DNA has some very old and largely deprecated legacy code that
needs initializing, which requires either explicit info or deduction
from an ID. The cache modifier creates keys in a new environment without
an existing known ID type to deduce the data element settings from.
cached strands data.
This modifier acts as a container for shape key data (i.e. as part of
the cache library). Such shape data can be applied to cached strands
data, which allows tweaking of animated hair shapes.
This could work both as a shape for the strand goals (adding dynamics
on top) or as a final modification after dynamics. Both methods have
their advantages (pre-sim allows control and sim at the same time,
post-sim gives accurate shapes but becomes rigid).
color layers.
The layer data is added to the strands after creation, but can be
existing already. It should only be reallocated when the number of
layers changes, and must be freed cleanly in that case.
When using external forces to implement goal springs to explicit
positions in space, the system cannot calculate a stable solution based
on the goal velocities. The new approach implements goals as vertices
in the simulation, which are constrained to their animated locations.
This way they are treated like regular vertices, but the springs only
affect their non-constrained counterpart (the actual vertices).
When using the hair path deformation feature of the modifier, each copy
of the mesh can now be rotated around the path, in addition to changing
length and offset. This includes a constant phase angle as well as a
randomization factor.
simulation.
The hair sim skips the first frame (velocities are undefined there), but
still has to create the motion state data in order to have it written
as a sample. Otherwise the strands motion state samples start one sample
short and frames get mapped to one sample in the past. This then leads
to incorrect movement of all hair vertices other than the roots (which
are always clamped to the animated positions).
This fix should actually happen in wiggly-widgets branch but it's now
has some commits which are not in gooseberry and merging them night
be not really safe.
with a dupligroup and distributed rendering.
The particle duplis were generated using the global RNG (BLI_rand). This
is unstable once scenes become a bit more complex and objects are
evaluated in multiple threads.
This patch is not in master, because it changes the look of existing
particle setups.
This is experimental. It might work for simple collision-like features,
by keeping goal positions for hair above the mesh surface. However, the
nature of nearest-point lookup on meshes means that the goal position
can flip to the other side of a mesh rapidly, which would lead to
strong forces and unstable behavior.
This code should be seen as an experiment.
The UV and vertex color attribute export function expects the curve data
arrays to contain only a single attribute at a time. They must be
cleared before filling in data for each UV/vcol attribute.
content.
This group is only used for the baking operator (like the datatypes
filter), but has no effect when reading caches. The UI now uses a box
layout to reflect this.
Unlike particles, the Strands data stores UV coordinates and vertex
colors explicitly as a per-strand (const) property. This means Cycles
does not have to know the details of how particles map themselves to
a changing mesh surfaces (which is very complicated and fraught with
broken use cases and bugs). The UVs and colors are stored once for each
strand, since they are not expected to change over time (strands being
fixed to the mesh surface).
The idea is to evaluate derived meshes in multiple threads, which will
reduce overall write_sample() time. We can't make all routines threaded
in there because alembic's file IO can't be threaded.
In any case, this change gives 2x speedup of exporting render resolution
version of Franck here.
Experiment for animators to try - only do full notification of scene at
end of scrubbing, do only manual area update instead like we do for
animation. Also skip audio update unless we do audio scrubbing.
Alembic exceptions and prevent crashing.
Alembic can throw exceptions on relatively common and uncritical errors,
such as mismatching properties in files which don't fit the expected
schema. These cases should now be handled gracefully and simple reject
the cache file with an error message.
Customdata layers also store some base information, such as "active"
layer for rendering. This information is needed currently for syncing
particle UV and MCol data in cycles, even though the MTFACE layer type
itself is pretty much deprecated.
This otherwise prevents hair simulation motion showing up in renders.
The Alembic structure will have to be redesigned anyway. The current
concept of storing two separate entire scenes for viewport/render will
be replaced by a per-object detail level system. This allows performing
hair simulation only once and still use different resolutions etc.
where appropriate.
The idea is simple: make it so scene_update_for_newframe is only doing updates
of the stuff which is really needed for the currently baking group.
Implementation is a bit tricky since we don't have parent relations after the
DAG is built, so doing some graph traversal there.
This code is also now using simplified version of scene_update_for_newframe()
which means in theory we can try de-duplicating some pieces of code, but that
can be done later.
Additionally, the same approach can be used to optimize motion path calculation.
Apparently, even if the smoke is backed to an external files it still could be
tried to be simulated at the frames outside of the baked range.
Kinda weird feature which isn't really safe, but better not to crash here.
Not totally happy with the code yet, will check with Lukas or Daniel how it
could be improved further.
influence explicitly.
The previous code would only allow using the layers of the given object.
This is not useful for dupli overrides, because even finding which
layers an object in a dupligroup is in is cumbersome. It makes more
sense to use the layers of the duplicator instead.
There is an '_ex' version of the pdInitEffectors function now, with
an explicit layers argument. The simple version now also skips the
'precalc' argument, because this was true in every case except the
depsgraph dependency building anyway.
step.
This calculate can take a bit of time (some seconds to a minute in
tests), due to having to loop through all array samples. In many cases
only the basic structure is wanted.
The issue was caused by RNA passing ownership from the cache data used by Blender
to Cycles. This lead to situations when all of a sudden blender looses data it
was expecting to have.
Now instead of passing ownership we're just copying strands from the data, so both
Blender and Cycles are having it's own local data.
Ideally this data will be shared between viewport and Cycles, but that's a bit
more tricky to do without modifying RNA API. Would happen eventually tho.
A new panel is added for showing the structure and contents of archives
of a cache library (instead of simply dumping on the terminal). The
archive structure is stored in a lightweight tree structure, mirroring
the hierarchy of objects and properties in Alembic. These
object/property nodes can be expanded individually for easier navigation
through the archive.
This implements roughly the same functionality as the equivalent mesh
shape propagate operator, with some tweaks specific to hair editing.
The operator copies selected points to all the shape keys. Unlike meshes
these points may be shifted afterward to achieve consistent segment
length (this can be avoided by selected entire strands before using the
operator).
when switching the active key.
Particle shapes were always using the relative blending of keys when
evaluating hair data. This works ok for meshes, because their edit data
is entirely separate from the mesh data (using bmesh). For particles,
however, the edit mode is hardwired to the hair data, so when switching
keys in edit mode we must take care to replace everything with the
shape. Otherwise the hair data will be using blender shapes and the edit
data will overwrite any shape keys with that.
Placeholder images, means that the image sequence reserves a range for
images, displaying black while we wait for them to finish rendering.
This is meant as a feature to not break the layout of gooseberry
sequence edits while we wait for those frames to arrive.
This commit replaces name-based node matching in the mesh device update
with check of special type. It is more robust approach for adding new
nodes which could use image slots for storage. it's also adds updates
of environment texture nodes used in bump shader (this was missing in
the original image sync reshuffle commit).
Note that currently this has virtually no effect, because the mirror
option relies on exact positions of vertices, which does not happen with
random hair placement (the add brush has no mirror option yet).
Eventually topological mirroring should help with this case, but is not
implemented for either old or new strand edit yet.
These are a modified version of their BMEditMesh counterparts.
use_topology is not yet implemented for strands. Native strand topology
is not very useful for this. Instead, the topology of the scalp mesh
should be used for finding mirrored strand roots, then the arc- or
parametric length of a vertex from the root to find mirrored verts.
Conflicts:
source/blender/blenkernel/BKE_editstrands.h
When copying mesh data to bmesh the MVERT and similar customdata types
have to be omitted. Otherwise the bmesh instance ends up with NULL
pointers in customdata layers, but entries in the typemap != -1. The
effect was that when storing new steps after one or more undo, the
resulting original data would be copied, and subsequent undo steps
are ignored.
data masks explicitly.
A dummy mesh is used for strand edit undo storage like in mesh edit
to prevent unnecessary code duplication. However, when copying from/to
BMesh only the mesh data layers are copied by default, omitting the new
data layers for strands (currently only MSurfaceSample hair root data).
This uses the generalized undo stack system which is also used for
object data edit. An extension is necessary to tell the undo functions
which object is specifically used for generating undo data and
identifying the stack, since strand editing does not edit the obdata
itself.
consistent segment lengths when transforming vertices.
Warning: The implementation is not correct yet, but all the steps should
be there.
The main idea is to treat strands as a sequence of joints that are
displaced out of their original locations by a transform or other tool.
The solver then tries to find a global per-strand solution that keeps
the segment lengths unmodified, with a minimum change in angles from
the original starting shape. Such a solution is much more usable and
efficient than the current O(n^2) attempt of "spreading the error"
across the strand.
The inverse kinematics method is very flexible. It can also include
stretching, which would be very welcome for features like the length
tool. Different parts of the strand could be weighted separately using
scaling factors for the angle/stretch parameters.
Conflicts:
source/blender/physics/intern/implicit.h
mode.
This method is simple, but not really very usable. It works by
successively relaxing segments that are too long or too short, moving
both vertices along the edge between them. This is repeated N^2 times
(N: number of vertices on the strand).
A true IK solver could give a lot better results, as well as providing
many opportunities to apply weighting for targets (e.g. preferring to
move non-selected over selected vertices). Many different methods for
simple IK solvers exist, so there should be one that works well for
large number of simple strands. See e.g.
http://www.math.ucsd.edu/~sbuss/ResearchWeb/ikmethods/iksurvey.pdf
vertex positions.
This works a lot better with strokes perpendicular to the general hair
strand direction. With the previous comb tool such strokes would only
make a small dent in the hair curve and then vertices would slip out
of the tool circle. The edge combing affects the local direction of
strands, which acts as a kind of grabbing functionality by moving
further vertices in front of the tool circle. The result is that
drawing a curvy hair shape with the comb becomes much easier.
In addition, the new tool also uses edge filtering and weighting rather
than vertices. This means that small brushes also work well, instead
of having no effect when hitting the edge segments between vertices.
Further improvement could be achieved later by using a global strand
solver, which adjusts vertex positions based on an error metric along
the whole of the strand.
direction vector for the hair stroke tool.
This is necessary for directional tools such as combing, where the
stroke direction can lead to unwanted results if it changes too
abruptly.
The rationale behind this is that the BKE code could be used for
modeling hair and fur as well as a number of other features such as
grass. The primary addition to BMesh is the limitation to strand-like
topology (simple vertex chains, optionally rooted on a mesh surface).
The editor code OTOH is quite hair specific, since the result should
be suitable for hair simulation, and the workflow should mimick actual
hair grooming intuitively. Eventually the hair edit mode could become
a generalized strand edit mode with slightly different tool sets for
various purposes, but for now it is quite specifically built for hair.
Conflicts:
source/blender/blenkernel/CMakeLists.txt
source/blender/blenkernel/intern/particle.c
drawing code for edit mode.
The lighting uses the same method as the particle hair, interpreting
strand direction as the normal. This is nowhere near realistic hair
shading, but doing this with line shading alone in the fixed-function
pipeline is probably not possible. A GLSL shader could be used instead
for a more realistic alternative as part of the viewport project. At
least this simple shading mode gives some orientation feedback while
editing hair.
Currently uses the same approach as old particle edit mode (rescale
hair segments from the root on). A more sophisticated approach using
least-square error minimization of the displacement could yield better
results.
Stroke tools will be categorized by the hair elements they operate on:
vertices, segments or strands (roots). In addition to that a filter
function defines the influence of the brush. This should be defined by
the other brush settings and be largely independent of the main tool
mode.
This requires converting the old messy particle num/num_dmcache/fuv/foffset
data into the new mesh samples, which can potentially introduce floating
point errors and inaccuracies due to lack of face index mapping in the
new system. However, in any well-constructed particle system the hair
roots should be nearest to their num face, so mapping would be accurate
enough. If necessary face index data could be added to samples as a
legacy code hack, but probably it's best to eventually replace the
hair system as a whole anyway.
Currently particles only have a single weight value, controlling
pinning in the simulation (for root and virtual root verts) and goal
force scaling. This will be replaced by a full vertex group weight
system eventually.
This is more for testing purposes, since currently there is only a
single mass property for the psys as a whole. This should change in the
future though, to allow variable mass per strand or vertex.
Conflicts:
source/blender/bmesh/intern/bmesh_interp.c
Hair/Strand editing will only use a subset of the bmesh topology and
expect a specific topology that needs to be verified and enforced.
However, this extra requirement is much less work than reimplementing a
whole edit data system with the same feature set as bmesh and avoids
much redundant code.
Conflicts:
source/blender/blenkernel/intern/customdata.c
source/blender/makesdna/DNA_customdata_types.h
This means using mempools to store curve and vertex data, which allows
arbitrary addition and removal of data more easily. Also this includes
an iterator system similar to bmesh iterators (although the simpler
topology makes it a lot less complex).
Conflicts:
source/blender/blenkernel/intern/customdata.c
source/blender/makesdna/DNA_customdata_types.h
particle hairs remain visible.
Note that currently the hair root location (num/num_dmcache, fuv, foffset)
is not stored from edit data, so all hairs end up in a default location.
This makes it work more like editmesh, and avoid the awkward and
basically bad-level approach in particles, where the edit data is an
anonymous pointer in particle systems together with a callback for
freeing.
Conflicts:
source/blender/blenkernel/CMakeLists.txt
sampling system to keep the code simple.
Now there is a MSurfaceSampleStorage struct that encodes the storage
details, which the algorithms don't have to care about.
Disabling group -> duplicator dependencies would in theory make caches
more attractive because the invisible objects don't have to be updated
just because of the group.
However, the viewport and render starts to behave unpredictably without
these updates, because the dupli cache relies in many ways on the
objects it is supposed to override.
This option makes it so BVH does not use pre-aligned storage for triangle vertex
coordinates which was originally needed for faster coordinate lookup when doing
triangle intersection test. This array gives up to 10% performance comparing to
fetching individual coordinates but it also used 12 floats per BVH primitive,
which might translate to quite huge array in a complex scene.
Intention of this option is to investigate if this is a right direction to make
gooseberry files being able to render on local farm which is not totally great
in memory.
Current approach is not totally cheap, meaning even the case with the storage
enabled might be slower, but currently it's within 1%.
This works like the existing effectors when using "surface" mode: It
selects the closest point on the mesh surface and creates a force based
on distance and falloff.
Unlike the old effectors it also supports signed distance, based on
the surface normal, which gives more control over the allowable distance
of strands to the mesh surface.
cached group itself.
This allows using the dupli group objects themselves as force fields,
in particular for collision objects in strand simulation. Without this
feature only the original objects of are recognized by the effectors
system, and even if duplis worked properly their settings would not be
accessible due to linking and dupli group hiding. The cache modifier
circumvents this issue by providing a new force field system, separate
from the current effectors.
These classes were used previously to write parts of the Blender DNA
directly to Alembic. Their code is shared with other classes that are
still in use, so none of the really substantial parts have been deleted.
This only removes trivial C function wrappers for parts of the C++ code
that are not used directly any more from outside the bl_pointcache
library. Some internal parts may be removed later, this cleanup is
mostly to reduce code noise.
Using two full scene nodes for render/realtime data creates a lot of
duplication in both storage and processing time. Instead storing
specific data types like meshes in different versions keeps duplication
to the necessary minimum and allows easier fallback where available.
There used to be quite huge 3D arrays stored in memory which are only needed for
simulation steps and not used at all if the smoke is baked to file. That memory
is unmanaged by blender and not visible in the info space header and in practice
they could easily be gigabytes for hires smoke.
This commit only addressed hires smoke since it's the more important at this
point, and non-hires smoke wouldn't have such noticeable benefit.
In the case of tornado file it gives around 20% memory saving (which is about
26GB vs. 33GB on later frames of 01.03.02.A3 shot.
This means no motion state data is generated for render mode, this
requires some larger changes to the way render/realtime data is stored
in the caches.
Before this the hair sim would be applied indiscriminately to all the
strands in the cache data. Now an object/psys combination from the dupli
cache must be selected. Note that this is not the actual target of the
hair simulation, which still operates on the cache overrides instead.
The Object/Psys only functions as a selectable key, if no matching data
is found in the cache no hairsim will be applied.
This way we can get rid of inefficient memory usage caused by BVH boundbox
part being unused by leaf nodes but still being allocated for them. Doing
such split allows to save 6 of float4 values for QBVH per leaf node and 3
of float4 values for regular BVH per leaf node.
This translates into following memory save using 01.01.01.G rendered
without hair:
Device memory size Device memory peak Global memory peak
Before the patch: 4957 5051 7668
With the patch: 4467 4562 7332
The measurements are done against current master. Still need to run speed tests
and it's hard to predict if it's faster or not: on the one hand leaf nodes are
now much more coherent in cache, on the other hand they're not so much coherent
with regular nodes anymore.
Reviewers: brecht, juicyfruit
Subscribers: venomgfx, eyecandy
Differential Revision: https://developer.blender.org/D1236
when not needed.
The pathcache data is only required in the first frame to construct the
child strand shapes. For subsequent frames the children are calculated
using the initial shape and parent deformation, so particles don't need
to recalculate all the time.
This gives a significant performance increase of the baking process when
using a lot of complicated child hairs.
The basis for this optimization is that child strands have a shape that
does not actually change over time: the particle code does not enforce
this formally, but in fact all the clump/kink/roughness algorithms take
great care to not introduce changes of shape over time.
The final look of child hairs is achieved only by following the parent
hair deformation, i.e. by applying the offset of the matching parametric
position on the parent strand.
This means that for caching child strands it is sufficient to store the
full vertex data only in the first sample, and then apply the parent
deformation on subsequent frames using only the animated root matrices.
In test cases this seems to reduce the cache size to something between
10-20 % of the original size (down to some hundred MB from some 10 GB).
Further optimizations could be achieved by using smaller data types,
such as quaternions instead of 3x3 or 4x4 matrices for the remaining
child data.
Particles use 0..100 range for parent particle "times" (curve parameter)
and 0..1 for children. For caches this is now uniformly calculated in
the 0..1 range to avoid breaking child interpolation.
caches.
This allows child strands to follow the motion of parent strands as
calculated by the hair simulation. Unlike the particle system path
caching, this method does not regenerate all the children every frame.
Instead, it relies on an initial child setup that defines the base
shape, then applies the parent motion as a weighted linear deformation.
The result is in fact the same as with regular particle clumping, kink,
roughness etc.. Particles also take care to not change the child shape
when the parent moves, but this is not formalized anywhere (which makes
changing particle code quite difficult). So ignoring the particle code
is both permissible and efficient.
The idea is to avoid having original BVH and BVH packed into scene BVH and hence
reduce memory footprint.
This change doesn't give any measurable differences on it's own, but together
with D1215 and D1217 it gives quite nice results.
Reviewers: campbellbarton, brecht, juicyfruit
Subscribers: eyecandy
Differential Revision: https://developer.blender.org/D1218
This way memory overhead caused by the BVH building is not so visible and peak
memory usage will be reduced.
Implementing this idea is not so straightforward actually, because we need to
synchronize images used for true displacement before meshes. Detecting whether
image is used for true displacement is not so striaghtforward, so for now all
all displacement types will synchronize images used for them.
Such change brings memory usage from 4.1G to 4.0G with the 01_01_01_D scene
from gooseberry. With 01_01_01_G scene it's 7.6G vs. 6.8G (before and after
the patch).
Reviewers: campbellbarton, juicyfruit, brecht
Subscribers: eyecandy
Differential Revision: https://developer.blender.org/D1217
The idea is to give artists a simplier way to control memory usage in such
scenes as grass fields by doing automatic object culling based on whether
they're visible in the frame or not.
This is controlled on per-object level. In order to use this option few
steps are required:
- Allow renderer to use camera culling (Performance panel of render settings)
- Set camera cull margin (measured in relative value to the render resolution)
- Enable Camera Cull for objects which are desired to be culled
(object culling option could be found in Option panel in object buttons).
Interface decisions are not final for sure and to be refined if this
option is considered good enough for production.
Reviewers: juicyfruit, brecht, campbellbarton
Subscribers: venomgfx, eyecandy
Differential Revision: https://developer.blender.org/D1230
popup and clipboard output.
For a production-size archive the info string can become really large,
and putting it into a buffer takes a long time. As a debugging tool the
stdout printing is sufficient for now and can be done as a stringstream.
Arbitrary wind force "density" factor has been moved out of the force
function now, so cloth simulation can do this on it's own without
affecting the strand sim.
In addition to the original particle hair sim settings the parameters
for strand sim have a curve to define the strength of goal springs along
strands. This useful because currently there is no way to override the
weights of strands. This would require a new particle edit mode that can
work with non-particle data in a sane way ...
This takes an arbitrarily large amount of space in the UI and does
currently not have a good purpose.
Eventually this information could be displayed in a dedicated part of
the UI, with proper scrolling.
This code works much like the main object drawing code and has to take
the same precautions regarding replacing the derivedFinal pointer in
Objects. This pointer may be replaced during drawing calls, regardless
of the "ownership" indicated by needsFree.
Reading from an empty object in Alembic will throw an exception, which
is not easily handled yet. So rather avoid annoying crashes the clumsy
way for now.
The code for syncing Strands and StrandsChildren data to Cycles curves
is almost identical, so to avoid code duplication a little templated
traits type has been defined, which provides the appropriate accessors
for both cases. This is very local and should not be too confusing.
This creates child strands along with the parents. These are not yet
deformed by parents or simulation, and are also not written back into
subsequent caches yet.
data.
Child strands have their own struct type for vertices and curves to
avoid unnecessary overhead (parent curves store a full 3x3 matrix after
all).
Child strands may be constructed from particle pathcaches, or read from
curves data in alembic files.
These are dummy values currently. The UV and vcol values will be stored
in strands directly, unlike particles which evaluate them every time;
this is a much safer and more efficient approach that makes strand data
truely independent from the "emitter" mesh. Such data will be made
available later, primarily for child curves.
Still buggy, but the basic workflow is there:
When a cache library is used in conjunction with a group duplicator and
it has strands that match the particle systems, Cycles will use the
new Strands structure to construct its internal curves.
This is not strictly needed because the strands data is only created
at runtime at this point. However, it might be useful to later include
this in blend files.
Also the C++ RNA API has a bug that makes non-DNA collection properties
difficult (the length function is not implemented properly).
particles.
This is currently the same as the ParticlePathcache writer class, but
will be modified somewhat later to store additional information such as
parent weights. The purpose of this cache object is not to populate the
particle child caches, but to store renderable data associated to dupli
cache strands.
This is a separate struct to avoid storing unnecessary data (this means
a bit of code duplication for iterators). The child strands can be used
as the actual render data, while the parent strands usually would be
used only for simulation/deformation purposes.
This matrix is needed to create a stable and smooth coordinate system
along the strands, for things such as bending forces and child hair
deformation. The base matrix is defined on the "scalp" mesh and should
be stable wrt. deformation and animation of the mesh. For any point on
the strand a matrix is then calculated using parallel transport, i.e.
by rotating the base matrix successively along each curve corner.
The overall world space matrix for dupli sims has to be the combined
obmat of the cached object as well as the duplicator for the dupligroup
instance. The cached strand data is generally in object space, so the
matrix has to be applied to get all the forces and interactions right.
This compensates the transformation of the dupli instance being cached,
so that forces such as gravity are in the correct reference frame.
Note that this does not yet generate fictitious forces resulting from
a moving and/or accelerated frame of reference. For this we would also
have to calculate linear/angular velocity and acceleration of the dupli
object.
This is the first actual drawing/display setting for smoke, so a new
panel "Display" has been added to give it a home in the UI.
The name "thickness" has been chosen deliberately to distinguish it from
density, since the parameter only affects OpenGL drawing but not the
actual physical density values.
The strong stretch forces lead to awkward lengthening of segments when
the damping factor is too high.
In fact, in a natural hair strand the stretching usually means the hair
itself is curly and stretches as a helical spring. In that case
the damping is quite small compared to the stiffness, and little energy
is lost to the stretching (the hair becomes "bouncy"). When the hair is
very stiff the damping should also be small in order to avoid extreme
forces resulting from the hair root location constraint.
The solver can calculate the de-facto velocity of the root using the
difference of the already-updated vertex location and the state from the
previous step.
This will allow debug visualization data to be stored inside caches
along the actual simulation state.
While the regular simulation allows examining simulation data on-the-fly
by stepping through the regular timeline, the new baking mechanism used
for caches does not easily allow simulation data to be visualized frame
by frame. Caching this data will allow adding it to the global sim debug
data.
This would also still be very handy if the baking process itself becomes
more interactive again. Unlike the current simulation progression, the
cached data can be accessed randomly, so scrubbing throught the timeline
becomes possible.
This color is currently only displayed in the viewport (when enabling
"Texture" color mode in the Display settings). It will be used for
controlling the smoke color when using particles for smoke emission.
Conflicts:
source/blender/blenkernel/intern/particle.c
source/blender/editors/space_view3d/drawobject.c
source/blender/makesdna/DNA_particle_types.h
source/blender/makesdna/DNA_texture_types.h
source/blender/makesrna/intern/rna_particle.c
This extends the point density evaluation to also store RGB color values
in the array provided by Cycles.
Since internally all Cycles textures are 4-float RGBA textures anyway,
it does not make a lot of sense to store density and color separately.
This could eventually be preferable to avoid unnecessary storage.
This is basically copied from existing cloth sim settings, with a good
deal of cleanup and sanitization.
Unlike cloth parameters, the damping values are described as relative
to their associated stiffness values for springs. This is much easier
to use in practice and less prone to lead to unstable simulations.
Damping simply works best when the force is in the same order of
magnitude as the stiffness, so having a relative factor requires much
less adjustment on the user side.
Also renamed StrandSimParams to HairSimParams, since strands are only
the agnostic data structure, while these parameters are specifically
used to simulate hair-like behavior. Other simulations could use strands
but implement completely different kinds of physics (e.g. grass, ropes).
Allows mattieu's masterplan to keep overdrop at higher sequencer space
and all strips lower.
This won't align text so well though, but will leave it simple and he
can give feedback later.
This function is used in every part of the code, and has been causing
great pain when deleting 15 hours worth of baking in an instant.
The complexity of the code makes it unfeasible to fix this selectively,
so for now simply disallow the code from deleting any of the files.
For selected well-defined cases the deleting can be reenabled later,
e.g. when the user presses the `Bake` button and confirms deletion in
a popup.
Seems like there is currently now way to get the full data path
reliably. So instead of the old "..." this patch just displays
(null) which also isn't really useful.
modification.
The bake operator will now generate output based on the 'source_mode'
setting:
* When the source is SCENE it uses the basic object data as stored in
the Object ID datablocks (DerivedMesh, particle hair strands etc.)
* When the source is CACHE it fills a temporary DupliCache with data
from the cache archive (like it would for the display dupli_cache,
but also supporting render data). This dupli cache is carried over
between frames during baking, which allows modifiers to act as
simulations and sequentially advance motion states.
This commit implements point density texture for Cycles shading nodes.
It's done via creating voxel texture at shader compilation time, Not
totally memory efficient, but avoids adding sampling code to kernel
(which keeps render time as low as possible), In the future this will
be compensated by using OpenVDB for more efficient storage of sparse
volume data.
Sampling of the voxel texture is happening at blender side and the
same code is used as for Blender Internal's renderer.
This texture is controlled by only object, particle system and radius.
Linear falloff is used and there's no turbulence. This is because
falloff is expected to happen using Curve Mapping node. Turbulence
will be done as a distortion on the input coordinate. It's already
possible to fake it using nose textures and in the future we can add
more proper turbulence distortion node, which then could also be used
for 2D texture mapping.
OSL is not currently supported. There's no actual stoppers for support
it, but so far it's not really a priority at this point.
Reviewers: campbellbarton, juicyfruit, brecht
Subscribers: campbellbarton, dingto, eyecandy
Differential Revision: https://developer.blender.org/D1208
Note that the fixed-function OpenGL pipeline does not support true
strand normals. The shading model used copies the method implemented for
particles, using the edge directions as normals. This is totally
unphysical but at least gives some indication of direction. Viewport
refactor and programmable shaders are needed to make this work nicer.
This is used to prevent empty dupli lists when reading the cache fails.
In that case the duplilist function will now revert to default scene
evaluation.
libraries.
Having a cache archive output in each modifier is not really practical.
Now the cache library has at most 2 file paths. These are used based on
2 associated settings: source mode/path and display mode/path.
* The SOURCE mode determines whether the original scene data is used as
input or a cache archive. If the scene input is used the dupli group
objects will be evaluated as usual with Mesh data, modifiers, proxy
armatures, etc.. With cache input the data stored in a cache is used to
override the scene data instead.
* The DISPLAY mode is essentially a toggle for the whole cache modifier
stack. If it is set to 'source' the respective source data is used
without further modification. If set to 'result' the data from the
output cache archive is used, which can be generated using the bake
operator. During baking the data will be passed through the cache
modifiers to create a variation of the original source data.
Now the filtering is simply based on the data types that should go into
the cache, instead of selecting each object and component individually.
This is slightly more limited and may need to be revisited again later,
but for the time being it is much less confusing and clumsy. Filtering
by objects can be accomplished by creating groups accordingly. Making
groups specifically for caching is totally acceptable.
The draw code was restoring the derivedFinal mesh unconditionally after
drawing duplis. However, the drawing code can in some cases replace the
derivedFinal pointer during drawing (bad!). This then leads to invalid
pointer access.
Fix is to only touch the derivedFinal when it's actually a dupli cache
mesh.
The job invocation should happen only in editors/. Also it makes sense
to modify the bake operator such that it supports both the basic cache
library and an optional modifier output.
The design for this area is in progress anyway, probably the cache
library archive paths will be changed so that there is always one input
and optionally an output path, with all the modifiers working in
sequence.
Theoretically we could write a stitcher that writes only the parts of
the data affected by a modifier and simply copies the rest, but this
is much more involved and not necessary at this point.
Unlike the particle hair writer this one does not work with the particle
system data but the new Strands data. This will allow successive
manipulations of cached hair data by first storing the initial particle
grooming data ("hair edit"), read this data from the cache into a
Strands instance, then apply hair simulation and write the result back
into a second Strands cache.
Cache baking means the cache modifier should read input caches over the
given frame range and produce the correct output for writing into the
output cache file.
Cache modifiers can optionally implement a 'bake' function. This is used
through a bake operator from the cache modifier panel. Baking is
executed in a job, which will help with cancelling bakes and displaying
status info.
This is implemented badly: when using disk caches i reads the *entire*
cache from the disk to create the trail. In the process it also clears
the cached_frames array, which makes it look like the cache was deleted.
dupli cache.
This makes the cache modifier stack work more like actual modifiers:
Each will take the previous result and write it into its own output
file. The last output is the final result used for viewport display.
When using a particle inflow object, there is now an option "Set Color"
which uses particle texture color to override the uniform smoke emission
color.
Particles must have one or more textures with the "Color" influence
enabled (the resulting color can be visualized in the viewport using the
"Texture" display option). The color will then replace the default smoke
flow setting and allows variable color per particle.
Combined with "Strand/Particle" texture mapping this can also be used
to change the particle emission color over the lifetime of particles or
make variable colors for each particle in the system.
Note that this currently does not work with child particles.
This color is currently only displayed in the viewport (when enabling
"Texture" color mode in the Display settings). It will be used for
controlling the smoke color when using particles for smoke emission.
The rationale here is that we don't try to replace particle data
temporarily like we do with DerivedMesh. This would be needed for
drawing cached data overrides in the viewport and pass them to renderers
(unless these read Alembic directly) and later for performing dupli
simulation.
This data will be used for dupli simulations later on. Child path
generation could also be done based on this data without involvement of
particle systems.
This feature is mostly useful for the "view selected" operator. It is
also used in the "set origin to geometry" operator, but since the
cache overrides the object geometry anyway the effect may not be what
users expect (which is acceptable).
Note that Cycles currently only supports tessellated triangle/quad
meshes. Alembic PolyMesh generally has ngons, so external tessellation
is required until cycles gets a proper tessellation implementation of
its own.
standalone.
The cycles XML files now can refer to Alembic (.abc) files. This will
call the default alembic reader to read in scene data. Currently it
simply prints the Alembic file structure.
Eventually a proper schema needs to be defined for both xml and abc.
Also care has to be taken to handle potential conflicts between settings
both within xml/abc and between them.
This adds Alembic to the standalone application version of cycles.
Files can be specified as XML or Alembic, or use automatic detection
based on the filename extension.
Currently the Alembic reader just dumps the file structure to stdout
as a test.
This means we don't bluntly store all the data in a group, but use the
selection from the cache library. It also helps to avoid issues with
object visibility which is not yet stored in the cache.
overriden data that is owned by the cache.
This is not at all nice ... Hopefully it doesn't get too complicated to
work around all these drawing code and depsgraph issues, so the code
can be understood and replaced at some point in the future.
Caches now create 2 new roots below the main 'top' object of Alembic:
root and root_render for realtime and render data respectively.
This makes it easy to switch the whole archive to either of the modes
during baking and for constructing dupli caches.
Alternatively individual objects could store hires versions of their
data. This would also be more efficient if the cache contains many
simple objects which don't have 2 different variants. However, such
design decisions are difficult to make at this point and the
implementation can be modifier later.
This is the default now. It should make workflow a lot more foolproof
and convenient, since having only one of these modes active at a time
very easily leads to broken renders and confusing situations.
The problem is mostly due to the complicated way the depsgraph layer
feature is used to handle duplicator visibility. The duplicator is
declared as a child of its group's objects (even though no real
dependency exists!), so that a visible duplicator triggers updates of
invisible group objects, making instances of hidden groups possible.
However, dupli caches have to disable this dependency in order to avoid
unnecessary costly updates in hidden layers which are overridden by
cached data anyway. At the point where these dependencies are created
the evaluation context is unknown though, which means we cannot
distinguish between render and realtime evaluation for the purpose of
cache reading ...
duplis.
Particle systems can not be overridden from caches easily, there are too
many strings attached to the data and code to make this reliable.
Instead, a new simplified data structure for reading hair from caches
will be added, which replaces drawing and rendering of particle data.
The original particle data is not updated through duplis, so is usually
out of sync and should not be displayed.
pointers.
Relationship between CacheLibrary, duplicator Objects and Groups is
difficult. There are a number of somewhat conflicting goals:
- CacheLibraries write out data for objects and dupli groups. Multiple
objects can be stored in the same cache: CL *->* GR
- Objects can override a dupli group with different caches: OB *->1 CL
- As before, each object can be the duplicator for one group: OB *->1 GR
To combine these requirements, the first relationship will be made
indirect. Only the Object -> Group/CacheLib relations are explicit
pointers in the DNA. For finding all objects contained in a cache
library the usual recursive DNA tagging system must then be used.
unnecessary dependencies.
This flag will replace the current "read" mode on cache libraries.
Beside enabling cache reading, it also disables the current "fake"
dependencies between duplicators and their group objects. This is
exploiting the layer visibility mechanism in depsgraph to ensure that
animated group objects get evaluated when used by a visible duplicator,
even when they are not themselves visible. These dependencies cause
group object updates even if the duplicator is using cached results.
To avoid this unnecessary overhead and make caching worthwhile we
rebuild depsgraph without these relations when using the cache instead.
If a duplicator has cached data it will now replace the derivedFinal
mesh of objects with the cached version for drawing.
This is a compromise atm: It would be better to actually draw derived
meshes directly, so that we don't have to modify objects. Then we could
also have multiple different instances of the same orignal object
(in whatever way these might be defined). DNA Objects would then be
totally separate from duplis, but at this point the drawing and render
code makes this unfeasible.
The layout of the Alembic files resembles the DNA structure in Blender:
- On the top level (under the top/root node) there are Abc::Objects for
Blender ID datablocks (currently Object and Group)
- Objects store final data (DerivedMesh) and later simulation results
etc.
- Groups store their full duplilists for instancing. Currently there is
no recursive nesting of groups, since this would limit duplis to
dupligroups and exclude e.g. duplifaces.
On reading the duplilist gets reconstructed and stored in the DupliCache
for a duplicator empty (the group instance). DerivedMesh data is stored
in a hash table for each instanciated object and can later replace
finalDM in drawing and rendering.
Now the base types for readers/writers are not nominally forming the
interface any more (they may be removed entirely later). This makes
possible a cleaner init method directly in the Abc Writer/Reader
classes.
Further work may be required in this area.
This is more in line with how readers work in the Blender Alembic
implementation elsewhere.
Generally, readers are less persistent than writers: they are created
whenever cache results need to be updated (e.g. on frame changes) and
discarded afterward. Writers OTOH stay alive during the whole baking
job, since they keep the references to Alembic Writer instances and can
only be deleted once the writing for that part is done.
Note that the cache stores dupli matrices without the final parent
transform, since it only knows about the group itself. The duplicator
obmat is applied to the duplis after reading the cache.
Now we store duplilists entirely inside a single alembic object for the
group. This allows using all the generated duplis, which would be
difficult if the alembic file had to define all the possible recursive
dupli relations that Blender allows, beside straightforward Group duplis.
Uses the duplilist generated by Blender to define instances, instead of
recreating the group layout. This omits some information about actual
structure of the DNA, which might be useful later on. The main problem
is that the duplilist itself does not encode this, so it's a tradeoff
between either including the Group structure or omitting the other
dupli types, like face, vertex, particle duplis.
creation call.
This is executed *a lot* (even for simple things such as viewport
panning), so the code is probably not suitable in this form. At least
it could do a frame comparison to avoid unnecessary updates.
This is necessary for the current viewport drawing and rendering code,
which expected each dupli instance to represent a copy of data in the
DNA.
The code maps base-level objects in the cache to DNA Objects inside the
dupligroup by name. Only objects that can be found in the blend data
will be allowed for DupliObjects.
This is a placeholder for proper depsgraph integration. Eventually frame
changes and some other updates should rebuild dupligroup caches
automatically using the depsgraph. Until then this operator is a quick
way to test the IO from caches and the further drawing and rendering
(TODO).
BKE (DerivedMesh, particles, cloth).
The new cache implementation will be used for constructing dupli data
instead, bypassing the complexities of the modifier stack.
Conflicts:
source/blender/blenkernel/intern/cloth.c
source/blender/blenkernel/intern/particle.c
source/blender/blenkernel/intern/particle_system.c
Conflicts:
source/blender/blenkernel/intern/particle.c
source/blender/blenkernel/intern/particle_system.c
Alembic caches.
This creates representations (Abc::Object) for Blender Object and Group
datablocks in the Alembic files and uses Alembic instancing to define
the dupligroup hierarchy. This leads to a relatively flat hierarchy in
Alembic files:
Top -> Object/Group -> DerivedMesh/Particles/Hair/Cloth/Smoke/...
The dupligroup structure can not be represented by a hierarchical
structure such as the Alembic object nesting, because of the
many-to-many relationship between objects and groups (a group can
contain multiple objects, multiple objects can instance the same group).
Instead we use the instancing feature of Alembic to represent
dupligroups. This is created in 2 stages to ensure all the main ID
blocks have been serialized before creating references.
This is not clearly documented for Alembic, but apparently properties
inside compounds still need to be name uniquely for the whole object.
This is somewhat involved for Blender's CustomData, because we have
5 different customdata elements for meshes (vert, edge, face, poly,
loop) and each of these can have the same types and multiple layers of
the same type ...
CustomData layers are pruned by the CDDM_copy function when they have
CD_FLAG_NOCOPY set. This is based on later modifiers in the stack, which
can specify that they require certain data layers - but the caching
modifier itself should store only what is needed by later modifiers.
It means we cannot easily keep a full copy of the DM in the cache
modifier to writing into caches later.
For now the hackish solution is to temporarily disable NOCOPY flags when
copying the DM. This is really not nice and needs a better solution.
This is a skeleton feature that provides a general way of storing
CustomData types. Currently only ORIGINDEX layers are implemented, the
code is designed to make extension easy.
Storing CustomData layers in Alembic is a bit involved because the
complex structs often used as customdata need to be de-interleaved for
Alembic into a set of POD (plain-old-datatype) array properties.
In addition the property names should be unique, so that mapping abc
properties back to customdata layers is safe. This works by using 2
levels of compound properties: the first level stored per CD type
properties, with a number of properties for each layer of this type,
using either a name or index (for unnamed layers). The internal
properties can then in turn be compounds, if structs need to be
serialized into PODs.
Abc property readers/writers are created dynamically for the CustomData
compounds. This is necessary because we don't know in advance what kind
of data layers a DerivedMesh or other CD user will have, and this can
change each frame in theory. Alembic is easier with state data schemes,
but using it this way is possible (if somewhat cumbersome).
This is totally weird and convoluted, no idea if correct.
Conflicts:
source/blender/blenkernel/intern/particle_system.c
Conflicts:
source/blender/blenkernel/intern/particle_system.c
This creates a string with information about all the objects and
properties contained within a (Alembic) archive, used by a cache
library.
The operator has 3 modes of presenting the info string:
- stdout, ie. printing to the terminal
- popup window (not very useful usually due to size limits and lack of
scrolling)
- clipboard copy
result read from the cache.
Mixing this with the output_dm used for writing leads to undefined
situations where the DM was released but should actually be passed on.
This designates a cache library to be used either for the viewport or
for renders. A "Render" cache library will evaluate modifiers with
render settings and a "Viewport" cache library will use viewport
(realtime) settings.
When reading the cache, the library will only be
used for the assigned purpose, i.e. a Render cache does not work for
viewport caching and vice-versa (although a cache can be baked for one
setting and then switched afterward).
Note that render results will never be visible in the realtime viewport
due to the way object evaluation is handled in Blender at this point
(render settings are only evaluated explicitly during renderer sync).
Conflicts:
source/blender/blenkernel/intern/particle_system.c
Conflicts:
source/blender/blenkernel/intern/particle_system.c
out of pointcache into blenkernel.
This is quite simple and repetitive and there is not need to have this
in the main pointcache/Alembic API. The code is mostly concerned with
logic of DNA data, so pointcache shouldn't have to deal with it.
The pointcache library is now pretty much independent from ID blocks and
should not be responsible for handling file paths. The path construction
is also fairly straightforward now compared to the old point cache
system, with only basic conversion of relative to absolute paths for
loading archive files.
python code to unify symbols.
This is necessary because the operator for adding new items (as opposed
to the enable/disable button) cannot be shown with the same checkbox
button. The UI template function can display a custom button for the
operator, so the look becomes less confusing.
This now appends a default extension based on the cache backend used.
At this point this is always Alembic (.abc) but would allow other
backend formats in the future.
In addition to the object name filtering, items in cache library groups
can now be filtered by type as well.
Existing items in the cache are always displayed, so as to not hide
important information (what gets stored in the cache). The filtering
is primarily a utility to simplify searching inside the group for things
that should be added to the cache.
This requires a better design and is not so easy to implement properly
within the limits of RNA definition. These collections don't physically
exist in the DNA, they are mainly utilities for looping over
//potential// items in a cache library. For now the python code has to
be adapted to only show valid items, until the RNA provides a good
solution.
This is also used during the bake process to avoid confusion: The read
flag gets disabled for the baking cachelib, so that objects don't try
to read cache data that is supposed to be generated.
CacheLibrary datablock has a generic unlink function now, but currently
nothing actually links to cache libs themselves, so it's empty. Still
good to keep this in mind for the future.
The `bf_pointcache_alembic` code is a separate library, to avoid
muddling up core code with alembic includes and preprocessor defines.
Alembic stuff only belongs strictly into alembic code and can be
disabled cleanly.
The `bf_pointcache` and `bf_pointcache_alembic` libraries had a circular
dependency, because the alembic implementation functions were called
directly. Now there is a "Factory" class to abstract the creation of
concrete implementations for readers and writers.
`bf_pointcache_alembic` defines this factory and is registered
//outside// of the core `bf_pointcache` lib, so there is no linker
circularity.
This was an artifact from the previous way of mapping non-0 start frame
to time 0.0. Now we start at times > 0.0 to match with the start frame.
Both ways are possible, can be changed later if needed.
Writers were always starting at time 0.0, which means that for start
frames > 1 the readers would always be off. Now match the writer start
frame to the actual Blender start frame.
cache library system.
The Cache modifier is now an optional "break point" of the modifier
stack:
- Without a cache modifier the stack works as before. Baking will write
the final stack result. After baking the cache replaces the whole
stack.
- With a cache modifier the stack result at the modifier's position is
stored. The cache is then applied as the output of that modifier,
skipping preceding modifiers. That way additional modifiers can be
applied on top of the cache.
- When using multiple cache modifiers, only the last (active) one will
be used, since all previous cache results would be discarded anyway.
This can happen if an object gets deleted or isn't loaded for some
reason. The item should just be ignored in that case and removed at the
next opportunity (cleanup function).
of constructing names internally.
This helps prevent name collisions and guarantees a consistent naming
scheme for putting multiple items in the same cache.
refactoring simpler.
Since the new approach is not tied to the old set of point cache data
types there is really no need to carry their empty husks around. Once
the new caching system is defined in detail we can add back whatever
makes sense.
Now instead of each reader/writer creating its own archive, the archive
is created by the caller in advance and passed as a constructor
argument. This means that multiple items can be stored together in the
same archive.
iterating over all objects eligible for caching in the group.
This allows for nested dupligroup instances as well. All objects that
are instantiated at least once by any group can be included in the
respective cache, but get represented only once because the cached data
is the same anyway.
subtype/index.
This omits possible instancing and recursion by dupli groups, but since
the data of instances is always the same in Blender at this point, there
is little need to take them into account for caching.
At some point in the future it may become desirable to store full dupli
hierarchies in the cache, but it doesn't make sense to try to design a
fully-fledged path descriptor for this hypothetical case now.
object group.
This is somewhat experimental, eventually we may want to link to Objects
instead, or allow multiple cache targets for the same cache, etc.
and files.
At it's core this is just a file path, but many different cache users
may refer to this, so having a dedicated ID block helps. It can be
compared to Image datablocks, which also primarily are used for data
on storage, but can be packed with the blend file, and carry some
additional information that would be cumbersome to keep sync'ed
otherwise.
The name 'CacheLibrary' deliberately resembles the 'Library' datablock:
just as a Library stores ID blocks in a physical file, a CacheLibrary
stores cached data (in Alembic HDF5/Ogawa or other formats).
Conflicts:
source/blender/blenkernel/BKE_pointcache.h
source/blender/blenkernel/intern/pointcache.c
source/blender/makesdna/DNA_pointcache_types.h
Conflicts:
source/blender/makesrna/intern/rna_main_api.c
When writing DerivedMesh data, the modifier result at the point of the
modifier will be used for writing to the cache. If this modifier is not
used the derivedFinal mesh is used instead.
The pointcache API is designed to keep the general concept of caching
separate from the concrete Alembic implementation. Other types of file
export/import would be possible, e.g. for OpenVDB data.
Adds Alembic as a dependency and build support in cmake and scons.
Also extends the install_deps.sh script with a temporary installation
procedure for the Alembic libraries. This is still very hackish and
needs to be improved.
This lets users set a start frame at which a cached smoke simulation
starts. This offset is simply subtracted from the current scene frame
for evaluating the cache.
The rationale here is that we don't try to replace particle data
temporarily like we do with DerivedMesh. This would be needed for
drawing cached data overrides in the viewport and pass them to renderers
(unless these read Alembic directly) and later for performing dupli
simulation.
This data will be used for dupli simulations later on. Child path
generation could also be done based on this data without involvement of
particle systems.
This feature is mostly useful for the "view selected" operator. It is
also used in the "set origin to geometry" operator, but since the
cache overrides the object geometry anyway the effect may not be what
users expect (which is acceptable).
Note that Cycles currently only supports tessellated triangle/quad
meshes. Alembic PolyMesh generally has ngons, so external tessellation
is required until cycles gets a proper tessellation implementation of
its own.
standalone.
The cycles XML files now can refer to Alembic (.abc) files. This will
call the default alembic reader to read in scene data. Currently it
simply prints the Alembic file structure.
Eventually a proper schema needs to be defined for both xml and abc.
Also care has to be taken to handle potential conflicts between settings
both within xml/abc and between them.
This adds Alembic to the standalone application version of cycles.
Files can be specified as XML or Alembic, or use automatic detection
based on the filename extension.
Currently the Alembic reader just dumps the file structure to stdout
as a test.
This means we don't bluntly store all the data in a group, but use the
selection from the cache library. It also helps to avoid issues with
object visibility which is not yet stored in the cache.
overriden data that is owned by the cache.
This is not at all nice ... Hopefully it doesn't get too complicated to
work around all these drawing code and depsgraph issues, so the code
can be understood and replaced at some point in the future.
Caches now create 2 new roots below the main 'top' object of Alembic:
root and root_render for realtime and render data respectively.
This makes it easy to switch the whole archive to either of the modes
during baking and for constructing dupli caches.
Alternatively individual objects could store hires versions of their
data. This would also be more efficient if the cache contains many
simple objects which don't have 2 different variants. However, such
design decisions are difficult to make at this point and the
implementation can be modifier later.
This is the default now. It should make workflow a lot more foolproof
and convenient, since having only one of these modes active at a time
very easily leads to broken renders and confusing situations.
The problem is mostly due to the complicated way the depsgraph layer
feature is used to handle duplicator visibility. The duplicator is
declared as a child of its group's objects (even though no real
dependency exists!), so that a visible duplicator triggers updates of
invisible group objects, making instances of hidden groups possible.
However, dupli caches have to disable this dependency in order to avoid
unnecessary costly updates in hidden layers which are overridden by
cached data anyway. At the point where these dependencies are created
the evaluation context is unknown though, which means we cannot
distinguish between render and realtime evaluation for the purpose of
cache reading ...
duplis.
Particle systems can not be overridden from caches easily, there are too
many strings attached to the data and code to make this reliable.
Instead, a new simplified data structure for reading hair from caches
will be added, which replaces drawing and rendering of particle data.
The original particle data is not updated through duplis, so is usually
out of sync and should not be displayed.
pointers.
Relationship between CacheLibrary, duplicator Objects and Groups is
difficult. There are a number of somewhat conflicting goals:
- CacheLibraries write out data for objects and dupli groups. Multiple
objects can be stored in the same cache: CL *->* GR
- Objects can override a dupli group with different caches: OB *->1 CL
- As before, each object can be the duplicator for one group: OB *->1 GR
To combine these requirements, the first relationship will be made
indirect. Only the Object -> Group/CacheLib relations are explicit
pointers in the DNA. For finding all objects contained in a cache
library the usual recursive DNA tagging system must then be used.
unnecessary dependencies.
This flag will replace the current "read" mode on cache libraries.
Beside enabling cache reading, it also disables the current "fake"
dependencies between duplicators and their group objects. This is
exploiting the layer visibility mechanism in depsgraph to ensure that
animated group objects get evaluated when used by a visible duplicator,
even when they are not themselves visible. These dependencies cause
group object updates even if the duplicator is using cached results.
To avoid this unnecessary overhead and make caching worthwhile we
rebuild depsgraph without these relations when using the cache instead.
If a duplicator has cached data it will now replace the derivedFinal
mesh of objects with the cached version for drawing.
This is a compromise atm: It would be better to actually draw derived
meshes directly, so that we don't have to modify objects. Then we could
also have multiple different instances of the same orignal object
(in whatever way these might be defined). DNA Objects would then be
totally separate from duplis, but at this point the drawing and render
code makes this unfeasible.
The layout of the Alembic files resembles the DNA structure in Blender:
- On the top level (under the top/root node) there are Abc::Objects for
Blender ID datablocks (currently Object and Group)
- Objects store final data (DerivedMesh) and later simulation results
etc.
- Groups store their full duplilists for instancing. Currently there is
no recursive nesting of groups, since this would limit duplis to
dupligroups and exclude e.g. duplifaces.
On reading the duplilist gets reconstructed and stored in the DupliCache
for a duplicator empty (the group instance). DerivedMesh data is stored
in a hash table for each instanciated object and can later replace
finalDM in drawing and rendering.
Now the base types for readers/writers are not nominally forming the
interface any more (they may be removed entirely later). This makes
possible a cleaner init method directly in the Abc Writer/Reader
classes.
Further work may be required in this area.
This is more in line with how readers work in the Blender Alembic
implementation elsewhere.
Generally, readers are less persistent than writers: they are created
whenever cache results need to be updated (e.g. on frame changes) and
discarded afterward. Writers OTOH stay alive during the whole baking
job, since they keep the references to Alembic Writer instances and can
only be deleted once the writing for that part is done.
Note that the cache stores dupli matrices without the final parent
transform, since it only knows about the group itself. The duplicator
obmat is applied to the duplis after reading the cache.
Now we store duplilists entirely inside a single alembic object for the
group. This allows using all the generated duplis, which would be
difficult if the alembic file had to define all the possible recursive
dupli relations that Blender allows, beside straightforward Group duplis.
Uses the duplilist generated by Blender to define instances, instead of
recreating the group layout. This omits some information about actual
structure of the DNA, which might be useful later on. The main problem
is that the duplilist itself does not encode this, so it's a tradeoff
between either including the Group structure or omitting the other
dupli types, like face, vertex, particle duplis.
creation call.
This is executed *a lot* (even for simple things such as viewport
panning), so the code is probably not suitable in this form. At least
it could do a frame comparison to avoid unnecessary updates.
This is necessary for the current viewport drawing and rendering code,
which expected each dupli instance to represent a copy of data in the
DNA.
The code maps base-level objects in the cache to DNA Objects inside the
dupligroup by name. Only objects that can be found in the blend data
will be allowed for DupliObjects.
This is a placeholder for proper depsgraph integration. Eventually frame
changes and some other updates should rebuild dupligroup caches
automatically using the depsgraph. Until then this operator is a quick
way to test the IO from caches and the further drawing and rendering
(TODO).
BKE (DerivedMesh, particles, cloth).
The new cache implementation will be used for constructing dupli data
instead, bypassing the complexities of the modifier stack.
Conflicts:
source/blender/blenkernel/intern/cloth.c
source/blender/blenkernel/intern/particle.c
source/blender/blenkernel/intern/particle_system.c
Alembic caches.
This creates representations (Abc::Object) for Blender Object and Group
datablocks in the Alembic files and uses Alembic instancing to define
the dupligroup hierarchy. This leads to a relatively flat hierarchy in
Alembic files:
Top -> Object/Group -> DerivedMesh/Particles/Hair/Cloth/Smoke/...
The dupligroup structure can not be represented by a hierarchical
structure such as the Alembic object nesting, because of the
many-to-many relationship between objects and groups (a group can
contain multiple objects, multiple objects can instance the same group).
Instead we use the instancing feature of Alembic to represent
dupligroups. This is created in 2 stages to ensure all the main ID
blocks have been serialized before creating references.
This is not clearly documented for Alembic, but apparently properties
inside compounds still need to be name uniquely for the whole object.
This is somewhat involved for Blender's CustomData, because we have
5 different customdata elements for meshes (vert, edge, face, poly,
loop) and each of these can have the same types and multiple layers of
the same type ...
CustomData layers are pruned by the CDDM_copy function when they have
CD_FLAG_NOCOPY set. This is based on later modifiers in the stack, which
can specify that they require certain data layers - but the caching
modifier itself should store only what is needed by later modifiers.
It means we cannot easily keep a full copy of the DM in the cache
modifier to writing into caches later.
For now the hackish solution is to temporarily disable NOCOPY flags when
copying the DM. This is really not nice and needs a better solution.
This is a skeleton feature that provides a general way of storing
CustomData types. Currently only ORIGINDEX layers are implemented, the
code is designed to make extension easy.
Storing CustomData layers in Alembic is a bit involved because the
complex structs often used as customdata need to be de-interleaved for
Alembic into a set of POD (plain-old-datatype) array properties.
In addition the property names should be unique, so that mapping abc
properties back to customdata layers is safe. This works by using 2
levels of compound properties: the first level stored per CD type
properties, with a number of properties for each layer of this type,
using either a name or index (for unnamed layers). The internal
properties can then in turn be compounds, if structs need to be
serialized into PODs.
Abc property readers/writers are created dynamically for the CustomData
compounds. This is necessary because we don't know in advance what kind
of data layers a DerivedMesh or other CD user will have, and this can
change each frame in theory. Alembic is easier with state data schemes,
but using it this way is possible (if somewhat cumbersome).
This creates a string with information about all the objects and
properties contained within a (Alembic) archive, used by a cache
library.
The operator has 3 modes of presenting the info string:
- stdout, ie. printing to the terminal
- popup window (not very useful usually due to size limits and lack of
scrolling)
- clipboard copy
result read from the cache.
Mixing this with the output_dm used for writing leads to undefined
situations where the DM was released but should actually be passed on.
This designates a cache library to be used either for the viewport or
for renders. A "Render" cache library will evaluate modifiers with
render settings and a "Viewport" cache library will use viewport
(realtime) settings.
When reading the cache, the library will only be
used for the assigned purpose, i.e. a Render cache does not work for
viewport caching and vice-versa (although a cache can be baked for one
setting and then switched afterward).
Note that render results will never be visible in the realtime viewport
due to the way object evaluation is handled in Blender at this point
(render settings are only evaluated explicitly during renderer sync).
Conflicts:
source/blender/blenkernel/intern/particle_system.c
out of pointcache into blenkernel.
This is quite simple and repetitive and there is not need to have this
in the main pointcache/Alembic API. The code is mostly concerned with
logic of DNA data, so pointcache shouldn't have to deal with it.
The pointcache library is now pretty much independent from ID blocks and
should not be responsible for handling file paths. The path construction
is also fairly straightforward now compared to the old point cache
system, with only basic conversion of relative to absolute paths for
loading archive files.
python code to unify symbols.
This is necessary because the operator for adding new items (as opposed
to the enable/disable button) cannot be shown with the same checkbox
button. The UI template function can display a custom button for the
operator, so the look becomes less confusing.
This now appends a default extension based on the cache backend used.
At this point this is always Alembic (.abc) but would allow other
backend formats in the future.
In addition to the object name filtering, items in cache library groups
can now be filtered by type as well.
Existing items in the cache are always displayed, so as to not hide
important information (what gets stored in the cache). The filtering
is primarily a utility to simplify searching inside the group for things
that should be added to the cache.
This requires a better design and is not so easy to implement properly
within the limits of RNA definition. These collections don't physically
exist in the DNA, they are mainly utilities for looping over
//potential// items in a cache library. For now the python code has to
be adapted to only show valid items, until the RNA provides a good
solution.
This is also used during the bake process to avoid confusion: The read
flag gets disabled for the baking cachelib, so that objects don't try
to read cache data that is supposed to be generated.
CacheLibrary datablock has a generic unlink function now, but currently
nothing actually links to cache libs themselves, so it's empty. Still
good to keep this in mind for the future.
The `bf_pointcache_alembic` code is a separate library, to avoid
muddling up core code with alembic includes and preprocessor defines.
Alembic stuff only belongs strictly into alembic code and can be
disabled cleanly.
The `bf_pointcache` and `bf_pointcache_alembic` libraries had a circular
dependency, because the alembic implementation functions were called
directly. Now there is a "Factory" class to abstract the creation of
concrete implementations for readers and writers.
`bf_pointcache_alembic` defines this factory and is registered
//outside// of the core `bf_pointcache` lib, so there is no linker
circularity.
This was an artifact from the previous way of mapping non-0 start frame
to time 0.0. Now we start at times > 0.0 to match with the start frame.
Both ways are possible, can be changed later if needed.
Writers were always starting at time 0.0, which means that for start
frames > 1 the readers would always be off. Now match the writer start
frame to the actual Blender start frame.
cache library system.
The Cache modifier is now an optional "break point" of the modifier
stack:
- Without a cache modifier the stack works as before. Baking will write
the final stack result. After baking the cache replaces the whole
stack.
- With a cache modifier the stack result at the modifier's position is
stored. The cache is then applied as the output of that modifier,
skipping preceding modifiers. That way additional modifiers can be
applied on top of the cache.
- When using multiple cache modifiers, only the last (active) one will
be used, since all previous cache results would be discarded anyway.
This can happen if an object gets deleted or isn't loaded for some
reason. The item should just be ignored in that case and removed at the
next opportunity (cleanup function).
of constructing names internally.
This helps prevent name collisions and guarantees a consistent naming
scheme for putting multiple items in the same cache.
refactoring simpler.
Since the new approach is not tied to the old set of point cache data
types there is really no need to carry their empty husks around. Once
the new caching system is defined in detail we can add back whatever
makes sense.
Now instead of each reader/writer creating its own archive, the archive
is created by the caller in advance and passed as a constructor
argument. This means that multiple items can be stored together in the
same archive.
iterating over all objects eligible for caching in the group.
This allows for nested dupligroup instances as well. All objects that
are instantiated at least once by any group can be included in the
respective cache, but get represented only once because the cached data
is the same anyway.
subtype/index.
This omits possible instancing and recursion by dupli groups, but since
the data of instances is always the same in Blender at this point, there
is little need to take them into account for caching.
At some point in the future it may become desirable to store full dupli
hierarchies in the cache, but it doesn't make sense to try to design a
fully-fledged path descriptor for this hypothetical case now.
object group.
This is somewhat experimental, eventually we may want to link to Objects
instead, or allow multiple cache targets for the same cache, etc.
and files.
At it's core this is just a file path, but many different cache users
may refer to this, so having a dedicated ID block helps. It can be
compared to Image datablocks, which also primarily are used for data
on storage, but can be packed with the blend file, and carry some
additional information that would be cumbersome to keep sync'ed
otherwise.
The name 'CacheLibrary' deliberately resembles the 'Library' datablock:
just as a Library stores ID blocks in a physical file, a CacheLibrary
stores cached data (in Alembic HDF5/Ogawa or other formats).
Conflicts:
source/blender/blenkernel/BKE_pointcache.h
source/blender/blenkernel/intern/pointcache.c
source/blender/makesdna/DNA_pointcache_types.h
Conflicts:
source/blender/makesrna/intern/rna_main_api.c
When writing DerivedMesh data, the modifier result at the point of the
modifier will be used for writing to the cache. If this modifier is not
used the derivedFinal mesh is used instead.
The pointcache API is designed to keep the general concept of caching
separate from the concrete Alembic implementation. Other types of file
export/import would be possible, e.g. for OpenVDB data.
Adds Alembic as a dependency and build support in cmake and scons.
Also extends the install_deps.sh script with a temporary installation
procedure for the Alembic libraries. This is still very hackish and
needs to be improved.
This reverts commit 91d0f6f113.
Artists here seem to dislike the interface being stuck after a change to
modifiers, reverting until an interactive solution is coded.
**Warning: WIP!** Basic functionality works fine and there shouldn't be any crashes, however, if you occur issues, feel free to report them to me (not in the bug tracker!)
Some more effort is needed to get this ready for master, but I thought it would be nice as a little present for the Gooseberry-team (I heard some of you were requesting this?) and the ones using the Gooseberry-builds :)
Oh, and better not look at that code, it's ugly and a lot of changes are planned, I just wanted to get something working before getting into details.
screen.
Apparently the screen on the given file did not have a scene attached.
Not sure how this is possible exactly, but for now just guard against it
at load time by assigning default scene in that case.
It was only happening on 32bit platforms because of alignment
differences when allocating class.
Now got rid of copy of eigen matricies stored by value in the
residual block which solves aligment issues and should also
give some unmeasurable speedup.
This hit animators already, basically when using sound sync we can
hit negative frames. This happens because we always subtracted the full
range of the triple buffer size from the timing, even when buffers were
flushed.
Also, old code read the offset from the offset of the reader. The
problem here, is that due to threading there is a time offset between
the when this offset is set and when it is offloaded to the buffers,
which means that we could get quite some variance between time
reporting.
Now sounds keep a private byte offset which is incremented right before
invalidating old buffers. This should make the combination of OpenAL
time report + byte offset more accurate. For even more accuracy we might
spinlock while updating those values but for now left it as is for fear
of the lock interfering with frame update performance. We can try to be
smarter here, storing old value while update is happening, and use
trylock and the old values if it fails but for now commit the simple
version.
The issue was caused by recent DM to mesh conversion which was taking ownership,
and it was failing if there are any referenced layers in the custom data.
Now disable passing ownership if there are any referenced layers.
The idea is pretty simple: instead of making temporary copy of all the
related custom data layers just pass the ownership from the DM to the
mesh.
This is really handy in cases when you've got DM which you need to
convert to Mesh datablock and wouldn't need that DM after conversion
anyway.
Foe example, render database conversion, exporters and even Modifier
Apply will benefit from this option.
Reviewers: campbellbarton
Differential Revision: https://developer.blender.org/D1127
The purpose of this change is to add extra possibility to render engines and
export scripts to reduce peak memory footprint during their operation.
This new argument should be used with care since it'll leave mesh in not really
compatible with blender format, but it's ok to be used on temp meshes.
Unfortunately, it's hard to get scene where it'll show huge benefit because
in my tests with cycles peak memory is reached in MEM_printmemlist_stats().
However, in the file with sintel dragon it gives around 1gig of memory benefit
after removing the polys which would allow other heavy to compute stuff such as
hair (or even pointiness calculation) to not be a peak memory usage.
In any case, this change is nice to have IMO, and only means more parts of
scene export code should be optimized memory-wise.
Reviewers: campbellbarton
Differential Revision: https://developer.blender.org/D1125
Issue this commit is addressed to is that particle system and particle modifier
will contain caches once derived mesh was requested and this cached data will
never be freed.
This could easily lead to unwanted memory peaks during synchronization stage
of rendering.
The idea is to have RNA function in object which would free caches which can't
be freed otherwise. This function is not intended to deal with derived final
since it might be used by other objects (for example by object with boolean
modifier).
This cache freeing is only happening in the background rendering and locked
interface rendering.
From quick tests with victor file this change reduces peak memory usage by
command line rendering by around 6% (1780MB vs. 1883MB). For rendering from
the interface it's about 12% (1763MB vs. 1998MB).
Reviewers: campbellbarton, lukastoenne
Differential Revision: https://developer.blender.org/D1121
for clumping instead of deformed location.
The orco location must be used for roughness to ensure a stable shape,
but for moving child keys toward the parent (clumping) the deformed
location must be used.
This randomisation was using the deformed parent coordinates for
calculating noise vectors, which causes unwanted movement as the parent
hair deforms. Instead the `orco` coordinates must be used for all noise
vectors, since these are from the undeformed mesh.
Rationale for this feature goes like this:
In meshes the influence of shape keys can be limited to certain vertex
groups. This allows shapes to be applied selectively on parts of the
mesh and possibly even be animated by changing vertex group weights over
time.
With hair shape keys there is no equivalent feature to vertex groups
(defining a vertex group on hair keys would be rather tedious anyway).
It would be possible to use the emitter mesh vgroups to assign a
per-strand blending weight, but since hairs usually are much more dense
than mesh vertices this would sacrifice accuracy.
A better option is to allow textures to influence the hair shape key
blending. For this purpose there is now a new texture influence.
A shape key name needs to be assigned to the MTex in addition to a
factor. It will then act as a multiplier for the shape key weight,
i.e. the base shapekey value still works as an overall control.
Draw motionpaths after scene. They are not really XRays since they don't
require any depth info so add separate list for them. Feature could be a
little more polished to avoid too many state changes but for now just
commit something that works (tm)
problematic (still!)
It seems due to floating point precision issues we have to account for
even more cases.
Added colinear check for one more triangle edge-vertex and assumed that
if no collision test is found then triangle is completely inside box.
Such cases are mostly from float precision not catching triangle just
outside bucket. Now tests are much better for simple tests (paint/flood
fill cube from ortho view, paint sphere), however there are still some
rare cases of stray pixels.
By default a surface-based force will push things away from the surface
in both directions, regardless of whether a point is "inside" or
"outside" (judging by the surface normal). The new option makes a force
field always push things away in the direction of the normal, so that
it becomes less likely to get points "trapped" inside a mesh object.
building the BVH.
A long long time ago (rB3816554cbc1a40dc5199c8e56e45817ec09128d5) it
was coded such that edges would be used instead of faces for surface
distance calculation. This is pretty much never what you actually want.
modifiers, for use in shaders.
The particle instance modifier can now store 2 custom data layers:
- Index: the integer index of the particle
- Value: a randomized floating point number for shader variation
These layers have user-defined names, so multiple particle instance
modifiers can be used without overwriting customdata layers. In Cycles
the data can be accessed using an Attribute node with the same names.
This allows using the same particle system for multiple objects without
creating too much repetitiveness. Each instance object can select a
range of the particles to actually use for instancing (default 1.0 means
all particles are used). To further avoid overlap with multiple
instancing objects, the offset value can be used to make each system
use a specific range of particles.
This commit adds a few generalizations to the VBO
code so that modifiers can create and populate their own GPU objects.
The VBO code originally supported CDDerivedMesh only. The design moves slightly
towards the viewport refactor where the rendering system requests data from the
modifiers.
In this commit only basic support for vertices and normals is provided and
some features from blender's VBO system, suchs as mapping to original faces,
unique element indices for vertices and loose vertex/edge support are missing.
Also, the quick navigation feature of the subsurf modifier won't be supported
for now.
What we do have is full support for solid shading with multiple materials,
flat/smooth shading and a big performance boost.
It's a bit messy, but that is to be expected: all of the areas involved
(particles, shapekeys, depsgraph, drivers) are badly designed and
supposed to get major refactoring ...
Auto View automatically adjusts the view based on selection, so that the view is
always focused on the current selection.
A checkbox in the header is used to access it and it works for the following
selection methods: Toggle All, Border, Circle, Lasso, Left, Right, More, Less,
Linked, Column (so all except of single selection, in which this can be a bit
annoying)
Reviewed by @Aligorith (thanks for that :) )
Only recompute if cost is below -FLT_EPSILON, we can get cases where both cases generate
very tiny negative costs (see 'Cylinder.004' mesh in .blend attached to report).
space the particle data should be interpreted in.
By default the space will now be `Local`, meaning that copies of the
mesh are made with the offset of the respective particle in the particle
object's local space, rather than using the world space offset of the
particle //inside the modified object space//. This behavior is much
more intuitive and consistent with true duplicators, such as face duplis.
Old files will still have `World` setting, so existing blend files are
not changed.
This is obviously total madness, this feature is totally unusable when
the coder is not sitting next to you ... But so are the rest of particle
settings, the only solution is a future node system.
The internal cloth modifier effector weights get replaced temporarily
to make the cloth sim use the particle weight settings instead. But
the particle sim was not putting back the original weights, which can be
non-NULL in case the cloth sim allocated these already. Messy design ...
between stroke and single-add mode.
This was requested especially for use with tablets. With a pen it is
difficult to set a precise point and the tool ends up making multiple
hairs close together all the time. Disabling the stroke option helps
adding individual hairs.
Avoid opening the file when doing uncached reading because ffmpeg takes
quite a long time to read the first frame. Instead, avi jpeg is almost
instantaneous here.
This will leave the anim struct without an Imbuf and that can cause
other unexpected issues elsewhere so commiting on gooseberry first
to try.
Speedup is quite impressive though.
Needless to say, code here quickly becomes a mess and the proxy system
should be made saner but for now just commit to keep things working at
the studio.
This is incompatible with particle data structures in their current form.
Hull drawing requires sorting child particles based on the distance from
their primary parent. However, this changes the order of children, which
is the main method of generating random numbers for them. In the
viewport this is not a problem, but when rendering the children are
constantly recreated, using the respective deformed emitter mesh each
time. This leads to changing child orders when using the convex hull
sorting, and therefore different randomisation values.
To properly implement child hull drawing we would have to generate a
stable parent-child offset metric as well as avoiding any resorting in
the actual data arrays. This in turn requires terribly inefficient
drawing iteration, which could become a bottleneck by itself even before
taking cache coherence or OpenGL optimization requirements into account
(collecting child data into a dedicated buffer for drawing).
Failed proxies will now attempt to fetch a smaller proxy first before
fetching the full resolution. This might allow us to see if we are
hitting a bandwidth issue (due to high res) or a file query issue (due
to more file queries)
to the hair dynamics input DM.
Also this calculation has to happen before caching paths, otherwise
there is a 1-update lag because parent paths still use old values.
and when switching the active key during edit mode.
This is different from how shape key editing works on meshes: there the
edit data (BMEditMesh) stores customdata layers for shape keys, then
writes them back to the mesh. The hair edit mode is a lot more messy and
confusing, it has to update the base data immeditately (with potential
consequences for undo ...)
The code was assuming that child 0 is always valid, but in fact has to
be treated like all others that can be disable due to preview percentage,
virtual parents, etc.
The current algorithm generates a triangle strip to fill the end of
each child bundle. This is not quite ideal, because even though the
base child groups are constructed to be convex, the resulting child
paths can become concave again by clumping, roughness, hair dynamics
etc.. This can create face inversion as well as ugly shading artifacts,
so a nicer algorithm may be desirable.
child hulls.
Note that the perpendicular direction around the hair bundles is not
smooth-shaded. Smooth shading doesn't make as much sense there because
large angles are common in this direction and give awkward shading
results.
This was used as part of the "sim preview" feature, where some amount of
vertices were tagged as disabled in the particle system. Due to the
mind twisting complexity of using the nested cloth modifier to simulate
hair strands indirectly it became necessary to still store all the
vertices, but then disable them again on the solver level ... If this
ever gets reimplemented it must be done in a sane way, avoiding the
cloth step altogether.
mode.
This was caused by variation of the number of keys on child hairs due
to shortening of hair curves based on euclidian distances. The other
kink modes also shorten hairs, but use the parametric distance instead,
which does not vary with deformation of hairs.
By default this now copies from one object's local space to another
object's local space (instead of the previous world space). This is
more useful when transferring particles between objects, because it
doesn't require moving objects on top of each other, as long as they
have similar shapes.
another, including edit data (grooming).
This uses basically the same method as the existing connect/disconnect
feature. The main difference is that it allows working with multiple
objects and transferring the //particle/hair data// instead of the
//mesh// data (which is what connect/disconnect expects). This is a much
more realistic workflow when rigging, topology etc. changes and
groomed hair has to be transferred to the changed model.
This is BAD code, but the particle kinking does not make it easy to
write a non-local modifier that requires neighboring positions,
curvature, etc. The feature is needed for Gooseberry.
This adds another level of clumping on child hairs. When enabled, child
hairs chose a secondary clumping target using a Voronoi pattern. This
adds visual detail on a smaller scale, which is useful particularly when
the number of parents is relatively small.
Natural fibres behave in a similar way when they become sticky and
intertwined. Hairs close to each other form a first twisted strand, then
combine into larger strands. Similar features can be found in ropes:
http://en.wikipedia.org/wiki/Hair_twistshttp://en.wikipedia.org/wiki/Rope
Scope update is very slow for high resolutions, and currently blocks
the UI thread(!). This is especially terrible in paint modes, where
each stroke causes a scope update and unacceptable freezing.
The scopes update method tries to avoid this somewhat by skipping if the
toolbar is disabled, but this doesn't help when painting where brush
tools etc. are frequently needed. It's also a bad-level poll, with the
core system accessing a UI element.
Eventually scope updates should become a low-priority background job,
as well as becoming threaded. Until then this polling provides a usable
workaround to the most outrageous cases.
This is an alternative method to the current fixed function with a
clump factor and "shape" parameter. This function is quite limited and
does not give the desired result in many cases (e.g. long, parallel
rasta strands are problematic). So rather than trying to add more
parameters there is now a fully user-defined optional curve for setting
the tapering shape.
This contains a few pieces of code for a future "modifier" system that
would allow more flexible combination of effects. Eventually a node
system is the way to go, but the current code makes that impossible.
to prevent double-freeing/invalid mem access.
This can happen with the "virtual parents" feature, which generates both
parent and child paths. Each task free function also freed the shared
context, leading to double freeing.
and graph editor.
This was a tricky commit that was not so straightforward to make work.
The information for bones is not easy to come by in the animation curves,
however we do have some string manipulation tricks to make it happen.
For now committing to gooseberry branch for testing, it will be ported
to master after most usual use cases have been confirmed as working.
(Current test with all rotation modes and translation seems to work, but
Not sure how well this will hold up with drivers etc)
Cloth data is used both for hair and actual cloth, which makes things
really difficult. The face number was used for distinguishing the two
types (no faces == hair mesh), but the extra hair data necessary for
hair sim is generated by particles and not available for edge-only cloth
meshes. This really needs to be sanitized ...
Now simply add a flag to the armature modifier to use facemaps. Facemaps
on the mesh object with the same name as the armature bones will get
displayed as widgets.
This is necessary because the hair dynamics settings are not part of the
ParticleSettings datablock, but part of the convoluted cloth modifier
inside the particle system struct. In the future this will be recoded
properly, but in the meantime presets provide a simple an unobtrusive
way to have reusable dynamics settings for hair.
approach does not work very well.
Using a cross section estimate still causes large oscillations due to
varying hair force based on angles. It also requires a sensible hair
thickness value (particle radius) which is difficult to control and
visualize at this point.
The new model is based purely on per-vertex forces, which seems to be
much more stable. It's also somewhat justified by the fact that each
hair vertex represents a certain mass.
* Test display code that displays the active facemap. Will not work on
subsurf modifier yet (crash)
* Fix loading of files with facemaps
* Facemap data now reside on polys instead of tessfaces.
The previous calculation was modulated with the angle between the wind
direction and the segments, which leads to very oscillating behavior.
Now the formula includes an estimate for the geometric cross section
of a hair segment based on the incident angle and the hair thickness
(currently just the particle size). This gives a more stable behavior
and more realistic response to wind.
This commit adds a few generalizations to the VBO
code so that modifiers can create and populate their own GPU objects.
The VBO code originally supported CDDerivedMesh only. The design moves
slightly towards the viewport refactor where the rendering system
requests data from modifiers.
What we do have is support for solid shading with multiple
materials, flat/smooth shading and a big performance boost.
Performance could
In this commit only basic support for vertices and normals is provided
and some features from blender's VBO system, such as mapping to
original faces, unique element indices for vertices and loose vertex/
and can be added later. The one feature that is missing now is the quick
navigation feature of the multires modifier (which uses the same code).
- Why this commit is made on the widget branch -
Facemap widgets need to avoid drawing the whole mesh when we do
collision detection on them. For subsurf drawing we need to iterate
through the whole mesh every time. What we need instead is sort
indexed elemet drawing according to the facemap each face belongs to.
This screams VBOs but since I was going to add that, I thought that
I could push it a bit further and add proper solid shading support.
This commit adds a few generalizations to the VBO
code so that modifiers can create and populate their own GPU objects.
The VBO code originally supported CDDerivedMesh only. The design moves
slightly towards the viewport refactor where the rendering system
requests data from modifiers.
In this commit only basic support for vertices and normals is provided
and some features from blender's VBO system, such as mapping to
original faces, unique element indices for vertices and loose vertex/
and can be added later. The one feature that is missing now is the quick
navigation feature of the multires modifier (which uses the same code).
What we do have is full support for solid shading with multiple
materials, flat/smooth shading and a big performance boost.
Performance could
- Why this commit is made on the widget branch -
Facemap widgets need to avoid drawing the whole mesh when we do
collision detection on them. For subsurf drawing we need to iterate
through the whole mesh every time. What we need instead is sort
indexed elemet drawing according to the facemap each face belongs to.
This screams VBOs but since I was going to add that, I thought that
I could push it a bit further and add proper solid shading support.
This module will contain mirrored functions for calculating and applying
weights for points on a mesh. This includes barycentric and UV weighting
and possibly more advanced global weighting such as harmonic weights.
The naming should follow this scheme:
<type>_{2d,3d}_{calc,apply}
e.g.
poly_2d_calc
poly_2d_apply
uv_3d_calc
...
Reviewers: campbellbarton
Differential Revision: https://developer.blender.org/D939
A bone using a facemap will not be displayed at all, rather it will use
the facemap of the mesh as an interaction area. The facemap still does
nothing since it's not yet hooked up to the customdata and widget code.
That will come next.
They are set when a highlighted widget intersection is detected and on
area level.
There's a discussion about using owners and a stack or array for cursors
but that needs some more thought and design.
Due to this we -obviously- rename this to overdrop
I am not sure if others would agree here, but it's pretty safe to do
this on a branch first. Rationale is that it's much nicer to work on
full screen with this.
backdrop is now functional.
The widget could communicate with the sequencer properties directly, but
here we set it to control the operator properties as a demonstration.
After some minor stress testing, time to give this to gooseberry team.
Areas support more than one widgetmap - will be useful to partition area
in different transform regions like data, pixel, 3D (will be done in a
separate refactor).
Also get rid of the context widget hack in favour of adding the
widgetmap handler to the operator handler itself, so handling will
be unified when the widget controls a property or spawns an operator
Handle events for widgetmap first when and pass to operator when
operator is active.
This is still a pretty confusing design, working on a simpler one now.
This commit makes it possible to use scenes as a kind of
multi-user metastrip (with their own time).
Currently this supports rendering & drawing nested strips,
but no convenient way to tab-enter into a scene strip.
* Minor cleanup - propname does not need to be stored anymore.
* Code to support modal widgetmap registration for operators - still
untested, but will be tested soon on sequencer cage widget
* Widgets take parameters to initialize property slots that are used for
feedback or control.
Cage now uses this to feed offset/scale to different properties.
* Initialize common properties during widget registration.
remove and add widgetgroup types from the system.
Start operator that will control sequencer backdrop. Architecture here
still needs a little work so good compatibility with Add-ons can be
ensured.
commit and operators to set faces in a group in edit mode.
Face maps are groupings of faces, similar to vertex groups, however,
each face can only belong to one face map.
They will be used for widget interaction, but there will possibly be
more uses for them in the future.
Code needs some cleanup still here, will apply as work is being done.
There is some code here for object mode interaction but it hasn't been
tested yet. This will probably be done with the widgets themselves.
The file wasn't being comiling by CMake, remove it from SCons compilation as well.
Not sure if the same is required in original branch, maybe it'll be redone anyway.
For now let's make it so gooseberry branch is compiling nicely on the buildbot.
transform the x direction of the backdrop in sequencer when a viewer
node is active.
There are still some issues with property transform not being a proper
operator, but we can fix that easily.
To make full 2D transform work we need positions stored in one property.
* Get rid of invalid flag (we avoid drawing widgets by dereferencing RNA
now so it should be OK)
* Add intersect function for 2D widgets (still needs correct mapping to
be any useful though)
A stupid hack is needed here, changing the way the factor is applied to
angular bending springs. In cloth sim the bending factor of individual
springs is applied as a mix value between the bending stiffness and a
max value, but this max value isn't even used in hair sim so that
approach becomes useless.
This helps to create some variation in a hair system, which can
otherwise become very uniform and boring. It's yet another confusing
setting in a system that should have been nodified, but only option for
now (broken windows ...)
Fix interaction when pivot is beside the camera. basically we now do all
calculations in 3D space and we do not scale correct the offset in 3D -
this creates a nasty dependency loop between drawing and calculation for
widgets bound to properties - ie offset depends on scale but scale
depends on screen position - ie offset.
Also added an extra callback to get the final position of the widget in
3d space and use that to calculate the scale. This takes care of
bringing a close to the camera widget to the background and the widget
keeping its initial size. The final position can be different from the
initial position, especially when the widget is offset from the original
position.
framing method instead of the Frenet frame.
The Frenet frame is very succeptible to sudden twists along straight
sections of a curve where the second derivative (curvature) becomes 0.
We now use plane/line intersection which seems to work quite well and
agrees to great extent with the transform system as well. There still
some small discrepancy but may also be related to scaling.
to support multiple hash identifiers.
Using explicit hashing functions for every sim debug call defeats the
purpose of having a quick feedback system. Now this can be done simply
by passing an arbitrary number of hash inputs (integers) at the end of
the function calls, which are then combined by a system of variadic
macros (based on the ELEM feature). Up to 8 identifiers are supported
currently, but more could be added easily if needed.
solver input and output.
This uses the central difference method (instead of combined forward/
backward difference), which makes it easier to correctly account for
grid borders.
samples.
This is just an intermediate method to make sure the density is valid.
Eventually the closest-point method should be used, but for testing
the poisson solver this is easier to debug.
This allows setting a target density which the fluid simulation will
take into account as an additional term in the pressure Poisson
equation. Based on two papers
"Detail Preserving Continuum Simulation of Straight Hair" (McAdams et al. 2009)
and
"Two-way Coupled SPH and Particle Level Set Fluid Simulation" (Losasso et al. 2008)
Currently the target pressure is specified directly, but it will be
a lot more convenient to define this in terms of a geometric value such
as "number of hairs per area" (combined with hair "thickness").
It may be better to use transform code for widgets but this will cause
nested operators for widget/operator and needs some consideration.
For transform e.g. we would have one transform operator taking care of
transformation of the widget, then feeding this back to regular object
transform.
Now widgetmaps get created and destroyed with the area. Not sure if it's
the best design but it avoids too many creations and recreations.
arrow offset now calculated at property binding time - no RNA access
during interaction means no chance of crash during undo.
There are several more or less new customdata layers that currently cannot be accessed via Python (or some are exposed via BMesh API but not via Mesh API). This patch aims to expose one of those layers - CD_PAINT_MASK.
It'd be pretty useful for existing and future addons targeting sculpt workflow. Currently to get some access to the mask we employ a hackish approach via hiding/revealing masked vertices, this adds mode switches and doesn't allow to get actual mask values which could be useful if converted into vertex group weights for later use in e.g. modifiers. With this patch we'd be able to access mask data directly.
Reviewers: campbellbarton
Reviewed By: campbellbarton
Subscribers: radcapricorn
Projects: #bf_blender
Differential Revision: https://developer.blender.org/D782
on the grid.
This uses the Eigen conjugate-gradient solver to solve the implicit
Poisson equation for the pressure Laplacian:
div(grad(p)) = div(v)
As described in "Detail Preserving Continuum Simulation of Straight Hair"
(McAdams, Selle, 2009).
To make this work, we need separate type/instance for the widgets. This
is a bit ugly but it is necessary if we want widgets on each editor to
depend of per editor-options or visibility state.
This is a bit more awkward for artists to use, but necessary for
a stable solution of the hair continuum calculation. The grid size is
defined by the user, the extent of the grid is then calculated based on
the hair geometry. A hard upper limit prevents bad memory allocation
in case too small values are entered.
This is a leftover from previous approach of hair collisions (with
insufficient results). The hair volumetrics actually implements
"collision" with solid objects as well, but uses a Neumann boundary
condition on the main grid for this purpose.
This is based on the paper
"Detail Preserving Continuum Simulation of Straight Hair"
(McAdams, Selle, Ward, 2009)
The main difference is that hair line segments are used rather than only
rasterizing velocity at the vertices. This gives a much better coverage
of the hair volume grid, otherwise gaps can be produced at smaller grid
cell sizes and the distribution is uneven along the hair curve.
The algorithm for rasterizing is a variation of Bresenham's algorithm
extended onto 3D grids.
Still pretty much exploring the best way to move here but currently this
works by wrapping objects in RNA pointers, pretty much like buttons do.
Also modified the arrow widget for lamps to use the new code. The new
"unmbrella" widget works now as expected for lamps - of course fine
tuning is also needed.
* Add gpu options for depth of field to camera - still inactive.
* GPUFXOptions now passed to the compositing system startup
so users can swap options if they wish to
This commit changes read-write data again, people should reset their
values again (sorry for that but this is still WIP)
* Add gpu options for depth of field to camera - still inactive.
* GPUFXOptions now passed to the compositing system startup
so users can swap options if they wish to
This commit changes read-write data again, people should reset their
values again (sorry for that but this is still WIP)
This is working mostly from the 3D viewport currently, rendering from
the render buttons or the sequencer will not use the FX yet. Still it's
a solid step towards that direction. Also moved options out of the v3d
struct. Old options will be lost and people may get some warnings,
however simply enableing and disabling one of the compositing settings
will bring the controls back.
Algorithm used is now a full implementation of
"Practical Post-Process depth of field", presented here [1],
with a few modifications as to how circle of confusion is
applied at the final pass.
[1] http://http.developer.nvidia.com/GPUGems3/gpugems3_ch28.html
solver step.
Calculating forces and jacobians from linearly interpolated grid values
is problematic due to discontinuities at the grid boundaries. The new
approach of modifying velocities after the backward euler solver step
was suggested in a newer paper
"Detail Preserving Continuum Simulation of Straight Hair"
(McAdams, Selle 2009)
more intuitive.
Also removed a couple of unused or useless features from the UI:
* collider friction is unused and replaced in favor of true collision
* spring damping refers to structural springs (stretch), which is
not noticable in hair due to extreme stiffness atm.
* pressure factors are not sure since this feature is too unstable atm.
Interpolating hairs in a meaningful way is a quite difficult problem.
It works ok-ish for child hairs as long as they don't move, but with
motion the direction of interpolated hairs can become quite nonsensical
quickly.
Probably a better way to create simulation previews is to limit
simulation to a particular well-defined area (like render preview
borders), but run it in full detail in that area. The error from
omitting some interaction should still be much less than orientation
errors from interpolation.
easier.
This code is badly broken and needs to be replaced, but at least having
a workable code structure might help with quick hacks to fix the worst
cases.
shape instead of a brush tool.
The brush cutting tool for hair, while useful, is not very accurate and
often requires rotating the model constantly to get the right trimming
on every side. This makes adjustments to a hair shape a very tedious
process.
On the other hand, making proxy meshes for hair shapes is a common
workflow. The new operator allows using such rough meshes as boundaries
for hair. All hairs that are outside the shape mesh are removed, while
those cutting it at some length are shortened accordingly.
The operator can be accessed in the particle edit mode toolbar via the
"Shape Cut" button. The "Shape Object" must be set first and stays
selected as a tool setting for repeatedly applying the shape.
more intuitive.
Also removed a couple of unused or useless features from the UI:
* collider friction is unused and replaced in favor of true collision
* spring damping refers to structural springs (stretch), which is
not noticable in hair due to extreme stiffness atm.
* pressure factors are not sure since this feature is too unstable atm.
Interpolating hairs in a meaningful way is a quite difficult problem.
It works ok-ish for child hairs as long as they don't move, but with
motion the direction of interpolated hairs can become quite nonsensical
quickly.
Probably a better way to create simulation previews is to limit
simulation to a particular well-defined area (like render preview
borders), but run it in full detail in that area. The error from
omitting some interaction should still be much less than orientation
errors from interpolation.
* Scons builds correctly
* Separate vertex shader file for DOF, will make handling main easier
* Downsample color buffer + blur it. Circle of confusion is calculated
from zplane instead of length of distance from camera.
* Cleanup compositing buffers better when not needed.
* Support passes for the dof effect through defines in main - easy
solution to allow not having too many files for one shader effect.
easier.
This code is badly broken and needs to be replaced, but at least having
a workable code structure might help with quick hacks to fix the worst
cases.
Issue here was position reconstruction could use depths from pixels off
the initial position. Using linear filtering here eliminates the issue.
Usually this is not correct however given that for depth discontinuities
we will get different depths anyway, the cases where we get smooth
interpolation of depths helps a lot in image quality.
Basically change the way SSAO samples the screen by using a circular
filter instead. This is inspired by HBAO though we still don't use this.
The quality settings change the number and density of samples. We can
definitely improve things here though.
We also jitter the sampling locations per pixel. Generally this adds a
kind of noise. Some banding is, unfortunately still apparent and small
distances will introduce noise. This can be fixed but I will have to see
if it can be done without much of a cost.
shape instead of a brush tool.
The brush cutting tool for hair, while useful, is not very accurate and
often requires rotating the model constantly to get the right trimming
on every side. This makes adjustments to a hair shape a very tedious
process.
On the other hand, making proxy meshes for hair shapes is a common
workflow. The new operator allows using such rough meshes as boundaries
for hair. All hairs that are outside the shape mesh are removed, while
those cutting it at some length are shortened accordingly.
The operator can be accessed in the particle edit mode toolbar via the
"Shape Cut" button. The "Shape Object" must be set first and stays
selected as a tool setting for repeatedly applying the shape.
explicit attenuation control (by using just max distance).
Also it now supports orthographic cameras properly. The maximum distance is
now defined in world space units so it should be slightly easier to understand
and uniform across perspective and orthographic cameras.
The optimizations can be generalized for orthographic cameras but the code is
more complex and the runtime advantage is questionable in that case
(in the end a matrix transform is 4 dot products which are not that terrible)
so commented out the code there for now.
This needed some adjustments from the formula I found on the article on
view space reconstruction, since OpenGL makes different assumptions about its
projection matrix.
SSAO does not work yet correctly under new model assumptions but we are going
to substitute with nice horizon based ambient occlusion.
* highlighted vs active widget.
Active widget takes all input and basically highjacks drawing of all
other widgets in the widgetmap.
* operator names for widgets. Now widgets will spawn operators based on
names that are passed on them. They will also modify a named property of
the operator and pass it on, after which operator can process it.
The sound system really needs better design for the sequencer and
patching it up now won't really help us in the long run.
Also the sequencer kept leaking file descriptors when preserving
the audio handlles on undo making the branch unusable on mac and
windows.
It's almost certain patching over this will lead to more trouble, better
handle it separately and keep the system working for now.
Kept threaded loading of waveforms since we control the file input
directly there and leads to nice speedup, but we still clear the whole
sound + the waveform on undo, which is pretty bad.
* Move lamp operators to a proper place
* Store a list of widgets to avoid polling too many times
* Fix wrong indices when widgetgroup was not used.
* Lamp widget now uses an arrow widget. This will be refined to make a
nice prototype for operator/widget interaction.
Avoid moving the 'from' pointer around in DNA. This way files stay back-
as well as forward-compatible. Only the additional type enum and
sub-index is stored in a new from_extra struct in shape keys now.
This generates a somewhat believable overall look for the hair system
by blending simulated hairs. For small number of hairs and small
preview ratios this can get quite inaccurate, but that's the tradeoff
for getting preview functionality ...
This is still terribly messy, especially because every change of the
particle system has to be piped through the dummy cloth modifier somehow.
The automatic caching system also tends to get in the way a lot; after
changing the preview factor one basically always has to start at frame 1
currently because the cloth modifier has to be reconstructed and forgets
about all caching.
* Add option for shaded/non shaded widgets
* Normalize arrow widget size to 1.0 so size is predictable. Probably
will use the same scheme for other widgets as well
format (vertices + normals + triangle vertex indices)
This will allow us to create widgets in blender instead of defining them
programmatically.
May also support facegroups for different colors in the future though
they must be somehow tied to the UI colors.
Especially if we have many widgets on screen, sorting them will be
difficult and for mesh-armature widgets it will be unavoidable if we
want to avoid tweaking "back" surfaces.
There is first code for display of an arrow widget. The manipulator will
use those when they are ready.
Basically the manipulator is now a WidgetGroup. WidgetGroups are a new entity and
they manage a group of related widgets. They should be responsible for state update,
drawing and polling of their widgets and can be bound to specific rules.
There is a big design plan on a whiteboard beside me, it will take a few
big commits still to take shape though. One thing at a time, I am first making
reusable arrow widgets that will allow us replace the translate widgets properly.
There are a few reusable widgets planned, more to come later.
distribution and path caching for child particles.
This gives a significant improvement of viewport playback performance
with higher child particle counts. Particles previously used their own
threads and had a rather high limit for threading. Also threading
apparently was disabled because only 1 thread was being used ...
This is not necessary: the implicit solver data can keep track instead
of how many off-diagonal matrix blocks are in use (provided the
allocation limit is calculated correctly). Every time a spring is
created it then simply increments this counter and uses the block index
locally - no need to store this persistently.
Without this the particle system only shows the actual non-simulated
hairs ("guide hairs") during edit mode. These hairs are used for goals
as well, so showing them in the regular viewport is pretty important.
Also the usual hair curves are interpolated along the entire length,
which makes it very difficult to see exact vertex positions, unless
using exact powers of 2 for the segment number and match the display
steps.
too.
Rationale - widgets might be useful to have even on rendered scenes -
we'll have to see about how this will work
Also use area initialization code to register region specific widgets once.
lala
The curl radius for children in interpolated mode was calculated using
the total offset from the parent particle. This leads to very large
radii when the distance is large due to sparse parents. Such behavior is
also very unrealistic because the curl radius is mostly constant and
defined by the material properties.
All the child hairs are roughly parallel by default. To simulate the
agglomeration of children into hair wisps the "flatness" parameter is
now used to clump them together.
This should use an "active" system, similar to buttons to detect if last
widget has changed and act accordingly. Currently, when the active
widget changes, there is no change in highlights.
Code that handles 3D object intersection via GPU selection. We might
want to use raycasting here, but leaving that for when we have it.
Manipulator code now uses this API to do intersection testing
Widgets use a similar notification system as operators. It might be
worth using WM_BREAK messages like the rest of the handlers. For now
this should do.
With the default 5 substeps the simulation can otherwise still become
unstable. This is just a preliminary measure anyway until the length
variance can be fixed properly.
This is more involved than using simple straight bending targets
constructed from the neighboring segments, but necessary for restoring
groomed rest shapes.
The targets are defined by parallel-transporting a coordinate frame
along the hair, which smoothly rotates to avoid sudden twisting (Frenet
frame problem). The rest positions of hair vertices defines the target
vectors relative to the frame. In the deformed motion state the frame
is then recalculated and the targets constructed in world/root space.
This commit moves sound loading out of library linking. The approach
chosen here is to do lazy initialization by basically validating the
sound before attaching to the scene.
Loading the sound buffer requires a valid Main pointer, which means we
have to pass this around in many places. We might also consider caching
the waveforms in this way too, but for now we're still using the
threaded update.
We'll have to test this somewhat to see if it behaves well..There are
corner cases which might not be covered. Sounds are muted if we undo
during an animation, but this bug existed in master as well.
Sound loading can take too much so move it to a job and continue. This makes
interaction much better. A similar system should be usable for movie previews.
derivatives for stabilization.
The bending forces are based on a simplified torsion model where each
neighboring point of a vertex creates a force toward a local goal. This
can be extended later by defining the goals in a local curve frame, so
that natural hair shapes other than perfectly straight hair are
supported.
Calculating the jacobians for the bending forces analytically proved
quite difficult and doesn't work yet, so the fallback method for now
is a straightforward finite difference method. This works very well and
is not too costly. Even the original paper ("Artistic Simulation of
Curly Hair") suggests this approach.
This returns a general status (success/no-convergence/other) along with
basic statistics (min/max/average) for the error value and the number
of iterations. It allows some general estimation of the simulation
quality and detection of critical settings that could become a problem.
Better visualization and extended feedback can follow later.
waveform display.
Basically, we store the waveform. This is not really recommended but
until we have nice threaded creation for the strips that doesn't block
the interface it's a nice solution for gooseverry.
May be wise to limit the undo memory for people who use that because
basically each operation will store a copy of the waveform in the file.
This makes the bending a truely local effect. Eventually target
directions should be based in a local coordinate frame that gets
parallel transported along the curve. This will allow non-straight
rest shapes for hairs as well as supporting twist forces. However,
calculating locally transformed spring forces is more complicated.
This commit has first WIP code for widgets API.
Generally widgets should be registered similar to dropboxes, that is,
any code can query for a widgetmap for a certain space and area type
and the window manager will take care of initializing the handlers
automatically.
The owner will be able remove and readd widgets to a certain widgetmap
dynamically. Generally owners should cleanup their widgets though initial code
cleans up leftover widgets automatically.
All this is the theory still, some parts are in that commit, more to come
later, but it's a solid base to build from for an initial commit.
These are much better suited for creating stiff hair. The previous
bending springs are based on "push" type spring along the hypothenuse
of 3 hair vertices. This sort of spring requires a very large force
in the direction of the spring for any angular effect, and is still
unstable in the equilibrium.
The new bending spring model is based on "target" vectors defined in a
local hair frame, which generates a force perpendicular to the hair
segment. For further details see
"Artistic Simulation of Curly Hair" (Pixar technical memo #12-03a)
or
"A Mass Spring Model for Hair Simulation" (Selle, Lentine, Fedkiw 2008)
Currently the implementation uses a single root frame that is not yet
propagated along the hair, so the resulting rest shape is not very
natural. Also damping and derivatives are still missing.
single transform matrix.
Dynamic properties of the transformation are only needed during the
setup phase when they should be read from external data (hair system
roots) and generate fictitious forces on each point.
This is part of the original method from "Volumetric Methods for
Simulation and Rendering of Hair". The current filter is a simple box
filter. Other energy-preserving filters such as gaussian filtering
can be implemented later.
The filter size is currently given as a cell count. This is not ideal,
rather it should use a geometrical length value, but this is too
abstract for proper artistical use. Eventually defining the whole grid
in terms of spatial size might work better (possibly using an external
object).
at a the margin distance ("outer" softbody margin).
This has to be clamped arbitrarily unfortunately, otherwise the
repulsion force can add too much energy into the system. A factor of
4 * restitution impulse seems to give good results for now, this can
be refined later on if necessary.
This is now also decoupled from the internal solver data. The grid is
created as an opaque structure, filled with vertex or collider data
(todo), and then forces can be calculated by interpolating the grid at
random locations. These forces and derivatives are then fed into the
solver.
The hair solver needs sane input to converge within reasonable time
steps. In particular the spring lengths must not be too difference
(factor 0.01..100 or so max, this is comparable to rigid body simulation
of vastly different masses, which is also unstable).
The basic hair system generate strands with equally spaced points, which
is good solver material. However, the hair edit operators, specifically
the cutting tool, can move points along the strands, creating tightly
packed hair points. This puts the solver under enormous stress and
causes the "explosions" observed already during the Sintel project.
The simple solution for now is to exclude very short hairs from the
simulation. Later the cutting tool should be modified such that it
keeps the segments roughly at the same length and throws away vertices
when the hair gets too short (same goes for the extension tool).
The hair system should have a general mechanism for making sure that
situations such as this don't occur. This will have to be a design
consideration for replacements in any future hair system.
Also avoid recreating the framebuffers and gpu/color textures each
frame.
Happiness :)
There are still some glithes when scaling the areas, I'll look at those
next.
* Expose attenuation value - allows to change influence of far objects
* Change influence of far objects to quadratic, eliminates some extreme
fringing from far occluding surfaces.
This works by using the calculated view space normal and accumulating
coverage of a certain area by nearby pixels. There are two sliders to
control the effect:
* Scale controls the area around each pixel that the shader "collides"
against
* Darkening scales the occlusion effect.
The effect works, but due to the way the normals are calculated, the
normals are never smooth shaded (that would require a separate render
target to store them) and the edges or polygons can be too apparent.
This is not really fixable at the moment unless we move to deferred
pipeline.
The FX system is stll not well optimized and rendering does not always
work correctly, but it's good to have this out for people to play with.
moving hair root reference frames.
This calculates Euler, Coriolis and Centrifugal forces which result
from describing hair in a moving reference frame.
http://en.wikipedia.org/wiki/Fictitious_force
frames.
These forces don't have to be calculated for each individual
contribution. Rather they can be split off and be calculated on top of
the basic force vector rotation (todo).
There are currently 3 types of springs: basic linear springs, goal
springs toward a fixed global target (not recommended, but works) and
bending springs.
These are agnostic to the specific spring definition in the cloth system
so hair systems can use the same API without converting everything to
cloth first.
code.
The implicit solver itself should remain agnostic to the specifics of
the Blender data (cloth vs. hair). This way we could avoid the bloated
data conversion chain from particles/hair to derived mesh to cloth
modifier to implicit solver data and back. Every step in this chain adds
overhead as well as rounding errors and a possibility for bugs, not to
speak of making the code horribly complicated.
The new subfolder is named "physics" since it should be the start of a
somewhat "unified" physics systems combining all the various solvers in
the same place and managing things like synchronized time steps.
This commit introduces offscreen rendering for the 3D viewport.
There is test code that calculates depth and normal data
in screen space.
Currently just DOF is supported with a few parameters to control the
effect
This code recreates the framebuffer and texture objects every frame
which will make it -slow-. Use at your own risk.
handle only one collision contact at a time.
Collision still randomly explodes, even with differing results on the
same file. This could indicate a threading issue, possibly also related
to the dependency graph since multiple objects are involved in
collisions.
This reverts commit c52b8ae818.
Sadly, at this point solver convergence is an exception rather than the
rule... Individual hairs can "explode" easily and thus disable the whole
simulation, which isn't helpful either.
This helps keep the simulation stable as long as there are only a few
substeps that become too constrained for the solver.
Eventually we need better feedback about these solver results, so that
artists can tweak situations specifically to resolve bad solver results.
This is somewhat similar to the camera tracker, which also can run into
cases that cannot be resolved and have to be fixed manually.
The Eigen solver is not quite stable currently (possibly due to
incorrect porting of force calculations). It also still lacks threading
support and optimized matrix construction, making it slower in
comparison. Eventually would still like to switch, but fixing these
issues takes time.
This adds transformations for each hair from world to "root space".
Currently positions and velocities are simply transformed for the solver
data and inverse-transformed when copying the results back to the cloth
data. This way the hair movement becomes independent from the movement
of the emitter object. Eventually the "fictitious" forces originating
from emitter movement can be added back in a controlled way.
http://en.wikipedia.org/wiki/Fictitious_force
Ignoring these fictitious forces or scaling their effect is physically
correct, because in the absence of external forces the hair will always
return to rest position in this root frame.
External forces currently are not yet transformed into the root space.
mode.
Eigen can become very slow in debug mode, which is a bit of a problem.
It relies heavily on compiler optimizations to remove function calls
etc. More optimizations may be desirable, possibly putting the implicit
solver into its own little library and enabling optimizations in debug
mode there could help.
This will allow us to implement moving reference frames for hair and
make "fictitious" forces optional, aiding in creating stable and
controllable hair systems.
Adding data in this place is a nasty hack, but it's too difficult to
encode as a DM data layer and the whole cloth modifier/DM intermediate
data copying for hair should be removed anyway.
This is not nice at all, but for some reason (possibly time scale) the
old force values are much too high and cause the solver to become
unstable. These will be revisited later anyway, so for now such scaling
should be fine.
constraining.
The algorithm is described in the paper "Large Steps in Cloth Simulation"
(Baraff/Witkin 1998). The same method was (incorrectly) implemented in
the old cloth solver.
It is based on restricting the degrees of freedom (ndof) of vertices
using a block matrix and a vector of target velocity deltas.
See chapter 5 of the paper for details.
Note that goal springs currently are really bad ... They have a factor
on hairs that "fades" goal influence from the root to the tip. The last
point on the hair is completely free, which makes the goal springs
pretty much useless on their own without supporting bend stiffness.
Can only assume this was added to compensate unphysical behavior of
goal springs when using uniform weight, but it's a poor replacement for
true localized bending forces ...
a custom built solver.
The old cloth solver is broken unfortunately. Eigen is a designated
linear algebra library and very likely their implementation is a lot
better (can't compare until it's implemented though).
Only basic gravity is active atm, spring forces, external force fields,
damping and volumetric friction have to be added back by converting
the data into the Eigen format.
responses.
The S matrix together with the z Vector encodes the degrees of freedom
of a colliding hair point and the target velocity change. In a collision
the hair vertex is restricted in the normal direction (when moving
toward the collider) and the collision dynamics define target velocity.
This simply uses the position above the triangle instead of the
intersection point of the vertex path. The other method was broken
anyway, but also has a problem catching all the contacts reliably. The
new method might have a few false positives but that is acceptable.
the contents of the strip around.
Only tested with movie strips now, metastrips won't work. Also there
might be some lingering bugs. Sonetimes refresh is needed to get
contents showing up correctly.
solver that properly supports constraints with some degrees-of-freedom.
The previous solver implementation only used the S matrix (constraint
filter matrix) for pinning vertices, in which case all elements are
zero and the error doesn't show up. With partial constraints (useful for
collision contacts) the matrix has non-zero off-diagonal elements and
the algorithm easily diverges.
There are also initial steps for implementing collision prevention as
described in the Baraff/Witkin paper "Large Steps in Cloth Simulation"
(http://www.cs.cmu.edu/~baraff/papers/sig98.pdf).
This is a first test, the contacts are very explosive atm because they
basically pin hair vertices globally on collision, which leads to
stretching of the springs which is then suddenly released in the next
frame.
Instead of handling contact tests and collision response in the same
function in collision.c, first generate contact points and return them
as a list, then free at the end of the stepping function. This way the
contact response can be integrated into the conjugate gradient method
properly instead of using the hackish and unstable double evaluation
that is currently used.
The original BLI method for line/triangle intersection returns false
in case the line does not actually intersect, but in order to generate
repulsion forces we need to also handle contacts inside the margin.
timestep segment.
This ensures the distance for a collision pair is the one of the current
point position, and the response gets calculated accordingly.
simulation.
Note that this currently generates an extreme amount of points, by
making a edit pathcache curve for each hair in every frame! But at least
doesn't simply crash now.
This implements a penalty force as well as a repulsion force to avoid
further penetration, as suggested in
"Simulating Complex Hair with Robust Collision Handling"
(http://graphics.snu.ac.kr/publications/2005-choe-HairSim/Choe_2005_SCA.pdf)
Friction forces are still missing. More problematic is handling of
moving colliders, when face swap places with the hair vertex and a
collision is missed, putting the vertex inside the mesh volume. Larger
margins might help, but ultimately using Bullet collision detection is
probably more reliable and failsafe.
as forces, velocities, contact points etc.
This uses a hash table to store debug elements (dots, lines, vectors at
this point). The hash table allows continuous display of elements that
are generated only in certain time steps, e.g. contact points, while
avoiding massive memory allocation. In any case, this system is really
a development feature, but very helpful in finding issues with the
internal solver data.
This is still using the old BVH tree collision methods to generate
contact points, similar to what cloth does. This should be replaced
by a Bullet collision check, but generating contacts in this way is
easier for now, and lets us test responses and stability (although in
more complex collision cases the BVH method fails utterly, beside being
terribly inefficient with many colliders).
on itself.
This uses the same voxel structure as the hair smoothing algorithm.
A slightly different method was suggested in the original paper
(Volumetric Methods for Simulation and Rendering of Hair), but this is
based on directing hair based on a target density, which is another
way of implementing global goals. Our own approach is to define a
pressure threshold above which the hair is repelled in the density
gradient direction to simulate internal pressure from collisions.
This is an important hair interaction feature that simulates friction
between hairs in an efficient way. The method is based on the paper
"Volumetric Methods for Simulation and Rendering of Hair"
( http://graphics.pixar.com/library/Hair/paper.pdf )
It was partially implemented already, but didn't work in this simplified
version. The same voxel structure can be used for implemeting repelling
forces on hair based on density, which can help a hair system maintain
volume instead of collapsing in on itself.
the suggested tent function from the original paper.
Plain float->int conversion for the grid location otherwise leads to
skewed data and unnecessary loss of information.
COMMAND${CMAKE_COMMAND}-Ecopy"${PYTHON_OUTPUTDIR}/python${PYTHON_SHORT_VERSION_NO_DOTS}${PYTHON_POSTFIX}.lib"${BUILD_DIR}/python/src/external_python/run/libs/python${PYTHON_SHORT_VERSION_NO_DOTS}.lib#missing postfix on purpose, distutils is not expecting it
COMMAND${CMAKE_COMMAND}-Ecopy"${PYTHON_OUTPUTDIR}/python${PYTHON_SHORT_VERSION_NO_DOTS}${PYTHON_POSTFIX}.lib"${BUILD_DIR}/python/src/external_python/run/libs/python${PYTHON_SHORT_VERSION_NO_DOTS}${PYTHON_POSTFIX}.lib#other things like numpy still want it though.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
