Summary:
Behaves very much the same as cache for Movie Clip datablock:
- Image now have `MovieCache *cache` field which replaced
legacy `ListBase ibufs`.
This allows image datablock to easily keep of image
buffers which are owned by itself. This field isn't
saved to the file and getting restored on undo steps.
However, cache limit is global for movies, sequences
and image datablocks now. So overall cached image buffers
size will not go above cache limit size in user
preferences.
- Image buffers which are marked as BITMAPDIRTY will never
be freed from the cache.
- Added utility function to iterate over image buffers
saved in movie cache.
- Movie cache cleanup check callback now have ImBuf argument
which can be used in a condition of cleanup.
- Added some utility functions which replaces legacy ibufs
iterations with image cache iteration which happens from
inside a lock.
- Fixed `image_mem_size()` which was only counting one of
the buffers if both float and byte buffer present.
Additional notes:
- `BKE_image_get_first_ibuf()` is rather stupid, but direct
access to ibufs->first was also the same stupid idea.
Would consider avoid this function is another project.
- There are some places which doesn't look threadsafe, but
they already were not so much threadsafe anyway before.
So think not a big deal with solving this later.
Finally solves infinite memory usage by image sequences! :)
Reviewers: brecht, campbellbarton
Reviewed By: brecht
CC: sebastian_k
Differential Revision: http://developer.blender.org/D95
- Made tooltip for compression a bit more clear.
- Use default compression of 15%. Gives around
two times boost on export movie file here in
test with only 10% increased file size.
Reviewers: Jonathan Williamson
generally speaking, if multilayer image fails to load for current
frame doesn't mean anything bad. It might be used to make it so
image sequence is being alpha-overed somewhere in the middle of
scene time.
Made it so if the whole file fails to load, image node will
deliver black transparent color, the same what happens for
regular (non-multilayer images).
Also needed to tweak code in load_multilayer_sequwnce to make
sure no cached frames are pointing to a freed memory.
We now support the combined layer of Photoshop files (stored as layer 0
in the file). This way users can keep their files as multilayer PSD and
Blender always handle them as flat images.
For perfect alpha this requires an OpenImageIO update:
342cc2633f
Photoshop sample files:
https://github.com/OpenImageIO/oiio-images
Brecht has some pending fixes to push for OIIO as well, so we may as
well wait to update our libraries.
What works:
===========
* 8bit images (with or without alpha)
* 16bits images (alpha discarded)
* Photoshop files saved with 'Maximum Compatibility'
* Cycles, Blender internal, BGE (and player)
Known limitations
(due to OIIO dependency):
=========================
* Images with less than 4 channels show a wrong thumbnail (bug may be in OIIO)
* Packed images are not supported
* We do not write PSD files.
Note: old Blenders have support for PSD via Quicktime library. But due
to license issues this was discontinued.
Many thanks for Brecht van Lommel for reviewing the patch, suggesting
multiple improvements and to help solving the alpha issue.
forgetting to change it from RGB when painting images or saving renders.
This makes more sense in the new alpha pipeline, renders no longer contain an
alpha channel that does not match the RGB channels, so saving files as RGBA
should give the expected results when opening them in other applications.
Issue was caused by reload caused by input colorspace change.
For generated images generated alpha flags weren't saved to
DNA, which lead to fallback from 32 bit depth to 24 when
doing any kind reload of generated image.
The same alpha loss happens when you save .blend file with
generated images.
Now added generated depth to DNA, so reload image and .blend
file wouldn't loss alpha.
were no properly updating when rendering animations.
The render engine was only updating the image user current frame on images used
by material textures. Now moved the function that updates all from the editors
to blenkernel level and do it on all frame changes.
- pass string size to BLI_timestr() to avoid possible buffer overrun.
- quiet warning for mingw.
- include guards for windows utf conversion funcs.
- fix for mistage in edge-angle-selection check.
- some style cleanup.
Issue was caused by delayed or missing image user frame
number update, which lead to image loading failure in
cases node is updating from image signal callback.
Solved in a way that file from image datablock is used
for sockets detection instead of loading image for
current frame.
PyNodes opens up the node system in Blender to scripters and adds a number of UI-level improvements.
=== Dynamic node type registration ===
Node types can now be added at runtime, using the RNA registration mechanism from python. This enables addons such as render engines to create a complete user interface with nodes.
Examples of how such nodes can be defined can be found in my personal wiki docs atm [1] and as a script template in release/scripts/templates_py/custom_nodes.py [2].
=== Node group improvements ===
Each node editor now has a tree history of edited node groups, which allows opening and editing nested node groups. The node editor also supports pinning now, so that different spaces can be used to edit different node groups simultaneously. For more ramblings and rationale see (really old) blog post on code.blender.org [3].
The interface of node groups has been overhauled. Sockets of a node group are no longer displayed in columns on either side, but instead special input/output nodes are used to mirror group sockets inside a node tree. This solves the problem of long node lines in groups and allows more adaptable node layout. Internal sockets can be exposed from a group by either connecting to the extension sockets in input/output nodes (shown as empty circle) or by adding sockets from the node property bar in the "Interface" panel. Further details such as the socket name can also be changed there.
[1] http://wiki.blender.org/index.php/User:Phonybone/Python_Nodes
[2] http://projects.blender.org/scm/viewvc.php/trunk/blender/release/scripts/templates_py/custom_nodes.py?view=markup&root=bf-blender
[3] http://code.blender.org/index.php/2012/01/improving-node-group-interface-editing/
Before this change only old flag "Premultiply" was used to
detect alpha mode, which is not enough actually.
Now the logic here is:
- If "Premultiply" was enabled it is likely float image with
straight alpha, which shall be premultiplied before usage.
In this case image/sequence Alpha Mode is set to Straight.
- Otherwise use default alpha mode for image format based on
an extension. This could fail in some cases like TIFF, but
this wasn't handled fully correct in older blender anyway.
Initial discovered issue was that EXR images saved in older
Blender versions were set to Straight alpha mode, which is
obviously a straight way to lots of headache.
The use alpha option moved from the texture datablock to the image, and now it
will duplicate the image datablock in case you have one texture using alpha and
the other not.
Issue was caused by couple of circumstances:
- Normal Map node requires tesselated faces to compute tangent space
- All temporary meshes needed for Cycles export were adding to G.main
- Undo pushes would temporary set meshes tessfaces to NULL
- Moving node will cause undo push and tree re-evaluate fr preview
All this leads to threading conflict between preview render and undo
system.
Solved it in way that all temporary meshes are adding to that exact
Main which was passed to Cycles via BlendData. This required couple
of mechanic changes like adding extra parameter to *_add() functions
and adding some *_ex() functions to make it possible RNA adds objects
to Main passed to new() RNA function.
This was tricky to pass Main to RNA function and IMO that's not so
nice to pass main to function, so ended up with such decision:
- Object.to_mesh() will add temp mesh to G.main
- Added Main.meshes.new_from_object() which does the same as to_mesh,
but adds temporary mesh to specified Main.
So now all temporary meshes needed for preview render would be added
to preview_main which does not conflict with undo pushes.
Viewport render shall not be an issue because object sync happens from
main thread in this case.
It could be some issues with final render, but that's not so much
likely to happen, so shall be fine.
Thanks to Brecht for review!
Issue was caused by FILE multilayer ImBuf sharing buffers with render
result, and SEQUENCE multilayer ImBufs duplicating buffers. Which is
nice by it's own. But, changing image source wouldn't remove any loaded
image buffers, meaning if you've got loaded FILE multilayers they'll
likely became invalid.
That behavior of handling multilayers on changing source was done as
a fix for #24976, which is now actually not needed (removing check
for multilayer doesn't change behavior at all).
Just to be sure added check to RNA, so signal wouldn't be fired if
source wasn't actually changed form a menu.
It was caused by image threading safe commit and it was noticeable
only on really multi-core CPU (like dual-socket Xeon stations), was
not visible on core i7 machine.
The reason of slowdown was spinlock around image buffer referencing,
which lead to lots of cores waiting for single core and using image
buffer after it was referenced was not so much longer than doing
reference itself.
The most clear solution here seemed to be introducing Image Pool
which will contain list of loaded and referenced image buffers, so
all threads could skip lock if the pool is used for reading only.
Lock only needed in cases when buffer for requested image user is
missing in the pool. This lock will happen only once per image so
overall amount of locks is much less that it was before.
To operate with pool:
- BKE_image_pool_new() creates new pool
- BKE_image_pool_free() destroys pool and dereferences all image
buffers which were loaded to it
- BKE_image_pool_acquire_ibuf() returns image buffer for given
image and user. Pool could be NULL and in this case fallback to
BKE_image_acquire_ibuf will happen.
This helps to avoid lots to if(poll) checks in image sampling
code.
- BKE_image_pool_release_ibuf releases image buffer. In fact, it
will only do something if pool is NULL, in all other case it'll
equal to DoNothing operation.
This adds support of movie textures for Cycles rendering.
Uses the same builtin images routines as packed/generated images,
but with some extra non-rna hookups from blender_session side.
Basically, it's not so clear how to give access to video frames
via C++ RNA -- it'll require exposing ImBuf to API, doing some
threading locks and so. Ended up adding two more functions which
are actually bad level call, but don't consider it's so much bad
-- we have few bad calls already, which are actually related.
Changed a bit how builtin images names are passing to image
manager. Now it's not just an ID datablock name, but also a frame
number concatenated via '@' character, which makes itpossible to
easily know frame number to be used for movie images, without
adding extra descriptors to image manager.
Decoding of builtin name is a bit slower now, but it should be
still nothing in comparison with rendering complexity.
Also exposed image user's frame_current to python API, which
is needed to get absolute frame number of movie from node's
image user.
P.S. Generated/packed images are also using bad level call but
only does it to make things more clear here. Either all images
are using C++ RNA here or no images does. That's the most clear
for now.
This assumptions are now made:
- Internally float buffers are always linear alpha-premul colors
- Readers should worry about delivering float buffers with that
assumptions.
- There's an input image setting to say whether it's stored with
straight/premul alpha on the disk.
- Byte buffers are now assumed have straight alpha, readers should
deliver straight alpha.
Some implementation details:
- Removed scene's color unpremultiply setting, which was very
much confusing and was wrong for default settings.
Now all renderers assumes to deliver premultiplied alpha.
- IMB_buffer_byte_from_float will now linearize alpha when
converting from buffer.
- Sequencer's effects were changed to assume bytes have got
straight alpha. Most of effects will work with bytes still,
however for glow it was more tricky to avoid data loss, so
there's a commented out glow implementation which converts
byte buffer to floats first, operates on floats and returns
bytes back. It's slower and not sure if it should actually
be used -- who're using glow on alpha anyway?
- Sequencer modifiers should also be working nice with straight
bytes now.
- GLSL preview will predivide float textures to make nice shading,
shading with byte textures worked nice (GLSL was assuming straight
alpha).
- Blender Internal will set alpha=1 to the whole sky. The same
happens in Cycles and there's no way to avoid this -- sky is
neither straight nor premul and doesn't fit color pipeline well.
- Straight alpha mode for render result was also eliminated.
- Conversion to correct alpha need to be done before linearizing
float buffer.
- TIFF will now load and save files with proper alpha mode setting
in file meta data header.
- Remove Use Alpha from texture mapping and replaced with image
datablock setting.
Behaves much more predictable and clear from code point of view
and solves possible regressions when non-premultiplied images were
used as textures with ignoring alpha channel.
This commit adds a support of saving 16bit PNG files.
Alpha for such files would be premultiplied, would be corrected
with an upcoming alpha premul cleanup (it's not the only format
which will output 16bit image with premul alpha).
This codec is absolutely needed to generate DCP using OpenDCP,
before that external application to convert JP2 to J2K was used
which slowed down export a lot.
New codec is exposed to image format settings panel and called
Codec. Default one is JP2 which creates files with .jp2 extension,
new one is called J2K which creates with .j2c extension.
Other changes:
- Fixed avi jpeg warning which was treating as error here.
- Made it so extension is detecting from ImageFormatData instead
of image file type, which makes it possible to have different
extension for the same file type depending on it's settings.
IRIS format should still be changed (depending on number of
channels it'll be .bw, .rgb or .rgba extension)
- Default image format settings would be set from image buffer
when re-saving it. Makes it possible to easily open .j2c file
and save it using J2K codec (without this change it'll save as
.jp2 using JP2 codec)
now blenlib/BLI doesn't depend on any blenkern/BKE functions,
there are still some bad level includes but these are only to access G.background and the blender version define.
