Summary:
Made objects update happening from multiple threads. It is a task-based
scheduling system which uses current dependency graph for spawning new
tasks. This means threading happens on object level, but the system is
flexible enough for higher granularity.
Technical details:
- Uses task scheduler which was recently committed to trunk
(that one which Brecht ported from Cycles).
- Added two utility functions to dependency graph:
* DAG_threaded_update_begin, which is called to initialize threaded
objects update. It will also schedule root DAG node to the queue,
hence starting evaluation process.
Initialization will calculate how much parents are to be evaluation
before current DAG node can be scheduled. This value is used by task
threads for faster detecting which nodes might be scheduled.
* DAG_threaded_update_handle_node_updated which is called from task
thread function when node was fully handled.
This function decreases num_pending_parents of node children and
schedules children with zero valency.
As it might have become clear, task thread receives DAG nodes and
decides which callback to call for it.
Currently only BKE_object_handle_update is called for object nodes.
In the future it'll call node->callback() from Ali's new DAG.
- This required adding some workarounds to the render pipeline.
Mainly to stop using get_object_dm() from modifiers' apply callback.
Such a call was only a workaround for dependency graph glitch when
rendering scene with, say, boolean modifiers before displaying
this scene.
Such change moves workaround from one place to another, so overall
hackentropy remains the same.
- Added paradigm of EvaluaitonContext. Currently it's more like just a
more reliable replacement for G.is_rendering which fails in some
circumstances.
Future idea of this context is to also store all the local data needed
for objects evaluation such as local time, Copy-on-Write data and so.
There're two types of EvaluationContext:
* Context used for viewport updated and owned by Main. In the future
this context might be easily moved to Window or Screen to allo
per-window/per-screen local time.
* Context used by render engines to evaluate objects for render purposes.
Render engine is an owner of this context.
This context is passed to all object update routines.
Reviewers: brecht, campbellbarton
Reviewed By: brecht
CC: lukastoenne
Differential Revision: https://developer.blender.org/D94
It was only used by opengl render and in fact it needed just to
set DISPLAY_BUFFER_INVALID flag for the image buffer.
In theory it wouldn't make any change to opengl render speed
(because this change just moved rect_from_float from color
management code to image save code). And could not see any speed
changes on my laptop.
were no properly updating when rendering animations.
The render engine was only updating the image user current frame on images used
by material textures. Now moved the function that updates all from the editors
to blenkernel level and do it on all frame changes.
of editmode on the child object.
Problem was that the object custom data mask was not taken into account when
rebuilding the derivedmesh in some cases, which is needed for the derivedmesh
to contain the mapping back to the original vertices. Now this data mask is
used for any derivedmesh build that will be cached.
Also problematic was that the datamask for the active object was applied to
all objects in the scene, which caused the parent object to be recalculated
when it didn't need to be. Now this datamask is only used for the active object.
Doing linearization with GLSL was already faster, but even faster is to just read the
bytes instead of floats and convert those to linear, since byte => float is just a quick
256 entry table lookup. Also made it assign the bytes directly to the image buffer so
they do not need to be converted back from float to byte for file saving, and made sky
render write the background color with OpenGL instead of doing it on the CPU.
Uses GLSL for drawing image in Image Editor space.
This requires change in image_buffer_rect_update, so
original float buffer is being updated as well. This
is unlikely be something bad, but will keep an eye
on this change.
Also no byte buffer allocation happens there, this
is so because byte buffer used for display only
and in case of GLSL display such allocation and
partial update is just waste of time.
Also switched OpenGL render from using CPU color
space linearization to GLSL color space transform.
Makes OpenGL rendering pretty much faster (but
still slower than in 2.60).
internal changes:
- Added functions to setup GLSL shader for color
space conversion in colormanagement.c. Currently
conversion form a colorspace defined by a role to
linear space is implemented. Easy to extend to
other cases.
- Added helper functions to glutil.c which does
smarter image buffer draw (calling all needed OCIO
stuff, editors now could draw image buffer with a
single function call -- all the checks are done in
glutil.c).
- Also added helper function for buffer linearization
from a given role to glutil.c. Everyone now able to
linearize buffer with a single call.
This function will do nothing is GLSL routines fails
or not supported.
And one last this: this function uses offscreen
drawing, could potentially give issues on some
cards, also will keep an eye on this.
with transparency, would show as too dark colors on edges.
Found a strange issue here though, the alpha value in the OpenGL render result
is not the same as the one specified in the material. It's not clear to me why
this happens, color space conversions should not influence the alpha channel.
without hurting quick texture painting
- ED_view3d_draw_offscreen will now output buffer with
transparent alpha, if sky needed it should be alpha-undered
later.
- ED_view3d_draw_offscreen_imbuf now accepts alpha mode as an
argument which could be either R_ADDSKY or R_PREMULALPHA
- OpenGL render and sequencer's opengl preview will now reflect
scene's Alpha Mode
- Quick Edit will use OpenGL with transparent alpha mode
This codec is absolutely needed to generate DCP using OpenDCP,
before that external application to convert JP2 to J2K was used
which slowed down export a lot.
New codec is exposed to image format settings panel and called
Codec. Default one is JP2 which creates files with .jp2 extension,
new one is called J2K which creates with .j2c extension.
Other changes:
- Fixed avi jpeg warning which was treating as error here.
- Made it so extension is detecting from ImageFormatData instead
of image file type, which makes it possible to have different
extension for the same file type depending on it's settings.
IRIS format should still be changed (depending on number of
channels it'll be .bw, .rgb or .rgba extension)
- Default image format settings would be set from image buffer
when re-saving it. Makes it possible to easily open .j2c file
and save it using J2K codec (without this change it'll save as
.jp2 using JP2 codec)
This commit makes BKE_image_acquire_ibuf referencing result, which means once
some area requested for image buffer, it'll be guaranteed this buffer wouldn't
be freed by image signal.
To de-reference buffer BKE_image_release_ibuf should now always be used.
To make referencing working correct we can not rely on result of
image_get_ibuf_threadsafe called outside from thread lock. This is so because
we need to guarantee getting image buffer from list of loaded buffers and it's
referencing happens atomic. Without lock here it is possible that between call
of image_get_ibuf_threadsafe and referencing the buffer IMA_SIGNAL_FREE would
be called. Image signal handling too is blocking now to prevent such a
situation.
Threads are locking by spinlock, which are faster than mutexes. There were some
slowdown reports in the past about render slowdown when using OSX on Xeon CPU.
It shouldn't happen with spin locks, but more tests on different hardware would
be really welcome. So far can not see speed regressions on own computers.
This commit also removes BKE_image_get_ibuf, because it was not so intuitive
when get_ibuf and acquire_ibuf should be used.
Thanks to Ton and Brecht for discussion/review :)
Screencast recording stopped on a undo/redo. This was because all thread jobs
were killed then. Now it leaves screen jobs (screen cast) running, that's
data that doesn't change on undos.
Also renamed jobs_stop_all() to jobs_kill_all() - it terminates threads.
Issue was caused by buffer shadows were binding buffer after offscreen buffers
was bind which lead to some unpredictable results.
Made it so ED_view3d_draw_offscreen wouldn't bind any buffers and for proper
shadows ED_view3d_draw_offscreen_init should be manually be called before
drawing to an offscreen.
This should also make open gl render with AA enabled a bit faster.
Also fixed missing sequencer cache invalidation when open gl render type is
changing.
Material and Rendered modes are still a TODO for sequencer.
Color management would be applied on both of float and byte buffers on image
save in cases if file format doesn't require linear float buffer and if image
is saving as render result.
This solves both initial report issue and TODO marked in previous fix.
Also de-duplicated image buffer color managing code and gave some more
meaningful names for few functions. Also wrote documentation around this
function, so current assumptions about spaces should be clear enough.
Made regression tests by saving EXR/PNG images to all supported format and
rendering OpenGL/Normal animation, in all cases seems everything is fine,
but more tests for sure would be welcome.
Replace old color pipeline which was supporting linear/sRGB color spaces
only with OpenColorIO-based pipeline.
This introduces two configurable color spaces:
- Input color space for images and movie clips. This space is used to convert
images/movies from color space in which file is saved to Blender's linear
space (for float images, byte images are not internally converted, only input
space is stored for such images and used later).
This setting could be found in image/clip data block settings.
- Display color space which defines space in which particular display is working.
This settings could be found in scene's Color Management panel.
When render result is being displayed on the screen, apart from converting image
to display space, some additional conversions could happen.
This conversions are:
- View, which defines tone curve applying before display transformation.
These are different ways to view the image on the same display device.
For example it could be used to emulate film view on sRGB display.
- Exposure affects on image exposure before tone map is applied.
- Gamma is post-display gamma correction, could be used to match particular
display gamma.
- RGB curves are user-defined curves which are applying before display
transformation, could be used for different purposes.
All this settings by default are only applying on render result and does not
affect on other images. If some particular image needs to be affected by this
transformation, "View as Render" setting of image data block should be set to
truth. Movie clips are always affected by all display transformations.
This commit also introduces configurable color space in which sequencer is
working. This setting could be found in scene's Color Management panel and
it should be used if such stuff as grading needs to be done in color space
different from sRGB (i.e. when Film view on sRGB display is use, using VD16
space as sequencer's internal space would make grading working in space
which is close to the space using for display).
Some technical notes:
- Image buffer's float buffer is now always in linear space, even if it was
created from 16bit byte images.
- Space of byte buffer is stored in image buffer's rect_colorspace property.
- Profile of image buffer was removed since it's not longer meaningful.
- OpenGL and GLSL is supposed to always work in sRGB space. It is possible
to support other spaces, but it's quite large project which isn't so
much important.
- Legacy Color Management option disabled is emulated by using None display.
It could have some regressions, but there's no clear way to avoid them.
- If OpenColorIO is disabled on build time, it should make blender behaving
in the same way as previous release with color management enabled.
More details could be found at this page (more details would be added soon):
http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Color_Management
--
Thanks to Xavier Thomas, Lukas Toene for initial work on OpenColorIO
integration and to Brecht van Lommel for some further development and code/
usecase review!
