* Division by zero fix for TNT SVD code.
* Sound fix, in case ffmpeg decode fails, don't use the samples.
* Fix for incorrect bounds of transformed objects in new raytracing code.
* Gave memory arena's a name used for allocations for easier memory
usage debugging.
* Dupligroup no_draw option was using layers but not restrict view/render
setting. (not a bugfix exactly but would do display list context switching
while drawing for no reason).
* Fix objects instanced on hair particles not giving consistent results
when the object is transformed.
* New math functions: madd_v4_v4fl, len_squared_v3v3, interp_v4_v4v4v4,
mul_v4_m4v4, SH and form factor functions, box_minmax_bounds_m4.
* mul_m4_m4m4 and mul_m3_m3m3 now accept the same pointers for multiple
arguments.
* endjob callback for WM jobs system.
* Geometry node uv/color layer now has search list/autocomplete.
* Various small buildsystem tweaks, not strictly needed yet in trunk.
* Viewer node could free image while it is being redrawn, viewer image
buffers now need acquire/release to be accessed as was already the
case for render results.
* The Composite node could free the image buffers outside of a lock,
also causing simultaneous redraw to crash.
* Especially on Windows, re-rendering could crash when drawing an image
that was freed. When RE_RenderInProgress was true it would access the
image buffer and simply return it while it could still contain a pointer
to a render result buffer that was already freed. I don't understand
why this case was there in the first place, so I've removed it.
Possibly fixes bugs #20174, #21418, #21391, #21394.
to wait for an item to be put in the queue and then pop immediately without,
this makes it possible to avoid sleep() while waiting for the results of a
thread.
* Rendering twice or more could crash layer/pass buttons.
* Compositing would crash while drawing the image.
* Rendering animations could also crash drawing the image.
* Compositing could crash
* Starting to rendering while preview render / compo was
still running could crash.
* Exiting while rendering an animation would not abort the
renderer properly, making Blender seemingly freeze.
* Fixes theoretically possible issue with setting malloc
lock with nested threads.
* Drawing previews inside nodes could crash when those nodes
were being rendered at the same time.
There's more crashes, manipulating the scene data or undo can
still crash, this commit only focuses on making sure the image
buffer and render result access is thread safe.
Implementation:
* Rather than assuming the render result does not get freed
during render, which seems to be quite difficult to do given
that e.g. the compositor is allowed to change the size of
the buffer or output different passes, the render result is
now protected with a read/write mutex.
* The read/write mutex allows multiple readers (and pixel
writers) at the same time, but only allows one writer to
manipulate the data structure.
* Added BKE_image_acquire_ibuf/BKE_image_release_ibuf to access
images being rendered, cases where this is not needed (most
code) can still use BKE_image_get_ibuf.
* The job manager now allows only one rendering job at the same
time, rather than the G.rendering check which was not reliable.
Preview renders:
- Added proper button type (BUT_EXTRA) for preview buttons, to
handle drawing better. It now first draws an alpha mask, to
ensure the preview is correctly fitting inside the widget style.
It then draws the outline.
- Added protection for executing preview renders while regular
rendering, that's not going to work...
Removed all limitations from render code for maximum threads. The only
define for this now is in BLI_threads.h, and currently set to 8.
Note that each thread renders an entire tile, and also allocates the
buffers for the tiles, so; more threads might work better with smaller
tiles.
IMPORTANT: node system won't work yet with more than 2 threads! So, don't
try material nodes or compositing with over 2 threads. That I'll commit
later today.
What does work (should work :) is AO and soft shadow now.
module itself, replacing the special MEM_mallocT/MEM_freeT functions.
Mutex locking is only enabled when threads are running.
There was no good reason to have these separate, it just led to ugly
hacks when calling functions with non-threadsafe malloc from threads.
- Renderwin still used a thread-unsafe malloc, in the header text print
- Setting clipping flags in vertices for parts required a mutex lock after
all... I thought it would go fine, but noticed on renders with small
amounts of faces that sometimes faces disappear from a render.
(was doing movie credits, so all faces are visible! Otherwise it would
have hardly been noticable...)
For some reason I thought SDL thread handling would be much simpler... but
the migration to posix pthread went very smooth and painless. Less code
even, and I even notice a slight performance increase!
All threading code is still wrapped in blenlib/intern/threads.c
Only real change was making the callback functions to return void pointer,
instead of an int.
The mutex handling is also different... there's no test anymore if a
mutex was initialized, which is a bit confusing. But it appears to run
all fine still. :)
Nathan Letwory has been signalled already to provide the Windows pthread
library and make/scons linking. For MSVC we might need help from someone
else later though.
In Orange we've been fighting the past weeks with memory usage a lot...
at the moment incredible huge scenes are being rendered, with multiple
layers and all compositing, stressing limits of memory a lot.
I had hoped that less frequently used blocks would be swapped away
nicely, so fragmented memory could survive. Unfortunately (in OSX) the
malloc range is limited to 2 GB only (upped half of address space).
Other OS's have a limit too, but typically larger afaik.
Now here's mmap to the rescue! It has a very nice feature to map to
a virtual (non existing) file, allowing to allocate disk-mapped memory
on the fly. For as long there's real memory it works nearly as fast as
a regular malloc, and when you go to the swap boundary, it knows nicely
what to swap first.
The upcoming commit will use mmap for all large memory blocks, like
the composit stack, render layers, lamp buffers and images. Tested here
on my 1 GB system, and compositing huge images with a total of 2.5 gig
still works acceptable here. :)
http://www.blender.org/bf/memory.jpg
This is a silly composit test, using 64 MB images with a load of nodes.
Check the header print... the (2323.33M) is the mmap disk-cache in use.
BTW: note that is still limited to the virtual address space of 4 GB.
The new call is:
MEM_mapalloc()
Per definition, mmap() returns zero'ed memory, so a calloc isn't required.
For Windows there's no mmap() available, but I'm pretty sure there's an
equivalent. Windows gurus here are invited to insert that here in code! At
the moment it's nicely ifdeffed, so for Windows the mmap defaults to a
regular alloc.
I noticed still several cases where the Imbuf library was called within a
thread... and that whilst the Imbuf itself isn't threadsafe. Also the
thread lock I added in rendering for loading images actually didn't
work, because then it was still possible both threads were accessing the
MEM_malloc function at same time.
This commit nearly fully replaces ImBuf calls in compositor (giving another
nice speedup btw, the way preview images in Nodes were calculated used
clumsy imbuf scaling code).
I've also centralized the 'mutex' locking for threading, which now only
resides in BLI_threads.h. This is used to secure the last ImBuf calls
I cannot replace, which is loading images and creating mipmaps.
Really hope we get something more stable now!
for compositing code.
Officially malloc/calloc/free is threadsafe, but our secure malloc system
requires all memory blocks to be stored in a single list, so when two
threads write in this list you get conflicts.
- Compositor now is threaded
Enable it with the Scene buttons "Threads". This will handle over nodes to
individual threads to be calculated. However, if nodes depend on others
they have to wait. The current system only threads per entire node, not for
calculating results in parts.
I've reshuffled the node execution code to evaluate 'changed' events, and
prepare the entire tree to become simply parsed for open jobs with a call
to node = getExecutableNode()
By default, even without 'thread' option active, all node execution is
done within a separate thread.
Also fixed issues in yesterdays commit for 'event based' calculations, it
didn't do animated images, or execute (on rendering) the correct nodes
when you don't have Render-Result nodes included.
- Added generic Thread support in blenlib/ module
The renderer and the node system now both use same code for controlling the
threads. This has been moved to a new C file in blenlib/intern/threads.c.
Check this c file for an extensive doc and example how to use it.
The current implementation for Compositing allows unlimited amount of
threads. For rendering it is still tied to two threads, although it is
pretty easy to extend to 4 already. People with giant amounts of cpus can
poke me once for tests. :)
- Bugfix in creating group nodes
Group node definitions demand a clear separation of 'internal sockets' and
'external sockets'. The first are sockets being linked internally, the latter
are sockets exposed as sockets for the group itself.
When sockets were linked both internal and external, Blender crashed. It is
solved now by removing the external link(s).
