Removed all limitations from render code for maximum threads. The only
define for this now is in BLI_threads.h, and currently set to 8.
Note that each thread renders an entire tile, and also allocates the
buffers for the tiles, so; more threads might work better with smaller
tiles.
IMPORTANT: node system won't work yet with more than 2 threads! So, don't
try material nodes or compositing with over 2 threads. That I'll commit
later today.
What does work (should work :) is AO and soft shadow now.
- Now baking itself is threaded too (like for render, max 2 cpus. Moving
this to 4 cpus is on todo. Goes twice as fast!
- fix: ESC from bake was broken...
- other fix: toolbox menus didn't treat sublevel string lengths OK,
truncating items like for Group library names.
- New perspective projection capability. If a camera is used as the
projection object, the modifier detects whether to do perspective or
orthographic projection based on the camera type. If any other object
type is used as the projection object, orthographic projection is used.
- Orthographic projection actually works properly now.
- The projected UVs are scaled and offset so that the image is centred in
the projecting camera's view.
- AspX and AspY inputs have been added to control the aspect ratio of the
projected UVs.
Also:
- I have added the Mat4MulVec3Project() function to BLI_arithb.h; this
function converts a 3-dimensional vector to homogeneous coordinates
(4-dimensional, with the 4th dimension set to 1), multiplies it with the
given matrix, then projects it back to 3 dimensions by dividing through
with the 4th dimension. This is useful when using projection matrices.
Random seeding is still not perfect in render, especially lack of good
thread support still.
- VectorBlur node was calling seed for each exec, causing other nodes to
get fixed random too.
- added seed in non-OSA main loop for render
- use BLI_srandom, is better than BLI_srand
2006/11/19 00:07:32 CET
Fix for bug #5250: inaccurate conversion between edit and pose mode bones.
Two very bad bugs:
- replacing atan() with atan2() should also remove the M_PI correction!
This is the equivalent:
angle=atan(x/y); if(y<0) angle+=M_PI;
angle= atan2(x, y);
- the new NormalizedVecAngle2() call was negating an input vector, causing
calling code to screw up. All arithb.c calls should not alter input.
Using acos(dot(u, v)) to find the angle between two vectors is quite
inaccurate, and there's a better way to do it, as explained here:
http://www.plunk.org/~hatch/rightway.php
Also changed the use of atan for computing roll to atan2 in some places,
the latter avoids accuracy and division by zero issues.
Loadsa new compo nodes, most added in new menu "Mattes".
- Seperate into YCC
- Combine YCC
- Seperate into YUV
- Combine YUV
- (Chroma) Difference Matte
- Chroma Key
- Luminance Key
- Color Spill correction
Main problem is missing docs still... Bob is working on it, he'll also
make tooltips for all buttons.
Some UI things might change too, like more clear names.
duplicate code. Also removed redundant files from the bsp module,
that where replaced by boolop last year, no sense in updating them
for these changes. On the user level things should still work the
same, this is only preparation work.
Not counting the removed files, -1501 lines of code, not too bad :)
Creates a 4x4 matrix from location, size and euler angles components.
Coded by Joshua Leung (aligorith) for one of his patches. Integrating right now because it's useful in and of itself.
Made nice crashes that way... but not in !@%@$ OSX because it seems to
clear memory unwanted. Solve dit with adding API call in MemArena to default
to callocs.
Also removed malloc() from MemArena, replaced with MEM_mallocN().
with regards to the noise functions in yafray based on Ken Musgrave's
original code.
I had left the 'Copyright' notice in the comments, and according to
Cyril Brulebois this is a problem. In fact, from what I understand this
makes it even impossible to use or modify the code in other software.
But since it is not a verbatim copy of the code but rather based on Musgrave's
work, he suggested I change it to explicitely state that it is in fact
based on the code from the 'Texturing & Modeling' book.
And since the yafray code is in turn based on the blender code, I better
adapt the blender code too.
This reminded me that I also have forgotten to include the copyright notice
in the mersenne twister rng code I used for the Python Noise module.
This does clearly state to include the original notice with any resdistributed
code, in modified form or not. So I added that too.
I hope that solves the problems.
- code for adding 'c:\' on Windows missed fact that BLI_convertstringcode is also used for databrowse.
- now only replacing when path starts with '/', which isn't the case with databrowse
module itself, replacing the special MEM_mallocT/MEM_freeT functions.
Mutex locking is only enabled when threads are running.
There was no good reason to have these separate, it just led to ugly
hacks when calling functions with non-threadsafe malloc from threads.
structured and more maintainable. The old algorithm could leave the mesh in an
inconsistent state during recursive calls, making it quite fragile. The new
algorithm keeps the mesh in a consistent state, and should be more robust.
This commit fixes the issues EdgeSplit was having with non-manifold meshes
(bug #4946, also reported by several other people). EdgeSplit now works
correctly with all the test files I have.
This commit also fixes the BLI_ghash_remove function to correctly decrement
gh->nentries (the number of entries in the hashtable) on a successful removal.
[ #4337 ] Cant refresh the C:\
[ #4710 ] Wrong paths in file selector under user prefs
[ #4353 ] Using ^ char + click on Open/Load = Blender crash
Details:
Fixes for root paths like C:\ on Windows, where Blender still used '/'.
Also contains fixes for relative paths:
- no relative paths for the default dirs (forced to absolute)
- message if using relative paths when .blend file hasn't been saved.
Lastly also added '.' for refresh in root paths. Windows
FindFirstFile/FindNextFile also return '.' and '..', but not in root paths like C:\
This is a new incarnation of compatible_eul(), a function called:
void Mat3ToCompatibleEul(float mat[][3], float *eul, float *oldrot)
It uses the two euler extraction methods as added by Brecht a while ago,
and checks for compatibility each, and then picks the best of the two
based on minimal difference with 'oldrot'.
Gives for rotation key inserting a much higher hitrate for OK eulers.
The 'edge fill' code failed on filling tiny small polygons. It has a limit
check for double points, which was hardcoded set to 0.0003.
That is (commented in code too) a weak part. Better would be to define a
bounding box first, and then derive the limit from that.
Further, the edge fill code uses blender EditEdge data, which fails when
the filling code removes edges. Certainly a topic to work on once, this
code is from the 80ies!
In windows, without temp path set, the 'save buffers' render option crashes.
I've coded a blenlib BLI_is_writable(char *filename) to check for such
cases. This is not much needed in Blender, since the open() command is
checked for. However, file saving happens deep inside the C++ exr lib, and
it throws an exception crash when a file cannot be written.
Small fix, but results are at least less frustrating now. It uses the
"compatible euler" function from inserting key positions here, preventing
euler values to be constrainted that differ weirdly.
I've tried several other approaches to get a definite rotate constraint,
but only constraining a single axes seems to me impossible magic still...
so you can select 2 faces and then get all coplaner faces around both.
Added fp to face and edge edit data so as to store edge length and face area/perimeter. (in the union so minimal overhead)
current implimentation is fairly optimized.
Change to arithb.c was just to command Inpf as being the dot producr.
Added VecAngle2 - needed for grouping by normal and planer faces.
Made select face & edge group ignore hidden faces.
fixed a bug where hiding faces didnt update G.tot*sel
Fixed BLI_exist: In Windows stat doesn't recognize
a dirname ending is a slash, exept when it's
the root dir ("C:\\"), where it is required.
So trailing slashes are only removed when filename
is longer than 3 chars.
Also fixed Python Sys.c that now uses BLI_exist
instead of calling stat directly.
- Renderwin still used a thread-unsafe malloc, in the header text print
- Setting clipping flags in vertices for parts required a mutex lock after
all... I thought it would go fine, but noticed on renders with small
amounts of faces that sometimes faces disappear from a render.
(was doing movie credits, so all faces are visible! Otherwise it would
have hardly been noticable...)
This modifier allows to make arrays of meshes, with multiple offset types:
- constant offset
- offset relative to object width
- offset with scale and rotation based on another object
The number of duplicates can be computed based on a fixed count, fixed length
or length of a curve. Duplicate vertices can be automatically merged.
Nice docs and example files available in the wiki:
http://mediawiki.blender.org/index.php/BlenderDev/ArrayModifier
For some reason I thought SDL thread handling would be much simpler... but
the migration to posix pthread went very smooth and painless. Less code
even, and I even notice a slight performance increase!
All threading code is still wrapped in blenlib/intern/threads.c
Only real change was making the callback functions to return void pointer,
instead of an int.
The mutex handling is also different... there's no test anymore if a
mutex was initialized, which is a bit confusing. But it appears to run
all fine still. :)
Nathan Letwory has been signalled already to provide the Windows pthread
library and make/scons linking. For MSVC we might need help from someone
else later though.
without initialization.
For Brecht:
source/blender/blenkernel/intern/subsurf_ccg.c:329: warning: left-hand operand of comma expression has no effect
This line I don't understand...
In Orange we've been fighting the past weeks with memory usage a lot...
at the moment incredible huge scenes are being rendered, with multiple
layers and all compositing, stressing limits of memory a lot.
I had hoped that less frequently used blocks would be swapped away
nicely, so fragmented memory could survive. Unfortunately (in OSX) the
malloc range is limited to 2 GB only (upped half of address space).
Other OS's have a limit too, but typically larger afaik.
Now here's mmap to the rescue! It has a very nice feature to map to
a virtual (non existing) file, allowing to allocate disk-mapped memory
on the fly. For as long there's real memory it works nearly as fast as
a regular malloc, and when you go to the swap boundary, it knows nicely
what to swap first.
The upcoming commit will use mmap for all large memory blocks, like
the composit stack, render layers, lamp buffers and images. Tested here
on my 1 GB system, and compositing huge images with a total of 2.5 gig
still works acceptable here. :)
http://www.blender.org/bf/memory.jpg
This is a silly composit test, using 64 MB images with a load of nodes.
Check the header print... the (2323.33M) is the mmap disk-cache in use.
BTW: note that is still limited to the virtual address space of 4 GB.
The new call is:
MEM_mapalloc()
Per definition, mmap() returns zero'ed memory, so a calloc isn't required.
For Windows there's no mmap() available, but I'm pretty sure there's an
equivalent. Windows gurus here are invited to insert that here in code! At
the moment it's nicely ifdeffed, so for Windows the mmap defaults to a
regular alloc.
I noticed still several cases where the Imbuf library was called within a
thread... and that whilst the Imbuf itself isn't threadsafe. Also the
thread lock I added in rendering for loading images actually didn't
work, because then it was still possible both threads were accessing the
MEM_malloc function at same time.
This commit nearly fully replaces ImBuf calls in compositor (giving another
nice speedup btw, the way preview images in Nodes were calculated used
clumsy imbuf scaling code).
I've also centralized the 'mutex' locking for threading, which now only
resides in BLI_threads.h. This is used to secure the last ImBuf calls
I cannot replace, which is loading images and creating mipmaps.
Really hope we get something more stable now!
Fixing case where BLI_make_file_string reads one byte before passed in string when the string is empty.
Leftover from patch tracker item #2740 - thanks Andrew Zabolotny.
Also fixed the part that made sure exactly one slash is appended at the end of the dir - comment wasn't telling what the code did here :)
- Enabled Groups to execute in Compositor. They were ignored still.
Note; inside of groups nothing is cached, so a change of a group input
will recalculate it fully. This is needed because groups are linked
data (instances use same internal nodes).
- Made Composit node "Viewer" display correctly input for images with
1/2/3/4 channels.
- Added pass rendering, tested now with only regular Materials. For
Material nodes this is quite more complex... since they cannot be
easily separated in passes (each Material does a full shade)
In this commit all pass render is disabled though, will continue work on
that later.
Sneak preview: http://www.blender.org/bf/rt.jpg (temporal image)
- What did remain is the 'Normal' pass output. Normal works very nice for
relighting effects. Use the "Normal Node" to define where more or less
light should be. (Use "Value Map" node to tweak influence of the
Normal node 'dot' output.)
- EVIL bug fix: I've spend almost a day finding it... when combining AO and
mirror render, the event queue was totally screwing up... two things not
related at all!
Found out error was in ray-mirror code, which was using partially
uninitialized 'ShadeInput' data to pass on to render code.
- Another fix; made sure that while thread render, the threads don't get
events, only the main program will do. Might fix issues reported by
people on linux/windows.
for compositing code.
Officially malloc/calloc/free is threadsafe, but our secure malloc system
requires all memory blocks to be stored in a single list, so when two
threads write in this list you get conflicts.
- Compositor now is threaded
Enable it with the Scene buttons "Threads". This will handle over nodes to
individual threads to be calculated. However, if nodes depend on others
they have to wait. The current system only threads per entire node, not for
calculating results in parts.
I've reshuffled the node execution code to evaluate 'changed' events, and
prepare the entire tree to become simply parsed for open jobs with a call
to node = getExecutableNode()
By default, even without 'thread' option active, all node execution is
done within a separate thread.
Also fixed issues in yesterdays commit for 'event based' calculations, it
didn't do animated images, or execute (on rendering) the correct nodes
when you don't have Render-Result nodes included.
- Added generic Thread support in blenlib/ module
The renderer and the node system now both use same code for controlling the
threads. This has been moved to a new C file in blenlib/intern/threads.c.
Check this c file for an extensive doc and example how to use it.
The current implementation for Compositing allows unlimited amount of
threads. For rendering it is still tied to two threads, although it is
pretty easy to extend to 4 already. People with giant amounts of cpus can
poke me once for tests. :)
- Bugfix in creating group nodes
Group node definitions demand a clear separation of 'internal sockets' and
'external sockets'. The first are sockets being linked internally, the latter
are sockets exposed as sockets for the group itself.
When sockets were linked both internal and external, Blender crashed. It is
solved now by removing the external link(s).
A full detailed description of this will be done later... is several days
of work. Here's a summary:
Render:
- Full cleanup of render code, removing *all* globals and bad level calls
all over blender. Render module is now not called abusive anymore
- API-fied calls to rendering
- Full recode of internal render pipeline. Is now rendering tiles by
default, prepared for much smarter 'bucket' render later.
- Each thread now can render a full part
- Renders were tested with 4 threads, goes fine, apart from some lookup
tables in softshadow and AO still
- Rendering is prepared to do multiple layers and passes
- No single 32 bits trick in render code anymore, all 100% floats now.
Writing images/movies
- moved writing images to blender kernel (bye bye 'schrijfplaatje'!)
- made a new Movie handle system, also in kernel. This will enable much
easier use of movies in Blender
PreviewRender:
- Using new render API, previewrender (in buttons) now uses regular render
code to generate images.
- new datafile 'preview.blend.c' has the preview scenes in it
- previews get rendered in exact displayed size (1 pixel = 1 pixel)
3D Preview render
- new; press Pkey in 3d window, for a panel that continuously renders
(pkey is for games, i know... but we dont do that in orange now!)
- this render works nearly identical to buttons-preview render, so it stops
rendering on any event (mouse, keyboard, etc)
- on moving/scaling the panel, the render code doesn't recreate all geometry
- same for shifting/panning view
- all other operations (now) regenerate the full render database still.
- this is WIP... but big fun, especially for simple scenes!
Compositor
- Using same node system as now in use for shaders, you can composit images
- works pretty straightforward... needs much more options/tools and integration
with rendering still
- is not threaded yet, nor is so smart to only recalculate changes... will be
done soon!
- the "Render Result" node will get all layers/passes as output sockets
- The "Output" node renders to a builtin image, which you can view in the Image
window. (yes, output nodes to render-result, and to files, is on the list!)
The Bad News
- "Unified Render" is removed. It might come back in some stage, but this
system should be built from scratch. I can't really understand this code...
I expect it is not much needed, especially with advanced layer/passes
control
- Panorama render, Field render, Motion blur, is not coded yet... (I had to
recode every single feature in render, so...!)
- Lens Flare is also not back... needs total revision, might become composit
effect though (using zbuffer for visibility)
- Part render is gone! (well, thats obvious, its default now).
- The render window is only restored with limited functionality... I am going
to check first the option to render to a Image window, so Blender can become
a true single-window application. :)
For example, the 'Spare render buffer' (jkey) doesnt work.
- Render with border, now default creates a smaller image
- No zbuffers are written yet... on the todo!
- Scons files and MSVC will need work to get compiling again
OK... thats what I can quickly recall. Now go compiling!
Credits go to Gernot Ziegler, who originally coded EXR support, and to
Austin Benesh for bringing it further. Kent Mein provided a lot of code
for integrating float buffers in Blender imbuf and ImBuf API cleanup,
and provided Make and Scons and static linking.
At this moment; the EXR libraries are a *dependency*, so you cannot get
the Orange branch compiled without having OpenEXR installed. Get the
(precompiled or sources) stuff from www.openexr.com. Current default is
that the headers and lib resides in /user/local/
Several changes/additions/fixes were added:
- EXR code only supported 'half' format (16 bits per channel). I've added
float writing, but for reading it I need tomorrow. :)
- Quite some clumsy copying of data happened in EXR code.
- cleaned up the api calls already a bit, preparing for more advanced
support
- Zbuffers were saved 16 bits, now 32 bits
- automatic adding of .exr extensions went wrong
Imbuf:
- added proper imbuf->flags and imbuf->mall support for float buffers, it
was created for *each* imbuf. :)
- found bugs for float buffers in scaling and flipping. Code there will
need more checks still
- imbuf also needs to be verified to behave properly when no 32 bits
rect exists (for saving for example)
TODO:
- support internal float images for textures, backbuf, AO probes, and
display in Image window
Hope this commit won't screwup syncing with bf-blender... :/
