added some missing functions too - which are not used yep but should be there for api completeness.
* CDDM_set_mloop
* CDDM_set_mpoly
* BLI_mempool_count
- move some larger vars into a nested scope.
- replace memset with zero initializer.
- rempace VECCOPY macros with copy_v3v3
- change function args to give the float array length.
- use 'const float *' and array size in some function declarations.
- replace macros for BLI_math functions INPF, VECCOPY, VECADD etc.
- remove unused VertRen.clip struct member.
- remove static squared_dist() from 2 files, replace with BLI_math function len_squared_v3v3().
- use vertex arrays for drawing clipping background in the 3D viewport.
These should not have any effect on render results, except in some cases with
you have overlapping faces, where the noise seems to be slightly reduced.
There are some performance improvements, for simple scenes I wouldn't expect
more than 5-10% to be cut off the render time, for sintel scenes we got about
50% on average, that's with millions of polygons on intel quad cores. This
because memory access / cache misses were the main bottleneck for those scenes,
and the optimizations improve that.
Interal changes:
* Remove RE_raytrace.h, raytracer is now only used by render engine again.
* Split non-public parts rayobject.h into rayobject_internal.h, hopefully
makes it clearer how the API is used.
* Added rayintersection.h to contain some of the stuff from RE_raytrace.h
* Change Isect.vec/labda to Isect.dir/dist, previously vec was sometimes
normalized and sometimes not, confusing... now dir is always normalized
and dist contains the distance.
* Change VECCOPY and similar to BLI_math functions.
* Force inlining of auxiliary functions for ray-triangle/quad intersection,
helps a few percentages.
* Reorganize svbvh code so all the traversal functions are in one file
* Don't do test for root so that push_childs can be inlined
* Make shadow a template parameter so it doesn't need to be runtime checked
* Optimization in raytree building, was computing bounding boxes more often
than necessary.
* Leave out logf() factor in SAH, makes tree build quicker with no
noticeable influence on raytracing on performance?
* Set max childs to 4, simplifies traversal code a bit, but also seems
to help slightly in general.
* Store child pointers and child bb just as fixed arrays of size 4 in nodes,
nearly all nodes have this many children, so overall it actually reduces
memory usage a bit and avoids a pointer indirection.
Ambient occlusion: multiplied with direct lighting by default, add
is also still available and more blending methods might be added if
they are useful. This is fundamentally a non physical effect.
Environment lighting: always added as you would expect (though you can
subtract by specifying negative energy). This can be just white or take
colors or textures from the world.
Indirect lighting: only supported for AAO at the moment (and is still
too approximate), and also is always added. A factor is available to
specify how much is added, though value 1.0 is correct.
Also:
* Material ambient value now defaults to 1.0.
* Added Environment, Indirect and Emit pass.
* "Both" blending method is no longer available.
* Attenuation, sampling parameters are still shared, some could be split
up, though if they are different this would affect performance.
computers, probably due to slow multithreaded malloc. Now each render
thread keeps a list of qmc samplers that it fills as needed while
rendering (since it is hard to predict the actual amount needed in
advance due to ray recursion).
Fix for bug #7418: texture ipo's didn't show for textures in node materials.
Fix for part of bug #6758: node materials in other node materials could
miss texture coordinates.
They now also store a list of samples per pixel, and then get
shaded together with the ztransp samples. This comes with a
slight speed hit, but mainly memory might be a concern. However,
testing some peach scenes I haven't problems.
This completes the pipeline make-over, as started in 2006. With this
option, during rendering, each sample for every layer and pass is being
saved on disk (looks like non-antialiased images). Then the composite
and color correction happens, then a clip to 0-1 range, and only in end
all samples get combined - using sampling filters such as gauss/mitch/catmul.
This results in artefact-free antialiased images. Even Z-combine or
ID masks now work perfect for it!
This is an unfinished commit btw; Brecht will finish this for strands.
Also Halo doesnt work yet.
To activate FSA: press "Save Buffers" and the new button next to it. :)
=================
Big commit, but little user visible changes.
- Dupliverts and duplifaces are now rendered as instances, instead
of storing all of the geometry for each dupli, now an instance is
created with a matrix transform refering to the source object.
This should allow us to render tree leaves more memory efficient.
- Radiosity and to some degree raytracing of such objects is not
really efficient still. For radiosity this is fundamentally hard
to solve, but raytracing an octree could be created for each object,
but the current octree code with it's fixed size doesn't allow this
efficiently.
- The regression tests survived, but with I expect that some bugs will
pop up .. hopefully not too many :).
Implementation Notes
====================
- Dupligroups and linked meshes are not rendered as instances yet,
since they can in fact be different due to various reasons,
instancing of these types of duplis that are the same can be added
for them at a later point.
- Each ObjectRen now stores it's own database, instead of there being
one big databases of faces, verts, .. . Which objects that are actually
rendered are defined by the list of ObjectRenInstances, which all refer
to an ObjectRen.
- Homogeneous coordinatess and clipping is now not stored in vertices
anymore, but instead computed on the fly. This couldn't work for
instances. That does mean some extra computation has to be done, but
memory lookups can be slow too, and this saves some memory. Overall
I didn't find a significant speed impact.
- OSA rendering for solid and ztransp now is different. Instead of e.g.
going 8 times over the databases times and rendering the z-buffer, it
now goes over the database once and renders each polygon 8 times. That
was necessary to keep instances efficient, and can also give some
performance improvement without instances.
- There was already instancing support in the yafray export code, now it
uses Blender's render instances for export.
- UV and color layer storage in the render was a bit messy before, now
should be easier to understand.
- convertblender.c was reorganized somewhat. Regular render, speedvector
and baking now use a single function to create the database, previously
there was code duplicated for it.
- Some of these changes were done with future multithreading of scene
and shadow buffer creation in mind, though especially for scene creation
much work remains to be done to make it threadsafe, since it also involves
a lot of code from blenkernel, and there is an ugly conflict with the way
dupli groups work here .. though in the render code itself it's almost there.
two separate files, raytrace.c and rayshade.c. The tracing code can now
be used separately from the renderer (will be used in a later commit),
and the raytracing acceleration structure can now also be easily replaced,
if someone wants to experiment with that.
Removed all limitations from render code for maximum threads. The only
define for this now is in BLI_threads.h, and currently set to 8.
Note that each thread renders an entire tile, and also allocates the
buffers for the tiles, so; more threads might work better with smaller
tiles.
IMPORTANT: node system won't work yet with more than 2 threads! So, don't
try material nodes or compositing with over 2 threads. That I'll commit
later today.
What does work (should work :) is AO and soft shadow now.
Irregular shadow buffer, for solid OSA case, suffered aliasing. This caused
by optimized storage of shadow factors in 1 pixel (averaged for all sub
samples). That didn't work well really...
Now the code uses same method as for transparent faces, storing results for
each sample.
- RenderLayers with 'view layers' set, now also take visible lights into
account. Works just like for scene layer settings.
- On ESC from render, compositing (if set) is being skipped too
- While rendering with multiple RenderLayers it will end with a display
of the current RenderLayer (as in Scene buttons)
- Enabled Groups to execute in Compositor. They were ignored still.
Note; inside of groups nothing is cached, so a change of a group input
will recalculate it fully. This is needed because groups are linked
data (instances use same internal nodes).
- Made Composit node "Viewer" display correctly input for images with
1/2/3/4 channels.
- Added pass rendering, tested now with only regular Materials. For
Material nodes this is quite more complex... since they cannot be
easily separated in passes (each Material does a full shade)
In this commit all pass render is disabled though, will continue work on
that later.
Sneak preview: http://www.blender.org/bf/rt.jpg (temporal image)
- What did remain is the 'Normal' pass output. Normal works very nice for
relighting effects. Use the "Normal Node" to define where more or less
light should be. (Use "Value Map" node to tweak influence of the
Normal node 'dot' output.)
- EVIL bug fix: I've spend almost a day finding it... when combining AO and
mirror render, the event queue was totally screwing up... two things not
related at all!
Found out error was in ray-mirror code, which was using partially
uninitialized 'ShadeInput' data to pass on to render code.
- Another fix; made sure that while thread render, the threads don't get
events, only the main program will do. Might fix issues reported by
people on linux/windows.
- Live scanline updates while rendering
Using a timer system, each second now the tiles that are being processed
are checked if they could use display.
To make this work pretty, I had to use the threaded 'tile processor' for
a single thread too, but that's now proven to be stable.
Also note that these updates draw per layer, including ztransp progress
separately from solid render.
- Recode of ztransp OSA
Until now (since blender 1.0) the ztransp part was fully rendered and
added on top of the solid part with alpha-over. This adding was done before
the solid part applied sub-pixel sample filtering, causing the ztransp
layer to be always too blurry.
Now the ztransp layer uses same sub=pixel filter, resulting in the same
AA level (and filter results) as the solid part. Quite noticable with hair
renders.
- Vector buffer support & preliminary vector-blur Node
Using the "Render Layer" panel "Vector" pass button, the motion vectors
per pixel are calculated and stored. Accessible via the Compositor.
The vector-blur node is horrible btw! It just uses the length of the
vector to apply a filter like with current (z)blur. I'm committing it anyway,
I'll experiment with it further, and who knows some surprise code shows up!
A full detailed description of this will be done later... is several days
of work. Here's a summary:
Render:
- Full cleanup of render code, removing *all* globals and bad level calls
all over blender. Render module is now not called abusive anymore
- API-fied calls to rendering
- Full recode of internal render pipeline. Is now rendering tiles by
default, prepared for much smarter 'bucket' render later.
- Each thread now can render a full part
- Renders were tested with 4 threads, goes fine, apart from some lookup
tables in softshadow and AO still
- Rendering is prepared to do multiple layers and passes
- No single 32 bits trick in render code anymore, all 100% floats now.
Writing images/movies
- moved writing images to blender kernel (bye bye 'schrijfplaatje'!)
- made a new Movie handle system, also in kernel. This will enable much
easier use of movies in Blender
PreviewRender:
- Using new render API, previewrender (in buttons) now uses regular render
code to generate images.
- new datafile 'preview.blend.c' has the preview scenes in it
- previews get rendered in exact displayed size (1 pixel = 1 pixel)
3D Preview render
- new; press Pkey in 3d window, for a panel that continuously renders
(pkey is for games, i know... but we dont do that in orange now!)
- this render works nearly identical to buttons-preview render, so it stops
rendering on any event (mouse, keyboard, etc)
- on moving/scaling the panel, the render code doesn't recreate all geometry
- same for shifting/panning view
- all other operations (now) regenerate the full render database still.
- this is WIP... but big fun, especially for simple scenes!
Compositor
- Using same node system as now in use for shaders, you can composit images
- works pretty straightforward... needs much more options/tools and integration
with rendering still
- is not threaded yet, nor is so smart to only recalculate changes... will be
done soon!
- the "Render Result" node will get all layers/passes as output sockets
- The "Output" node renders to a builtin image, which you can view in the Image
window. (yes, output nodes to render-result, and to files, is on the list!)
The Bad News
- "Unified Render" is removed. It might come back in some stage, but this
system should be built from scratch. I can't really understand this code...
I expect it is not much needed, especially with advanced layer/passes
control
- Panorama render, Field render, Motion blur, is not coded yet... (I had to
recode every single feature in render, so...!)
- Lens Flare is also not back... needs total revision, might become composit
effect though (using zbuffer for visibility)
- Part render is gone! (well, thats obvious, its default now).
- The render window is only restored with limited functionality... I am going
to check first the option to render to a Image window, so Blender can become
a true single-window application. :)
For example, the 'Spare render buffer' (jkey) doesnt work.
- Render with border, now default creates a smaller image
- No zbuffers are written yet... on the todo!
- Scons files and MSVC will need work to get compiling again
OK... thats what I can quickly recall. Now go compiling!
(WIP, don't bugs for this in tracker yet please!)
- New Panel "Layers" in Material buttons, allows to add unlimited amount
of materials on top of each other.
- Every Layer is actually just another Material, which gets rendered/shaded
(including texture), and then added on top of previous layer with an
operation like Mix, Add, Mult, etc.
- Layers render fully independent, so bumpmaps are not passed on to next
layers.
- Per Layer you can set if it influences Diffuse, Specular or Alpha
- If a Material returns alpha (like from texture), the alpha value is
used for adding the layers too.
- New texture "Map To" channel allows to have a texture work on a Layer
- Each layer, including basis Material, can be turned on/off individually
Notes:
- at this moment, the full shading pass happens for each layer, including
shadow, AO and raytraced mirror or transparency...
- I had to remove old hacks from preview render, which corrected reflected
normals for preview texturing.
- still needs loadsa testing!
- The ortho render recode had a weak method to calculate the Z value of the
render coordinate (shi.co[2]). Worked OK for normal render, but not in
unified, giving big errors in Ortho Unfied render.
Solved with proper math, using face equation a*x+b*y+c*y=d. Dunno why I
didn't do it in first place. :)
- Bug fix#2493. Edge render gave much different result for Ortho. This
was due to overflows in the integer math filtering zbuffer values.
Cleaned it op properly, preventing any integer overflow now.
- removed ugly pointerhack from OSA render (negative indices denoted
pointers). this should solve memory errors when using >1.5 gig mem
- cleaned up usage of zbuffer values. These are signed in Blender, and
treated as unsigned all over, giving confusing code
- fixed incorrect gamma-adding for halos (caused in after xmas commit)
And bugfix #2101; wire render didn't give correct rendering for mist.
This caused by fact wires are 2D pixel lines, and not correctly filled
in faces. Retrieving the 3d coordinate while render cannot use a face-
equation then. Solved by retrieving 3D coordinate based on zbuffer value.
Still todo here: calculating correct texture coordinates for wire-edges
that are no faces.
Render:
- New; support for dual CPU render (SDL thread)
Currently only works with alternating scanlines, but gives excellent
performance. For both normal render as unified implemented.
Note the "mutex" locks on z-transp buffer render and imbuf loads.
- This has been made possible by major cleanups in render code, especially
getting rid of globals (example Tin Tr Tg Tb Ta for textures) or struct
OSA or using Materials or Texture data to write to.
- Made normal render fully 4x32 floats too, and removed all old optimizes
with chars or shorts.
- Made normal render and unified render use same code for sky and halo
render, giving equal (and better) results for halo render. Old render
now also uses PostProcess options (brightness, mul, gamma)
- Added option ("FBuf") in F10 Output Panel, this keeps a 4x32 bits buffer
after render. Using PostProcess menu you will note an immediate re-
display of image too (32 bits RGBA)
- Added "Hue" and "Saturation" sliders to PostProcess options
- Render module is still not having a "nice" API, but amount of dependencies
went down a lot. Next todo: remove abusive "previewrender" code.
The last main global in Render (struct Render) now can be re-used for fully
controlling a render, to allow multiple "instances" of render to open.
- Renderwindow now displays a smal bar on top with the stats, and keeps the
stats after render too. Including "spare" page support.
Not only easier visible that way, but also to remove the awkward code that
was drawing stats in the Info header (extreme slow on some ATIs too)
- Cleaned up blendef.h and BKE_utildefines.h, these two had overlapping
defines.
- I might have forgotten stuff... and will write a nice doc on the architecture!
When tracing a mirror with AO, the rendering was extremely slow due to
each mirror sample (like 8 per pixel) taking a full range of AO samples.
Now it uses for mirror samples a corrected amount, which makes sure for
a single pixel still a full AO range is used.
Makes mirror+AO render 5-6 times faster, at least.
- Ambient Occlusion is a more sophisticated ambient trick, which takes
nearby faces into account by firing a hemisphere of shadow-rays
around. AKA 'dirt shader'.
- Eeshlo made it a Lamp type, which doesn't fit well. I've moved the
settings to the World menu, and let the Material->ambient value control
the amount it contributes
- currently, the AO value is added/subtracted/mixed with the 'diffuse'
factor while shading, before it is multiplied with Material color
Buttons are in new Panel 'Amb Occ" in F8 menu. Note:
- "Dist:" by shortening the length of rays you get subtler effects and it
renders faster too
- "DistF:" the attennuation factor gives control over how the 'shadow'
spreads out.
Further it's just raytracing, so tends to be slooooow.... :)
Here same tricks as for other raytraced scenes apply, especially try to
keep the environment as small as possible (exclude faces from Octree by
giving them no Material Traceable).
I still have to think over a couple of aspects, will await feedback on it:
- AO color? Now it just adds 'white'
- other sampling patterns? I tried dithering, which was so-so
- method of controlling final 'samples' in F10? Might be useful for other
oversampling too (area light) to have it reacting to a percentage or so..
(1 lamp, shadow). The 'coherence' check gets reset now for each new
pixel rendered, which remains efficient for oversampling.
- small cleanups in code, prototype added, less globals.
already gave noise with area size of 0.1.
Limited buttons to minimum value of 0.01 for area light. For people
who want smaller they can scale it down in 3d, effectively reducing
the energy then as well.
At last irc meeting, eeshlo pointed to an error in the code. It didn't
use the IOR value correctly. This has been solved. So how it works now:
- the IOR button value influences (very subtle) the fresnel effect.
Only for realism diehards.
- the Fresnel value (slider) now denotes the power in the function
rf + (1-rf) * (1-c)^5
where rf= rf = ((ior-1)/(ior+1))^2
and c the dot-product ray/normal.
- so, set the slider at '5' and you have real fresnel. Lower values
for interesting artistic effects.
- put back the forgotten code for gaussian corrected sampling during
antialising render. Normally, each sub-pixel sample in Blender counts
equally, and together make up the pixel color.
With 'Gauss' option set (F10 menu) each sub-pixel sample creates a small
weighted mask with variable size, which (can) affect neighbouring pixels
as well. The result is smoother edges, less sensitive for gamma, and
well suited to reduce motion-aliasing (when things move extreme slow).
This is result of *long* period of research in NeoGeo days, and based on
every scientific sampling/reconstructing theory we could find. Plus a
little bit of our selves. :)
- I should write once how blender constructs Jitter tables for sub-sampling.
this is a very nice method, and superior to normal block filter or random
jittering... time!
Main target was to make the inner rendering loop using no globals anymore.
This is essential for proper usage while raytracing, it caused a lot of
hacks in the raycode as well, which even didn't work correctly for all
situations (textures especially).
Done this by creating a new local struct RenderInput, which replaces usage
of the global struct Render R. The latter now only is used to denote
image size, viewmatrix, and the like.
Making the inner render loops using no globals caused 1000s of vars to
be changed... but the result definitely is much nicer code, which enables
making 'real' shaders in a next stage.
It also enabled me to remove the hacks from ray.c
Then i went to the task of removing redundant code. Especially the calculus
of texture coords took place (identical) in three locations.
Most obvious is the change in the unified render part, which is much less
code now; it uses the same rendering routines as normal render now.
(Note; not for halos yet!)
I also removed 6 files called 'shadowbuffer' something. This was experimen-
tal stuff from NaN days. And again saved a lot of double used code.
Finally I went over the blenkernel and blender/src calls to render stuff.
Here the same local data is used now, resulting in less dependency.
I also moved render-texture to the render module, this was still in Kernel.
(new file: texture.c)
So! After this commit I will check on the autofiles, to try to fix that.
MSVC people have to do it themselves.
This commit will need quite some testing help, but I'm around!
- implemented tracing of transparency for shadows. This is a material
option, in the new RayTrace panel.
it only traces color and alpha, not shading. So the results of some
transparant colored unlit faces can look odd. I will look onto that.
- changed fresnel formula (got hint from eeshlo!). this simplifies the UI,
now only one button needed. The fresnel value "should" be identical as
the refraction index, but that is booooring! So i added a special fresnel
refraction slider for both mirroring and transparency. By setting all 3
sliders equal, you get 'realism'.
- fresnel for transparancy works for Ztra rendering too. Same for transpa-
rent shadow. But then you need to set 'ray' on in F10 menu.
- uploaded new monkey_glass blend in download.blender.org/demo/test/
Next stage: killing the globals from render, and implement "translucency"
which is effectively allowing faces being lit from behind, as paper or
cloth.
Changelog:
- enable refraction with button "Ray Transp" in Material buttons.
- set "Angular Index" value for amount of refraction.
- use the "Alpha" value to define transparency.
- remember to set a higher "Depth" too... glass can bounce quite some
more than expected.
- for correct refraction, 3D models MUST have normals pointing in the
right direction (consistently pointing outside).
- refraction 'sees' the thickness of glass based on what you model. So
make for realistic glass both sides of a surface.
- I needed to do some rewriting for correct mirroring/refraction,
especially to prevent specularity being blended away.
Solved this with localizing shading results in the rendercore.c.
Now specularity correctly is added, and reduces the 'mirror' value.
- Localizing more parts of the render code is being planned. The old
render heavily relies on struct Render and struct Osa to store globals.
For scanline render no problem, but recursive raytracing dislikes that.
- done test with gamma-corrected summation of colors during tracing, is
commented out still. But this will give more balanced reflections. Now
dark reflections that are reflected in a bright surface seem incorrect.
- Introduced 'Fresnel' effect for Mirror and Transparency. This
influences the amount of mirror/transparency based at viewing angle.
Next to a new Fresnel slider, also a 'falloff' button has been added to
define the way it spreads.
- Fresnel also works for Ztransp rendering
- created new Panel for Raytrace options
I have to evaluate still where it all should be logically located.
- material preview shows fake reflection and fake refraction as well.
This is a revision of the old NeoGeo raytracer, dusted off, improved quite
a lot, and nicely integrated in the rest of rendering pipeline.
Enable it with F10-"Ray", and set either a 'ray-shadow' lamp or give the
Material a "RayMirror" value.
It has been added for 2 reasons:
- get feedback on validity... I need artists to play around with it if it's
actually useful. It still *is* raytracing, meaning complex scenes will
easily become slow.
- for educational purposes. All raytracing happens in ray.c, which can be
quite easily adjusted for other effects.
When too many disasters pop up with this, I'll make it a compile #ifdef.
But so far, it seems to do a decent job.
Demo files: http://www.blender.org/docs/ray_test.tgz
An article (tech) about how it works, and about the new octree invention
will be posted soon. :)
Note: it doesn't work with unified render yet.
do a make clean in source/blender/ to be sure!
- Included the new shaders from Cessen... well, only the shader calls
themselves. To make sure the shaders work I nicely integrated it
- MaterialButtons: layout changed a bit, but still resembles the old
layout. The 'shader' options now are located together.
- Shaders are separated in 'diffuse' and 'specular'. You can combine them
freely.
- diffuse Lambert: old shader
diffuse Oren Nayar: new shader, gives sandy/silky/skinny material well
diffuse Toon: for cartoon render
- specular Phong: new spec, traditional 70ies spec
specular CookTorr: a reduced version of cook torrance shading, does
off specular peak well
specular Blinn: new spec, same features as CookTorr, but with extra
'refraction' setting
specular Toon: new spec for cartoon render
- default blender starts with settings that render compatible!
- works in shaded view and preview-render
- works in unified render
Further little changes:
- removed paranoia compile warnings from render/loader/blenlib
- and the warnings at files I worked at were removed.
