Thanks to Lguillaume for helping with this one.
Had to go back to bug: [#17631] PIL_dynlib_get_error_as_string() returns NULL, causes crash
and revert the fix and make some new updates.
function needs to return NULL so fix the functions that were assuming it
always returns a string.
Kent
could be 0, hanging yafray. This commits removes the separate yafray
number of processor setting and simply using the blender threads
button, there is no reason for a separate setting.
minus one small include file that was commented out, I'm not
sure why it was commented out but I'm pretty sure its needed.
If there are still problems later we can sort it out, everything
else is pretty simple.
Kent
Also, duplis are now taking into account, the proper way to exclude
them is to set the material to be not traceable.
Removed an unnecessary pointer from the VlakRen struct to save some
memory, not really that significant, but still, saves 70 mb for 10
million faces.
=================
Big commit, but little user visible changes.
- Dupliverts and duplifaces are now rendered as instances, instead
of storing all of the geometry for each dupli, now an instance is
created with a matrix transform refering to the source object.
This should allow us to render tree leaves more memory efficient.
- Radiosity and to some degree raytracing of such objects is not
really efficient still. For radiosity this is fundamentally hard
to solve, but raytracing an octree could be created for each object,
but the current octree code with it's fixed size doesn't allow this
efficiently.
- The regression tests survived, but with I expect that some bugs will
pop up .. hopefully not too many :).
Implementation Notes
====================
- Dupligroups and linked meshes are not rendered as instances yet,
since they can in fact be different due to various reasons,
instancing of these types of duplis that are the same can be added
for them at a later point.
- Each ObjectRen now stores it's own database, instead of there being
one big databases of faces, verts, .. . Which objects that are actually
rendered are defined by the list of ObjectRenInstances, which all refer
to an ObjectRen.
- Homogeneous coordinatess and clipping is now not stored in vertices
anymore, but instead computed on the fly. This couldn't work for
instances. That does mean some extra computation has to be done, but
memory lookups can be slow too, and this saves some memory. Overall
I didn't find a significant speed impact.
- OSA rendering for solid and ztransp now is different. Instead of e.g.
going 8 times over the databases times and rendering the z-buffer, it
now goes over the database once and renders each polygon 8 times. That
was necessary to keep instances efficient, and can also give some
performance improvement without instances.
- There was already instancing support in the yafray export code, now it
uses Blender's render instances for export.
- UV and color layer storage in the render was a bit messy before, now
should be easier to understand.
- convertblender.c was reorganized somewhat. Regular render, speedvector
and baking now use a single function to create the database, previously
there was code duplicated for it.
- Some of these changes were done with future multithreading of scene
and shadow buffer creation in mind, though especially for scene creation
much work remains to be done to make it threadsafe, since it also involves
a lot of code from blenkernel, and there is an ugly conflict with the way
dupli groups work here .. though in the render code itself it's almost there.
Added support for multiple UVs in the render engine. This also involved
changing the way faces are stored, to allow data to be added optionally
per 256 faces, same as the existing system for vertices.
A UV layer can be specified in the Map Input panel and the Geometry node
by name. Leaving this field blank will default to the active UV layer.
Also added sharing of face selection and hiding between UV layers, and at
the same time improved syncing with editmode selection and hiding.
Still to do:
- Multi UV support for fastshade.
- Multires and NMesh preservation of multiple UV sets.
All data layers, including MVert/MEdge/MFace, are now managed as custom
data layers. The pointers like Mesh.mvert, Mesh.dvert or Mesh.mcol are
still used of course, but allocating, copying or freeing these arrays
should be done through the CustomData API.
Work in progress documentation on this is here:
http://mediawiki.blender.org/index.php/BlenderDev/BlenderArchitecture/CustomData
Replaced TFace by MTFace:
This is the same struct, except that it does not contain color, that now
always stays separated in MCol. This was not a good design decision to
begin with, and it is needed for adding multiple color layers later. Note
that this does mean older Blender versions will not be able to read UV
coordinates from the next release, due to an SDNA limitation.
Removed DispListMesh:
This now fully replaced by DerivedMesh. To provide access to arrays of
vertices, edges and faces, like DispListMesh does. The semantics of the
DerivedMesh.getVertArray() and similar functions were changed to return
a pointer to an array if one exists, or otherwise allocate a temporary
one. On releasing the DerivedMesh, this temporary array will be removed
automatically.
Removed ssDM and meshDM DerivedMesh backends:
The ssDM backend was for DispListMesh, so that became obsolete automatically.
The meshDM backend was replaced by the custom data backend, that now figures
out which layers need to be modified, and only duplicates those.
This changes code in many places, and overall removes 2514 lines of code.
So, there's a good chance this might break some stuff, although I've been
testing it for a few days now. The good news is, adding multiple color and
uv layers should now become easy.
solution could have been much simpler, but since that would have required
altering blender code (only a single line though)...
also fixed a minor bug reported in the yafray forums, when spotlights had the
shadowbuf flag set and then switching to yafray, yafray still rendered
shadows which to user was unexpected since there was no shadow flag enabled
in the yafray lamp panel, so now ignores the blender flag.
are already defined locally, probably does not work fully yet.
Added extra 'threads' parameter as requested by Lynx3d.
Optimized drawing of rendered tiles, so that the entire image doesn't have
to be redrawn every time a tile is complete.
The blender code that handles this part was not yet complete and could only
draw (sets of) scanlines. I extended the renderwin_progress() function in
renderwin.c to handle a given subrectangle.
This code needs review!
For the limited test I did it seems to work at least...
strand texcoords as orco coords, so yafray doesn't have to be adapted for this.
bugfix #4254 added support for dupligroups, but might not work completely
correct yet at this point, more testing needed.
Also added some missing parts from the code apparently removed at some time.
Mainly having to do with dupliverts, cam.info for aspect ratio/ortho mode/etc.
Header stats (render window) should now work again too.
Fixed missing last tile draw of render window.
Added the missing const_cast in the win32 part of the xml export code.
to take over now, but I'm available for help. Main notes for completing:
- Yafray module uses old global R all over... is now a pointer handle.
It can be temporally bypassed by straight copying, which I do now.
- I am not sure in what pixel format Yafray renders... Blender now only
uses float buffers. In the code, marked with XXX I've added the
rudimentary code for retrieving buffers.
- This integration will skip compositing when Yafray render is used.
A full detailed description of this will be done later... is several days
of work. Here's a summary:
Render:
- Full cleanup of render code, removing *all* globals and bad level calls
all over blender. Render module is now not called abusive anymore
- API-fied calls to rendering
- Full recode of internal render pipeline. Is now rendering tiles by
default, prepared for much smarter 'bucket' render later.
- Each thread now can render a full part
- Renders were tested with 4 threads, goes fine, apart from some lookup
tables in softshadow and AO still
- Rendering is prepared to do multiple layers and passes
- No single 32 bits trick in render code anymore, all 100% floats now.
Writing images/movies
- moved writing images to blender kernel (bye bye 'schrijfplaatje'!)
- made a new Movie handle system, also in kernel. This will enable much
easier use of movies in Blender
PreviewRender:
- Using new render API, previewrender (in buttons) now uses regular render
code to generate images.
- new datafile 'preview.blend.c' has the preview scenes in it
- previews get rendered in exact displayed size (1 pixel = 1 pixel)
3D Preview render
- new; press Pkey in 3d window, for a panel that continuously renders
(pkey is for games, i know... but we dont do that in orange now!)
- this render works nearly identical to buttons-preview render, so it stops
rendering on any event (mouse, keyboard, etc)
- on moving/scaling the panel, the render code doesn't recreate all geometry
- same for shifting/panning view
- all other operations (now) regenerate the full render database still.
- this is WIP... but big fun, especially for simple scenes!
Compositor
- Using same node system as now in use for shaders, you can composit images
- works pretty straightforward... needs much more options/tools and integration
with rendering still
- is not threaded yet, nor is so smart to only recalculate changes... will be
done soon!
- the "Render Result" node will get all layers/passes as output sockets
- The "Output" node renders to a builtin image, which you can view in the Image
window. (yes, output nodes to render-result, and to files, is on the list!)
The Bad News
- "Unified Render" is removed. It might come back in some stage, but this
system should be built from scratch. I can't really understand this code...
I expect it is not much needed, especially with advanced layer/passes
control
- Panorama render, Field render, Motion blur, is not coded yet... (I had to
recode every single feature in render, so...!)
- Lens Flare is also not back... needs total revision, might become composit
effect though (using zbuffer for visibility)
- Part render is gone! (well, thats obvious, its default now).
- The render window is only restored with limited functionality... I am going
to check first the option to render to a Image window, so Blender can become
a true single-window application. :)
For example, the 'Spare render buffer' (jkey) doesnt work.
- Render with border, now default creates a smaller image
- No zbuffers are written yet... on the todo!
- Scons files and MSVC will need work to get compiling again
OK... thats what I can quickly recall. Now go compiling!
Absorption and Dispersion parameters now only visible when 'Ray Transp'
enabled. WardIso specular amount scale to match Blender output.
Updated halo spotlight 'samples' to use new yafray syntax.
Quick addition for access to another yafray feature:
When using HDR backgrounds for lighting ('SkyDome' of 'Full' GI methods),
it is currently not always possible to get smooth lighting results.
Especially HDR images with small lightsource can be very noisy,
because currently yafray still relies on brute force random sampling.
As a temporary simple solution (better options will be available in the
'next generation' yafray), yafray can do some processing on the
image to smooth out all (or most) noise.
Besides smooth lighting, this also has
the advantage that AA will have less work to do,
GI quality can be set to the lowest level and still get reasonably
good results. Disadvantage however is that shadow definition is lost.
To switch on this option, set the world image texture filter parameter
to any value greater than 1.0
When 'filter' is 1.0 or less, normal hdr sampling is done as before.
So, current fastest possible render settings for IBL:
set texture image filter parameter of the background image to any value
greater than 1.0, set GI to 'SkyDome' type, enable 'Cache',
(possibly enable 'NoBump' when scene uses lots of bumpmapping),
set 'Quality' menu to 'Use Blender AO settings',
make sure AO is enabled in blender World buttons and set there the number
of AO samples to 1.
Should at least be good enough for previews.
sqeeze in a minor bugfix.
I wouln't be surprised it is too late,
but since it really is sort of minor, it doesn't matter really.
I had mistakenly put the export of dispersion and absorption parameters in the
ray_mirror block instead of ray_transp, which means that both dispersion and
absorption can only be used if both ray mirror and ray transp are enabled.
Anyway, I don't want to make everyone recompile release builds if already
done, it is minor, but users should take this into account at least.
parameters the material preset menu won't be as useful. Both glass presets will look the same
because there is no 'filter' parameter in the old yafray for instance.
So using the new Blender version with an old yafray version should work a bit better,
though the other way around, using the new yafray with an old blender version, will generally
not work as well.
I added a few extra things. In 'yafray' panel re-arranged some buttons, and added a new
button 'Clamp RGB'. This button will be enabled by default and helps to improve AA on
high contrast edges in the image. When using bokeh however, it is best to switch this off,
otherwise lens shaped highlights will be quite a bit less visible.
Changed the 'extinction' parameter name to the probably more correct term 'absorption',
though mathematically it works out the same. Also changed the behaviour of this color,
it no longer specifies a color that will be removed as I wrote in the previous commit,
but instead the actual color at one (blender) unit of distance. The 'Ds' (distance scale)
button below the color sliders controls the scaling of this unit distance.
What this means is that if you take the standard blender cube, which covers two units of
distance by default, setting the distance scale button to 2.0 will make sure that the color
you specified is exactly that color at that distance (provided the base color itself is white
of course, or 'filter' is 0, otherwise it will be filtered by the base color too).
Beyond this distance the color will get darker.
The glow option for point/soft/sphere lights has a new parameter 'GloOfs', or glow offset.
Setting this to a higher value then 0 will soften the central peak of the glow.
Another unreported bug fix: For xml export, when yafray failed to render the xml file
for some unknown reason, or because of other problems, the export code would still load
the previously rendered image, this causes problems however if the image resolution is
not the same as the current Blender buffer, and so could cause memory corruption or crashes.
This is now taken into account.
World image backgrounds now use the blender mapping settings as well, but only the
'AngMap', 'Sphere' and 'Tube' settings. But in yafray those last two, unlike Blender, cover
the whole view, not just the upper half, so is not really fully compatible with yafray.
So now you have to set one of these buttons too when loading a hdr lightprobe image.
btw, something I forgot to mention in previous commits is that the exposure control using
the texture brightness slider is no longer restricted to integer values. It is now a
floating point value, so you're not restricted to the 0 1 and 2 slider positions anymore,
anything in between will work too.
And finally, display updating is now more like Blender, using the mouse cursor as frame
counter for animation, etc.
of course...)
Not quite complete, but due to lack of time as good as it will get for now.
From the previous commit, forgot to report that basic fog is supported as well. Though because I had not much time to complete the code, it is sort of unfinished, and you will have
to tweak parameters specifically for yafray again. It uses only the world horizon color, and
only uses the Blender mist distance setting.
Textures now support checker clip mode.
Fixed possibly all 'duplilist non-empty' errors, though it could hide the real cause of the
error.
AA is no longer enabled automatically for certain GI quality settings, I thought it best to
leave it to the user to decide.
SkyDome GI mode now supports cache as well. There is a new option in the GI quality menu 'Use
Blender AO settings', which will as it says use the most important AO settings for the
skydome parameters. The only AO parameters used are 'Samples', 'Dist' and the random sampling
switch, which unlike in Blender you might want to use more often, since the QMC sampling used
in yafray can result in visible patterns or a dithering type look. 'Random' is not completely
random in yafray however, it is actually jittered (stratified) sampling.
Using an occlusion cache, doesn't necessarily mean that you will always get much shorter
render times. As with 'full' GI and cache, one problem is bumpmaps, when using bump (or
normal) maps, the sampling will be much more dense, using lots more rendertime.
As a temporary fix there is a button 'NoBump', but this also has the side effect that in
areas of total indirect light (or when used with SkyDome cache) no bumpmapping will be
visible. It is therefor best used with some direct light as well.
For SkyDome with cache, and strong bumpmapping it might actually not make much difference,
since for low distance values you can usually get away with low sample values as well.
The entire material panel is now replaced by another panel to show only the parameters
important to yafray and add some new ones as well.
Since lots of users (especially yafray beginners) have had problems getting certain material
aspects right, there is now a material preset menu available to hopefully solve some of the
most common "How do I do this? It doesn't work!" questions seen in various forums.
Choosing an option from this menu will set the required parameters to default
values for yafray, and you can work your way from there to tweak it something you want.
Most buttons are copies of the same Blender parameters, with some variations. Just like
Blender 'Ray Mirror' enables reflection, 'Ray Transp' enables refraction. You can use
'ZTransp' for materials that have texture maps with alpha channels.
Again, same as Blender 'rayMir' sets the amount of reflection. Next button 'frsOfs' however
controls fresnel offset, meaning that when this is set to 1, you will get no fresnel effect
and when set to 5, reflection is totally determined by fresnel, which is important for
realistic glass/metals/etc.
IOR is self-explanatory (...), same as Blender.
When you have 'Ray Transp' enabled, the blender 'filter' button will appear next to the IOR
button. This has the same effect as in Blender.
Below that there are some new parameters, 'Ext.Color' sets the extinction color for
transparent materials. Usually, in real transparent materials, light loses some of it's
energy the further it has to travel through the object. This effect can be simulated with
this parameter. Thing to look out for is that it specifies the color which will be
REMOVED after traveling through the object. What this means is that say you have a clear
white glass sphere, and set the extinction color to a strong blue, the result will be a
very yellow object when rendered.
Next to the color sliders, there is another set of three parameters, with which you can
enable color dispersion for transparent objects. 'Pwr' sets the amount of dispersion,
the higher, the more dispersion (the more colorful the result).
(For real world materials, this number can be found or derived from data in various glass catalogues)
The 'Samples' button below that sets the number of samples used, minimum values are around
7-10, and for very strong dispersion you might need a lot more.
As usual, this also means an increase in render time of course, but to simulate
realistic materials, you shouldn't really need more than 25 samples.
In addition to that, when using low sample numbers, but to still get a good spread of colors,
you can enable the jitter button, but this will also add noise.
Point/soft(point with shadowbuffer) or sphere lights (light with radius), have a new option
to add a simple glow effect, so that lights can be made visible.
NOTE: just like spotlight halo's, glow is not visible against the background, there must be
another object behind it. Simplest solution is to use a large black shadeless plane behind
your scene.
The glow intensity can be set with the 'GlowInt' parameter (use very low values around 0.01
even lower), and you can choose from two different types with the 'GlowType' button (which
don't look much different, but type 1 is probably better, type 0 faster).
And that's it, with apologies for the still missing features and
full support in general, but this will have to do for now.
Totally updated blender shader in yafray, hopefully better matches blender
results. Though ramps are now partially supported, they cannot work in all
cases properly in yafray, and in fact are a bit useless probably as far as
yafray is concerned. In fact the 'Result' ramp input mode is not supported
at all, because it works on the total lighting result, and in a yafray
shader this is not possible since it works per light.
Also, since Blender and Yafray have totally different lighting models,
the 'Energy' ramp input mode also won't generally give the same results
as in Blender, since it works with light energy and in yafray this is
different from Blender. Even worse, the only ramp shader that will work
properly when used with GI is the 'Normal' ramp input mode.
As contradictory as this might seem, at various stages of the GI process,
lighting is not known, so properly getting light (ramp 'energy' mode)
or shader information (ramp 'shader' mode, which depends on lighting)
is not possible. Which all means that when the ramp is in 'energy' or
'shader' mode and using it with GI enabled, yafray can only 'see' the
underlying material color, not the ramps, which results in a mix of the
ramp colors (from direct light) with the material color (from indirect light).
There is currently nothing that can be done about that.
The supported texture mapping modes now includes raymir as well, transparency
as far as texturing is concerned now works similar to Blender, with the
exception that you still have to set alpha to a low value to get any
transparency effect at all in yafray. So the Blender 'filter' parameter
now also will affect yafray.
All texture blending modes are now supported (same for ramps).
'Translu' and 'Amb' texture modulation are not supported.
Texture interpolation can be switched off ('InterPol' switch in blender
image texture button section).
All Blender brdf models (aka 'shaders' for the Blender users) are now supported,
and again, you won't necessarily get the same results as in Blender.
The reason for that is partially of course the lighting differences, but also,
not all Blender 'shader' implementations are actually correct, and copying
those errors just for the sake of matching Blender results doesn't really
seem like a good idea...
Though this really is only the case for WardIso, less so for Minnaert and
Blinn, which in yafray are more or less (but not totally) a copy of
the Blender code.
In any case, in practice those differences might not be
too noticable at all (I hope).
Continue to the next part...
Fixed an as of yet unreported bug, linked data objects which have materials
linked to object instead of data have to be exported as normal, they cannot
be instanciated, otherwise they will all share the same material.
parented linked data lamps made the export crash.
fix for two bugs reported by Diego Pino, an endian issue with vertex colors,
and the plugin code didn't export multiple colorbands correctly.
Updated the ortho camera export code for the new Blender ortho camera model.
Render:
- New; support for dual CPU render (SDL thread)
Currently only works with alternating scanlines, but gives excellent
performance. For both normal render as unified implemented.
Note the "mutex" locks on z-transp buffer render and imbuf loads.
- This has been made possible by major cleanups in render code, especially
getting rid of globals (example Tin Tr Tg Tb Ta for textures) or struct
OSA or using Materials or Texture data to write to.
- Made normal render fully 4x32 floats too, and removed all old optimizes
with chars or shorts.
- Made normal render and unified render use same code for sky and halo
render, giving equal (and better) results for halo render. Old render
now also uses PostProcess options (brightness, mul, gamma)
- Added option ("FBuf") in F10 Output Panel, this keeps a 4x32 bits buffer
after render. Using PostProcess menu you will note an immediate re-
display of image too (32 bits RGBA)
- Added "Hue" and "Saturation" sliders to PostProcess options
- Render module is still not having a "nice" API, but amount of dependencies
went down a lot. Next todo: remove abusive "previewrender" code.
The last main global in Render (struct Render) now can be re-used for fully
controlling a render, to allow multiple "instances" of render to open.
- Renderwindow now displays a smal bar on top with the stats, and keeps the
stats after render too. Including "spare" page support.
Not only easier visible that way, but also to remove the awkward code that
was drawing stats in the Info header (extreme slow on some ATIs too)
- Cleaned up blendef.h and BKE_utildefines.h, these two had overlapping
defines.
- I might have forgotten stuff... and will write a nice doc on the architecture!
of 'envmap', 'magic', and 'plugin' modes.
The stucci texture also is not exact match, since it cannot be fully
emulated in yafray because of implementation issues. It will work best
for low turbulence values (which is actually not taken into account
in the export code).
Also, since Blender's static noise is basically just direct random number
output, don't expect the exact same result when rendered in yafray, but in this
case that probably shouldn't be that much of a problem...
(needs yafray from cvs)
(btw, on a side note, I put this in the comments once when working on the noise
stuff for Blender, but noise is affected by the 'depth' parameter, and there
is no way to control this directly from the GUI, can only be done by temporarily
switching to 'clouds' for instance.)
orco coordinates were assumed to be valid when pointers were non-null,
this seems to have been a wrong assumption, pointers were probably unitialized
(but only for some objects like surfaces?), so when the export code tried to
access them, it crashed.
Now the rendermaterial texco flag is tested instead, which probably is how
it should have been done in the first place...
with the zblur plugin for faster dof (or other seq.plugins that need the
zbuffer).
I don't think the conversion to a blender zbuffer value is quite correct,
but at least it does produce usable results for zblur without too much
differences with the blender render (at least not for the short tests that I
could do in this short time...)
Texture matrix bug in plugin code reported by Mel_Q.
Vertex colors, this was basically the same as the previous uv coord
splitting bug, for xml export, uv coord splitting was actually not quite
complete either (reported by richie).
Added:
Camera Ipo curves for DoF aperture and focal distance.
Aspect ratio set with AspX & AspY are now taken into account as well.
(needs yafray from cvs)
Bokeh parameters for DoF (also needs yafray from cvs).
'Bokeh' controls the shape of out of focus points when rendering
with depth of field enabled.
This is mostly visible on very out of focus highlights in the image.
There are currently seven types to choose from.:
'Disk1' is the default, the same as was used before.
'Disk2' is similar, but allows you to modify the shape further with the 'bias'
parameter, see below.
Triangle/Square/Pentagon/Hexagon, in addition to the bias control, you can
offset the rotation with the 'Rotation' parameter (in degrees).
'Ring', a weird ring shaped lens, no additional controls.
The 'bias' menu controls accentuation of the shape.
Three types available, uniform, center or edge, with uniform the default.
Although based on an actual phenomenon of real camera's, the current
code is bit of a hack and not physically based, and doesn't work all that
well yet (in yafray anyway). Since this is also mostly visible in the very
out of focus parts of the image, it usually also means that you need lots
of samples to get a reasonably smooth result.
Blender hemilight shadow flag now ignored (reported by varuag).
Texture axes were not exported for procedural textures.
Duplicate armatures were not handled correctly (reported by richie).
Triangle uv-coord splitting (reported by anael, richie & Alvaro).
Additions:
Material 'TexFace' mode now works too, as in Blender it functions as an extra
first texture channel, replacing the base color.
The new noise functions for procedural textures are now supported in yafray,
but is not quite completed yet, still undergoing changes.
(needs yafray from cvs).
The 'power' button has been renamed to 'EmitPwr', since it controls background,
arealight (including lamp with radius) & material emit power.
This button can now be used with the 'SkyDome' method as well to control
background lighting.
To control indirect lighting power, a button called 'GI pwr' has been added,
only use this when really necessary, first try modifying 'EmitPwr' instead.
Removed:
The 'gradient' button. This includes the python code to set
this parameter as well.
Bugfix for anim crash with xml export and no export dir set.
Alternate relative path using blender function, but wouldn't be surprised
if it still is a problem in some cases for some (windows) users.
