Basically, due to new 'virtual' libraries & 'path' assets, we consider those 'virtualmy linked' IDs as:
* Local for editing purpose (i.e. they are editable).
* Linked for datablock management purposes (i.e. they can be made local, etc.).
Note: all this is more like a quick hack to test 'file-based' asset repositories (like cloud)
in comming weeks. I really do not think we should use that in the end, an full-featured
'overriding ID' system (as proposed in replacement of proxies for 2.8) would be much saner imho.
Since this file mostly defines/handles interactions with asset engines...
Asset handling itself is more done in library area (will split it too in next commit).
Note that a few sub-data (like animdata) needs to keep option to unlink their
own ID usages themselves for now, this is not ideal, but need some more time
to mumble on this and find an elegant fix. Nothing blocking here anyway.
Idea is to avoid the need for asset engines to generate whole .blend files
with relevant datablocks in case they just need/want to provide files (images, videos, sounds...).
This commit introduces the concept of 'virtual' library, which in fact only exists to store
asset repository data. Datablocks from those virtual libs are then fully written in .blend file.
Much to do still, this is only preliminary/experimental work.
Asset engines are now only selectable and used in link/append case. Makes no real sense to have
those in other cases (like regular file opening, or even worse when saving a file!).
Also allows to get back asset engine persistance accross filebrowser usage.
Dead dummy simplistic test runs (nearly) OK here (only have a memleak from versionning code...),
but obviously will need much serious tests to validate everything.
Also, many things in code will need cleanup/refactor. Not to mention UI/UX.
But we do have asset version check & reload on file opening now! :D
Process summary:
I) Open .blend file exactly as in master, load assets just like any other linked data (add placeholders in case not found).
II) Check all asset engines and ask them whether each asset is OK, needs to be updated/reloaded, is missing...
III) User then can decide to actually reload the asset, or not.
Step II) happens in an async job (since engines using remote storage/database could take some time to answer),
step III) is a locking task (just like initial linking). This should allow user to not be blocked at all by that
asset update process.
With current master this is not an issue, but with all the reload stuff,
a same main could end up going several time into 'do_version', which is absolutely
not desired (multi-allocations, repeating conversions over some values, etc.).
Whole 'database' is rebuilt later anyway, and in more complex 'asset reload' cases
(where we may actually be reloading same asset with same uuid from same lib file),
this creates confusion...
Idea here is to both factorize it seriously (reload and relocate are very similar, let's
avoid as much as possible twice the same code...), and make new `lib_relocate_do()`
much more flexible - now it should be able to reload individual IDs as well
(plan is to use it to reload assets in relelvant branch too).
So far, we were accepting the idea of several repositories per asset engine (e.g. different
directories in case of Amber). However, if this works OK during listing/initial linking,
it makes things way more complicated later in asset management, since asset uuid is no more
enough to unically indentify an asset!
Now an asset UUID is assumed to be really unique inside an asset engine.
For Amber, we 'salt' assets' uuids with new repo uuids (both being only 2 ints long now),
and keep a config 'cache' of mapping repo uuid -> path to repo.
Fetching assets' uuids by engines is now in own func (will be used by update op itself too).
Also, each AssetRef has only *one* asset ID, the first one, avoids us a useless loop!
And early out of update_check op in case we have no asset in current Main, no need to start
job in this case.
So now, jobs callback (async ones) of asset may return a specific job id value in case they actually
complete (or fail) on the first run.
This allows engines that do not need slow async stuff to perform an action to use simpler code
(e.g. imagine an engine able to list its assets from a cached DB, in most cases its list_dir
callback can execute instantaneously (from a user PoV), no need for a listing job then).
Reason is, even if paths returned by `list_dir` are OK, we will also use `load_pre`
during 'reload'/'update' scenarii, where we only have uuids, to retrieve final
.blend datablock paths.
Code only reports which datablocks/assets shall be reloaded etc. so far, actual
reloading needs id-remap code (and hence will be implemented in asset-experiment
branch later).
This commit:
* Fixes bad handling of 'stop iteration' (by adding a status flag, so that we can actually
stop in helper functions too, and jumping to a finalize label instead of raw return, to
allow propper clean up).
* Adds optional recursion into 'ID tree' - callback can also decide to exclude current id_pp
from recursion. Note that this implies 'readonly', modifying IDs while recursing is not
something we want to support!
* Changes callback signature/expected behavior: return behavior is now handled through flags,
and 'parent' ID of id_pp is also passed (since it may not always be root id anymore).
Used this new recursive behavior in ID preview generation as an example, would obviously be committed separately.
Reviewers: sergey, campbellbarton
Differential Revision: https://developer.blender.org/D1869
This commit:
* Fixes bad handling of 'stop iteration' (by adding a status flag, so that we can actually
stop in helper functions too, and jumping to a finalize label instead of raw return, to
allow propper clean up).
* Adds optional recursion into 'ID tree' - callback can also decide to exclude current id_pp
from recursion. Note that this implies 'readonly', modifying IDs while recursing is not
something we want to support!
* Changes callback signature/expected behavior: return behavior is now handled through flags,
and 'parent' ID of id_pp is also passed (since it may not always be root id anymore).
Used this new recursive behavior in ID preview generation as an example, still needs more testing!
We need that info (knowing which non-asset IDs are used by which asset-IDs),
otherwise managing reloading, updates etc. of assets would be impossible
(or rather, would leave a mess of unused IDs behind them).
Only partially done, non-functional yet.
This commit also adds asset repository references in libraries.
Note that this is only very first steps, I think we'll need to 'propagate'
uuid to all data used/linked by asset itself, otherwise most things won't get
updated/reloaded. Still have to think about this.
Note that this is only very first steps, I think we'll need to 'propagate'
uuid to all data used/linked by asset itself, otherwise most things won't get
updated/reloaded. Still have to think about this.
Issue here is again with NEVER_NULL usages: obdata (& co) would be dereferenced twice.
I'm not totally happy with this solution, ideally remap should never leave Main
in invalid state, but for now it will do...
* We need to clear the 'user-one' flags and potential usercount for old_id.
* We need to consider ID usages inside a same lib as **never** being indirect ones.
Also re-enabled proxy checking for now (not sure why I commented it, think testing stuff or so...).
Seems to work for simple cases, but deletion of complex libs from complex files
(Gooseberry ones) still generates lots of assert failures, and crashes in some cases.
This is hacky to make work on Linux, and seems to totally break on Windows.
So now, instead, we define a CollectionListBase (exact copy of ListBase), to be used inside RNA...
Seems to work nicely on Linux, lets see what win buildbot says.
issues with unlinking ParticleSettings.
We now have the option to still decrement user count of old_id, even if we could not
replace it with NULL new_id (aka unlinking), needed to keep proper user count when
unlinking and ID from datablocks it uses, to delete it.
Conflicts:
source/blender/blenkernel/BKE_library.h
source/blender/blenkernel/intern/library.c
Also clears now useless 'bool is_user_one' flag from editors' ID remap callback.
Idea is to add two new flags, one saying 'we need to ensure a real user exists',
the other 'we had to increment user count to ensure we have a real user'.
This allows us to easily control the extra user in release/delete/remap cases,
and also fixes the infamous 'add new image to texture, open image in ImageEditor,
delete image from texture, have a zero-user red image in Image Editor' issue.
There is still much to be done here, more places where we can use those flags,
also clear them when we force usercount to zero, etc.
All this allows us to fix unsolvable issues (like Group being ensure_user'ed
in loading code, but only if they do have objects in them), and to avoid returning
ugly bool from editors' callbacks (this is still to be cleaned up in the branch too).
Bad news - this means we cannot use short ID->flag anymore (not enough flags), for now
added a new int ID->flag2 (replacing pad int), not sure how to best manage change here,
maybe for 2.8 we can totally wipe ID->flag? But this would totally break forward compat.
So, idea is, since mat/tex/scene/etc. nodetrees are owned by their respective IDs
(those nodetree do not exist in Main, they are systematically freed with their ower IDs, etc.),
we should not treat them as IDs in IDlooper, but rather as mere sub-data, and hence directly
loop over the IDs of those nodetrees.
From quick check it seems to work, but this needs to be confirmed as a valid idea!
Heavily simplfies that freeing area, but again a rather risky and likely-to-break change.
At least, NodeTree is known to be a trouble-maker here, due to how it seems to be
'owned' by its mat/tex/sce/etc. Have to dig deeper here, this is still quite unclear
what exactly happens with those, and how to handle them correctly.
(instead of whole Main content).
Could replace maybe things like constraint, rigisbody world, etc. '_relink' func,
but for now it's only intended to replace custom 'id releasing' code in _free()
funcs of all IDs!
Previous situation was pretty much horrible, a few data types having their own coocking to
unlink, often doing more than only unlinking, often doing the same thing two times or more,
often messing with areas they should not have touched (like editors from withing BKE)...
Now we hope to have something generic, working the same way for all ID types
(we do have to add some specific handling for groups/objects/scene unfortunately,
but this remains reasonable).
Needless to say such a change is calling for troubles - I tried to follow and reproduce
as best as I could previous code, but most likely some areas will become buggy. Do not think
previous code was 100% correct anyway, things like Objects have a really big and complicated
usage of IDs...
Also, foreach_id tool has been enhanced, again for complex types like objects & co, we should
cover much better all possible IDs now (e.g. rigidbody & logic bricks were
totally missing from there).
And there is more to come...
Think core of the system is getting pretty much OK, now we'll likely have to deal
with tons of specific cases, given how Blender is totally inconsistent when it comes to
handling user counts (e.g. spaceimage...).
Also, still have to make generic ID_unlink and ID_release_datablocks func to replace
redundant code in BKE we have currently.
Still missing: at least modifiers (have to change all modifiers callbacks, yuck).
AFAIK constraints never refcount thier IDs so we can keep current code here.
Similar changed to those done for mesh/curve/mball.
Also, systematically nullify pointers in `_free()` functions, this does not cost
much and can help troubleshooting later.
And tag `BKE_object_unlink()` as an horrible piece of code...
Renamed their `_unlink()` functions to `_release_datablocks()`, since they do not
unlink anything (not in the sense `BKE_object_unklink()` does, at least)!
Also, added a `const bool do_id_user` to their `_free()`, for consistency, now
relevant `_release_datablocks()` func only gets called when this option is set.
They would be removed from bmain, but never added again nor freed - in this case
we can simply explicitely reload them in fact, non-linkable is only a user-related status...
We need to append to linklist, or our library indices would be reversed compared to lib list...
As for locking, we need reentrant one here if we want to use it. :(
Performances are not an issue at all here, but it makes code slightly simpler, avoids
a loop over ghash of libs, avoids some potential mem fragmentation, and will be easier
with lib relocate code too in future...
Reason is, on next fileread, proxy will be assigned with its target's data again...
Note that this may not be true about all its datablocks pointers - but proxy objects
are really brittle in current code anyway, they could use a serious work as well.
Basics seem to be working, proving idea is valid. There is much to do yet though,
handling correctly all possible scenarii (especially those including indirect libs
used by several libraries, or libs used both directly and indirectly, etc.)
is not going to be a piece of cake...
Making the code in WM area more generic (will be used by relocate libs feature too in future).
And multi-append/link shall no more close & re-open lib files for each and every idcode!
Also, serious cleanup of BLO_append_... API (renamed BLO_link_..., since it links and never appends!),
main real changes there are removing the bContext arg in favor of scene/v3d (a bit more verbose, but does
not requires a valid context anymore to be able to instanciate ob/groups!). And logic behind instanciating
or not has been simplified, previous code was really obscure and sometimes redondant, from quick tests
it still works as expected.
This is temp branch for until 2.76 is over, shall be merged in master asap then.
Note that this op will have to be moved to WM area in fact, it's way too generic for Outliner only...
and we need to re-use most of append/link operator code!
We have to take care of Object->data aside from main foreach loop, since we can
only replace that pointer if object is **not** in Edit mode!
Also, added a "Remap" action to ID context menu of outliner. This is **very far**
from nice and pretty code/feature, but it allows to quickly test the code.
Only did very quick tests with meshes so far...
This seems to be OK, but will obviously need much more testing - and is useless as-is, we need
the big ID remapping code itself still.
Also, Nodetrees' remapping of Scene IDs is still TODO, not sure yet how to do this.
Needed, since trying to (re)init data already set could lead to many issues,
and handling this is not in the scope of this function.
Note that definition of BLI_memcmp_null stuff is rather bad currently, no good idea where to place this. :|
Just having 'NULL' data as placeholder is not really possible, too much places
to check against that.
WIP commit, still much to be done!
See T43351 for details.
Differential Revision: https://developer.blender.org/D1394
Idea is, instead of ignoring completely missing linked datablocks, to
create void placeholders for them.
That way, you can work on your file, save it, and find again your missing data once
lib becomes available again.
Plans are also to be able to locate missing libs and reload them at runtime.
To support all that, we must be able to make Blender survive those missing data
(i.e. empty datablocks) all over the place. This commit contains some early work
in that direction, but this will need much much more work!
Indeed, though rather unlucky, we may end up handling same source file from
different preview threads, which could lead to conflicts and bugs.
So idea is to add a simple way of locking a given source file path, since
thumb handling of different paths shall never conflict.
Note that this adds some 'generic' stuff to GHash and Threads area, that are to
be committed separately of course.
The task was not so annoying (workers are just sleeping anyway), but having the timer
always checking for updates was not so nice. This remains quite minor optimization probably...
So now, we do not start anymore preview task when enabling previews. This is deferred
to the moment we really need it (i.e. when we cache block of entries currently visible).
Preview timer is started at the same time.
Further more, when preview task has been inactive for about one second (i.e. all
queued previews have been handled, and no new one have been queued), we free it
(switching to some kind of 'standby' mode).
This also allows to avoid stoping/restarting preview tasks on each redraw when user is scrolling.
Classical bool vs char value mismatch... Here it was leading to a constant re-filtering
of the list, which invalidates cached previews.
Strange thing is, it would look as if Windows CMake builds are using 'real' bool,
while scons ones are using fake char-based one?
Instead of computing MD5sum of paths, just use incremental UUIDs!
There meaning and lifetime for internal listing is very limited anyway,
they shall only be unique for a given listing.
Note about using atomic op here: did it because it's nice use case,
not sure it's worth adding atomic stuff to /editors area though?
This commit replaces `filelist_numfiles()` by `filelist_files_ensure()`. The later
also returns number of visible files (i.e. filtered ones), but also ensures filelist
is filetered and sorted.
Otherwise, during listing process, operators could be executed between an update of the listing job,
and the refresh from the file space - i.e. at a time filelist is virtually empty (from filebrowser PoV).
Also, got rid of 'need_refresh', this ended up being unused...
* Do not systematically clear selection_state's uuids GHash when updating filelist
during listing process - uuids used here should remain valid.
Allows to not lose selections during listing background job!
* Use new BLI_ghash_lookup_p to handle selection state setting, symbolic optimization but...
A file which does not have *any* preview is perfectly valid, error is only when there
are some previews, but not the same number as actual datablocks...
Conflicts:
source/blender/editors/space_file/file_draw.c
source/blender/editors/space_file/file_ops.c
source/blender/editors/space_file/space_file.c
NOTE: Tagged some parts in new code to be fixed, rather do that in a separate commit.
This means this commit is broken and won't compile.
Also adds some 'state' flags to AssetEngine, so that custom AE's setting changes
can be notified to filebrowser, as default internal ones are.
For now, we only have 'dirty filtering' and 'dirty sorting'.
With big screens and tiny drawing, we can show much more than 1k items at once...
Also allows to reduce number of cached items in case we show few things big!
Mismatch in 'owner' of job, not giving right one to stoping helper, sigh.
Note that this was also affecting any 'interrupting' task (like changing level of flat listing...).
We cannot keep 'stable' list of items during listing process (each time new
entries are added, we have to filter and sort the whole lot again, which
means rebuilding 'draw' data from scratch - previews included).
Just wait until our filelist is complete to generate/use previews.
* Let asset engine draw most of header bar itself!
* Pass whole FileSelectParams to AE's sort_filter function.
* Move 'use_library_browsing' to params level (this way AE's sort_filter can be ware of it too).
Amber:
* Now supports real basic sorting/filtering modes
* Also basic support of tags (needs a way to refresh though, currently).
this is mandatory, since we do not keep previews for all listed items anymore
(with 10k datablocks would start to take too much mem). So we now generate (read)
them from thumbnailing threaded task as any other image, has several advantages:
* Quicker initial listing of data from .blend files.
* Since thumbnails are cached on disk, re-generating them is rather quick.
* General consistency!
Note thumbnail generation of datablocks could (should!) be enhanced, right now
it reads all previews for a given datatype in a file each time it has to generate
a single datablock thumbnail, we should generate all of those previews in a single
run. But this means handling cache dirty things (TBH_FAIL, outdating, etc.) on another
level so need some more work.
Also, still some memleaks issues with thumbnails, have to track those dose!
Since with internal listing we have no choice but to list everything, store
those data in the smallest possible struct - note we do not even store preview
here, these are only handled in full FileDirEntry cache (re-loading them from
cached thumbnails is really quick anyway).
Still not complete, have to re-do ID previews now, among other things...
And some memleaks again with previews, ugh.
Those were eating quite a bit of bites, not neglectable with thousands of entries...
Also, useless to print those info most of the time, they are not used!
So now, we only store 'raw' stat data, and expose funcs to convert them to readable
strings on demand.
Since we do not store anymore ell entries, we cannot use them to hold selection status.
So we now use a ghash in FileList struct, with entry UUID's as keys.
Also cleanuped up / refactored selection handling, filelist.c now features a nice intern
api and all other code uses it instead of accessing directly entries' selflag!
Conflicts:
source/blender/editors/space_file/filelist.c
source/blender/editors/space_file/filelist.h
source/blender/makesrna/intern/rna_internal.h
Notes:
* asset-experiments branch still needs some work (e.g. entires selection is to be reworked too),
but was more than time to do a first merge...
* This is raw merge, code is hence broken and needs more fixes to take into account new changes!
Get rid of cache iterator code (not needed, and don't think we'll need it later,
easy to add back if needed). Also comment some debug prints.
New preview (thumbnails) handling seems to be working pretty nice now.
* Full path is no more static, so we need dynamic tooltips to use it (still an
issue with drag stuff here :| ).
* Remove timer would attempt to free timer's customdata, which is mere int-in-pointer
in case of new notifier timer, so we need a special tweak on remove here.
Start to looks good... This commit adds a new timer helper, that only send notifiers
(kind of very restricted subset of wm_job stuff), since using jobs for previews
would be now counter-productive and way too much heavy.
Also fix/enhance how previews are generated (order), etc. etc.
Still WIP though, having some weird crashes and such lurking around.
Note this is rather stupid anyway, we need to rewrite this from scratch,
using a job for that makes no sense now, we should stick to a lighter
threading schema with a queue, or something like that!
Core part seems to be working, but this is still heavy WIP, much to do
to resume complete features from existing code. Mainly:
* Previews (those should only be ran on cached items now).
* Use block-caching to load in-display entries.
* Rework things like selection (cannot store selected states in entires anymore).
Also, internal listing is stupid currently (since it still stores everything
as FileDirEntry & co), not really crucial currently, but ultimately it'll
use its own, compact struct to keep full listing!
Also did some minor cleanup/renaming.
That code is currently pretty stupid, but we need to clearly separate
what is handled by filebrowser itself, and what is by listing code
(be it internal one or future asset engines).
Plan/idea is to have a dual cache system (one 'block' of entries - centered
on currently viewed ones -, and one 'random' FIFO for random access to any entry).
This implies filebrowser itself shall now only be aware of the number of (filtered)
entries, and it then requests entries as needed.
Also, made size of columns in filebrowser fixed (ultimately, user configurable),
since it has no more access to the whole list of entries!
Still heavy WIP.
We cannot do that correctly with current listdir system, have to rework again
in asset-experiments brnach first, so that listing, filtering and sorting is
completely "delocalized" to asset engine (will also try to solve scalability issues).
Also, make clear tags are only informative here.
Reason is, if the project is a success, we have to think big and
imagine asset engines managing catalogs of tens or hundreds of thousands
of entries (web ones e.g.), we cannot hold list of those in filebrowser!
So most of filetring will be 'deffered' to asset-engine.
We'll also probably have to implement a 'chunck listing' behavior
(to only list at most n entries in a row), for same reasons.
This will make things slightly more complicated :|
Think issue also exists in master code actually, where some op properties
could be left uninitialized in 'params_to_op' helper func... But it does not
backfire like that for sure.
This will allow asset engines to:
* Actually ensure requested data is available (downloading it, generating it, etc. as needed).
* Change entries generated for user by actual data paths (allows e.g. to present
a set of different paths to load as a single 'asset' to user).
As usual, had to change existing filebrowser code... Seems to be working fine
from quick tests (i.e. regular filebrowser not being broken).
Basically, filelisting code in space_file becomes default, internal 'asset engine',
and user can switch to any other available engine instead.
Note filelisting with asset engines is fully main-thread from C code PoV,
parallelization/asynchronism is responsability of the py code!
Also, added a basic (dummy currently) AssetEngine py implementation.
Not much there, but helps checking things works as expected before
writing whole interface!
All this is very basic early code yet, much much to do before we get anything really usable...
Remove everything not really related to OS representation of file entries.
FileBrowser will now use a complete decoupled version of file representation,
much easier to keep sane behavior this way, easy integration with RNA, etc.
This allows us to greatly simplifies sorting (and avoid recomputing UI name each and everytime!).
Also, correctly free entries/variants names/descriptions.
Was not sure about that (adds two pointers to entries, variants **and** revisions :/ ),
but with the incremental process of listing we are using now, it makes things sooooo
much easy to handle (and also avoids tons of (deep)copy and mem (re)alloc), that it's
definitively worth it.
And since most (if not all) of 'public' access to filelist is done through filtered ones,
we can easily keep this a mere array of pointers, and hence keep quick index lookup.
Also, quite noticiably simplified the whole listing code (less functions, less levels of subcalls...).
Note: for now we stick to mere alloc, we may benefit from either memarena or mempool,
but those are not threadsafe (which means we could not so easily pass mem from worker
thread to main one), so not quite sure it would be worth it.
Note: thumbnail stuff is to be reworked from scratch too, have the feeling it's not yet
100% thread safe, and using a job here is probably way overkill, since we do not show
any progress in UI.
Also, reduce revision's uuid to 64bytes, should be more than enough!
Thing is, we can expect tens (if not hundreds) of thousands of revisions from a
big assets repository with history storage (vcs). So we want that struct
to be as small as possible.
Also, we do not expect to handle several variants/revisions of a same asset within
a same 'context' (there is always only one active variant & rev). So storing
path & co at entry level shall be enough. And helps make code simpler/nicer.
We may suffer a little performance loss here, but would not expect it to be really important,
since rebuilding a full abspath is not needed that often.
Also, get rid of modes/owner crap, this is quite advanced UNIX fs info, not much to
do in our filebrowser imho, and takes time, UI space and Mem space!
Move typeflag and selflag back to FileDirEntry, and add blentype there too,
this will be needed by asset engines, and avoids rechecking lib ID types over and over!
There's much more to do here though. Also, FileDirEntryRevision is going to be
used quite higly, we want to keep it as light as possible (much lighter than
its current state for sure!). Think we can move back to FileDirEntry paths
(and try to get rid of abspath on the run, too), but also most likely
preview (we won't store a preview for all possible revisions anyway),
owner/permissions, etc.
Rev should probably only have its uuid, and date/time.
Also, simplified filetype generation (from extensions for real files, and BLO helpers
for libdata), now it's nicely encapsulated into each reading callback.
Main idea is that direntry had quite a bit of things already absolutely useless
in the mere 'file listing' context (image pointer, void 'data' pointer, etc.).
On the other end, to handle future asset stuff, we need *more* 'meta data' for our 'files'
(tags, variants, revisions...).
Further more, we need all this to be easily usable through RNA, i.e. we need it to
be defined in DNA.
So instead of trying to use direntry in DNA, mixing low-level system stuff like stat
where we do not want to see it, we define our own 'file' representation,
with only what we really need (who cares about chars or block or dev files in filebrowser?
we only need to know whether it's a regular file or a directory, period).
That way, low-level dirlisting remains clean, and we can easily extend our own
data as needed.
Note this new code works, does not crash (so far...) nor memleak, but it
needs quite a bit of work still, we are still storing useless stuff in direntry
(all those strings representing modes, size etc.!), and mem handling
during dir/lib listing is confusing at best (would like to use a memarena
or mempool here, but it's not that easy).
Passing around ID's themselves is not safe - they may change their mem addresses.
Yet, I had no issue on linux, but on win build this is still not enough,
have to use a super strange hack (add an 'import sys' line just before where error happens)
to get things running...
Batch preview generation can now also generate 'internal' previews (materials, textures etc.),
as well as scene ones. Also, some basic Cycles handling is there now.
You can now chose which kind of data to generate previews for, and whether the files are
trusted or not (for py autorun and drivers).
And rendering happens in background again, issue was actually a bug in creator.c, fixed in master.
Main TODOs reamining here: OpenGL rendering, and find a way to prevent '.blend1' when
saving over (either as a new option of save op, or simply by removing those files in script?).
Added a batch cleanup of previews too.
Also, added path of files as their tooltip 'image' button in main filebrowser view,
quite useful with the 'recursive' option...
Now, recursion 0 = previous behavior,
recursion 1 = only show content of immediate .blend files, do not recurse in actual dir hierarchy,
recursion >1 = complete recursion!
This will allow us to keep op panel independent, and have tabs for future assets tools!
Note 'conversion' from previous file area (channels) is not perfect, you have to hit 'T'
twice currently if you had a filebrowser in your .blend file...
Consider blendfiles and 'categories' inside them as regular dirs, but with this hierachy:
real dir > blende file lib > category in a blend lib.
This gives a much better sorting (among other things, gives more importance to
blend files!).
Got completely rid of C code for those panels.
Added full path in ttips of each 'bookmark' entry, as in current master.
Note this forced me to make tip in uiBut dynamic (like label). Not so sure
about this, but seems to me forcing tooltips to be static is actually quite
bad and limitating? we'll see...
Only remaining TODO (new feature) is reordering of bookmarks, should be easy now.
Also fixed some stupid (again!) mistake with RNA access to paths of bookmark entries.
* When overlaying icon, we have to also ensure background pict is always the same size
(else you get usgly icon size changes). Still not perfect though, suspect some more issues
in filrdraw code, probably should systematiacally run the 'ensure size' code during thumbnail
generation?
* Sharing of img ImBuf between thumbnail job and main thread could still happen,
leading to random crashes.
Note: unlike in current master, path is checked immediately when setting it
into filelist (simpler now that filelist building itself is threaded).
`filelist_setdir()` modifies in place given path if this one is not valid.
`BLI_fnmatch_strncpy_add_endswildcards()` will copy and add '*' wildcards on both ends of given string, if needed.
`BLI_fnmatch_strcmp_ignore_endswildcards()` is same as strcmp, but ignores any '*' wildcards on both ends of given strings.
New code shall be more easy to maintain and extend.
Sorting is now handled quite the same as filtering, and all filtering parameters
are now packed into a sub-struct to help extending it later.
Also done some optimizations in filelist refresh, and sorting/filtering area.
Now we should avoid re-sorting and re-filtering too often, also removed
calls to those in read_xxx funcs.
Note thumbnail job is still started basically on each call to `file_refresh()`,
will be addressed in next commit.
Doing it mostly because I need some update work done here so far to get py uilists/bookmarks working.
This exposes/moves part of bookmark code to ED_ area, so that it becomes available from e.g. RNA.
And adds RNA struct and such to represent a bookmark entry.
Also, Bookmark format is ammended to store optional custom names.
All this is still *very* crappy, should probably take some time soon to cleanup the whole branch!
See also T42777.
* Do not render cameras, lamps etc. for now.
* Generate bbox in camera local space, gives much better results.
* Add an op to clear previews from files.
Now for any object, we have an rna func to call with a set of (flatten :/)
coordinates, and we get back location this object should be to 'see'
all given points.
Would need more refinements, but will do for now.
First expose a method in Object RNA to get camera projection matrix (done).
Then, expose a method to get camera position (and scale for ortho) so that
it fits a given set of coordinates (still unfinished).
this among other things implied rewriting `BKE_camera_view_frame_fit_to_scene()`
underlying logic so that it uses `CameraParams` system, instead of `BKE_camera_view_frame()` one.
Note since this gives python possibility to change those previews,
we add flags to PreviewImage to tag as user_defined.
TODO: replace all those shorts-used-as-bools by bitflags!
Previuosly, we would only write existing data - and big previews were
nearly never generated by default.
Now, if saving previews is enabled, `wm_file_write()` will ensure
all 'common' previewable datablocks do have up-to-date previews
(brushes remain TODO here for now, they have both a preview **and**
an icon_imbuf, sigh...).
This implied adding an 'immediate' non-job-background version of
`ED_preview_icon_job()`.
Note this does not include objects/groups/etc. It's not simple to generate
previews for those, will probably go for a user-launched py operator here...
Also, cleanup some debug prints!
Previous implementation was uglyly relying on button using a specific
icon (VIEWZOOM) to enable this feature. Now we have a dedicated button flag
(UI_BUT_TEXTEDIT_UPDATE) and a dedicated RNA subtype (PROP_SEARCH) for that.
Also, if no icon is specified, it automatically set VIEWZOOM for it,
which saves us explicitely defining it in UI py code!
NOTE: this is to be committed asap in master, but has to wait for
after 2.73, so storing it here for now.
Also, try to make brushes also save their previews.
Anyway, something fishy is going on here, in theory we should get
previews for brushes, worlds, lamps... for free, but so far looks like
lib 'peek' code only handles mat/texture previews correctly?
Doing nothing useful yet, though!
Note: ultimately we may want to move that preview stuff to ID struct,
so that we get a real basic generic handling of it, but for now
simpler to keep it as is.
Also, definitively removed 'G.main.name protection' from lib reading
code. I do absolutely not see how listing a separate blend file handle
could possibly affect that anyway - and we do not want to access
G from non-main thread I'd bet...
Now it works reasonably OK for filesystem, and also for .blend libs.
Code is even quite simpler than previous one. Now it needs some serious cleanup,
and find where (new) memleak comes from too.
Only very partially working in this state, still needs some work.
Proof of concept seems rather interesting though.
Also, only pure filesystem stuff is implemented for now.
Also:
* Changed BLO_library_path_explode, now r_group and r_name must be pointers of char pointer,
if found they will point to right place in r_dir 'memspace'.
* Select everything in IDTypes filtering by default.
Nothing really magic here, just grouping the imports by libfile/grouptype.
We could probably get better code (loop less on given list of 'files' to import)
but not sure it'd be worth it, this is not supposed to be a performance-critical point.
* Delete filelist when changing flat setting (so that we get immediate update when changing that setting).
* Disable flat when going up in tree (we ;ight actually want to do this everytime).
Not much to add, we'll likely add more filtering options (tags, categories...) later,
maybe for now we just need an 'exclude' option for name filter...
Main goal is to see whether we can use existing filelist to handle that
(by default it is 100% tree-oriented).
Also, getting some strange issues with ID name currently...
2014-12-03 15:29:18 +01:00
66 changed files with 4866 additions and 422 deletions
WM_jobs_timer(wm_job,0.1,0,0/*NC_SPACE | ND_SPACE_FILE_LIST, NC_SPACE | ND_SPACE_FILE_LIST*/);/* TODO probably outliner stuff once UI is defined for this! */
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.