Instead of handling mmap, compression etc. all directly in readfile.c, refactor
the code to use a generic FileReader.
This makes it easier to add new compression methods or similar, and allows to
reuse the logic in other places (e.g. thumbnail reading).
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
Keep braces balanced where possible, even with ifdef's as it avoids
confusions with editors calculating correct indentation level &
finding matching brackets.
This patch tries to sanitize the types of our size parameters across our read
and write code, which is currently fairly inconsistent (using `int`, `uint`,
`size_t`...), by using `size_t` everywhere. Since in Blender file themselves
we can only store chunk of size `MAX_INT`, added some asserts to ensure that
as well.
See {T79561} for details.
Differential Revision: https://developer.blender.org/D8672
Optimizations in rBcda15408582e8de5b405 do not guarantee anymore that
consecutive memchunks in two consecutive undo steps are actually about
the same data (and hence can share the same buffer when unchanged).
This buffer sharing can now happen without any particular order, so we
need to change the process when 'merging' two undo memfiles together.
Note that existing code was not logically correct either, even with
previous undo storage code, since it would blindly transfer ownership of
the buffer to the second memchunk, without checking whether the first
one was actually the owner of it or not (a same buffer can be shared by
matching memchunks in many consecutive memfiles/undo steps).
Most of the time current (based on order) system works fine, but when you add
or rename (i.e. re-sort) some ID, every data/memchunk afterwards would be out
of sync and hence re-stored in memory (and reported as changed).
Now we are storing the ID's session_uuid in the memchunks, which allows to
actually always find the first memchunk for an already existing ID stored in
previous undo steps, and compare the right memory.
Note that current, based-on-order system is still used almost all of the time,
search in the new ghash is only performed for a few data-blocks (when needed at all).
Reviewed By: brecht
Maniphest Tasks: T60695
Differential Revision: https://developer.blender.org/D7877
These changes only have an effect when the experimental Undo Speedup preference
is enabled.
* For DEG_id_tag_update, accumulate recalc flags immediately before the undo
push happens instead of afterwards. Otherwise the undo state does not
contain enough flags, and the current state may contain too many flags.
This also means we call DEG_id_tag_update after undo with the accumulated
flags to ensure they are flushed to other datablocks.
* For undo, accumulate recalc flags in id->recalc and clear accumulated flags
immediately. Not clearing would cause circular behavior where accumulated
flags may never end up being cleared.
This matches what happens after an undo push where these are also cleared,
indicating that the undo state and current in-memory state match exactly.
* Don't change id->recalc of identical datablocks, it should not be needed.
There is one exception for armatures where pointers across datablocks
exist which otherwise would cause problems. There may be a better solution
to this but it seems to work in agent 327 production files.
* This contains a change in undofile.c to avoid detecting all datablocks as
changed for the first of the two undo steps, where we restore to the state
of the last undo push before going to the one before.
Without this the whole system is much less efficient. However this is unsafe
in the sense that if an app handler or operators edits a datablock after an
undo push, that change will not be undone.
It can be argued that this is acceptable behavior, since a following undo push
will include that change and this may already have unexpected side effects.
Ref T60695
Differential Revision: https://developer.blender.org/D7339
Note this only changes cases where the variable was declared inside
the for loop. To handle it outside as well is a different challenge.
Differential Revision: https://developer.blender.org/D7320
Clear is_identical_future before adding a next undo step, to avoid wrong
values for cases where we undo and then add a step with different changes
than what was previously the next step.
Ref D7274
The feature is hidden behind an experimental option, you'll have to
enable it in the preferences to try it.
This feature is not yet considered fully stable, crashes may happen, as
well as .blend file corruptions (very unlikely, but still possible).
In a nutshell, the ideas behind this code are to:
* Detect unchanged IDs across an undo step.
* Reuse as much as possible existing IDs memory, even when its content
did change.
* Re-use existing depsgraphs instead of building new ones from scratch.
* Store accumulated recalc flags, to avoid needless re-compute of things
that did not change, when the ID itself is detected as modified.
See T60695 and D6580 for more technical details.
BF-admins agree to remove header information that isn't useful,
to reduce noise.
- BEGIN/END license blocks
Developers should add non license comments as separate comment blocks.
No need for separator text.
- Contributors
This is often invalid, outdated or misleading
especially when splitting files.
It's more useful to git-blame to find out who has developed the code.
See P901 for script to perform these edits.
That kind of implicit includes should really only be done when totally,
absolutely necessary, and ideally only with rather simple 'second-level'
headers.
Otherwise not being explicit with includes always end up biting in
unexpected ways...
- Move static undo variable into 'WriteData',
'memfile_chunk_add' used arguments in a confusing way,
sometimes to set/clear static var.
- Replace checks for 'wd->current' with 'wd->use_memfile'
move memfile vars into 'wd->mem' struct.
- Use a single undo history for all operations.
- UndoType's are registered and poll the context to check if they
should be used when performing an undo push.
- Mode switching is used to ensure the state is correct before
undo data is restored.
- Some undo types accumulate changes (image & text editing)
others store the state multiple times (with de-duplication).
This is supported by checking UndoStack.mode `ACCUMULATE` / `STORE`.
- Each undo step stores ID datablocks they use with utilities to help
manage restoring correct ID's.
Needed since global undo is now mixed with other modes undo.
- Currently performs each undo step when going up/down history
Previously this wasn't done, making history fail in some cases.
This can be optimized to skip some combinations of undo steps.
grease-pencil is an exception which has not been updated
since it integrates undo into the draw-session.
See D3113
- Get memory usage from MemFile instead of MEM API
avoids possible invalid when threads alloc memory.
- Use size_t instead of uint and uintptr_t to store size.
- Rename UndoElem.str -> filename
- Rename MemFileChunk.ident -> is_identical
`my_memcmp` didn't work properly comparing memory sizes not aligned to 4 bytes,
this worked while we used guarded-alloc (which always wrote a guard at the end of each allocation).
Since moving to lockfree allocator it could read uninitialized memory.
It also consistently performed ~10-30% worse then glibc's.
This is typically well optimized, no need to do ourselves.
NOTE: BLI_winstuff.h was meant to be a wrapper around windows.h to handle
undefining various crap that windows.h defines. Platform specific headers
should only have to be included in a few places. This reduces the number
of inclusions of BLI_winstuff.h to 16 which is a much more reasonable
number (than the 144 or whatever it used to be)
