Make the Windows version of `BLI_dir_create_recursive()` call
`BLI_path_slash_native()` before it tries to handle the path. This
should make it possible to call it with non-native path separators.
This change was provided by our Windows platform maintainer @LazyDodo in
P2414, so I assume he agrees with this change.
Compressing blendfiles can help save a lot of disk space, but the slowdown
while loading and saving is a major annoyance.
Currently Blender uses Zlib (aka gzip aka Deflate) for compression, but there
are now several more modern algorithms that outperform it in every way.
In this patch, I decided for Zstandard aka Zstd for several reasons:
- It is widely supported, both in other programs and libraries as well as in
general-purpose compression utilities on Unix
- It is extremely flexible - spanning several orders of magnitude of
compression speeds depending on the level setting.
- It is pretty much on the Pareto frontier for all of its configurations
(meaning that no other algorithm is both faster and more efficient).
One downside of course is that older versions of Blender will not be able to
read these files, but one can always just re-save them without compression or
decompress the file manually with an external tool.
The implementation here saves additional metadata into the compressed file in
order to allow for efficient seeking when loading. This is standard-compliant
and will be ignored by other tools that support Zstd.
If the metadata is not present (e.g. because you manually compressed a .blend
file with another tool), Blender will fall back to sequential reading.
Saving is multithreaded to improve performance. Loading is currently not
multithreaded since it's not easy to predict the access patterns of the
loading code when seeking is supported.
In the future, we might want to look into making this more predictable or
disabling seeking for the main .blend file, which would then allow for
multiple background threads that decompress data ahead of time.
The compression level was chosen to get sizes comparable to previous versions
at much higher speeds. In the future, this could be exposed as an option.
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
Instead of handling mmap, compression etc. all directly in readfile.c, refactor
the code to use a generic FileReader.
This makes it easier to add new compression methods or similar, and allows to
reuse the logic in other places (e.g. thumbnail reading).
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
The fseek() function on Windows only accepts a 32-bit long offset
argument. Because of this we have our own version, BLI_fseek(), which
will use 64-bit _fseeki64() on Windows. This patch just replaces some
fseek() calls with BLI_fseek().
Differential Revision: https://developer.blender.org/D11430
Reviewed by Brecht Van Lommel
Previously the return value of `ufopen` wasn't checked and if it failed,
`NULL` was passed into `fclose()` which resulted in a crash. This patch
avoids this by returning from `BLI_gzopen` when the file cannot be created.
Reviewed By: sebbas, iss
Differential Revision: https://developer.blender.org/D9576
This addresses warnings from Clang-Tidy's `readability-else-after-return`
rule in the `source/blender/blenlib` module. Not all warnings are
addressed in this commit.
No functional changes.
This patch implements dumping images from cache to HDD.
The main goal of this system is to provide a means to achieve consistent playback speed mainly for strips that are not possible to preview in real time.
How to use:
Disk cache has own settings in user preferences for path to storage, size limit and compression level.
To use disk cache, you need to check `Use Disk Cache` box, set `Disk Cache Directory`, `Disk Cache Limit` and save or open existing .blend file.
By default sequencer output will be cached only. Manual setting is possible in cache panel.
Uses:
- Replacement or alternative for proxies. Disk cache will work with any strip type, supports float images as well.
- Storage for strip thumbnails.
- Less RAM needs to be allocated for preview cache
How it works:
Disk cache is extension of RAM cache. Every image, that is stored or deleted in RAM will be stored or deleted on HDD as well. Images can be compressed to save space and for use on slower drives. Compressed images are slower to write and read though.
Images are stored in bulk of 100 rendered frames per one file. This is to overcome slow file access time for large amount of files. Drawback is, that if one frame needs to be redrawn, all 100 frames are deleted.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D5524
In Blender 2.7 delete would permanently delete files, now this function is back
but using more standard behavior.
This patch includes code contributed by Kris (Metricity).
Differential Revision: https://developer.blender.org/D4585
Preparing for the bigger changes which will be related on passing
dependency graph to various callbacks which need it.
Differential Revision: https://developer.blender.org/D5725
BF-admins agree to remove header information that isn't useful,
to reduce noise.
- BEGIN/END license blocks
Developers should add non license comments as separate comment blocks.
No need for separator text.
- Contributors
This is often invalid, outdated or misleading
especially when splitting files.
It's more useful to git-blame to find out who has developed the code.
See P901 for script to perform these edits.
The Issue
=======
For a long time now MinGW has been unsupported and unmaintained and at this point,
it looks like something that we should just leave behind and move on.
Why Remove
==========
One of the big motivations for MinGW back in the day is that it was free compared to MSVC which was licensed based.
However, now that this is no longer true we have basically stopped updating the need CMake files.
Along with the CMake files, there are several patches to the extern libs needed to make this work. For example, see:
https://developer.blender.org/diffusion/B/browse/master/extern/carve/patches/mingw_w64.patch
If we wanted to keep MinGW then we would need to make more custom patches to the external libs and
this is not something our platform maintainers are willing to do.
For example, here is the patches needed to build python: https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-python3
Fixes T51301
Differential Revision: https://developer.blender.org/D2648
* Fix BLI_file_touch, used to add one dummy byte ((unsigned char)EOF) to empty files!
* Get rid of static global temp string in WIN32 area (very bad, and useless!).
* Get rid of paranoid NULL checks in WIN32's BLI_gzopen().
* Add non-relative filename asserts to WIN32 file operations too.
* ifdef-out BLI_move and BLI_create_symlink, unused (and the later is not even implemented for windows).
Partly based on patch and points raised by Jason Wilkins (jwilkins) in T32870, thanks!
Missed adding return value in one place.
Thanks a lot to Thomas Szepe for reporting the issue and finding the solution!
Question remains: how could MSVC allow such a code to compile in some cases
(own win64 + scons buildbot win32 were OK with that it seems!).
Crappy compiler, not comming to the rescue of stupid dev...
