BLI_strncpy_wchar_from_utf8 made the assumption that
wchar_t is UTF-32 bit regardless of environment, while
this holds true on both mac and linux, on windows
wchar_t is actually actually UTF-16.
This resulted in the upper 16 bits being dropped from
from some string conversions and prevented blender
from starting when installed in a path with unicode
code-points over 0xffff.
There was also a fair bit of code duplication between
BLI_strncpy_wchar_from_utf8 and BLI_str_utf8_as_unicode_and_size
this change essentially removes all logic from
BLI_strncpy_wchar_from_utf8 and calls the right function
for the right environment.
Reviewed By: brecht , Robert Guetzkow
Differential Revision: https://developer.blender.org/D9822
This addresses warnings from Clang-Tidy's `readability-else-after-return`
rule in the `source/blender/blenlib` module. Not all warnings are
addressed in this commit.
No functional changes.
`BLI_strncpy_wchar_from_utf8` internally assumes `wchar_t` is 32 bits
which is not the case on windows.
The solution is to replace `wchar_t` with `char32_t`.
Thanks to @robbott for compatibility on macOS.
Differential Revision: https://developer.blender.org/D6198
BF-admins agree to remove header information that isn't useful,
to reduce noise.
- BEGIN/END license blocks
Developers should add non license comments as separate comment blocks.
No need for separator text.
- Contributors
This is often invalid, outdated or misleading
especially when splitting files.
It's more useful to git-blame to find out who has developed the code.
See P901 for script to perform these edits.
That code was a joke, letting some invalid utf8 bytes pass, returning
wrong offset for some invalid sequences, not to mention length and
pointer easily going out of sync, NULL final byte being 'forgotten' by
memcpy, etc. etc.
The miracle here is that we could survive using this for so long!
Probably because we do not use utf-8 sanitizing enough in Blender,
actually... :/
Own mistake in refactoring of `BLI_strncpy_wchar_as_utf8()`, if given size was exactly
the one needed, we'd lost last char (off-by-one error).
Many thanks to plasmasolutions (Thomas Beck) who found the issue and did
all the investigation work here!
- BLI_strncpy_wchar_from_utf8 wasn't NULL terminating the destination string, caused uninitialized memory use in BPY_python_start().
- BLI_strncpy_wchar_as_utf8 could write one byte past the buffer bounds.
the way Curve.len is used at the moment is really stupid, calculate string size on save for now, but should really store the length in bytes and total number of characters.
- pass string size to BLI_timestr() to avoid possible buffer overrun.
- quiet warning for mingw.
- include guards for windows utf conversion funcs.
- fix for mistage in edge-angle-selection check.
- some style cleanup.
This patch allows Blender to display i18n monospace font in the text
editor and the Python interactive console. Wide characters that occupy
multiple columns such as CJK characters can be displayed correctly.
Furthermore, wrapping, selection, suggestion, cursor drawing, and
syntax highlighting should work.
Also fixes a bug [#34543]: In Text Editor false color in comment on cyrillic
To estimate how many columns each character occupies, this patch uses
wcwidth.c written by Markus Kuhn and distributed under MIT-style license:
http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c
wcwidth.c is stored in extern/wcwidth and used as a static library.
This patch adds new API to blenfont, blenlib and blenkernel:
BLF_get_unifont_mono()
BLF_free_unifont_mono()
BLF_draw_mono()
BLI_wcwidth()
BLI_wcswidth()
BLI_str_utf8_char_width()
BLI_str_utf8_char_width_safe()
txt_utf8_offset_to_column()
txt_utf8_column_to_offset()
