Use i18n monospace font in Text editor and Python console #34373

New Issue

Shinsuke Irie · 2013-02-22T23:17:26+01:00

Shinsuke Irie commented

2013-02-22 23:17:26 +01:00

%%%Bunch of patches that allows Blender to display i18n monospace font in the text editor and the Python interactive console. Wide characters that occupy multiple columns such as CJK characters can be displayed correctly. Furthermore, wrapping, selection, suggestion, and cursor drawing should work.

Screenshot: https://plus.google.com/photos/108666457756457924138/albums/5847890778201826545/5847890782374975970?authkey=CIPZrc32pPzuTw

This patch uses wcwidth.c written by Markus Kuhn to estimate how many columns each character occupies:

http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c

wcwidth.c is distributed under MIT-style license:

Permission to use, copy, modify, and distribute this software
for any purpose and without fee is hereby granted. The author
disclaims all warranties with regard to this software.

I attached 3 files:

texteditor_console_i18n.tar.gz ... Patch files (See below for details)
texteditor_console_i18n.patch ... Same as .tar.gz file, but unified all patches into one file
WenQuanYiMicroHeiMono.ttf.gz ... Font file for testing CJK characters display

The tgz archive texteditor_console_i18n.tar.gz contains 7 patches:

(0) 00_i18n_mono_font.patch

Patch to load i18n monospace font when "International Fonts" in user preference is ON.
Adds function to blenfont:

BLF_get_unifont_mono()
BLF_free_unifont_mono()

(1) 10_extern_wcwidth.patch

Integrates wcwidth.c into extern/wcwidth/ directory. I made wcwidth.h to use it as a library.

(2) 20_BLI_str_utf8_char_width.patch

Adds primitive functions to blenlib to use wcwidth.c:

BLI_wcwidth()
BLI_wcswidth()
BLI_str_utf8_char_width()
BLI_str_utf8_char_width_safe()

Also adds BLI_UTF8_WIDTH_MAX macro. Its value is 2 (in columns).

(3) 30_BLF_draw_mono.patch

Adds a function BLF_draw_mono() to blenfont. This is an analogue of BLF_draw() but drows texts with fixed column width. It takes additional the 4th argument 'cwidth', and returns how many columns were used.

(4) 40_BKE_txt_utf8_offset_to_column.patch

Adds functions txt_utf8_offset_to_column() and txt_utf8_column_to_offset() to blenkernel. They are alternatives of txt_utf8_offset_to_index() and txt_utf8_index_to_offset(), respectively.
Also modifies txt_move_up() and txt_move_down() to take character widths into account.

(5) 50_space_text_i18n.patch

Fixes the text editor (space_text) to take character widths into account.
Renames some internal functions to more suitable names:

txt_utf8_get_nth()        -> txt_utf8_forward_columns()
text_pixel_x_to_index()   -> text_pixel_x_to_column()
flatten_len()             -> flatten_width()
flatten_index_to_offset() -> flatten_column_to_offset()

Also changes args/retval of some inernal functions.

(6) 60_space_console_i18n.patch

Fixes the interactive console (space_console + space_info) to take character widths into account.
Adds internal functions to compute offsets for wrapping:

console_wrap_offsets()
console_cursor_wrap_offset()

Also changes args/retval of some inernal functions.

To build blender with the patch, apply the above 7 patches or texteditor_console_i18n.patch, and save the font file into release/datafiles/fonts/ directory.

Note 1: WenQuanYi Micro Hei font is used only for testing. We should prepare more appropriate one, because the WQY font doesn't include all languages' characters which Blender supports. If you want to test the patch with the other font, save it as "WenQuanYiMicroHeiMono.ttf.gz" (file name is hard-coded). For example, to use DejaVu Sans Mono font on Ubuntu,

gzip -c9 /usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono.ttf > [PATH_TO_SOURCE_TREE]/release/datafiles/fonts/WenQuanYiMicroHeiMono.ttf.gz

Note 2: I'm testing it only on Ubuntu 12.10 (with both CMake and SCons), not tested on Win/Mac at all.
%%%

%%%Bunch of patches that allows Blender to display i18n monospace font in the text editor and the Python interactive console. Wide characters that occupy multiple columns such as CJK characters can be displayed correctly. Furthermore, wrapping, selection, suggestion, and cursor drawing should work. Screenshot: https://plus.google.com/photos/108666457756457924138/albums/5847890778201826545/5847890782374975970?authkey=CIPZrc32pPzuTw This patch uses wcwidth.c written by Markus Kuhn to estimate how many columns each character occupies: ``` http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c ``` wcwidth.c is distributed under MIT-style license: ``` Permission to use, copy, modify, and distribute this software for any purpose and without fee is hereby granted. The author disclaims all warranties with regard to this software. ``` I attached 3 files: - texteditor_console_i18n.tar.gz ... Patch files (See below for details) - texteditor_console_i18n.patch ... Same as .tar.gz file, but unified all patches into one file - WenQuanYiMicroHeiMono.ttf.gz ... Font file for testing CJK characters display The tgz archive texteditor_console_i18n.tar.gz contains 7 patches: (0) 00_i18n_mono_font.patch Patch to load i18n monospace font when "International Fonts" in user preference is ON. Adds function to blenfont: ``` BLF_get_unifont_mono() BLF_free_unifont_mono() ``` (1) 10_extern_wcwidth.patch Integrates wcwidth.c into extern/wcwidth/ directory. I made wcwidth.h to use it as a library. (2) 20_BLI_str_utf8_char_width.patch Adds primitive functions to blenlib to use wcwidth.c: ``` BLI_wcwidth() BLI_wcswidth() BLI_str_utf8_char_width() BLI_str_utf8_char_width_safe() ``` Also adds BLI_UTF8_WIDTH_MAX macro. Its value is 2 (in columns). (3) 30_BLF_draw_mono.patch Adds a function BLF_draw_mono() to blenfont. This is an analogue of BLF_draw() but drows texts with fixed column width. It takes additional the 4th argument 'cwidth', and returns how many columns were used. (4) 40_BKE_txt_utf8_offset_to_column.patch Adds functions txt_utf8_offset_to_column() and txt_utf8_column_to_offset() to blenkernel. They are alternatives of txt_utf8_offset_to_index() and txt_utf8_index_to_offset(), respectively. Also modifies txt_move_up() and txt_move_down() to take character widths into account. (5) 50_space_text_i18n.patch Fixes the text editor (space_text) to take character widths into account. Renames some internal functions to more suitable names: ``` txt_utf8_get_nth() -> txt_utf8_forward_columns() text_pixel_x_to_index() -> text_pixel_x_to_column() flatten_len() -> flatten_width() flatten_index_to_offset() -> flatten_column_to_offset() ``` Also changes args/retval of some inernal functions. (6) 60_space_console_i18n.patch Fixes the interactive console (space_console + space_info) to take character widths into account. Adds internal functions to compute offsets for wrapping: ``` console_wrap_offsets() console_cursor_wrap_offset() ``` Also changes args/retval of some inernal functions. To build blender with the patch, apply the above 7 patches or texteditor_console_i18n.patch, and save the font file into release/datafiles/fonts/ directory. Note 1: WenQuanYi Micro Hei font is used only for testing. We should prepare more appropriate one, because the WQY font doesn't include all languages' characters which Blender supports. If you want to test the patch with the other font, save it as "WenQuanYiMicroHeiMono.ttf.gz" (file name is hard-coded). For example, to use DejaVu Sans Mono font on Ubuntu, ``` gzip -c9 /usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono.ttf > [PATH_TO_SOURCE_TREE]/release/datafiles/fonts/WenQuanYiMicroHeiMono.ttf.gz ``` Note 2: I'm testing it only on Ubuntu 12.10 (with both CMake and SCons), not tested on Win/Mac at all. %%%

Shinsuke Irie commented

2013-02-22 23:17:26 +01:00

Changed status to: 'Open'

Shinsuke Irie commented

2013-02-23 08:32:52 +01:00

%%%texteditor_console_i18n_v2.tar.gz
texteditor_console_i18n_v2.patch

Minor change. In console_cursor_wrap_offset(), use BLI_str_utf8_char_width_safe() rather than BLI_str_utf8_char_width().
%%%

%%%texteditor_console_i18n_v2.tar.gz texteditor_console_i18n_v2.patch Minor change. In console_cursor_wrap_offset(), use BLI_str_utf8_char_width_safe() rather than BLI_str_utf8_char_width(). %%%

Shinsuke Irie commented

2013-02-27 18:07:45 +01:00

%%%texteditor_console_i18n_v3.tar.gz
texteditor_console_i18n_v3.patch

Minor optimization.

Can anyone test/review this patch?
%%%

%%%texteditor_console_i18n_v3.tar.gz texteditor_console_i18n_v3.patch Minor optimization. Can anyone test/review this patch? %%%

Shinsuke Irie commented

2013-02-27 18:13:40 +01:00

%%%Forgot to mention that console_draw_string() was modified.%%%

Campbell Barton commented

2013-03-02 07:02:27 +01:00

%%%Ill check on applying next week. (after 2.66a)%%%

Shinsuke Irie commented

2013-03-09 11:05:18 +01:00

%%%texteditor_console_i18n_v4.tar.gz
texteditor_console_i18n_v4.patch
bmonofont-i18n.ttf.gz
bmonofont-i18n-licenses.tar.gz

Update for code cleanup r55123.
Also replace "WenQuanYiMicroHeiMono.ttf.gz" with "bmonofont-i18n.ttf.gz".

bmonofont-i18n.ttf is a mixed font based on DejaVu Sans Mono and including M+1M Regular and Wen Quan Yi Micro Hei.

Versions and licenses of the included fonts are as follows:

DejaVu Sans Mono: version 2.33, Bitstream font license and Arev font license and public domain
M+1M Regular: TESTFLIGHT 54, M+ font license
Wen Quan Yi Micro Hei: version 0.2.0-beta, GPLv3 with font embedding exception or Apache2.0

See license files in bmonofont-i18n-licenses.tar.gz for the details.
%%%

%%%texteditor_console_i18n_v4.tar.gz texteditor_console_i18n_v4.patch bmonofont-i18n.ttf.gz bmonofont-i18n-licenses.tar.gz Update for code cleanup r55123. Also replace "WenQuanYiMicroHeiMono.ttf.gz" with "bmonofont-i18n.ttf.gz". bmonofont-i18n.ttf is a mixed font based on DejaVu Sans Mono and including M+1M Regular and Wen Quan Yi Micro Hei. Versions and licenses of the included fonts are as follows: DejaVu Sans Mono: version 2.33, Bitstream font license and Arev font license and public domain M+1M Regular: TESTFLIGHT 54, M+ font license Wen Quan Yi Micro Hei: version 0.2.0-beta, GPLv3 with font embedding exception or Apache2.0 See license files in bmonofont-i18n-licenses.tar.gz for the details. %%%

Campbell Barton commented

2013-03-10 08:53:39 +01:00

%%%Checked the patch, looks good to me.

Suggest Shinsuke Irie be given commit rights to commit this patch.%%%

%%%Checked the patch, looks good to me. Suggest Shinsuke Irie be given commit rights to commit this patch.%%%

Shinsuke Irie commented

2013-03-10 10:51:23 +01:00

%%%OK, I'll commit the patch. Please give me commit rights.

However I'm not sure where the license docs of the font should be stored. I couldn't find license docs of droidsans.ttf.gz.
%%%

%%%OK, I'll commit the patch. Please give me commit rights. However I'm not sure where the license docs of the font should be stored. I couldn't find license docs of droidsans.ttf.gz. %%%

Sergey Sharybin commented

2013-03-11 17:43:59 +01:00

%%%Looked over the patch. Looks fine to me as well. Just one note which isn't directly related on this patch. Would be nice to get rid of casting int<->wchar. it's possible that they're different size on different platform. Better to avoid this :) (for sure could be done as separate patch later).
%%%

%%%Looked over the patch. Looks fine to me as well. Just one note which isn't directly related on this patch. Would be nice to get rid of casting int<->wchar. it's possible that they're different size on different platform. Better to avoid this :) (for sure could be done as separate patch later). %%%

Shinsuke Irie commented

2013-03-12 08:33:18 +01:00

%%%Applied r55202+r55203.%%%

Shinsuke Irie commented

2013-03-12 08:33:18 +01:00

Changed status from 'Open' to: 'Resolved'

Shinsuke Irie closed this issue

2013-03-12 08:33:18 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Use i18n monospace font in Text editor and Python console #34373