WIP: UI: Bidirectional Text and Complex Shaping #104662

Harley Acheson · 2023-02-12T18:34:09+01:00

Harley Acheson commented

2023-02-12 18:34:09 +01:00

Adds an Experimental feature to support the output of UI text for
scripts that are right-to-left and require complex shaping.

This is progress toward supporting RTL languages and complex text shaping, as needed for Arabic, Aramaic, Azeri, Dhivehi/Maldivian, Hebrew, Kurdish (Sorani), Persian/Farsi, Urdu, Thai, and others.

Specifically this means allowing the entry, editing, and display of these languages in the UI. So text edit boxes, labels, hints, etc. This does not attempt to add such support to Text Editor, nor to Text Objects or String to Curve Node. Text Objects and Curves would be a logical next step, but Text Editor is currently not even considered (mostly because of how Syntax Highlighting is done).

Current status: It is demonstrable, looks cool, and doesn't crash. Mostly works.

Once compiled you need to enable it by going to Edit / Preferences / Interface, turning on "Developer Extras", then enabling "Complex Text Shaping" in the "Experimental" section. Once enabled it will use the complex shaping code for ALL text. The code includes means to only do so for runs of text that require it, but for testing everything goes through it. This means that regular Latin text will track slightly differently as this properly handles gpos kerning.

The impact on performance is minimal. Using this code for all text for default scene is 11.7 milliseconds, while it 9.6 using current code.

It handles text caret insertion properly. This is complex because of bidirectionality in that you are selecting something in visual order but needing an answer in logical order (and these might be differing lengths). Made more complex because the length of a subset of a complex string is not related to the full string as glyphs change as the content changes.

I handles text selection somewhat adequately. Selection with mixed directions gets complicated, but current code does so almost correctly and allows editing and deletion properly. The following might look odd, but is exactly how it should work for mixed-direction selection.

Implementation Details

Our regular text functions handle each character one-by-one as the string is traversed. With complex shaping however we need to process chunks of text at a time since this requires greater context. For RTL languages, for example, we need at least entire words at a time in order to select the correct glyphs for each position. Ligatures need to know of multiple characters in order to do substitutions. Even kerning using the gpos table uses rules that require more context.

In a nutshell this digests an entire string at a time, processing separate segments that have the same language and direction. Strings can contain an unlimited number of segments of differing language or direction. So you can combine English, Arabic, Hebrew, and English again. In that example the string will processed using specific fonts for Arabic and Hebrew from the font stack. This code selects the best font based on the font's language coverage.

A string is consumed at once in the constructor of a ShapingData object:

ShapingData text(str, str_len);

At this point we have versions of the string in u32string in the order we received it (logical order), a copy transformed by FriBiDi into visual order, along with arrays that index between the two.

We need to process each segment separately, so this is done like this:

  while (text.process(font, gc)) {
    // do stuff with text.segment
  }

The above will process each segment by Harfbuzz to transform it into its final form with appropriate glyphs, spacing, and positioning. The segment member has a glyph_count (which may vary from the character count), and other information about this portion of the string including the language. Most importantly it has a Vector of glyphs, a Vector of bounds, and other position and size information. Note that in the above, the font and gc arguments are just what is used as defaults and will change as needed.

Therefore outputting the text is very simple with just a simple loop to place each glyph. Measuring a string is just summing the bounds extents. "For each" processing works as it does now, but keep in mind that the callbacks are called in visual order, so don't assume that the offset will always be ascending.

One big change outside of shaping has to do with glyph caching. We currently locate cached glyphs by codepoint, and then we find the glyph id stored there. But with complex shaping we are given arrays of glyph ids. Therefore the hash index is changed glyph id. Searches for glyphs by codepoint (for simple path) is still fast because the FreeType caching system caches these lookups.

Adds an Experimental feature to support the output of UI text for scripts that are right-to-left and require complex shaping. --- This is progress toward supporting RTL languages and complex text shaping, as needed for Arabic, Aramaic, Azeri, Dhivehi/Maldivian, Hebrew, Kurdish (Sorani), Persian/Farsi, Urdu, Thai, and others. Specifically this means allowing the entry, editing, and display of these languages **in the UI**. So text edit boxes, labels, hints, etc. This does not attempt to add such support to Text Editor, nor to Text Objects or String to Curve Node. Text Objects and Curves would be a logical next step, but Text Editor is currently not even considered (mostly because of how Syntax Highlighting is done). Current status: **It is demonstrable, looks cool, and doesn't crash. Mostly works.** Once compiled you need to **enable it** by going to Edit / Preferences / Interface, turning on "Developer Extras", then enabling "Complex Text Shaping" in the "Experimental" section. Once enabled it will use the complex shaping code **for ALL text**. The code includes means to only do so for runs of text that require it, but for testing everything goes through it. This means that regular Latin text will track slightly differently as this properly handles gpos kerning. ![complex2.png](/attachments/cf3f66f8-bdc8-488b-8fa6-4b30db89627f) The impact on performance is minimal. Using this code for all text for default scene is 11.7 milliseconds, while it 9.6 using current code. It handles text caret insertion properly. This is complex because of bidirectionality in that you are selecting something in visual order but needing an answer in logical order (and these might be differing lengths). Made more complex because the length of a subset of a complex string is not related to the full string as glyphs change as the content changes. I handles text selection somewhat adequately. Selection with mixed directions gets complicated, but current code does so almost correctly and allows editing and deletion properly. The following might look odd, but is _exactly_ how it should work for mixed-direction selection. ![complex4.gif](/attachments/63500864-a228-4116-a246-f3ea709fec87) **Implementation Details** Our regular text functions handle each character one-by-one as the string is traversed. With complex shaping however we need to process chunks of text at a time since this requires greater context. For RTL languages, for example, we need at least entire words at a time in order to select the correct glyphs for each position. Ligatures need to know of multiple characters in order to do substitutions. Even kerning using the gpos table uses rules that require more context. In a nutshell this digests an entire string at a time, processing separate segments that have the same language and direction. Strings can contain an unlimited number of segments of differing language or direction. So you can combine English, Arabic, Hebrew, and English again. In that example the string will processed using specific fonts for Arabic and Hebrew from the font stack. This code selects the best font based on the font's language coverage. A string is consumed at once in the constructor of a `ShapingData` object: ``` ShapingData text(str, str_len); ``` At this point we have versions of the string in u32string in the order we received it (logical order), a copy transformed by FriBiDi into visual order, along with arrays that index between the two. We need to process each segment separately, so this is done like this: ``` while (text.process(font, gc)) { // do stuff with text.segment } ``` The above will process each segment by Harfbuzz to transform it into its final form with appropriate glyphs, spacing, and positioning. The `segment` member has a `glyph_count` (which may vary from the character count), and other information about this portion of the string including the language. Most importantly it has a Vector of glyphs, a Vector of bounds, and other position and size information. Note that in the above, the font and gc arguments are just what is used as defaults and will change as needed. Therefore outputting the text is very simple with just a simple loop to place each glyph. Measuring a string is just summing the bounds extents. "For each" processing works as it does now, but keep in mind that the callbacks are called in _visual order_, so don't assume that the offset will always be ascending. One big change outside of shaping has to do with glyph caching. We currently locate cached glyphs by codepoint, and then we find the glyph id stored there. But with complex shaping we are given arrays of glyph ids. Therefore the hash index is changed glyph id. Searches for glyphs by codepoint (for simple path) is still fast because the FreeType caching system caches these lookups.

complex2.png

33 KiB

complex4.gif

62 KiB

MixedSelection.gif

99 KiB

🎉 2 🚀 1

Harley Acheson force-pushed ComplexShaping from 3f9984cfa1 to 25826f4ee1

2023-02-14 22:08:04 +01:00

Compare

Harley Acheson force-pushed ComplexShaping from 25826f4ee1 to 8c469259bb

2023-02-15 21:39:48 +01:00