BLF: Subpixel Positioning, Anti-aliasing, Hinting #105441

Harley Acheson · 2023-03-04T18:57:58+01:00

Harley Acheson commented

2023-03-04 18:57:58 +01:00

Text output using subpixel positioning, subpixel anti-aliasing, and
typographically-correct hinting.

This patch enables three new text features to give more typographically-correct output.

"Subpixel positioning" means keeping the advancing pen position on 64th of pixels, rather than truncating at every glyph placement, for more accurate positioning. "Subpixel anti-aliasing" means using multiple renders of glyphs at different horizontal subpixel positions. And this PR also corrects our "Hinting", which is the altering of the glyphs to better align with the pixel grid. This then also makes "Slight Hinting" the default, rather than "None", which makes only vertical alignment changes.

Because this (necessarily) changes many things at once it is a bit difficult to compare before and after because of how strings can change length. So the following uses "Inter Regular" at various sizes, magnified 300% for clarity. The one in red uses no hinting and is without subpixel anti-aliasing or positioning, while the green one uses "Slight" hinting and subpixel anti-aliasing and positioning. You should notice a reduction in vertical blurriness (the hinting) and also a better placement of letters within words (because they are rendered for their ideal placement).

Note that this patch has had a round of review when it was here: https://archive.blender.org/developer/D16193

Text output using subpixel positioning, subpixel anti-aliasing, and typographically-correct hinting. --- This patch enables three new text features to give more typographically-correct output. "Subpixel positioning" means keeping the advancing pen position on 64th of pixels, rather than truncating at every glyph placement, for more accurate positioning. "Subpixel anti-aliasing" means using multiple renders of glyphs at different horizontal subpixel positions. And this PR also corrects our "Hinting", which is the altering of the glyphs to better align with the pixel grid. This then also makes "Slight Hinting" the default, rather than "None", which makes only vertical alignment changes. Because this (necessarily) changes many things at once it is a bit difficult to compare before and after because of how strings can change length. So the following uses "Inter Regular" at various sizes, magnified 300% for clarity. The one in red uses no hinting and is without subpixel anti-aliasing or positioning, while the green one uses "Slight" hinting and subpixel anti-aliasing and positioning. You should notice a reduction in vertical blurriness (the hinting) and also a better placement of letters within words (because they are rendered for their ideal placement). ![Hinting.gif](/attachments/44594b12-1d5f-40e9-a374-8829544f540d) Note that this patch has had a round of review when it was here: https://archive.blender.org/developer/D16193

Hinting.gif

138 KiB

InterComparison.gif

240 KiB

Inter.gif

69 KiB

👍 3 ❤️ 3 🎉 3

Harley Acheson force-pushed Subpixel from 4386dbfe35 to 38b20e7f9b

2023-03-05 21:41:13 +01:00

Compare

Harley Acheson commented

2023-03-05 21:43:33 +01:00

Fixed a dumb mistake. Now it has the same performance as current code.

Harley Acheson commented

2023-03-05 22:15:36 +01:00

@blender-bot build

Leon Schittek commented

2023-03-12 11:59:10 +01:00

I've been testing this in the node editor. Overall looks great!

Here are some observations (using the font Helvetica Neue) to maybe help get an idea of what this is useful for:

For normal nodes this patch reduces the "jiggliness" of text during zoom very noticably.
Video: main | patch
Frame nodes in particular benefit from this since it makes the line breaks of the rendered text more consistent when zooming - though the results are still a bit mixed.
Video: main | patch
It works "best", when hinting is set to None but even then it is still very sensitive to the width of the frame node. You can get it pretty stable as in this example. (Not the fault of this patch, but the line spacing for multi line text being in integer increments certainly doesn't help in this case.)

I've been testing this in the node editor. Overall looks great! Here are some observations (using the font *Helvetica Neue*) to maybe help get an idea of what this is useful for: * For normal nodes this patch reduces the "jiggliness" of text during zoom very noticably. *Video: [main](/attachments/788b493c-6b9a-4b65-8303-081031e06d3f) | [patch]([/attachments/e8ecfc5e-cb1e-42ad-915c-17908538cdd9)* * Frame nodes in particular benefit from this since it makes the line breaks of the rendered text more consistent when zooming - though the results are still a bit mixed. *Video: [main](/attachments/268999b5-bdf1-49e3-b3ec-12b30f63772e) | [patch](/attachments/3994e870-a773-4e6c-bb0d-591dba648ee3)* It works "best", when hinting is set to `None` but even then it is still very sensitive to the width of the frame node. You can get it pretty stable as in [this example](/attachments/5f3f6d09-ea48-4cc1-8755-51e14d188009). (Not the fault of this patch, but the line spacing for multi line text being in integer increments certainly doesn't help in this case.)

node_main_slight.mp4

530 KiB

frame_subpixel_slight.mp4

1.1 MiB

frame_main_slight.mp4

1.0 MiB

frame_subpixel_none_best-case.mp4

1.1 MiB

node_subpixel_slight.mp4

586 KiB

Harley Acheson commented

2023-03-12 20:11:45 +01:00

I've been testing this in the node editor. Overall looks great!

Thanks for checking it out!

I think some line-breaking issues are solvable after this patch but not solvable without. The "subpixel positioning" gets us close enough that we could introduce a 1 pixel wriggle room in some places that would solve some issues where current code has none because it would require one that is too large to be useful.

the line spacing for multi line text being in integer increments certainly doesn't help in this case

Yes, that looks worse (more than a pixel) than it probably needs to be. Haven't looked at that code but I'd guess that is calculating and then using an integer line-space. Instead it could calculate an ideal float one, accumulate that as you go, truncating to integer at every line. Right now the ideal space between lines might be 10.5 pixels so we use 11 for each and end up being out 5 pixels over 10 lines. But if we accumulate with floats you end up with 11, 10, 11, 10, etc. That result lines up better as you increase and decrease font size.

> I've been testing this in the node editor. Overall looks great! Thanks for checking it out! I think some line-breaking issues are solvable after this patch but not solvable without. The "subpixel positioning" gets us close enough that we could introduce a 1 pixel wriggle room in some places that would solve some issues where current code has none because it would require one that is too large to be useful. > the line spacing for multi line text being in integer increments certainly doesn't help in this case Yes, that looks worse (more than a pixel) than it probably needs to be. Haven't looked at that code but I'd guess that is calculating and then using an integer line-space. Instead it could calculate an ideal float one, accumulate that as you go, truncating to integer at every line. Right now the ideal space between lines might be 10.5 pixels so we use 11 for each and end up being out 5 pixels over 10 lines. But if we accumulate with floats you end up with 11, 10, 11, 10, etc. That result lines up better as you increase and decrease font size.

Harley Acheson force-pushed Subpixel from a71e90c34b to 6d41b4d2b1

2023-06-28 23:38:52 +02:00

Compare

Harley Acheson changed title from ~~WIP: BLF: Subpixel Positioning, Anti-aliasing, Hinting~~ to BLF: Subpixel Positioning, Anti-aliasing, Hinting

2023-06-28 23:39:21 +02:00

Harley Acheson added this to the Core project 2023-06-28 23:41:35 +02:00

Harley Acheson force-pushed Subpixel from e27bbb39c7 to 5aa78be7c9

2023-09-05 21:00:12 +02:00

Compare

Harley Acheson added 1 commit 2023-09-10 17:38:56 +02:00

Merge branch 'main' of projects.blender.org:blender/blender into Subpixel 5c3b34a9d0

Harley Acheson referenced this pull request

2023-09-10 23:47:51 +02:00

UI: Possible Change of Interface Font #112209

Julian Eisel referenced this pull request

2023-09-13 13:15:45 +02:00

UI: Possible Change of Interface Font #112209

Harley Acheson commented

2023-09-13 21:32:03 +02:00

The plan is looking like it will be to do this along with a font change. Therefore the code for "legacy" spacing can be removed - it was only here in case this went in before the font change.

🎉 1

Harley Acheson added 2 commits 2023-09-13 23:59:52 +02:00

Merge branch 'main' of projects.blender.org:blender/blender into Subpixel 2d319942f7

Removing Legacy spacing. Fixing blf_glyph_calc_rect_test for Full hinting. d704598d48

Harley Acheson added 1 commit 2023-09-14 00:56:37 +02:00

Adjusting string clip assert. And allowing out by one on the glyph rect test. 3272189a6c

Harley Acheson added 2 commits 2023-09-15 02:48:40 +02:00

Merge branch 'main' of projects.blender.org:blender/blender into Subpixel ba82a23680

Cleanup.

buildbot/vexp-code-patch-coordinator Build done.

Details

11340ec4eb

Harley Acheson commented

2023-09-15 03:56:10 +02:00

@blender-bot build

Harley Acheson added 2 commits 2023-09-18 21:15:13 +02:00

Merge branch 'main' of projects.blender.org:blender/blender into Subpixel 6906014b82

Merge branch 'main' of projects.blender.org:blender/blender into Subpixel 5ccc439bf7

Harley Acheson added 1 commit 2023-09-18 23:00:44 +02:00

Adding defines that enable subpixel positioning and subpixel antialiasing bd9e4ee23d

Harley Acheson referenced this pull request

2023-09-19 22:09:24 +02:00

UI: Replace UI Font with Inter Regular v3.19 #112603

Harley Acheson modified the project from Core to User Interface

2023-09-19 22:11:34 +02:00

Harley Acheson added 1 commit 2023-09-19 22:55:16 +02:00

Merge branch 'main' of projects.blender.org:blender/blender into Subpixel cfd32746da

Harley Acheson requested review from Brecht Van Lommel 2023-09-20 00:36:57 +02:00

Brecht Van Lommel commented

2023-09-20 12:59:42 +02:00

Have you measure the impact on (GPU) memory usage? Storing glyphs for different subpixel positions increases memory usage. I see the number of positions is limited which should help. But how many megabytes are we talking about with e.g. English and Chinese translation?

Harley Acheson commented

2023-09-21 02:36:32 +02:00

@brecht - Have you measure the impact on (GPU) memory usage?

I have not, but will try to get that information.

Note if that becomes a sticking point we could commit the font change and this, just with the Subpixel anti-aliasing turned off (enabled with a BLF_SUBPIXEL_AA define).

That would still get a lot of improvement, with the exception of something that bothers Campbell. With subpixel position, but without subpixel AA, entering a string like "iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii" shows a pattern in the spacing. This is because with subpixel positioning the letters go as close as possible to where they should go, without relation to the preceding characters. in a nutshell the spacing between the letters could go 4, 4, 5, 4, 4, 5. With the subpixel AA that pattern goes away since the glyphs are rendered for a subpixel placement.

> @brecht - Have you measure the impact on (GPU) memory usage? I have not, but will try to get that information. Note if that becomes a sticking point we could commit the font change and this, just with the Subpixel anti-aliasing turned off (enabled with a BLF_SUBPIXEL_AA define). That would still get a lot of improvement, with the exception of something that bothers Campbell. With subpixel position, but without subpixel AA, entering a string like "iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii" shows a pattern in the spacing. This is because with subpixel positioning the letters go as close as possible to where they should go, without relation to the preceding characters. in a nutshell the spacing between the letters could go 4, 4, 5, 4, 4, 5. With the subpixel AA that pattern goes away since the glyphs are rendered for a subpixel placement.

Brecht Van Lommel referenced this pull request

2023-09-21 16:28:17 +02:00

UI: Replace UI Font with Inter Regular v3.19 #112603

Harley Acheson referenced this pull request

2023-09-21 17:03:05 +02:00

UI: Replace UI Font with Inter Regular v3.19 #112603

Harley Acheson commented

2023-09-21 20:03:48 +02:00

Have you measure the impact on (GPU) memory usage? Storing glyphs for different subpixel positions increases memory usage. I see the number of positions is limited which should help. But how many megabytes are we talking about with e.g. English and Chinese translation?

I'm not seeing any measurable change.

I changed my VRAM usage stats to show 6 decimals and I am not seeing any measurable change between subpixel AA and not. As in I can start multiple times with the default scene, set to 1.0 scale, and the difference between hinting "None" (which does not use subpixel AA) and "Slight" (which does) and I see my VRAM usage being on average about 1.044 GB for the former and 1.038 GB for the latter.

Yes, it goes down for "Slight" hinting because it moves features vertically (only) which gets rid of the fuzziness at the top and bottom which results in a glyph bitmap being usually a pixel shorter.

Note that for Chinese, Japanese, and Korean there is almost always no change to strings with hinting changes or with subpixel AA. Because they have consistent width and advance they don't get subpixel placement changes unless there is also a Latin character earlier in the same string. And FreeType does not do any hinting on these characters. So in almost all cases you get just a single version of a glyph.

> Have you measure the impact on (GPU) memory usage? Storing glyphs for different subpixel positions increases memory usage. I see the number of positions is limited which should help. But how many megabytes are we talking about with e.g. English and Chinese translation? I'm not seeing any measurable change. I changed my VRAM usage stats to show 6 decimals and I am not seeing any measurable change between subpixel AA and not. As in I can start multiple times with the default scene, set to 1.0 scale, and the difference between hinting "None" (which does not use subpixel AA) and "Slight" (which does) and I see my VRAM usage being on average about 1.044 GB for the former and 1.038 GB for the latter. Yes, it goes down for "Slight" hinting because it moves features vertically (only) which gets rid of the fuzziness at the top and bottom which results in a glyph bitmap being usually a pixel shorter. Note that for Chinese, Japanese, and Korean there is almost always no change to strings with hinting changes or with subpixel AA. Because they have consistent width and advance they don't get subpixel placement changes unless there is also a Latin character earlier in the same string. And FreeType does not do any hinting on these characters. So in almost all cases you get just a single version of a glyph.

Brecht Van Lommel commented

2023-09-21 20:43:40 +02:00

VRAM stats like that are not really useful for this unless there was a really big problem. Other software, memory allocation strategies and fragmentation makes it difficult to see exactly what the impact is.

This is more precise:

--- a/source/blender/blenfont/intern/blf_glyph.cc
+++ b/source/blender/blenfont/intern/blf_glyph.cc
@@ -137,6 +137,8 @@ void blf_glyph_cache_release(FontBLF *font)
   BLI_mutex_unlock(&font->glyph_cache_mutex);
 }
 
+static size_t MEMORY_USAGE = 0;
+
 static void blf_glyph_cache_free(GlyphCacheBLF *gc)
 {
   for (uint i = 0; i < ARRAY_SIZE(gc->bucket); i++) {
@@ -145,6 +147,7 @@ static void blf_glyph_cache_free(GlyphCacheBLF *gc)
     }
   }
   if (gc->texture) {
+    MEMORY_USAGE -= GPU_texture_width(gc->texture) * GPU_texture_height(gc->texture);
     GPU_texture_free(gc->texture);
   }
   if (gc->bitmap_result) {
@@ -1239,11 +1242,14 @@ void blf_glyph_draw(FontBLF *font, GlyphCacheBLF *gc, GlyphBLF *g, const int x,
 
       /* Keep in sync with the texture. */
       if (gc->texture) {
+        MEMORY_USAGE -= GPU_texture_width(gc->texture) * GPU_texture_height(gc->texture);
         GPU_texture_free(gc->texture);
       }
       gc->texture = GPU_texture_create_2d(
           __func__, w, h, 1, GPU_R8, GPU_TEXTURE_USAGE_SHADER_READ, nullptr);
 
+      MEMORY_USAGE += w * h;
+      printf("memory usage %f MB\n", MEMORY_USAGE / (1024.0f * 1024.0f));
       gc->bitmap_len_landed = 0;
     }

Memory usage is maybe 20% higher in some cases. But we are talking about ~0.5MB and sometimes temporarily 2MB when zooming in and out a lot. So I don't think it's an issue.

VRAM stats like that are not really useful for this unless there was a really big problem. Other software, memory allocation strategies and fragmentation makes it difficult to see exactly what the impact is. This is more precise: ``` --- a/source/blender/blenfont/intern/blf_glyph.cc +++ b/source/blender/blenfont/intern/blf_glyph.cc @@ -137,6 +137,8 @@ void blf_glyph_cache_release(FontBLF *font) BLI_mutex_unlock(&font->glyph_cache_mutex); } +static size_t MEMORY_USAGE = 0; + static void blf_glyph_cache_free(GlyphCacheBLF *gc) { for (uint i = 0; i < ARRAY_SIZE(gc->bucket); i++) { @@ -145,6 +147,7 @@ static void blf_glyph_cache_free(GlyphCacheBLF *gc) } } if (gc->texture) { + MEMORY_USAGE -= GPU_texture_width(gc->texture) * GPU_texture_height(gc->texture); GPU_texture_free(gc->texture); } if (gc->bitmap_result) { @@ -1239,11 +1242,14 @@ void blf_glyph_draw(FontBLF *font, GlyphCacheBLF *gc, GlyphBLF *g, const int x, /* Keep in sync with the texture. */ if (gc->texture) { + MEMORY_USAGE -= GPU_texture_width(gc->texture) * GPU_texture_height(gc->texture); GPU_texture_free(gc->texture); } gc->texture = GPU_texture_create_2d( __func__, w, h, 1, GPU_R8, GPU_TEXTURE_USAGE_SHADER_READ, nullptr); + MEMORY_USAGE += w * h; + printf("memory usage %f MB\n", MEMORY_USAGE / (1024.0f * 1024.0f)); gc->bitmap_len_landed = 0; } ``` Memory usage is maybe 20% higher in some cases. But we are talking about ~0.5MB and sometimes temporarily 2MB when zooming in and out a lot. So I don't think it's an issue.

Brecht Van Lommel approved these changes 2023-09-21 20:44:00 +02:00

Harley Acheson commented

2023-09-21 21:38:30 +02:00

This is more precise:

Nice!

Memory usage is maybe 20% higher in some cases. But we are talking about ~0.5MB and sometimes temporarily 2MB when zooming in and out a lot. So I don't think it's an issue.

Yes we've gotten fairly efficient with this with the deletion of unused glyph caches and no longer pre-caching.

> This is more precise: Nice! > Memory usage is maybe 20% higher in some cases. But we are talking about ~0.5MB and sometimes temporarily 2MB when zooming in and out a lot. So I don't think it's an issue. Yes we've gotten fairly efficient with this with the deletion of unused glyph caches and no longer pre-caching.

Harley Acheson merged commit a0b4ead737 into main

2023-09-21 22:43:24 +02:00

Harley Acheson referenced this issue from a commit

2023-09-21 22:43:25 +02:00

BLF: Subpixel Positioning, Anti-aliasing, Hinting

Harley Acheson deleted branch Subpixel

2023-09-21 22:43:26 +02:00

Colin Basnett referenced this issue from a commit

2023-09-22 10:24:46 +02:00

BLF: Subpixel Positioning, Anti-aliasing, Hinting

Harley Acheson referenced this pull request

2023-09-28 18:38:07 +02:00