VSE: Text shadow blur / outline #121478

Merged
Aras Pranckevicius merged 10 commits from aras_p/blender:vse_text_effects into main 2024-05-08 11:13:33 +02:00

Implements small part of #112698 and possibly other existing requests (from right-click select: https://blender.community/c/rightclickselect/dJfbbc/ and https://blender.community/c/rightclickselect/xTdbbc/):

image

Text got configurable shadow angle and offset (previously shadow location relative to text was fixed). Currently the shadow position is rounded to integer pixel locations (sub-pixel positioning might come later).

Text got optional shadow blur amount:
Shadow blur 0.1: image
Shadow blur 0.4: image
Shadow blur 0.8: image

Text got optional outline option, with separate outline color and outline width. Outline is calculated using jump-flooding algorithm (similar to what compositor does for inpainting / double edge mask), which in my tests is faster than a blur based approach even at small outline widths. At some point it would allow even mapping a gradient or whatever across the outline.

Here's large font with 30px outline:
image
The same with 3px outline:
image
And a tiny (12pt) text with 2px outline:
image

TODO / Questions:

  • Test coverage.
  • Performance could be improved by doing all blurring / outline only in the area that the text actually occupies, not on the full image.
  • Found an issue in BLF where it accepts "how many color channels an image has" argument but proceeds to the the wrong thing afterwards. Not sure if should be fixed separately (or that functionality removed?)
Implements small part of #112698 and possibly other existing requests (from right-click select: https://blender.community/c/rightclickselect/dJfbbc/ and https://blender.community/c/rightclickselect/xTdbbc/): ![image](/attachments/32fcacd7-a299-433c-a84f-9a6a596de08a) Text got configurable **shadow angle and offset** (previously shadow location relative to text was fixed). Currently the shadow position is rounded to integer pixel locations (sub-pixel positioning might come later). Text got optional **shadow blur amount**: Shadow blur 0.1: ![image](/attachments/5ca3509a-97eb-424e-8969-231316278cd0) Shadow blur 0.4: ![image](/attachments/dcf6634f-710c-4730-8aad-867bf073cae2) Shadow blur 0.8: ![image](/attachments/1b371b09-fb78-4597-95aa-06d91ac4e450) Text got optional **outline option**, with separate outline color and outline width. Outline is calculated using jump-flooding algorithm (similar to what compositor does for inpainting / double edge mask), which in my tests is faster than a blur based approach even at small outline widths. At some point it would allow even mapping a gradient or whatever across the outline. Here's large font with 30px outline: ![image](/attachments/31e2905e-3c71-4055-9cfe-9a6d810252dd) The same with 3px outline: ![image](/attachments/c7dda39a-390e-45b7-b220-b9e98fc34c1b) And a tiny (12pt) text with 2px outline: ![image](/attachments/eb63aeb1-fae5-4cef-96a1-f56271a4b224) **TODO / Questions**: - Test coverage. - Performance could be improved by doing all blurring / outline only in the area that the text actually occupies, not on the full image. - Found an issue in BLF where it accepts "how many color channels an image has" argument but proceeds to the the wrong thing afterwards. Not sure if should be fixed separately (or that functionality removed?)
Aras Pranckevicius added 3 commits 2024-05-06 14:35:27 +02:00
First-time contributor

maybe or really i'm pushing too far , i'm dreaming having Shadows of the shadow meaning double outline ..(or more with custom color for each (color ramp ?))

maybe or really i'm pushing too far , i'm dreaming having Shadows of the shadow meaning double outline ..(or more with custom color for each (color ramp ?))
Aras Pranckevicius added 1 commit 2024-05-07 10:23:34 +02:00
Advantages:
- "More round" outlines especially at large widths,
- Actually faster, e.g. 4K VSE resolution, outline width 30: outline
  calculation 110ms -> 40ms. At small outline widths, about the same
  performance as blur based approach.
- If someone wants to make outlines "fancy" someday (e.g. mapping a
  gradient across outline radius), this is trivially doable with JFA.
Aras Pranckevicius added 1 commit 2024-05-07 11:53:23 +02:00
Gaussian blur code is not fully premultiplication-correct. So render
text as white, blur it, and then tint with shadow color when compositing.
Makes large blurs with small shadow opacity work better too.
Aras Pranckevicius added 2 commits 2024-05-07 13:19:16 +02:00
VSE: initialize new variables of text strips when loading older files
All checks were successful
buildbot/vexp-code-patch-darwin-x86_64 Build done.
buildbot/vexp-code-patch-darwin-arm64 Build done.
buildbot/vexp-code-patch-windows-amd64 Build done.
buildbot/vexp-code-patch-lint Build done.
buildbot/vexp-code-patch-linux-x86_64 Build done.
buildbot/vexp-code-patch-coordinator Build done.
d199cfe108
And shadow offset/angle defaults that better match the previous hardcoded values
Author
Member

@blender-bot build

@blender-bot build
First-time contributor

Great feature! Just a remark: is it possible to smooth these jagged edges/artefacts and put some antialiasing?
изображение

Great feature! Just a remark: is it possible to smooth these jagged edges/artefacts and put some antialiasing? ![изображение](/attachments/56244604-69ef-4bfa-851a-babfb6443d6d)
Author
Member

Just a remark: is it possible to smooth these jagged edges/artefacts and put some antialiasing?

@Mikamiel that "weird line" in the middle has nothing to do with shadow/outline (or even text), it's just that there was fractional scaling in the VSE preview window when I took the screenshots.

> Just a remark: is it possible to smooth these jagged edges/artefacts and put some antialiasing? @Mikamiel that "weird line" in the middle has nothing to do with shadow/outline (or even text), it's just that there was fractional scaling in the VSE preview window when I took the screenshots.
Aras Pranckevicius added 1 commit 2024-05-07 19:20:40 +02:00
Aras Pranckevicius changed title from WIP: VSE: Text shadow blur / outline to VSE: Text shadow blur / outline 2024-05-07 19:28:38 +02:00
Aras Pranckevicius added this to the Video Sequencer project 2024-05-07 19:28:50 +02:00
Aras Pranckevicius requested review from Sergey Sharybin 2024-05-07 19:28:56 +02:00
Aras Pranckevicius requested review from Richard Antalik 2024-05-07 19:29:06 +02:00
First-time contributor

That's what it was!)

That's what it was!)
Richard Antalik approved these changes 2024-05-07 21:20:32 +02:00
Richard Antalik left a comment
Member

I was following the JFA, untill it got to AA part, which is bit blurry (HA!) to me. but I got the gist.
Technically both JFA and shadow can be optimized by only working on part of buffer, because you know the position and size of these elements.

Quite some time ago I wanted to add support for blending buffers of non-uniform size, as this would make using text strip much faster overall, but that would be bit more complicated.

I was following the JFA, untill it got to AA part, which is bit blurry (HA!) to me. but I got the gist. Technically both JFA and shadow can be optimized by only working on part of buffer, because you know the position and size of these elements. Quite some time ago I wanted to add support for blending buffers of non-uniform size, as this would make using text strip much faster overall, but that would be bit more complicated.
Sergey Sharybin reviewed 2024-05-08 10:07:17 +02:00
Sergey Sharybin left a comment
Owner

While testing I've noticed that the shadow is only moved by integer pixels, so if you animate shadow rotation it kind of jumps rather than having smooth anti-aliased transition. It is not something that necessarily needs to be supported by this initial PR, but it worth stating it as a known limitation in the PR description.

Found an issue in BLF where it accepts "how many color channels an image has" argument but proceeds to the the wrong thing afterwards. Not sure if should be fixed separately (or that functionality removed?)

I didn't look into details of what exactly is going wrong there, but it would be good if we the broken functionality got addressed separately. it'll make it easier to review.

While testing I've noticed that the shadow is only moved by integer pixels, so if you animate shadow rotation it kind of jumps rather than having smooth anti-aliased transition. It is not something that necessarily needs to be supported by this initial PR, but it worth stating it as a known limitation in the PR description. > Found an issue in BLF where it accepts "how many color channels an image has" argument but proceeds to the the wrong thing afterwards. Not sure if should be fixed separately (or that functionality removed?) I didn't look into details of what exactly is going wrong there, but it would be good if we the broken functionality got addressed separately. it'll make it easier to review.
@ -2706,0 +2801,4 @@
IMB_freeImBuf(tmp_out1);
IMB_freeImBuf(tmp_out2);
BLF_buffer(font,
out->float_buffer.data,

Can we pass nullptr instead of out->float_buffer.data?
To me it will be much more clear indication of what we expect to be happening here: we only expect the text to operate o na byte buffers, and we only write to byte buffer. So when we pass float buffer somewhere it reads as if it is expected that it might be a valid buffer.

Can we pass `nullptr` instead of `out->float_buffer.data`? To me it will be much more clear indication of what we expect to be happening here: we only expect the text to operate o na byte buffers, and we only write to byte buffer. So when we pass float buffer somewhere it reads as if it is expected that it might be a valid buffer.
aras_p marked this conversation as resolved
Aras Pranckevicius added 2 commits 2024-05-08 10:22:54 +02:00
Author
Member

While testing I've noticed that the shadow is only moved by integer pixels, so if you animate shadow rotation it kind of jumps rather than having smooth anti-aliased transition. It is not something that necessarily needs to be supported by this initial PR, but it worth stating it as a known limitation in the PR description.

This seems to be general limitation of whole BLF system: the text effect code does pass floats to BLF_position, but then that one internally rounds them to nearest integer coordinate, and stores as integers for later rasterization. That is: all texts across whole blender are positioned at integer pixel coordinates, it seems.

I guess with some additional work we could rasterize the text for shadow at non-offset-at-all position, then blur it, and then manually place it where needed, with bilinear interpolation. But perhaps better done separately indeed (including perf optimizations that would do outline/blurring only on text area, and not the whole image).

> While testing I've noticed that the shadow is only moved by integer pixels, so if you animate shadow rotation it kind of jumps rather than having smooth anti-aliased transition. It is not something that necessarily needs to be supported by this initial PR, but it worth stating it as a known limitation in the PR description. This seems to be general limitation of whole BLF system: the text effect code does pass floats to `BLF_position`, but then that one internally rounds them to nearest integer coordinate, and stores as integers for later rasterization. That is: all texts across whole blender are positioned at integer pixel coordinates, it seems. I guess with some additional work we could rasterize the text for shadow at non-offset-at-all position, then blur it, and then manually place it where needed, with bilinear interpolation. But perhaps better done separately indeed (including perf optimizations that would do outline/blurring only on text area, and not the whole image).

That is: all texts across whole blender are positioned at integer pixel coordinates, it seems.

The BLF historically was done to draw UI texts indeed, where you do want to place it as pixel-perfect as possible. Is only much later when it started to be used for an effect type of a thing.

> That is: all texts across whole blender are positioned at integer pixel coordinates, it seems. The BLF historically was done to draw UI texts indeed, where you do want to place it as pixel-perfect as possible. Is only much later when it started to be used for an effect type of a thing.
Sergey Sharybin approved these changes 2024-05-08 10:54:02 +02:00
Sergey Sharybin left a comment
Owner

For the description I'd still suggest doing something like:

Text got configurable shadow angle and offset (previously shadow location relative to text was fixed). Currently it is rounded to integer pixels. It is possible to allow sub-pixel positioning for animation, but it is not planned for this first step.

For the description I'd still suggest doing something like: > Text got configurable shadow angle and offset (previously shadow location relative to text was fixed). Currently it is rounded to integer pixels. It is possible to allow sub-pixel positioning for animation, but it is not planned for this first step.
Aras Pranckevicius merged commit d66598c16f into main 2024-05-08 11:13:33 +02:00
Aras Pranckevicius deleted branch vse_text_effects 2024-05-08 11:13:35 +02:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#121478
No description provided.