LineArt: New module to generate lines over meshes using grease pencil as target #80194

Open
opened 2020-08-28 16:37:20 +02:00 by Antonio Vazquez · 45 comments

This is the main Line Art task, includes the implementation of following functions:

  • Line Art cpu calculation code and UI.
  • Line Art modifier for Grease Pencil.
  • Other additional data structures in Object/Collection/Materials.
This is the main Line Art task, includes the implementation of following functions: - Line Art cpu calculation code and UI. - Line Art modifier for Grease Pencil. - Other additional data structures in Object/Collection/Materials.
Author
Member

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'
Author
Member

Added subscriber: @antoniov

Added subscriber: @antoniov
YimingWu self-assigned this 2020-08-29 05:40:47 +02:00

Added subscriber: @mendio

Added subscriber: @mendio
Contributor

Added subscriber: @Eary

Added subscriber: @Eary

Added subscriber: @pafurijaz-4

Added subscriber: @pafurijaz-4

Added subscriber: @T.R.O.Nunes

Added subscriber: @T.R.O.Nunes
Member

Updated some new code, I'll be looking to clean up the style and comments as well.

Updated some new code, I'll be looking to clean up the style and comments as well.

Added subscriber: @slumber

Added subscriber: @slumber
Member

Added subscriber: @HooglyBoogly

Added subscriber: @HooglyBoogly
Member

I'm continuing the discussion from D8758 here, as it's really more of a design discussion and less code review.


It looks like I didn't properly understand the goal of the modifier, it's becoming clearer now. It seems like the goal of the modifier is:

  1. To connect line art to grease pencil to allow using all of grease pencil's functionality on top of it.
  2. As a container for the UI for per object / per collection strokes.

I think this is confusing because it fundamentally breaks the concept of evaluating the modifier stack. Here is a solution that makes more sense in my opinion:

  1. Move the settings from the modifier to collections and objects directly. Each object (edit: and collection) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there.
  2. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections.

I think the important part is not using the modifier stack as a UI container unless the line art is actually evaluated as a modifier.
Especially since we already have the collection for grouping settings for multiple objects, these settings currently in the modifier would fit much better there in my opinion.

I'm continuing the discussion from [D8758](https://archive.blender.org/developer/D8758) here, as it's really more of a design discussion and less code review. --- It looks like I didn't properly understand the goal of the modifier, it's becoming clearer now. It seems like the goal of the modifier is: 1. To connect line art to grease pencil to allow using all of grease pencil's functionality on top of it. 2. As a container for the UI for per object / per collection strokes. I think this is confusing because it fundamentally breaks the concept of evaluating the modifier stack. Here is a solution that makes more sense in my opinion: 1. Move the settings from the modifier to collections and objects directly. Each object (*edit: and collection*) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there. 2. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections. I think the important part is not using the modifier stack as a UI container unless the line art is actually evaluated as a modifier. Especially since we already have the collection for grouping settings for multiple objects, these settings currently in the modifier would fit much better there in my opinion.
Author
Member

Add a new object type for Line Art, which would basically be grease pencil without edit mode This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected.

Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers, using different values for each object. Also, you can "bake" the result just pressing apply to the modifier.

`Add a new object type for Line Art, which would basically be grease pencil without edit mode` This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected. Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers, using different values for each object. Also, you can "bake" the result just pressing apply to the modifier.
Member

This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected.

Yes, definitely not trivial. So maybe an easier solution with a similar result would be to just use a regular Gpencil empty.

Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers

This can still be possible if the UI for Line Art isn't in the modifier stack, Line Art could just be evaluated before grease pencil modifiers.

> This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected. Yes, definitely not trivial. So maybe an easier solution with a similar result would be to just use a regular Gpencil empty. > Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers This can still be possible if the UI for Line Art isn't in the modifier stack, Line Art could just be evaluated before grease pencil modifiers.

Added subscriber: @brecht

Added subscriber: @brecht

From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it.

But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output.

I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe.

From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it. But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output. I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe.

Added subscriber: @ZedDB

Added subscriber: @ZedDB

In #80194#1048778, @HooglyBoogly wrote:

It looks like I didn't properly understand the goal of the modifier, it's becoming clearer now. It seems like the goal of the modifier is:

  1. To connect line art to grease pencil to allow using all of grease pencil's functionality on top of it.
  2. As a container for the UI for per object / per collection strokes.

You missed one thing:

  1. Be able to bake and edit the strokes afterwards. (This was one of the major pitfalls with freestyle, you couldn't edit anything)

I think this is confusing because it fundamentally breaks the concept of evaluating the modifier stack. Here is a solution that makes more sense in my opinion:

  1. Move the settings from the modifier to collections and objects directly. Each object (edit: and collection) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there.

Moving the settings around to individual objects and collections would not really solve anything I think.
We still need to have a way to specify which GP object we want to save specific strokes to and live preview the strokes before baking.

The easiest way to archive this currently is by using modifiers.

If we go this route then the settings will be scattered all over the UI inside different objects and collections.
So if you want to change anything, it will be a PITA. Now you can just do it in the modifier or scene settings and everything updates.

However after talking to Yiming, this could be a way that we get rid of the scene settings.
So now the line crease and chaining settings for example would be per object or collection.

But I think we should keep the current modifier settings in the modifier.

  1. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections.

This would remove the major feature this has over freestyle, the ability to bake and edit the strokes in post.
Having different object type would only make things more cumbersome and tedious.

Note that the modifier does not replace any strokes, so you can draw things by hand and then use the modifier to get strokes from geometry.
So this does not prevent the artist from drawing things by hand in the same GP object.
You can even bake frame 1-5, edit them, then bake 6-10, and continue from there.

Having a new object type for this would destroy any fast and easy workflow imo.

> In #80194#1048778, @HooglyBoogly wrote: > It looks like I didn't properly understand the goal of the modifier, it's becoming clearer now. It seems like the goal of the modifier is: > 1. To connect line art to grease pencil to allow using all of grease pencil's functionality on top of it. > 2. As a container for the UI for per object / per collection strokes. You missed one thing: 3. Be able to bake and edit the strokes afterwards. (This was one of the major pitfalls with freestyle, you couldn't edit anything) > I think this is confusing because it fundamentally breaks the concept of evaluating the modifier stack. Here is a solution that makes more sense in my opinion: > 1. Move the settings from the modifier to collections and objects directly. Each object (*edit: and collection*) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there. Moving the settings around to individual objects and collections would not really solve anything I think. We still need to have a way to specify which GP object we want to save specific strokes to and live preview the strokes before baking. The easiest way to archive this currently is by using modifiers. If we go this route then the settings will be scattered all over the UI inside different objects and collections. So if you want to change anything, it will be a PITA. Now you can just do it in the modifier or scene settings and everything updates. However after talking to Yiming, this could be a way that we get rid of the scene settings. So now the line crease and chaining settings for example would be per object or collection. But I think we should keep the current modifier settings in the modifier. > 2. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections. > This would remove the major feature this has over freestyle, the ability to bake and edit the strokes in post. Having different object type would only make things more cumbersome and tedious. Note that the modifier does not replace any strokes, so you can draw things by hand and then use the modifier to get strokes from geometry. So this does not prevent the artist from drawing things by hand in the same GP object. You can even bake frame 1-5, edit them, then bake 6-10, and continue from there. Having a new object type for this would destroy any fast and easy workflow imo.

In #80194#1048812, @brecht wrote:
From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it.

Correct.

But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output.

Yes, it is done this way to not lock up the UI completely. This way we can jump around and scrub the timeline without having to wait for it to finish.
(Or just moving around objects in the scene will not lock up blender)

I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe.

As you noticed it is not perfect by any means. Perhaps we should draft something like this in general for heavy modifiers?

So for modifiers that is not "real time" we can have a job system of sorts.
Take mesh modifiers for example:

  1. I have the default cube with two modifiers (array and subdiv)
  2. The array modifier duplicates the cube 1000 times (nearly instant, no job required)
  3. The subdiv will probably be slow in this case so a job is created and the geometry from the array modifier is displayed, but doesn't lock up the UI while it waits.
  4. When the subdiv modifier job finishes, the result is displayed.

When the job is active you can pan around and scrub the time line, but I guess we would have to lock up the UI if the user does something the requires the full modifier stack to be evaluated.

> In #80194#1048812, @brecht wrote: > From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it. > Correct. > But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output. > Yes, it is done this way to not lock up the UI completely. This way we can jump around and scrub the timeline without having to wait for it to finish. (Or just moving around objects in the scene will not lock up blender) > I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe. As you noticed it is not perfect by any means. Perhaps we should draft something like this in general for heavy modifiers? So for modifiers that is not "real time" we can have a job system of sorts. Take mesh modifiers for example: 1. I have the default cube with two modifiers (array and subdiv) 2. The array modifier duplicates the cube 1000 times (nearly instant, no job required) 3. The subdiv will probably be slow in this case so a job is created and the geometry from the array modifier is displayed, but doesn't lock up the UI while it waits. 4. When the subdiv modifier job finishes, the result is displayed. When the job is active you can pan around and scrub the time line, but I guess we would have to lock up the UI if the user does something the requires the full modifier stack to be evaluated.
Member

In #80194#1048853, @ZedDB wrote:
3. Be able to bake and edit the strokes afterwards. (This was one of the major pitfalls with freestyle, you couldn't edit anything)

Fair enough. I imagined being able to "bake" or "apply" the freestyle, but that is indeed starting to look a lot like a modifier again.

Moving the settings around to individual objects and collections would not really solve anything I think.
We still need to have a way to specify which GP object we want to save specific strokes to and live preview the strokes before baking.
The easiest way to archive this currently is by using modifiers.

I suppose that the collections / objects could get a gpencil object target property. That sort of inverts the relationship though, and it might not be better that way.

If we go this route then the settings will be scattered all over the UI inside different objects and collections.
So if you want to change anything, it will be a PITA. Now you can just do it in the modifier or scene settings and everything updates.

I'm not sure about this one. There are already settings in every object and collection, so it would also be condensing them.

  1. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections.

Note that the modifier does not replace any strokes, so you can draw things by hand and then use the modifier to get strokes from geometry.
So this does not prevent the artist from drawing things by hand in the same GP object.

Okay, the way you explain this makes it clear that was not a good idea. Thanks.

You can even bake frame 1-5, edit them, then bake 6-10, and continue from there.

Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now.

> In #80194#1048853, @ZedDB wrote: > 3. Be able to bake and edit the strokes afterwards. (This was one of the major pitfalls with freestyle, you couldn't edit anything) Fair enough. I imagined being able to "bake" or "apply" the freestyle, but that is indeed starting to look a lot like a modifier again. > Moving the settings around to individual objects and collections would not really solve anything I think. > We still need to have a way to specify which GP object we want to save specific strokes to and live preview the strokes before baking. > The easiest way to archive this currently is by using modifiers. I suppose that the collections / objects could get a gpencil object target property. That sort of inverts the relationship though, and it might not be better that way. > If we go this route then the settings will be scattered all over the UI inside different objects and collections. > So if you want to change anything, it will be a PITA. Now you can just do it in the modifier or scene settings and everything updates. I'm not sure about this one. There are already settings in every object and collection, so it would also be condensing them. >> 2. Add a new object type for Line Art, which would basically be grease pencil without edit mode, where the strokes are dependent on the settings in the scene, objects, and collections. > Note that the modifier does not replace any strokes, so you can draw things by hand and then use the modifier to get strokes from geometry. > So this does not prevent the artist from drawing things by hand in the same GP object. Okay, the way you explain this makes it clear that was not a good idea. Thanks. > You can even bake frame 1-5, edit them, then bake 6-10, and continue from there. Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now.

In #80194#1048915, @HooglyBoogly wrote:
Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now.

I think this should be case as well. :)
Perhaps we could keep the "bake all" button though?
So it's kinda like the physics modifiers, they have a "bake all dynamics" button or so IIRC.

> In #80194#1048915, @HooglyBoogly wrote: > Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now. I think this should be case as well. :) Perhaps we could keep the "bake all" button though? So it's kinda like the physics modifiers, they have a "bake all dynamics" button or so IIRC.
Member

In #80194#1048787, @antoniov wrote:
Add a new object type for Line Art, which would basically be grease pencil without edit mode This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected.

Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers, using different values for each object. Also, you can "bake" the result just pressing apply to the modifier.

I think this idea is worth exploring, Not necessarily a new object type, but a sub category of Grease Pencil, make it a special flag that differentiate with normal ones, this way we have all the Grease Pencil functions available, but we can then approach the UI representation differently. Internally Line Art can still be a modifier, but for this special kind of object we can show the modifier differently, e.g. in a specific tab, and they can then be hidden in the regular modifier stack.

> In #80194#1048787, @antoniov wrote: > `Add a new object type for Line Art, which would basically be grease pencil without edit mode` This is not a trivial task. We would need review all places where we check edit mode and review all draw and operators code to be sure all it's working as expected. > > Also, to have a modifier for Lineart allows to include in the modifier stack and modify the data generated by this modifier with more modifiers, using different values for each object. Also, you can "bake" the result just pressing apply to the modifier. I think this idea is worth exploring, Not necessarily a new object type, but a sub category of Grease Pencil, make it a special flag that differentiate with normal ones, this way we have all the Grease Pencil functions available, but we can then approach the UI representation differently. Internally Line Art can still be a modifier, but for this special kind of object we can show the modifier differently, e.g. in a specific tab, and they can then be hidden in the regular modifier stack.
Member

In #80194#1048812, @brecht wrote:
From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it.

But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output.

I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe.

It is thread-safe, it locks main thread during loading, after that Blender data is not accessed while line art is running. The only thing called from the thread is the tag_update() call, if that is thread-safe then we are good to go.

This is currently the best way that we can run line art without blocking the UI. Because the update isn't called by an operator, it's kinda hard for me to use a Job like fluid/physics simulations do.

> In #80194#1048812, @brecht wrote: > From looking at the code, it seems that the modifier actually outputs the geometry, and further modifiers can be applied on top of it. > > But it does a custom delayed modifier evaluation. The modifier evaluations starts a thread, and then when that is done it tags the object for update, so that the depsgraph evaluates the modifiers again and the resulting strokes are output. > > I have not looked closely enough to see if we can actually do this in a way that is compatible with other Blender features and is thread safe. It is thread-safe, it locks main thread during loading, after that Blender data is not accessed while line art is running. The only thing called from the thread is the `tag_update()` call, if that is thread-safe then we are good to go. This is currently the best way that we can run line art without blocking the UI. Because the update isn't called by an operator, it's kinda hard for me to use a `Job` like fluid/physics simulations do.
Member

In #80194#1048862, @ZedDB wrote:

So for modifiers that is not "real time" we can have a job system of sorts.
Take mesh modifiers for example:

  1. I have the default cube with two modifiers (array and subdiv)
  2. The array modifier duplicates the cube 1000 times (nearly instant, no job required)
  3. The subdiv will probably be slow in this case so a job is created and the geometry from the array modifier is displayed, but doesn't lock up the UI while it waits.
  4. When the subdiv modifier job finishes, the result is displayed.

Yes! Job access for slow modifiers should be implemented at some point, so that we don't need that much hassle to asynchronously tag everything and check them. But for now, current "worker thread: might just be as good as we can get.

In #80194#1049012, @ZedDB wrote:

In #80194#1048915, @HooglyBoogly wrote:
Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now.

I think this should be case as well. :)
Perhaps we could keep the "bake all" button though?
So it's kinda like the physics modifiers, they have a "bake all dynamics" button or so IIRC.

This is technically possible, just bake stroke from certain modifiers, there are some small concerns related to it:

  1. You might just want to bake line art on top of the stack, as in some cases user might add another line art after some deformation modifiers.
  2. Line Art would need 1 entire evaluation for every frame, doesn't matter how many line art modifiers exist. So if you partially bake, it will run the whole thing once again, it isn't really efficient. You can output to different layers in order to separate them.
> In #80194#1048862, @ZedDB wrote: > > So for modifiers that is not "real time" we can have a job system of sorts. > Take mesh modifiers for example: > 1. I have the default cube with two modifiers (array and subdiv) > 2. The array modifier duplicates the cube 1000 times (nearly instant, no job required) > 3. The subdiv will probably be slow in this case so a job is created and the geometry from the array modifier is displayed, but doesn't lock up the UI while it waits. > 4. When the subdiv modifier job finishes, the result is displayed. Yes! Job access for slow modifiers should be implemented at some point, so that we don't need that much hassle to asynchronously tag everything and check them. But for now, current "worker thread: might just be as good as we can get. > In #80194#1049012, @ZedDB wrote: >> In #80194#1048915, @HooglyBoogly wrote: >> Could the baking settings be moved to the modifier? That way you could get editable strokes for only one collection / object? I just tested and it seems this isn't possible right now. > > I think this should be case as well. :) > Perhaps we could keep the "bake all" button though? > So it's kinda like the physics modifiers, they have a "bake all dynamics" button or so IIRC. This is technically possible, just bake stroke from certain modifiers, there are some small concerns related to it: 1. You might just want to bake line art on top of the stack, as in some cases user might add another line art after some deformation modifiers. 2. Line Art would need 1 entire evaluation for every frame, doesn't matter how many line art modifiers exist. So if you partially bake, it will run the whole thing once again, it isn't really efficient. You can output to different layers in order to separate them.
Member

In #80194#1048778, @HooglyBoogly wrote:

  1. Move the settings from the modifier to collections and objects directly. Each object (edit: and collection) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there.

This doesn't really apply. Currently you can only set "exclude" and such properties for individual ob/collections, because to calculate occlusion, you need everything in the scene right? or you might want to draw some isolate groups and don't care about the relationships with the rest of the scene. (If this is what you want, at the moment it's not really easy to do so, but possible in line art, but I can see there might be some uses to it, and this when exposed, can be used to make some Mesh->GP quick conversions, might be useful for making assets in animations.)

> In #80194#1048778, @HooglyBoogly wrote: > 1. Move the settings from the modifier to collections and objects directly. Each object (*edit: and collection*) would have a Line Art toggle in a panel header, and all of the settings currently configurable per-modifier would be there. This doesn't really apply. Currently you can only set "exclude" and such properties for individual ob/collections, because to calculate occlusion, you need everything in the scene right? or you might want to draw some isolate groups and don't care about the relationships with the rest of the scene. (If this is what you want, at the moment it's not really easy to do so, but possible in line art, but I can see there might be some uses to it, and this when exposed, can be used to make some Mesh->GP quick conversions, might be useful for making assets in animations.)
Member

And for the "transparency bits" thing, it might be too technical for the user, Me and @ZedDB are talking about if we should expose the "bits" as a list, so the user can add entries and name each one, in the modifier they can select the transparency component they want, but the length would be limited to 8 entries (line art internal limit). But this still looks pretty complicated to be honest.

In my opinion, we can think about the bits like the "bone layer" buttons, so it's not actually that hard to understand. (It's better if we make an icon to show in the toggle button if that one has been configured as viable transparency layer in the material panel, that way it's more clear)

To compare the UI of bone layer and line art transparency selection:

图片.png

图片.png

And for the "transparency bits" thing, it might be too technical for the user, Me and @ZedDB are talking about if we should expose the "bits" as a list, so the user can add entries and name each one, in the modifier they can select the transparency component they want, but the length would be limited to 8 entries (line art internal limit). But this still looks pretty complicated to be honest. In my opinion, we can think about the bits like the "bone layer" buttons, so it's not actually that hard to understand. (It's better if we make an icon to show in the toggle button if that one has been configured as viable transparency layer in the material panel, that way it's more clear) To compare the UI of bone layer and line art transparency selection: ![图片.png](https://archive.blender.org/developer/F9214980/图片.png) ![图片.png](https://archive.blender.org/developer/F9214949/图片.png)
Member

To further explain how Line Art transparency mask works, I made a comparison with Freestyle. The transparency setup in the mesh material is necessary, while calculating occlusion info, the transparency bit is registered every time a triangle occludes a line segment. Thus to save space, line art allow up to 8 bits as transparency info, combinations are also allowed to achieve "multiple layers of glass" effect. To achieve regional selection in freestyle like this, one needs to use composition.

image.png

image.png

To further explain how Line Art transparency mask works, I made a comparison with Freestyle. The transparency setup in the mesh material is necessary, while calculating occlusion info, the transparency bit is registered every time a triangle occludes a line segment. Thus to save space, line art allow up to 8 bits as transparency info, combinations are also allowed to achieve "multiple layers of glass" effect. To achieve regional selection in freestyle like this, one needs to use composition. ![image.png](https://archive.blender.org/developer/F9243899/image.png) ![image.png](https://archive.blender.org/developer/F9243912/image.png)

After looking at the code and some discussion with Sergey, I don't think the current deferred modifier evaluation is ok for master. It's too much of a hack and needs a proper design and implementation, which would not be specific to line art, but deferred depsgraph evaluation in general. I suggest to leave out that part of the code and do evaluation immediately, even with the associated downsides. A mechanism for deferred implementation can be added in a second step, and needs a proper design doc before going to code review.

There are a few immediate issues with the deferred evaluation code:

  • It's not obvious that it is thread safe, that the data structures being accessed can't be modified or removed while line art is reading from them.
  • It doesn't seem to work when multiple scenes are open at the same time (into two separate main windows).
  • It's not clear that the computation is properly cancelled if the modifier, object or scene is deleted.
  • The implementation should not use global variables, state should be stored per depsgraph / evaluated scene or object.
  • Tagging objects for depsgraph update affects the undo system and will put unrelated changes in some undo steps.

In general it's just unpredictable to have a mechanism like this that works outside the current design of the depsgraph and does computations on its own..

After looking at the code and some discussion with Sergey, I don't think the current deferred modifier evaluation is ok for master. It's too much of a hack and needs a proper design and implementation, which would not be specific to line art, but deferred depsgraph evaluation in general. I suggest to leave out that part of the code and do evaluation immediately, even with the associated downsides. A mechanism for deferred implementation can be added in a second step, and needs a proper design doc before going to code review. There are a few immediate issues with the deferred evaluation code: * It's not obvious that it is thread safe, that the data structures being accessed can't be modified or removed while line art is reading from them. * It doesn't seem to work when multiple scenes are open at the same time (into two separate main windows). * It's not clear that the computation is properly cancelled if the modifier, object or scene is deleted. * The implementation should not use global variables, state should be stored per depsgraph / evaluated scene or object. * Tagging objects for depsgraph update affects the undo system and will put unrelated changes in some undo steps. In general it's just unpredictable to have a mechanism like this that works outside the current design of the depsgraph and does computations on its own..

Some notes about from testing and looking at the code:

  • Modifier properties
    Thickness: what is the unit? can this be clarified? Pre Sample length, Multiple Levels, and more: it's unclear from description what the effect is for end users, descriptions need to be better.
    ** Multiple Levels: unclear from description what the effect is for end user
  • Scene properties
    ** Line Art panel should not be at the top in the order, but rather next to Grease Pencil or even in the same panel
  • Edge only meshes without faces result in line art but are invisible in renders otherwise, Ignore these for line art by default to match renders, maybe an option can be added to use them if needed.

Also, the implementation of modifiers should not be in editors/, but in modifiers/ or another module specifically for line art. Basically it should be possible to evaluate and render scenes without any code from editors/.

Some notes about from testing and looking at the code: * Modifier properties **Thickness: what is the unit? can this be clarified?** Pre Sample length, Multiple Levels, and more: it's unclear from description what the effect is for end users, descriptions need to be better. ** Multiple Levels: unclear from description what the effect is for end user * Scene properties ** Line Art panel should not be at the top in the order, but rather next to Grease Pencil or even in the same panel * Edge only meshes without faces result in line art but are invisible in renders otherwise, Ignore these for line art by default to match renders, maybe an option can be added to use them if needed. Also, the implementation of modifiers should not be in editors/, but in modifiers/ or another module specifically for line art. Basically it should be possible to evaluate and render scenes without any code from editors/.

For transparency masks. The simple solution would be just to have a single option to indicate a different look behind a transparent object, which may cover the majority of use cases.

For more clever ways to have a different look based on some combination of mask values, it would be good to see the actual use case, it's not clear to me what they are. It's tempting to add it because it's easy to do from a programming point of view, but are there really important use cases beyond the basic case?

For transparency masks. The simple solution would be just to have a single option to indicate a different look behind a transparent object, which may cover the majority of use cases. For more clever ways to have a different look based on some combination of mask values, it would be good to see the actual use case, it's not clear to me what they are. It's tempting to add it because it's easy to do from a programming point of view, but are there really important use cases beyond the basic case?
Member

In #80194#1051927, @brecht wrote:
After looking at the code and some discussion with Sergey, I don't think the current deferred modifier evaluation is ok for master. It's too much of a hack and needs a proper design and implementation, which would not be specific to line art, but deferred depsgraph evaluation in general. I suggest to leave out that part of the code and do evaluation immediately, even with the associated downsides. A mechanism for deferred implementation can be added in a second step, and needs a proper design doc before going to code review.

So if this is the case, I'd like to see if there's any way I can implement a deferred evaluation path for the depsgraph, just like @ZedDB mentiioned that we can have job accessible in the modifier. Line art isn't gonna be really usable if we evaluate multiple modifiers multiple times, unless we are only using it for extremely simple cases.

  • It doesn't seem to work when multiple scenes are open at the same time (into two separate main windows).

Humm that's what I haven't thought of. This surely needs per-depsgraph/per-evaluated scene kind of storage.

  • Tagging objects for depsgraph update affects the undo system and will put unrelated changes in some undo steps.

Yeah this is a problem.

In general it's just unpredictable to have a mechanism like this that works outside the current design of the depsgraph and does computations on its own..

Pleeeeease let us implement the Job system for modifiers. I'd love to take on this!

Also, the implementation of modifiers should not be in editors/, but in modifiers/ or another module specifically for line art. Basically it should be possible to evaluate and render scenes without any code from editors/.

This is due to the current nature of line art which needs shared data and stuff, If we do non-threaded updates, this could be fully contained in modifier/.

I'll look into ther UI description problems. For transparency... I can give it a try, where a simple "transparency" toggle would allow selecting those lines behind glass, and if we need more flexibility, we can check an "advanced" button and do it from there.

> In #80194#1051927, @brecht wrote: > After looking at the code and some discussion with Sergey, I don't think the current deferred modifier evaluation is ok for master. It's too much of a hack and needs a proper design and implementation, which would not be specific to line art, but deferred depsgraph evaluation in general. I suggest to leave out that part of the code and do evaluation immediately, even with the associated downsides. A mechanism for deferred implementation can be added in a second step, and needs a proper design doc before going to code review. So if this is the case, I'd like to see if there's any way I can implement a deferred evaluation path for the depsgraph, just like @ZedDB mentiioned that we can have job accessible in the modifier. Line art isn't gonna be really usable if we evaluate multiple modifiers multiple times, unless we are only using it for extremely simple cases. > * It doesn't seem to work when multiple scenes are open at the same time (into two separate main windows). Humm that's what I haven't thought of. This surely needs per-depsgraph/per-evaluated scene kind of storage. > * Tagging objects for depsgraph update affects the undo system and will put unrelated changes in some undo steps. Yeah this is a problem. > In general it's just unpredictable to have a mechanism like this that works outside the current design of the depsgraph and does computations on its own.. Pleeeeease let us implement the Job system for modifiers. I'd love to take on this! > Also, the implementation of modifiers should not be in editors/, but in modifiers/ or another module specifically for line art. Basically it should be possible to evaluate and render scenes without any code from editors/. This is due to the current nature of line art which needs shared data and stuff, If we do non-threaded updates, this could be fully contained in modifier/. I'll look into ther UI description problems. For transparency... I can give it a try, where a simple "transparency" toggle would allow selecting those lines behind glass, and if we need more flexibility, we can check an "advanced" button and do it from there.
Member

I think for this patch, I can make the modifiers self-contained and use no thread for doing the updates (If we are okay with the downsides). I'll keep a branched version on the side so there's still some usability. Meanwhile I'll look into the deferred evaluation stuff.

I think for this patch, I can make the modifiers self-contained and use no thread for doing the updates (If we are okay with the downsides). I'll keep a branched version on the side so there's still some usability. Meanwhile I'll look into the deferred evaluation stuff.
Member

@mendio Hi, I may also need some suggestion on the UI stuff here :D. I see linguistic description of some stuff may not align that well with western English speakers.

Thanks!

@mendio Hi, I may also need some suggestion on the UI stuff here :D. I see linguistic description of some stuff may not align that well with western English speakers. Thanks!

Added subscriber: @zeauro

Added subscriber: @zeauro

About UI, I suggest to move LineArt modifier settings into a sub-panel of Layers panel in Grease Pencil data properties.
Basically, Lineart modifier properties are properties of a group of strokes (a line set like in freestyle) created into a specific Grease Pencil Layer.
Re-using Grease Pencil Layers abilities helps a lot to define what line set is supposed to be rendered on top of the others.

Levels are similar to Freestyle QI range. The idea is to define Line Set according to occlusion in order to be able to give a different style to hidden lines of geometry.
Here is an example .blend file.
Lineart_usecase.blend

The idea was to be able re-use already existing grease pencil modifiers to create a style specific to this line set like in Freestyle.
As shown in .blend file attached above, that is possible to set multiple LineArt modifiers to define several line sets and give different styles to them.

Those differences may be due to Lineart modifier settings. But also to modifiers that are added afterwards, if layer restriction of modifier is used.

The fact to re-use grease pencil modifiers thought to modify lines in 3D space does not work with those strokes generated from camera angle and projected onto surface.
Something as basic as stroke thickness is not well controlled by a current thickness modifier.
Stroke_Thickness.blend
Maybe, that would not be the case, if strokes generation was thought to project them on a View plane at origin of 3D mesh instead of being projected on 3D Mesh surface.

I think that ideally, current Line Art modifier should become a sub-panel of layers.
We have a properties tab dedicated to 2D postprocessing effects that can not be restricted to layers.

Simplest UI would be to have a properties tab dedicated to Line Style modifiers with modifiers thought to give a satisfying result from Camera Angle.
Modifiers shown in this tab would be restricted to the ones defining active layer.
Each layer would have its own Line Art modifier stack.

But user could be satisfied by current modifiers with current restriction to layer ; at condition that those modifiers are adapted to have a LineArt mode to give satisfying results from Camera Angle.

About UI, I suggest to move LineArt modifier settings into a sub-panel of Layers panel in Grease Pencil data properties. Basically, Lineart modifier properties are properties of a group of strokes (a line set like in freestyle) created into a specific Grease Pencil Layer. Re-using Grease Pencil Layers abilities helps a lot to define what line set is supposed to be rendered on top of the others. Levels are similar to Freestyle QI range. The idea is to define Line Set according to occlusion in order to be able to give a different style to hidden lines of geometry. Here is an example .blend file. [Lineart_usecase.blend](https://archive.blender.org/developer/F9264089/Lineart_usecase.blend) The idea was to be able re-use already existing grease pencil modifiers to create a style specific to this line set like in Freestyle. As shown in .blend file attached above, that is possible to set multiple LineArt modifiers to define several line sets and give different styles to them. Those differences may be due to Lineart modifier settings. But also to modifiers that are added afterwards, if layer restriction of modifier is used. The fact to re-use grease pencil modifiers thought to modify lines in 3D space does not work with those strokes generated from camera angle and projected onto surface. Something as basic as stroke thickness is not well controlled by a current thickness modifier. [Stroke_Thickness.blend](https://archive.blender.org/developer/F9264071/Stroke_Thickness.blend) Maybe, that would not be the case, if strokes generation was thought to project them on a View plane at origin of 3D mesh instead of being projected on 3D Mesh surface. I think that ideally, current Line Art modifier should become a sub-panel of layers. We have a properties tab dedicated to 2D postprocessing effects that can not be restricted to layers. Simplest UI would be to have a properties tab dedicated to Line Style modifiers with modifiers thought to give a satisfying result from Camera Angle. Modifiers shown in this tab would be restricted to the ones defining active layer. Each layer would have its own Line Art modifier stack. But user could be satisfied by current modifiers with current restriction to layer ; at condition that those modifiers are adapted to have a LineArt mode to give satisfying results from Camera Angle.

Added subscriber: @GeorgiaPacific

Added subscriber: @GeorgiaPacific

Added subscriber: @heini

Added subscriber: @heini
Member

Added subscriber: @IRIEShinsuke

Added subscriber: @IRIEShinsuke

Added subscriber: @Qwerkie

Added subscriber: @Qwerkie

Added subscriber: @marcelp102

Added subscriber: @marcelp102

Added subscriber: @elbruswaters

Added subscriber: @elbruswaters

This issue was referenced by 3e87d8a431

This issue was referenced by 3e87d8a4315d794efff659e40f0bb9e34e2aec8a
Member

Updated to include the manual page created for line art.

Updated to include the manual page created for line art.

Added subscriber: @Zeirus

Added subscriber: @Zeirus

Added subscriber: @Okavango

Added subscriber: @Okavango

@ChengduLittleA Congrats on patch landing ! (pun intended :)

I tried it out a little in my spare time, looks like a really great potential for technical-like presentations, kudos!

I was wondering, are there any plans to make an option for zoom-relative line width? It works great as it is, but in some cases if i zoom out to a scene over-all, the lines almost disappear. Likewise, if you zoom in for a detail shot, lines become a little too thick.

Anyways, great effort, thanks a lot for your addition, waiting for a long time for such a feature.

@ChengduLittleA Congrats on patch landing ! (pun intended :) I tried it out a little in my spare time, looks like a really great potential for technical-like presentations, kudos! I was wondering, are there any plans to make an option for zoom-relative line width? It works great as it is, but in some cases if i zoom out to a scene over-all, the lines almost disappear. Likewise, if you zoom in for a detail shot, lines become a little too thick. Anyways, great effort, thanks a lot for your addition, waiting for a long time for such a feature.

This issue was referenced by f8f7c0ca6d

This issue was referenced by f8f7c0ca6db87e5f2613896156e595d2bea5fdde
Philipp Oeser removed the
Interest
Grease Pencil
label 2023-02-09 15:19:45 +01:00
Philipp Oeser added the
Interest
VFX & Video
label 2023-02-10 11:36:38 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
20 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#80194
No description provided.