Setup Tracking Scene needs an update #84769

New Issue

Sebastian Koenig · 2021-01-16T11:07:57+01:00

Sebastian Koenig commented

2021-01-16 11:07:57 +01:00

The Setup Tracking Scene operator needs an update. Since Blender has the option to make an object a shadow catcher, the current setup seems overly complex and elaborate.
Also, not always is a shadow catcher even needed. That's why I suggest to simplify the setup tracking scene operator and make it a bit more flexible.

Different Setup operators
Instead of the current two operators, I would break it up a bit more.

Set as Background (same as current operator)
Setup Camera: If no camera is available, add one. Setup the camera solver constraint. Set Background images.
Setup Tracking scene: see below
Setup Solved Object (not sure about the term): Add an Empty (drawtype Sphere maybe?) to the scene, setup the object solver constraint, set inverse.
optionally more operators can be added, maybe via addons, such as cleanplate setup etc.

Setup Tracking Scene
The current setup operator is designed for a very specific usecase. But now with the shadow catcher and Eevee available I find it is not suitable anymore in most cases. So I suggest to use a simplified setup instead. Should the user need a more elaborate solution it is much easier to extend the simplified setup, than to cleanup the complicated one that we have at the moment.
To make it more convenient for the user I suggest to let the user choose which render engine he wants to use and if he even needs a shadow catcher.
So first the user can switch the render engine from the currently active one to a different engine with a dropdown menu. Though for now I would suggest to only use Eevee and Cycles.
Then with a checkbox you can choose whether to use a shadow catcher or not.
If you then hit "Setup Tracking scene" the following happens:

Make Film transparent
Enable use_nodes in the compositor
Setup node_tree: Add Movieclip. Add scale node. Only if LensDistortion has been used for solving the Undistortion node will be added. Combine Viewlayer and the movieclip tree with an AlphaOver node.
If shadow catcher checkbox is enabled: Add a plane in a new collection named "shadow catcher". With Cycles as render engine, enable "Shadow Catcher" for the plane. With Eevee, add a shadow catcher material for the plane and set the Material to Alpha Blend, as suggested here: https://www.blendernation.com/2019/06/03/blender-2-8-eevee-tutorial-shadow-catcher
In both cases the shadow will appear in the alpha channel of the Viewlayer, so the node_tree from above won't have to be modified.
I also would suggest not to change the World shader anymore, as it feels quite invasive, especially with Eevee.

Python
Currently the setup tracking scene operator is made up of a bunch of functions all inside CLIP_OT_setup_tracking_scene. So I guess those functions can be pulled out of that class to make them more global, so that the new operators from above have access to them.

Feedback and suggestions are very welcome!

The Setup Tracking Scene operator needs an update. Since Blender has the option to make an object a shadow catcher, the current setup seems overly complex and elaborate. Also, not always is a shadow catcher even needed. That's why I suggest to simplify the setup tracking scene operator and make it a bit more flexible. **Different Setup operators** Instead of the current two operators, I would break it up a bit more. - Set as Background (same as current operator) - Setup Camera: If no camera is available, add one. Setup the camera solver constraint. Set Background images. - Setup Tracking scene: see below - Setup Solved Object (not sure about the term): Add an Empty (drawtype Sphere maybe?) to the scene, setup the object solver constraint, set inverse. - optionally more operators can be added, maybe via addons, such as cleanplate setup etc. **Setup Tracking Scene** The current setup operator is designed for a very specific usecase. But now with the shadow catcher and Eevee available I find it is not suitable anymore in most cases. So I suggest to use a simplified setup instead. Should the user need a more elaborate solution it is much easier to extend the simplified setup, than to cleanup the complicated one that we have at the moment. To make it more convenient for the user I suggest to let the user choose which render engine he wants to use and if he even needs a shadow catcher. So first the user can switch the render engine from the currently active one to a different engine with a dropdown menu. Though for now I would suggest to only use Eevee and Cycles. Then with a checkbox you can choose whether to use a shadow catcher or not. If you then hit "Setup Tracking scene" the following happens: - Make Film transparent - Enable use_nodes in the compositor - Setup node_tree: Add Movieclip. Add scale node. Only if LensDistortion has been used for solving the Undistortion node will be added. Combine Viewlayer and the movieclip tree with an AlphaOver node. - If shadow catcher checkbox is enabled: Add a plane in a new collection named "shadow catcher". With Cycles as render engine, enable "Shadow Catcher" for the plane. With Eevee, add a shadow catcher material for the plane and set the Material to Alpha Blend, as suggested here: https://www.blendernation.com/2019/06/03/blender-2-8-eevee-tutorial-shadow-catcher In both cases the shadow will appear in the alpha channel of the Viewlayer, so the node_tree from above won't have to be modified. I also would suggest not to change the World shader anymore, as it feels quite invasive, especially with Eevee. **Python** Currently the setup tracking scene operator is made up of a bunch of functions all inside CLIP_OT_setup_tracking_scene. So I guess those functions can be pulled out of that class to make them more global, so that the new operators from above have access to them. Feedback and suggestions are very welcome!

Sebastian Koenig commented

2021-01-16 11:07:57 +01:00

Added subscriber: @sebastian_k

Sebastian Koenig commented

2021-01-16 11:50:40 +01:00

Added subscriber: @SeanKennedy

Sean Kennedy commented

2021-01-20 01:17:32 +01:00

Would it also be possible to make the entire node tree optional? Many times when I click Setup tracking Scene, I only want the 3D stuff, not the node setup. I end up deleting and/or moving around the nodes to my own preferences, anyway. Could be a simple check box under the button "setup nodes", something simple like that

Sebastian Koenig commented

2021-01-20 08:21:59 +01:00

Sure! :)

Evan Wilson commented

2021-01-20 08:23:11 +01:00

Added subscriber: @EAW

Sebastian Koenig commented

2021-01-20 11:26:10 +01:00

Added subscriber: @Sergey

Sebastian Koenig commented

2021-01-20 11:26:10 +01:00

Here is a first draft of it:
https://github.com/blendfx/blender/blob/master/setup_tracking_scene_new.py
I couldn't get it to work with operators and UI separated in clip.py and space_clip.py like it is supposed to be, so arranged them in one file in the style of an addon.
@SeanKennedy if you want to try it out, open that file in the text editor and run it.
It also has a new operator, which could be useful for object tracking. It will create an Empty in the 3d Viewport at the position of the selected track, if it is part of a solved object track. The empty will have the objcet solver contraint already setup.
The setup for the Eevee shadow catcher still has be configured, I will do that in the next couple of days.
Also, the UI will probably have to be adjusted once the new tool system is in place.

@Sergey, if you're interested, have a look as well :)

Here is a first draft of it: https://github.com/blendfx/blender/blob/master/setup_tracking_scene_new.py I couldn't get it to work with operators and UI separated in clip.py and space_clip.py like it is supposed to be, so arranged them in one file in the style of an addon. @SeanKennedy if you want to try it out, open that file in the text editor and run it. It also has a new operator, which could be useful for object tracking. It will create an Empty in the 3d Viewport at the position of the selected track, if it is part of a solved object track. The empty will have the objcet solver contraint already setup. The setup for the Eevee shadow catcher still has be configured, I will do that in the next couple of days. Also, the UI will probably have to be adjusted once the new tool system is in place. ![scene_setup.png](https://archive.blender.org/developer/F9589141/scene_setup.png) @Sergey, if you're interested, have a look as well :)

Sean Kennedy commented

2021-01-21 05:41:40 +01:00

Got an error when I tried to run it. Probably super simple to fix, but I don't know anything about coding so I have no idea how to fix it.

Got an error when I tried to run it. Probably super simple to fix, but I don't know anything about coding so I have no idea how to fix it. ![scene_setup_fail_01.jpg](https://archive.blender.org/developer/F9590485/scene_setup_fail_01.jpg)

Sebastian Koenig commented

2021-01-21 09:19:13 +01:00

Okay, seems you did download the HTML file from github, not the raw file.
Try this one: https://raw.githubusercontent.com/blendfx/blender/master/setup_tracking_scene_new.py

Okay, seems you did download the HTML file from github, not the raw file. Try this one: https://raw.githubusercontent.com/blendfx/blender/master/setup_tracking_scene_new.py

Sean Kennedy commented

2021-01-22 06:32:50 +01:00

Nice! Seems to be working nicely.

Only if LensDistortion has been used for solving the Undistortion node will be added.

This isn't applicable yet, correct? I tried with and without having any lens distortion values, and the lens distortion node was there regardless.

Just to clarify, pressing Setup Camera is exactly the same as pressing Setup Tracking Scene with Nodes unchecked, right?

This is nice, great start!

Nice! Seems to be working nicely. > Only if LensDistortion has been used for solving the Undistortion node will be added. This isn't applicable yet, correct? I tried with and without having any lens distortion values, and the lens distortion node was there regardless. Just to clarify, pressing Setup Camera is exactly the same as pressing Setup Tracking Scene with Nodes unchecked, right? This is nice, great start!

Sebastian Koenig commented

2021-01-22 09:55:01 +01:00

Hey @SeanKennedy, the check for undistortion is working now. I replaced need_stabilization with need_undistortion, the check for stabilization never worked anyway, it was always set to False.
Setup Camera is not the same as Setup Tracking Scene without nodes.
Setup Tracking Scene also makes the Film transparent, creates a foreground collection with a Cube and can setup a shadow catcher, if needed.

Hey @SeanKennedy, the check for undistortion is working now. I replaced need_stabilization with need_undistortion, the check for stabilization never worked anyway, it was always set to False. Setup Camera is not the same as Setup Tracking Scene without nodes. Setup Tracking Scene also makes the Film transparent, creates a foreground collection with a Cube and can setup a shadow catcher, if needed.

Sebastian Koenig commented

2021-01-24 13:43:58 +01:00

@SeanKennedy The setup of Eevee shadow catcher should work now as well. So depending on which render engine you want to use, a different kind of shadow catcher will be setup.

Sean Kennedy commented

2021-01-26 06:16:15 +01:00

Setup Tracking Scene also makes the Film transparent, creates a foreground collection with a Cube and can setup a shadow catcher, if needed

If you leave the default cube in the scene, it does not move it to the foreground collection. If you delete the default cube, it will create a cube in that foreground collection.

Should the Backdrop in the node view be automatically turned on when using the Nodes checkbox? I don't ever use the Backdrop anymore, but if most people do, I guess it would make sense to leave on.

Checking both Nodes and Shadow Catcher with in both cycles and eevee, there weren't any nodes added for shadows. I haven't used Eevee enough to know if that's necessary, but for Cycles, should there be some nodes? Also, should Indirect be turned on for the "shadow catcher" collection?

Just typing these out as I mess around with it, so I may be talking nonsense. But this is MUCH better scene setup options than the current way!

> Setup Tracking Scene also makes the Film transparent, creates a foreground collection with a Cube and can setup a shadow catcher, if needed If you leave the default cube in the scene, it does not move it to the foreground collection. If you delete the default cube, it will create a cube in that foreground collection. Should the Backdrop in the node view be automatically turned on when using the Nodes checkbox? I don't ever use the Backdrop anymore, but if most people do, I guess it would make sense to leave on. Checking both Nodes and Shadow Catcher with in both cycles and eevee, there weren't any nodes added for shadows. I haven't used Eevee enough to know if that's necessary, but for Cycles, should there be some nodes? Also, should Indirect be turned on for the "shadow catcher" collection? Just typing these out as I mess around with it, so I may be talking nonsense. But this is MUCH better scene setup options than the current way!

Sebastian Koenig commented

2021-02-04 14:04:45 +01:00

In #84769#1100664, @SeanKennedy wrote:
If you leave the default cube in the scene, it does not move it to the foreground collection. If you delete the default cube, it will create a cube in that foreground collection.

Thanks for the hint, this should be fixed now. Instead of always creating a new collection, it will rename the default collection to "foreground", if there really is only 1 collection in your scene.

Should the Backdrop in the node view be automatically turned on when using the Nodes checkbox? I don't ever use the Backdrop anymore, but if most people do, I guess it would make sense to leave on.

I still use it. I didn't touch that setting, would leave it for now.

Checking both Nodes and Shadow Catcher with in both cycles and eevee, there weren't any nodes added for shadows. I haven't used Eevee enough to know if that's necessary, but for Cycles, should there be some nodes? Also, should Indirect be turned on for the "shadow catcher" collection?

No, the cycles setup uses the Shadow Catcher option. You don't need any nodes for that, it works via Alpha Channel. Not really fancy and not that much controls, but it's an easy setup.

Just typing these out as I mess around with it, so I may be talking nonsense. But this is MUCH better scene setup options than the current way!

Thanks! :)

> In #84769#1100664, @SeanKennedy wrote: > If you leave the default cube in the scene, it does not move it to the foreground collection. If you delete the default cube, it will create a cube in that foreground collection. > Thanks for the hint, this should be fixed now. Instead of always creating a new collection, it will rename the default collection to "foreground", if there really is only 1 collection in your scene. > Should the Backdrop in the node view be automatically turned on when using the Nodes checkbox? I don't ever use the Backdrop anymore, but if most people do, I guess it would make sense to leave on. > I still use it. I didn't touch that setting, would leave it for now. > Checking both Nodes and Shadow Catcher with in both cycles and eevee, there weren't any nodes added for shadows. I haven't used Eevee enough to know if that's necessary, but for Cycles, should there be some nodes? Also, should Indirect be turned on for the "shadow catcher" collection? No, the cycles setup uses the Shadow Catcher option. You don't need any nodes for that, it works via Alpha Channel. Not really fancy and not that much controls, but it's an easy setup. > Just typing these out as I mess around with it, so I may be talking nonsense. But this is MUCH better scene setup options than the current way! Thanks! :)

Alaska commented

2024-04-07 06:04:58 +02:00

I am removing the Needs Triage label. This is under the general rule that Design and TODO tasks should not have a status.

If you believe this task is no longer relevant, feel free to close it.

I am removing the `Needs Triage` label. This is under the general rule that Design and TODO tasks should not have a status. If you believe this task is no longer relevant, feel free to close it.

Alaska removed the

Status

Needs Triage

label 2024-04-07 06:05:09 +02:00

Alaska changed title from ~~Setup Tracking Scene needs an upate~~ to Setup Tracking Scene needs an update

2024-04-07 06:05:16 +02:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Setup Tracking Scene needs an update #84769