Photogrammetry Importer Addon Proposal #84577

New Issue

Sebastian · 2021-01-10T22:55:49+01:00

Sebastian commented

2021-01-10 22:55:49 +01:00

The proposed addon allows to import image-based reconstructions of several (state-of-the-art) Structure from Motion (SfM) and Multi-View Stereo (MVS) libraries into Blender. SfM allows to determine camera calibrations and camera motions from plain color images without manually defining feature tracks. MVS uses the reconstruction results of the SfM step to automatically compute dense point clouds and (textured) meshes.

The addon supports the following libraries:

You can find the latest release of the addon on the release page, which is compatible with Blender 3.0 onwards.
Please checkout [this github repository]] and the [ https:*blender-addon-photogrammetry-importer.readthedocs.io/en/latest/ | corresponding documentation for more information.

There are several (non-critical) Blender issues that are related to the usage of this addon:

blender/blender#85709 (Offscreen rendering) [has been fixed]
blender/blender#85750 (Camera background movieclips)
blender/blender#86106 (Bug in draw handler removal) [has been fixed]
blender/blender#77958 (Viewport rendering) [circumvented by exploiting the screenshot operator]

Currently, the docstrings do not document the parameters and return values. If required I can add the corresponding documentation.

If you have any questions, problems or suggestions how to proceed, please let me know.

Best regards,
Sebastian

The proposed addon allows to import image-based reconstructions of several (state-of-the-art) *Structure from Motion* (SfM) and *Multi-View Stereo* (MVS) libraries into Blender. SfM allows to determine camera calibrations and camera motions from plain color images without manually defining feature tracks. MVS uses the reconstruction results of the SfM step to automatically compute dense point clouds and (textured) meshes. The addon supports the following libraries: * [Metashape ](https://www.agisoft.com/) * [Colmap ](https://github.com/colmap/colmap) * [Meshroom ](https://alicevision.github.io/) * [MVE ](https://github.com/simonfuhrmann/mve) * [Open3D ](http://www.open3d.org/) * [OpenSfM ](https://github.com/mapillary/OpenSfM) * [OpenMVG ](https://github.com/openMVG/openMVG) * [Regard3D ](https://www.regard3d.org/) * [VisualSFM ](http://ccwu.me/vsfm/) You can find the latest release of the addon on the [release page](https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer/releases), which is compatible with Blender 3.0 onwards. Please checkout [this github repository]] and the [[ https:*blender-addon-photogrammetry-importer.readthedocs.io/en/latest/ | corresponding documentation ](https:*github.com/SBCV/Blender-Addon-Photogrammetry-Importer) for more information. There are several (non-critical) Blender issues that are related to the usage of this addon: - blender/blender#85709 (Offscreen rendering) [has been fixed] - blender/blender#85750 (Camera background movieclips) - blender/blender#86106 (Bug in draw handler removal) [has been fixed] - blender/blender#77958 (Viewport rendering) [circumvented by exploiting the screenshot operator] Currently, the docstrings do not document the parameters and return values. If required I can add the corresponding documentation. If you have any questions, problems or suggestions how to proceed, please let me know. Best regards, Sebastian

Sebastian commented

2021-01-10 22:55:49 +01:00

Added subscriber: @sbcv

Robert Guetzkow commented

2021-01-12 11:41:45 +01:00

Added subscriber: @rjg

Robert Guetzkow commented

2021-01-12 11:41:45 +01:00

The issue with bgl.GL_DEPTH_TEST could be the same as blender/blender#83482. Were the issues that you suspect to be in Blender already reported on our bug tracker? If yes, please link the tickets. This needs to be investigated especially if there are condition under which Blender crashes.

The issue with `bgl.GL_DEPTH_TEST` could be the same as blender/blender#83482. Were the issues that you suspect to be in Blender already reported on our bug tracker? If yes, please link the tickets. This needs to be investigated especially if there are condition under which Blender crashes.

Sebastian commented

2021-01-12 19:41:25 +01:00

Hi Robert, thank you for having a look.
Regarding bgl.GL_DEPTH_TEST I'll have another look. But (probably) it is a bug, since it works as expected under Linux, but not with Windows. The problem in this case is that the error did only appeared for large point clouds (and in a non-deterministic fashion).
Here an excerpt of an e-mail conversation I had with some reporting that bug:

I tried to build a simple test case (see attachement) to locate the error under Windows.

Unfortunately, I experienced several strange behaviors of Blender's gpu / bgl module - from freezing GUI elements to non-deterministic rendering results. Oftentimes I was not able to render the result - even without bgl.glEnable(bgl.GL_DEPTH_TEST).

I tested it with Blender 2.83.5, 2.83.6, 2.83.7, 2.90.1 and 2.91.alpha.

Because the behavior is not deterministic, it is hard to tell what is going on. But it looks like an memory issue ... and I'm aifraid that there is not much I can do about it ...

While searching the web for possible causes, I found this issue "BGL deprecation"
https://developer.blender.org/T80730

Quote: "For 2.91 we refactored the whole GPU module to be ready for vulkan. In this process we added a state tracking mechanism which removes unneeded state changes by only pushing the state before drawing commands. The issue is that BGL does not make use of this and directly use the GL API which can lead to unsafe states. ..."

So I guess, the most reasonable approach is too wait and check if this problem persists with the new api.

Do you know, if the transition of OpenGL to Vulcan has already been made? Could this potentially fix the problem? If possible I like to avoid to debug "dead" code that will be replaced anyways.

For the other problems mentioned in https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer/issues/26 , I'll screen available issues, open new ones if necessary, and link them here.

Hi Robert, thank you for having a look. Regarding `bgl.GL_DEPTH_TEST` I'll have another look. But (probably) it is a bug, since it works as expected under Linux, but not with Windows. The problem in this case is that the error did only appeared for large point clouds (and in a non-deterministic fashion). Here an excerpt of an e-mail conversation I had with some reporting that bug: > I tried to build a simple test case (see attachement) to locate the error under Windows. > > Unfortunately, I experienced several strange behaviors of Blender's gpu / bgl module - from freezing GUI elements to non-deterministic rendering results. Oftentimes I was not able to render the result - even without bgl.glEnable(bgl.GL_DEPTH_TEST). > > I tested it with Blender 2.83.5, 2.83.6, 2.83.7, 2.90.1 and 2.91.alpha. > > Because the behavior is not deterministic, it is hard to tell what is going on. But it looks like an memory issue ... and I'm aifraid that there is not much I can do about it ... > > While searching the web for possible causes, I found this issue "BGL deprecation" > https://developer.blender.org/T80730 > > Quote: "For 2.91 we refactored the whole GPU module to be ready for vulkan. In this process we added a state tracking mechanism which removes unneeded state changes by only pushing the state before drawing commands. The issue is that BGL does not make use of this and directly use the GL API which can lead to unsafe states. ..." > > So I guess, the most reasonable approach is too wait and check if this problem persists with the new api. Do you know, if the transition of OpenGL to Vulcan has already been made? Could this potentially fix the problem? If possible I like to avoid to debug "dead" code that will be replaced anyways. For the other problems mentioned in https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer/issues/26 , I'll screen available issues, open new ones if necessary, and link them here.

Robert Guetzkow commented

2021-01-12 20:02:11 +01:00

@sbcv I'm more concerned about the crash that happens after deleting the object. This needs to be investigated no matter if this is a bug in Blender or the add-on. I was referring to tickets on the official Blender bug tracker. If the problem wasn't previously reported on here, then it should be, so that we have all relevant information to reproduce and fix the issue.

The transition to Vulkan is not made, the implementation still has to be done. See blender/blender#68990 for the current progress. Whether or not the switch to Vulkan would fix anything is impossible to tell without knowing where the original problem came from.

@sbcv I'm more concerned about the crash that happens after deleting the object. This needs to be investigated no matter if this is a bug in Blender or the add-on. I was referring to tickets on the official Blender bug tracker. If the problem wasn't previously reported on here, then it should be, so that we have all relevant information to reproduce and fix the issue. The transition to Vulkan is not made, the implementation still has to be done. See blender/blender#68990 for the current progress. Whether or not the switch to Vulkan would fix anything is impossible to tell without knowing where the original problem came from.

Sebastian commented

2021-01-12 20:46:17 +01:00

I was referring to tickets on the official Blender bug tracker.

Me too, sorry for the imprecise description.

If the problem wasn't previously reported on here, then it should be, so that we have all relevant information to reproduce and fix the issue.

Because of the non-deterministic behaviour and the dependency of large point clouds it was difficult to create reasonable (simple) test cases. But like I said, I'll have another look.

> I was referring to tickets on the official Blender bug tracker. Me too, sorry for the imprecise description. > If the problem wasn't previously reported on here, then it should be, so that we have all relevant information to reproduce and fix the issue. Because of the non-deterministic behaviour and the dependency of large point clouds it was difficult to create reasonable (simple) test cases. But like I said, I'll have another look.

Robert Guetzkow commented

2021-01-12 21:25:13 +01:00

@sbcv I see, then we've been misunderstanding each other. I have photogrammetry software and data around so if the overall steps to reproduce the problem are a bit more clear then this should be fine.

Sebastian commented

2021-02-16 20:16:47 +01:00

Sorry for the delay ... - I had to meet several dealines in the past few weeks.

Anyways, I've ran more tests regarding the bgl.GL_DEPTH_TEST related bug and I found that it is not related to the ticket that you linked earlier in this thread (i.e. blender/blender#83482.
I've created a new ticket including a simple test script to to reproduce the error here: blender/blender#85709

I'll address the other issues in the next few days.

Bests,
Sebastian

Sorry for the delay ... - I had to meet several dealines in the past few weeks. Anyways, I've ran more tests regarding the `bgl.GL_DEPTH_TEST` related bug and I found that it is **not** related to the ticket that you linked earlier in this thread (i.e. blender/blender#83482. I've created a new ticket including a simple test script to to reproduce the error here: blender/blender#85709 I'll address the other issues in the next few days. Bests, Sebastian

Sebastian commented

2021-02-17 19:36:06 +01:00

Added another ticket regarding the visibility of background images here: blender/blender#85750

Sebastian commented

2021-02-20 18:54:18 +01:00

The problem that (large) point clouds can not be correctly represented with particle systems is discussed in this ticket: blender/blender#81103

Sebastian commented

2021-02-28 23:05:40 +01:00

I've investigated the crashing issue and found that they have been caused by bpy.types.SpaceView3D.draw_handler_remove(...).
I've added a corresponding ticket here: blender/blender#86106

As a temporary workaround, I've commented out the bpy.types.SpaceView3D.draw_handler_remove(...) calls in the photogrammetry addon.
That means: In the latest version of the addon the deletion of the anchor objects (i.e. the Blender empties that are used to control the pose of the point cloud) does not longer remove the callbacks. However, the callbacks do not longer draw the point clouds and produce only a small overhead (since they only check if two variables are None).

I've investigated the crashing issue and found that they have been caused by `bpy.types.SpaceView3D.draw_handler_remove(...)`. I've added a corresponding ticket here: blender/blender#86106 As a temporary workaround, I've commented out the `bpy.types.SpaceView3D.draw_handler_remove(...)` calls in the photogrammetry addon. That means: In the latest version of the addon the deletion of the anchor objects (i.e. the Blender empties that are used to control the pose of the point cloud) does not longer remove the callbacks. However, the callbacks do not longer draw the point clouds and produce only a small overhead (since they only check if two variables are `None`).

Sebastian commented

2021-02-28 23:13:29 +01:00

Please find the latest release of the addon here: https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer/releases

I think I've addressed now all critical issues.
I would be pleased, if someone could provide some guidance how to proceed.

Please find the latest release of the addon here: https://github.com/SBCV/Blender-Addon-Photogrammetry-Importer/releases I think I've addressed now all critical issues. I would be pleased, if someone could provide some guidance how to proceed.

Robert Guetzkow commented

2021-02-28 23:37:22 +01:00

@sbcv We will investigate and try to fix the reported bugs. As far as I'm aware the module owner(s) will take a look at your add-on and will let you know if it will be included in Blender.

Sebastian changed title from ~~Blender Photogrammetry Importer~~ to Photogrammetry Importer Addon Proposal

2021-03-01 15:23:21 +01:00

hujin commented

2021-03-11 11:54:38 +01:00

Added subscriber: @2046411367

Sign in to join this conversation.