Implement a PLY importer/exporter in C++ #101294
Closed
opened 2022-09-23 11:59:46 +02:00 by Nathan Rozendaal
·
22 comments
No Branch/Tag Specified
main
blender-v3.6-release
temp-sculpt-dyntopo
temp-sculpt-dyntopo-hive-alloc
asset-shelf
brush-assets-project
blender-v3.3-release
tmp-usd-python-mtl
asset-browser-frontend-split
node-group-operators
blender-v2.93-release
universal-scene-description
temp-sculpt-attr-api
blender-v3.5-release
realtime-clock
sculpt-dev
gpencil-next
bevelv2
microfacet_hair
blender-projects-basics
xr-dev
principled-v2
v3.3.7
v2.93.18
v3.5.1
v3.3.6
v2.93.17
v3.5.0
v2.93.16
v3.3.5
v3.3.4
v2.93.15
v2.93.14
v3.3.3
v2.93.13
v2.93.12
v3.4.1
v3.3.2
v3.4.0
v3.3.1
v2.93.11
v3.3.0
v3.2.2
v2.93.10
v3.2.1
v3.2.0
v2.83.20
v2.93.9
v3.1.2
v3.1.1
v3.1.0
v2.83.19
v2.93.8
v3.0.1
v2.93.7
v3.0.0
v2.93.6
v2.93.5
v2.83.18
v2.93.4
v2.93.3
v2.83.17
v2.93.2
v2.93.1
v2.83.16
v2.93.0
v2.83.15
v2.83.14
v2.83.13
v2.92.0
v2.83.12
v2.91.2
v2.83.10
v2.91.0
v2.83.9
v2.83.8
v2.83.7
v2.90.1
v2.83.6.1
v2.83.6
v2.90.0
v2.83.5
v2.83.4
v2.83.3
v2.83.2
v2.83.1
v2.83
v2.82a
v2.82
v2.81a
v2.81
v2.80
v2.80-rc3
v2.80-rc2
v2.80-rc1
v2.79b
v2.79a
v2.79
v2.79-rc2
v2.79-rc1
v2.78c
v2.78b
v2.78a
v2.78
v2.78-rc2
v2.78-rc1
v2.77a
v2.77
v2.77-rc2
v2.77-rc1
v2.76b
v2.76a
v2.76
v2.76-rc3
v2.76-rc2
v2.76-rc1
v2.75a
v2.75
v2.75-rc2
v2.75-rc1
v2.74
v2.74-rc4
v2.74-rc3
v2.74-rc2
v2.74-rc1
v2.73a
v2.73
v2.73-rc1
v2.72b
2.72b
v2.72a
v2.72
v2.72-rc1
v2.71
v2.71-rc2
v2.71-rc1
v2.70a
v2.70
v2.70-rc2
v2.70-rc
v2.69
v2.68a
v2.68
v2.67b
v2.67a
v2.67
v2.66a
v2.66
v2.65a
v2.65
v2.64a
v2.64
v2.63a
v2.63
v2.61
v2.60a
v2.60
v2.59
v2.58a
v2.58
v2.57b
v2.57a
v2.57
v2.56a
v2.56
v2.55
v2.54
v2.53
v2.52
v2.51
v2.50
v2.49b
v2.49a
v2.49
v2.48a
v2.48
v2.47
v2.46
v2.45
v2.44
v2.43
v2.42a
v2.42
v2.41
v2.40
v2.37a
v2.37
v2.36
v2.35a
v2.35
v2.34
v2.33a
v2.33
v2.32
v2.31a
v2.31
v2.30
v2.28c
v2.28a
v2.28
v2.27
v2.26
v2.25
Labels
Clear labels
Issues relating to security: https://wiki.blender.org/wiki/Process/Vulnerability_Reports
Apply labels
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
Eevee & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest/Import
Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest: Wayland
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Issues relating to security: https://wiki.blender.org/wiki/Process/Vulnerability_Reports
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Eevee & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
Eevee & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest/Import
Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest: Wayland
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Eevee & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
Milestone
Set milestone
Clear milestone
No items
No Milestone
Projects
Set Project
Clear projects
No project
Assignees
Assign users
Clear assignees
No Assignees
12 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#101294
Reference in New Issue
There is no content yet.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
The problem
There is an importer/exporter for PLY already implemented in Blender, but it is implemented in Python which means it is slow for bigger files. The importer is also missing support for point cloud PLY files with vertex colors. Since PLY is often used for laser scan data (which are big files that contain a point cloud and sometimes vertex colors) it would be a shame not to address these issues.
The idea
We are a group of software engineering students from the university of applied sciences Windesheim in the Netherlands. And we would like to rewrite the current Python based importer/exporter to C++, with a focus on performance, extendibility and implementing missing functionality.
Requirements
Importer
Exporter
Nonfunctional requirements
Priorities
Our main focus will be implementing a fully featured importer, since we feel that will benefit Blender users the most. But having the importer in C++, and the exporter in Python is not optimal So we will split the development team into two parts, so we can work on the importer and exporter in parallel.
Possible additions
The PLY file format supports user-defined properties; we could make use of this fact by exposing these properties to geometry nodes as attributes. This would allow us to generically support all user defined properties (which we personally think is very cool). But some research needs to be done to see if this is practical and able to be implemented.
Scope
Since this is a student project, we have a definite deadline (20th of January). And after that deadline we won’t be able to work on the importer/exporter as much, if at all. @aras_p is willing to maintain the code after the project ends (if we send him some cookies), so we will be counting on him.
Testing strategy
For this project we are planning on utilizing test driven development. Detailed unit tests will be written for a large part of the added code. We will also gather a large amount of PLY files from different sources, to keep track of which files get loaded correctly. Both fine grain and coarse grain tests will be written.
If there is functionality missing in this design, or if you have other feedback. Please let us know.
Added subscribers: @aras_p, @super_jo_nathan
Looks good! Questions I have:
Thank you for your feedback! We have discussed your questions within our team.
Added subscriber: @GeorgiaPacific
Added subscriber: @Arjan-van-Diest
Added subscriber: @MichaelAProstka
Possible thing to also keep in mind: this abandoned Python patch, that's meant for improving PLY import support for some software that produces "slightly odd" file syntax. https://developer.blender.org/D14718
Added subscriber: @mont29
Project looks good to me, I would just add to the requirements something like "Match as closely as possible the behavior and results of current python add-on. Divergences, if any, should be clearly justified and documented."
Working on the importer first makes sense, but I would strongly recommend also considering the exporter as mandatory for this project. Would really rather not have a situation where we have a fully finished working importer in C++, and exporter still in python add-on.
Added subscriber: @KeirCB
Very excited for this,
In the possible additions section you discuss other attributes.
In point clouds generated with photogrammetry you generate the normals of the points. This is incredibly useful for mesh generation and is for me an integral attribute, I hope this can make it into the final release.
Added subscriber: @Lilith-Houtjes
@MichaelAProstka contacted us and gave some helpful pointers about what the current importer struggles with. So we are definitely keeping this in mind!
Thanks for the feedback! Where should these divergences be documented?
We will set the exporter as part of the minimum viable product.
Thanks for the feedback! We will try our best to support this. But we are sadly not able to guarantee this functionality.
In the design task first, then in the release notes once pushed to master.
Added subscriber: @T.R.O.Nunes
Added subscriber: @softyoda
Added subscriber: @Yoran-Huzen
Added subscriber: @bashendriks10
The project got done, reviewed, feedback addressed and just landed to main branch for Blender 3.6 (
43e9c90061
).Please remember to update the release notes. ;)Yikes please ignore, as @super_jo_nathan reminded me, this is for 3.6, not 3.5!