Initial Particle Nodes #68636

New Issue

Jacques Lucke · 2019-08-14T11:16:31+02:00

Jacques Lucke commented

2019-08-14 11:16:31 +02:00

This document lists what needs to be done to get an initial version of Particle Nodes in master. The merge is currently scheduled to be part of Blender 2.82.

The function branch consists of four main parts:

C++ libraries in blenlib. (depends on nothing)
Python code that defines the user interface including the nodes. (depends on nothing)
The functions framework with a runtime type system that supports multiple backends. (depends on blenlib and the python ui)
The bparticles system that parses a particle node tree and performs the simulation. (depends on blenlib, functions and the python ui)

Libraries

While I have some changes planned (especially for the hash maps/sets), this code can go in at any time. A significant portion of this code is covered by unit tests. I happily continue improving and maintaining it when it is in master.

Also see:

Motivation for new data structures in blenlib: https://wiki.blender.org/wiki/User:JacquesLucke/Documents/BasicDataStructures

Nodes UI

As mentioned, nodes are currently implemented in Python. I happily move this to C/C++ at some point, but it does not seem worth it right now.

Also see:

Proposal for a node ui framework that is less static than the one used by other node systems in Blender: https://wiki.blender.org/wiki/Source/Nodes/NodeInterfaceFramework

Functions

The function system consists of a couple of components which can be categorized into:

Backend: This is purely C++ and does not depend on the user interface at all.
Frontend: This takes a node tree as input and creates an internal data flow graph from it that can be handled by the backend.

The backend can be merged as soon as the libraries are merged. It should not impact anything else.

For the frontend to make sense, the node ui needs to be merged as well. This could be merged without actually exposing a new node tree type to the user.
Currently, there are also two new modifiers that I use for testing of the function system. I do not expect them to be merged.

Also see:

Original document explaining the need for a function system: https://wiki.blender.org/wiki/Source/Nodes/EverythingNodes
Slightly outdated but still mostly accurate explanation of the function system: https://wiki.blender.org/wiki/Source/Nodes/InitialFunctionsSystem

Particle System

While these other parts could almost be merged as they are, I have to spend more time on the actual particle system.
One particle simulation backend that is sufficient for a variety of particle effects is working, but the user interface is still lacking.

These things still have to be worked out:

How to structure the node tree exactly? I had an original proposal, but there are some weak points that need to be solved (see link below).
Where does the output of the simulation go? Currently, the particles are just output on a mesh as vertices. This might depend on the introduction of a new point cloud data type. Also, the output of the particle simulation might be particles of different types, that users will want to use different materials for.
Which nodes do we actually want to have? Currently, most nodes are just for testing and don't offer enough options for artists. I'll create separate design tasks for different categories soon.
How to integrate this with the current cache system? The problems of the current caching system are well known. There is probably not enough time to implement a new system before Blender 2.82.

Possible compromises to get this to merge sooner (not saying that we should not do these things, but we might need to trade-off features vs. time-to-market here):

No node group support. While it is not too difficult to implement node groups, it is still unclear how they should work exactly (see link below). We could postpone that until we have a better idea of how artists want to group things.
No integration with the current caching system. There is already very simple caching in the modifier directly. That could be made more usable. Then we could implement a new caching system a bit later and use it when it is ready.
No ability to use separate materials for different particle types. It should still be possible to get the particle type in a shader via an attribute, to shade different types differently nevertheless.

Also see:

Original node ui proposal: https://wiki.blender.org/wiki/Source/Nodes/ParticleSystemNodes
Weak points of the original proposal explained and a possible solution: https://wiki.blender.org/wiki/Source/Nodes/UpdatedParticleNodesUI
Yet another updated UI: https://wiki.blender.org/wiki/Source/Nodes/UpdatedParticleNodesUI2
Rough design of the actual simulation code: https://wiki.blender.org/wiki/Source/Nodes/ParticleSystemCodeArchitecture
Overview of different kinds of caches we need: https://wiki.blender.org/wiki/Source/Nodes/Caching
Proposal for new object types: https://wiki.blender.org/wiki/Source/Nodes/Object_Types

This document lists what needs to be done to get an initial version of Particle Nodes in master. The merge is currently scheduled to be part of Blender 2.82. The function branch consists of four main parts: 1. C++ libraries in blenlib. (depends on nothing) 2. Python code that defines the user interface including the nodes. (depends on nothing) 3. The functions framework with a runtime type system that supports multiple backends. (depends on blenlib and the python ui) 4. The bparticles system that parses a particle node tree and performs the simulation. (depends on blenlib, functions and the python ui) ## Libraries While I have some changes planned (especially for the hash maps/sets), this code can go in at any time. A significant portion of this code is covered by unit tests. I happily continue improving and maintaining it when it is in master. Also see: - Motivation for new data structures in blenlib: https://wiki.blender.org/wiki/User:JacquesLucke/Documents/BasicDataStructures ## Nodes UI As mentioned, nodes are currently implemented in Python. I happily move this to C/C++ at some point, but it does not seem worth it right now. Also see: - Proposal for a node ui framework that is less static than the one used by other node systems in Blender: https://wiki.blender.org/wiki/Source/Nodes/NodeInterfaceFramework ## Functions The function system consists of a couple of components which can be categorized into: - Backend: This is purely C++ and does not depend on the user interface at all. - Frontend: This takes a node tree as input and creates an internal data flow graph from it that can be handled by the backend. The backend can be merged as soon as the libraries are merged. It should not impact anything else. For the frontend to make sense, the node ui needs to be merged as well. This could be merged without actually exposing a new node tree type to the user. Currently, there are also two new modifiers that I use for testing of the function system. I do not expect them to be merged. Also see: - Original document explaining the need for a function system: https://wiki.blender.org/wiki/Source/Nodes/EverythingNodes - Slightly outdated but still mostly accurate explanation of the function system: https://wiki.blender.org/wiki/Source/Nodes/InitialFunctionsSystem ## Particle System While these other parts could almost be merged as they are, I have to spend more time on the actual particle system. One particle simulation backend that is sufficient for a variety of particle effects is working, but the user interface is still lacking. These things still have to be worked out: - How to structure the node tree exactly? I had an original proposal, but there are some weak points that need to be solved (see link below). - Where does the output of the simulation go? Currently, the particles are just output on a mesh as vertices. This might depend on the introduction of a new point cloud data type. Also, the output of the particle simulation might be particles of different types, that users will want to use different materials for. - Which nodes do we actually want to have? Currently, most nodes are just for testing and don't offer enough options for artists. I'll create separate design tasks for different categories soon. - How to integrate this with the current cache system? The problems of the current caching system are well known. There is probably not enough time to implement a new system before Blender 2.82. Possible compromises to get this to merge sooner (not saying that we should not do these things, but we might need to trade-off features vs. time-to-market here): - No node group support. While it is not too difficult to implement node groups, it is still unclear how they should work exactly (see link below). We could postpone that until we have a better idea of how artists want to group things. - No integration with the current caching system. There is already very simple caching in the modifier directly. That could be made more usable. Then we could implement a new caching system a bit later and use it when it is ready. - No ability to use separate materials for different particle types. It should still be possible to get the particle type in a shader via an attribute, to shade different types differently nevertheless. Also see: - Original node ui proposal: https://wiki.blender.org/wiki/Source/Nodes/ParticleSystemNodes - Weak points of the original proposal explained and a possible solution: https://wiki.blender.org/wiki/Source/Nodes/UpdatedParticleNodesUI - Yet another updated UI: https://wiki.blender.org/wiki/Source/Nodes/UpdatedParticleNodesUI2 - Rough design of the actual simulation code: https://wiki.blender.org/wiki/Source/Nodes/ParticleSystemCodeArchitecture - Overview of different kinds of caches we need: https://wiki.blender.org/wiki/Source/Nodes/Caching - Proposal for new object types: https://wiki.blender.org/wiki/Source/Nodes/Object_Types

Jacques Lucke self-assigned this 2019-08-14 11:16:31 +02:00

Jacques Lucke commented

2019-08-14 11:16:31 +02:00

Added subscribers: @JacquesLucke, @brecht

Philipp Oeser commented

2019-08-14 11:21:00 +02:00

Added subscriber: @lichtwerk

Omar Emara commented

2019-08-14 11:38:13 +02:00

Added subscriber: @OmarEmaraDev

Demeter Dzadik commented

2019-08-14 11:57:25 +02:00

Added subscriber: @Mets

Demeter Dzadik commented

2019-08-14 11:57:25 +02:00

I absolutely love the amount of information you're making available about how these systems work/will work and how everything is coming along. I played around with the build yesterday and it looked pretty awesome! ^^ I managed to make some particles bounce off a plane :)

I feel like to answer some of these questions, you should put a "beta version" in front of artists, and advertise it(Blender Today), and see what they do with it and what they have to say about it.

My understanding was always that this is planned to be merged no sooner than 2.82, if we are lucky. It sounds like you want to get it in sooner, but I for one, am happy to wait for something that's more polished. Take your time man.

I absolutely love the amount of information you're making available about how these systems work/will work and how everything is coming along. I played around with the build yesterday and it looked pretty awesome! ^^ I managed to make some particles bounce off a plane :) I feel like to answer some of these questions, you should put a "beta version" in front of artists, and advertise it(Blender Today), and see what they do with it and what they have to say about it. My understanding was always that this is planned to be merged no sooner than 2.82, if we are lucky. It sounds like you want to get it in sooner, but I for one, am happy to wait for something that's more polished. Take your time man.

Adam Friesen commented

2019-08-14 18:56:53 +02:00

Added subscriber: @Ace_Dragon

Adam Friesen commented

2019-08-14 18:56:53 +02:00

The compromises to get this in sooner seem a bit harsh in my book, so waiting until 2.82 at the earliest (especially with the 3 month release schedule) appears reasonable.