Prototype for portal/level/pages node groups #86583

New Issue

Dalai Felinto · 2021-03-15T14:23:59+01:00

Dalai Felinto commented

2021-03-15 14:23:59 +01:00

Mockup will eventually be available here:#86586

For the purpose of this prototype we will be using the terminology "pages". They were also referred as "layers", "levels" before.

Pages are local to a nodetree, a portal connects a page from another page inside the same nodetree.
Users should be able to click 1-20 and see different kind nodes.
Users should be able to move a node to a different page.
Users can create pairs of portals, to connect sockets from a page to another.
Each portal clearly indicates where its counterpart is.
Double-clicking (or tab) in a portal (or clicking on a button on it) changes to the page (and position) of the portal.

Sample file:
pebble_scattering_portals.blend

{F9898329, size=full}

{F9898326, size=full}

Branch: temp-node-tree-pages-prototype

Mockup will eventually be available here:#86586 For the purpose of this prototype we will be using the terminology "pages". They were also referred as "layers", "levels" before. * Pages are local to a nodetree, a portal connects a page from another page inside the same nodetree. * Users should be able to click 1-20 and see different kind nodes. * Users should be able to move a node to a different page. * Users can create pairs of portals, to connect sockets from a page to another. * Each portal clearly indicates where its counterpart is. * Double-clicking (or tab) in a portal (or clicking on a button on it) changes to the page (and position) of the portal. --- Sample file: [pebble_scattering_portals.blend](https://archive.blender.org/developer/F9898322/pebble_scattering_portals.blend) {[F9898329](https://archive.blender.org/developer/F9898329/image.png), size=full} {[F9898326](https://archive.blender.org/developer/F9898326/image.png), size=full} Branch: `temp-node-tree-pages-prototype`

Dalai Felinto commented

2021-03-15 14:23:59 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Dalai Felinto commented

2021-03-15 14:23:59 +01:00

Added subscriber: @dfelinto

michael knubben commented

2021-03-16 10:35:18 +01:00

First-time contributor

Added subscriber: @michaelknubben

Simon Sommer commented

2021-03-16 20:34:29 +01:00

First-time contributor

Added subscriber: @CreatorSiSo

Simon Sommer commented

2021-03-16 20:34:29 +01:00

First-time contributor

I think it would make sence to use the TAB key for changing to the page linked to the selected (like diving into a group node).

Simon Sommer commented

2021-03-16 20:39:04 +01:00

First-time contributor

Is this a redesign/replacement of Node Groups or something additional?

Kenzie commented

2021-03-16 21:18:37 +01:00

First-time contributor

Added subscriber: @KenzieMac130

Kenzie commented

2021-03-16 21:18:37 +01:00

First-time contributor

I still have cautions about this concept and how having different parts of a node graph on different pages might be not helpful. Blender currently is in the right direction IMO to provide useful, organized node functionality between splitting functionality into managable layers with the modifier stack, to node groups, to the upcoming attribute processor design.

Modifiers would offer a way for the artist to layer and apply operations built out of nodes to their model.

Node groups already offer a way to define functions that can be re-used accross the project.

Attribute processors sort of cracks open the geometry itself and lets the user play with it's attributes as if it was a material.

These three levels offer a method of organization, modularity, and access that can be easily understood by anyone jumping into nearly node graph. And if the asset manager was able to store and link modifiers and nodes from these levels then scaling this to work with large projects would be straightforward.

Currently understanding logic is just about "diving deeper" into the graph not "flipping pages". Introducing the latter just because it might have sounded good on paper but may likely turn out making everything much more difficult and scattered. Portals just seem a lot like a goto

I still have cautions about this concept and how having different parts of a node graph on different pages might be not helpful. Blender currently is in the right direction IMO to provide useful, organized node functionality between splitting functionality into managable layers with the modifier stack, to node groups, to the upcoming attribute processor design. Modifiers would offer a way for the artist to layer and apply operations built out of nodes to their model. Node groups already offer a way to define functions that can be re-used accross the project. Attribute processors sort of cracks open the geometry itself and lets the user play with it's attributes as if it was a material. These three levels offer a method of organization, modularity, and access that can be easily understood by anyone jumping into nearly node graph. And if the asset manager was able to store and link modifiers and nodes from these levels then scaling this to work with large projects would be straightforward. Currently understanding logic is just about "diving deeper" into the graph not "flipping pages". Introducing the latter just because it might have sounded good on paper but may likely turn out making everything much more difficult and scattered. Portals just seem a lot like a `goto`

Leon Schittek commented

2021-03-16 21:35:34 +01:00

Added subscriber: @lone_noel

Simon Sommer commented

2021-03-17 00:00:46 +01:00

First-time contributor

How would make this work would be by having in and output nodes that allow you to select a page to import/export stuff to:

And a higher level graph for linking the pages inside a object (you would have to set your in and outputs for the object in the in and out pages):

But I actuall think that this not a system that should be added in geometry nodes because group nodes just work fine for the time beeing, but rather something that we should add as a scene/object tree to connect several geometry node trees over different objects in a procedural way (the over all everything nodes tree).

How would make this work would be by having in and output nodes that allow you to select a page to import/export stuff to: ![Generate Rocks.png](https://archive.blender.org/developer/F9895605/Generate_Rocks.png) And a higher level graph for linking the pages inside a object (you would have to set your in and outputs for the object in the in and out pages): ![Portal Graph.png](https://archive.blender.org/developer/F9895606/Portal_Graph.png) But I actuall think that this not a system that should be added in geometry nodes because group nodes just work fine for the time beeing, but rather something that we should add as a **scene**/object **tree** to connect several geometry node trees over different objects in a procedural way (the over all everything nodes tree).

Kenzie commented

2021-03-17 01:13:51 +01:00

First-time contributor

In #86583#1130914, @CreatorSiSo wrote:
How would make this work would be by having in and output nodes that allow you to select a page to import/export stuff to:

And a higher level graph for linking the pages inside a object (you would have to set your in and outputs for the object in the in and out pages):

But I actuall think that this not a system that should be added in geometry nodes because group nodes just work fine for the time beeing, but rather something that we should add as a scene/object tree to connect several geometry node trees over different objects in a procedural way (the over all everything nodes tree).

A scene/object graph makes a lot more sense as you could direct everything at a higher level. This opens up have importer/exporter scripts, manage cameras and texture baking/render to texture, simulations, applinks, geometry gen inputs, outputs, dependencies, etc.

This would be desperately needed inside blender as workflows can be rather manual and tedious. Automation is definitely the future. But yeah scattering geometry IO all over a bunch of objects throughout the scene with portals would still make more of a mess. Perhaps a "director" nodegraph is really what is needed here in the future.

> In #86583#1130914, @CreatorSiSo wrote: > How would make this work would be by having in and output nodes that allow you to select a page to import/export stuff to: > > ![Generate Rocks.png](https://archive.blender.org/developer/F9895605/Generate_Rocks.png) > > And a higher level graph for linking the pages inside a object (you would have to set your in and outputs for the object in the in and out pages): > > ![Portal Graph.png](https://archive.blender.org/developer/F9895606/Portal_Graph.png) > > But I actuall think that this not a system that should be added in geometry nodes because group nodes just work fine for the time beeing, but rather something that we should add as a **scene**/object **tree** to connect several geometry node trees over different objects in a procedural way (the over all everything nodes tree). A scene/object graph makes a lot more sense as you could direct everything at a higher level. This opens up have importer/exporter scripts, manage cameras and texture baking/render to texture, simulations, applinks, geometry gen inputs, outputs, dependencies, etc. This would be desperately needed inside blender as workflows can be rather manual and tedious. Automation is definitely the future. But yeah scattering geometry IO all over a bunch of objects throughout the scene with portals would still make more of a mess. Perhaps a "director" nodegraph is really what is needed here in the future.

Simon Sommer commented

2021-03-17 09:44:32 +01:00

First-time contributor

Maybe a feature for sending differnt versions of the same Geomtry between Objects would make sence but i think that this isnt a feature we should think about before any higher level structure is in place.

Dalai Felinto commented

2021-03-17 12:53:02 +01:00

I clarified the explanation above. Pages are local to the nodetrees. Portals are intended to connect sockets from a page to another page in the same nodetree. This was not clear before.

I clarified the explanation above. Pages are local to the nodetrees. Portals are intended to connect sockets from a page to another page in the *same* nodetree. This was not clear before.

Kenzie commented

2021-03-17 20:26:35 +01:00

First-time contributor

In #86583#1131168, @dfelinto wrote:
I clarified the explanation above. Pages are local to the nodetrees. Portals are intended to connect sockets from a page to another page in the same nodetree. This was not clear before.

Thank you for the clarification. My criticism about the portal/page concept being unfortunately similar to a goto in programming still stands. I feel as though node groups still offer the best way forward to a structured functional approach that can still scale to very large projects. Would be nicer to see the ability to define scoped/local node groups to not clutter the file. I don't really see the pages/portal approach as a clean solution to anything.

> In #86583#1131168, @dfelinto wrote: > I clarified the explanation above. Pages are local to the nodetrees. Portals are intended to connect sockets from a page to another page in the *same* nodetree. This was not clear before. Thank you for the clarification. My criticism about the portal/page concept being unfortunately similar to a `goto` in programming still stands. I feel as though node groups still offer the best way forward to a `structured` `functional` approach that can still scale to very large projects. Would be nicer to see the ability to define scoped/local node groups to not clutter the file. I don't really see the pages/portal approach as a clean solution to anything.

Brecht Van Lommel commented

2021-03-18 00:40:41 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2021-03-18 00:40:41 +01:00

I'm missing a problem statement here, it's not clear to me which problem this is intended to solve.

Dalai Felinto commented

2021-03-18 09:58:27 +01:00

Hi @brecht , it is intended to allow high levels of complexity without the excessive need of more and more scrolling to navigate the nodetree. While nodegroups can be used for that they don't allow for linking of sockets from its internal components to inside other node groups. So nodegroups are just abstraction/replacements of a few nodes, while this can be used to slice up the tree and let users think of a part of the problem at a time.

Jacques Lucke self-assigned this 2021-03-18 11:43:59 +01:00

Dalai Felinto commented

2021-03-18 18:04:42 +01:00

I created a test file from the temp-node-tree-pages-prototype branch and the pebbles distribution sample file:
pebble_scattering_portals.blend

{F9898329, size=full}

{F9898326, size=full}

I created a test file from the `temp-node-tree-pages-prototype` branch and the pebbles distribution sample file: [pebble_scattering_portals.blend](https://archive.blender.org/developer/F9898322/pebble_scattering_portals.blend) {[F9898329](https://archive.blender.org/developer/F9898329/image.png), size=full} {[F9898326](https://archive.blender.org/developer/F9898326/image.png), size=full}

Kenzie commented

2021-03-18 23:20:13 +01:00

First-time contributor

How and where is this any more effective at organization than properly utilizing the already existing solutions to graph organization?

Take frames for instance. It's a flat graph where the user can frame and comment blocks of functionality. These blocks can give the graph structure even when zoomed out and makes it easy for people to inspect graphs that they might not be familiar with for their overall structure at a glance. Pages break this structure of the graph.

it is intended to allow high levels of complexity without the excessive need of more and more scrolling to navigate the nodetree.

I feel as though most users would have a better time zooming out and tracing a large graph with 20 frames than they would tracing and flipping back and forward through 20 pages of smaller graphs every time they hit a portal. Scrolling isn't exactly the problem, either way you are going to have to navigate that graph. While the pebbles demo is small it is still given a little more clarity with frames.

Then you have groups which are basically just functions. Taking that existing pebble node graph and remove the node duplication by making effective use of groups it and you end up with something much cleaner and re-usable.

To me pages/portals seem most like they are trying to be groups, but with massive caveats. Besides the previously described: in the current flat/nested-node-graph approach if a user wants to take a snippet from another graph and use it elsewhere they just need to box select the region of the graph they are interested in and copy/paste. With pages you now have to deal with hunting down and tracing the connections to copy and paste or flatten them onto your graph. The user can still get lost in the current system but if they do they can always zoom out and follow their tracks with breadcrumbs. With portals/pages you have literally added new dimensions of complexity that the user has to keep track of.

While nodegroups can be used for that they don't allow for linking of sockets from its internal components to inside other node groups.

In most programming languages you don't exactly call only a second half of a function and mess with it's locals from the outside either. You can still nest groups inside of other groups when you need to minimize node duplication. It's probably better that larger projects be organized into blocks like this anyways.

Pages and portals seem to introduce a lot more problems than they solve in the current example. It would be nice to see a more clear demonstration of what real world problems they specifically solve alongside effective use of existing solutions rather than what they might just look like on their own.

How and where is this any more effective at organization than properly utilizing the already existing solutions to graph organization? Take frames for instance. It's a flat graph where the user can frame and comment blocks of functionality. These blocks can give the graph structure even when zoomed out and makes it easy for people to inspect graphs that they might not be familiar with for their overall structure at a glance. Pages break this structure of the graph. > it is intended to allow high levels of complexity without the excessive need of more and more scrolling to navigate the nodetree. I feel as though most users would have a better time zooming out and tracing a large graph with 20 frames than they would tracing and flipping back and forward through 20 pages of smaller graphs every time they hit a portal. Scrolling isn't exactly the problem, either way you are going to have to navigate that graph. While the pebbles demo is small it is still given a little more clarity with frames. ![image.png](https://archive.blender.org/developer/F9898642/image.png) Then you have groups which are basically just functions. Taking that existing pebble node graph and remove the node duplication by making effective use of groups it and you end up with something much cleaner and re-usable. ![Groups.PNG](https://archive.blender.org/developer/F9898650/Groups.PNG) ![GroupInterrior.PNG](https://archive.blender.org/developer/F9898653/GroupInterrior.PNG) To me pages/portals seem most like they are trying to be groups, but with massive caveats. Besides the previously described: in the current flat/nested-node-graph approach if a user wants to take a snippet from another graph and use it elsewhere they just need to box select the region of the graph they are interested in and copy/paste. With pages you now have to deal with hunting down and tracing the connections to copy and paste or flatten them onto your graph. The user can still get lost in the current system but if they do they can always zoom out and follow their tracks with breadcrumbs. With portals/pages you have literally added new dimensions of complexity that the user has to keep track of. > While nodegroups can be used for that they don't allow for linking of sockets from its internal components to inside other node groups. In most programming languages you don't exactly call only a second half of a function and mess with it's locals from the outside either. You can still nest groups inside of other groups when you need to minimize node duplication. It's probably better that larger projects be organized into blocks like this anyways. Pages and portals seem to introduce a lot more problems than they solve in the current example. It would be nice to see a more clear demonstration of what real world problems they specifically solve alongside effective use of existing solutions rather than what they might just look like on their own.

Charlie Jolly commented

2021-03-19 01:17:04 +01:00

Added subscriber: @CharlieJolly

Charlie Jolly commented

2021-03-19 01:17:04 +01:00

@KenzieMac130 At least with the prototype there can be practical feedback to say if it works better than node groups rather than just remain an idea. There are lots of ideas to improve nodes when working with large trees. From the top of my head, mini-maps, split views, collapsible frames, sticky inputs and outputs (edge of editor window), popup nodes when wiring etc....

Kenzie commented

2021-03-19 03:46:00 +01:00

First-time contributor

@CharlieJolly Those improvements would be very welcome, I see the minimap has already started. It just seems like this task is being designed in isolation from best practices of the existing organization tools. I would like to see some interesting results and insights come out of this prototype as well, it just seems the design needs to try to acknowledge where the other tools work and find out where it best fits in.

Albert commented

2021-03-20 18:07:45 +01:00

First-time contributor

Added subscriber: @wevon-2

Albert commented

2021-03-20 18:07:45 +01:00

First-time contributor

A few days ago I made a couple of proposals on rightclickselect.

The first was the use of portals:
https://blender.community/c/rightclickselect/LMgbbc/

And after reflecting, a second proposal, instantiate nodes and groups of nodes:
https://blender.community/c/rightclickselect/2Ngbbc/

I think instantiating nodegroups is equivalent to portals.
The layout of the proposal in right click select is not good, but I think it can be understood. Solution D in my opinion is the good one.

If any clarification or improvement of the document is missing, I can do it. For reflection I think that is enough.

A few days ago I made a couple of proposals on rightclickselect. The first was the use of portals: https://blender.community/c/rightclickselect/LMgbbc/ And after reflecting, a second proposal, instantiate nodes and groups of nodes: https://blender.community/c/rightclickselect/2Ngbbc/ I think instantiating nodegroups is equivalent to portals. The layout of the proposal in right click select is not good, but I think it can be understood. Solution D in my opinion is the good one. If any clarification or improvement of the document is missing, I can do it. For reflection I think that is enough.

Dalai Felinto commented

2021-03-22 14:56:28 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Dalai Felinto closed this issue

2021-03-22 14:56:28 +01:00

Hadrien Brissaud commented

2021-03-22 18:20:08 +01:00

First-time contributor

Added subscriber: @hadrien

Hadrien Brissaud commented

2021-03-22 18:20:08 +01:00

First-time contributor

I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ?

Duarte Farrajota Ramos commented

2021-03-22 18:36:38 +01:00

First-time contributor

Added subscriber: @DuarteRamos

Simon Sommer commented

2021-03-22 19:47:31 +01:00

First-time contributor

In #86583#1134241, @hadrien wrote:
I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ?

The minimap is already a WIP: https://developer.blender.org/D10776

> In #86583#1134241, @hadrien wrote: > I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ? The minimap is already a WIP: https://developer.blender.org/D10776

nameless commented

2021-03-23 19:39:48 +01:00

First-time contributor

Added subscriber: @tonpix

nameless commented

2021-03-23 19:39:48 +01:00

First-time contributor

In #86583#1134241, @hadrien wrote:
I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ?

The pathing between frames/reroutes can make it a visual clutter very easily.

Big +1 for portals from me.

> In #86583#1134241, @hadrien wrote: > I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ? The pathing between frames/reroutes can make it a visual clutter very easily. Big +1 for portals from me.

Charlie Jolly commented

2021-03-23 20:02:21 +01:00

In #86583#1134241, @hadrien wrote:
I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ?

Remember that using this feature will be optional.

> In #86583#1134241, @hadrien wrote: > I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ? Remember that using this feature will be optional.

Bartosz Moniewski commented

2021-03-23 22:01:05 +01:00

First-time contributor

Added subscriber: @BartekMoniewski

Kenzie commented

2021-03-23 23:17:13 +01:00

First-time contributor

In #86583#1134810, @CharlieJolly wrote:

In #86583#1134241, @hadrien wrote:
I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ?

Remember that using this feature will be optional.

It's "optional" until you have to figure out and work with somebody else's node graph.

> In #86583#1134810, @CharlieJolly wrote: >> In #86583#1134241, @hadrien wrote: >> I don't know, doesn't this seem more complicated than the problem it tries to solve ? Don't frames and reroutes already provide all the organizing functionality ? Is scrolling actually an issue ? Maybe a minimap could help in that regard ? > > Remember that using this feature will be optional. It's "optional" until you have to figure out and work with somebody else's node graph.

Paul Larson commented

2021-11-11 04:41:06 +01:00

First-time contributor

Added subscriber: @GeorgiaPacific

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Prototype for portal/level/pages node groups #86583