USD Hydra render engine #100892

New Issue

Details on the design for the Hydra render engine enablement:

This system will include a system for using the [USD Hydra render engine ]] inside of Blender. This will allowed simplified render addons, by renderers compiling a Hydra Render Delegate against the USD libs which will be included with Blender in [ https:*developer.blender.org/T99618 | 3.5 . Additionally, render export will be done on the C++ side, so it should be better export performance than using python.

Render Delegates will install like a normal render engine add-on (via python), but will subclass the HydraRenderEngine class instead of Render Engine. This will do a few things, but mainly point the Hydra system (on the C++ side) where to load a compiled Hydra render delegate from. Then at render time, that render delegate will be used for rendering.

Addons can specify UI and settings. An overridable method on HydraRenderEngine will allow engines to send USD settings to the Render Delegate.

Blender to Hydra data export will happen on the C++ side using a Hydra Scene Delegate object.

Pieces to make this work:

HydraRenderEngine type for python api - overridable for render addons, sets up render and points to Hydra plugin (dll) for renderer.
Scene Delegate (C++) which exports Blender data to Hydra

Hydra Engine (C++) when a Hydra renderer is selected, ties together the Scene Delegate and Render Delegate and executes the render.

Includes HDStorm (GL or Metal) render delegate which comes with Hydra. Users can install 3rd party render delegates via add-on preferences
Enables final and viewport rendering like other renderers in Blender. Users can select in render properties the render delegate to use. Users can select different render delegate for viewport vs final renders.
Blender Material Nodegraphs are automatically converted to USD just as exporting USD works now. Future changes (in a separate design) will use MaterialX for this
Lights and all geometry will be supported as well.

Details on the design for the Hydra render engine enablement: This system will include a system for using the [USD Hydra render engine ]] inside of Blender. This will allowed simplified render addons, by renderers compiling a Hydra Render Delegate against the USD libs which will be included with Blender in [[ https:*developer.blender.org/T99618 | 3.5 ](https:*graphics.pixar.com/usd/files/Siggraph2019_Hydra.pdf). Additionally, render export will be done on the C++ side, so it should be better export performance than using python. Render Delegates will install like a normal render engine add-on (via python), but will subclass the `HydraRenderEngine` class instead of `Render Engine`. This will do a few things, but mainly point the Hydra system (on the C++ side) where to load a compiled Hydra render delegate from. Then at render time, that render delegate will be used for rendering. Addons can specify UI and settings. An overridable method on HydraRenderEngine will allow engines to send USD settings to the Render Delegate. Blender to Hydra data export will happen on the C++ side using a Hydra Scene Delegate object. Pieces to make this work: - HydraRenderEngine type for python api - overridable for render addons, sets up render and points to Hydra plugin (dll) for renderer. - Scene Delegate (C++) which exports Blender data to Hydra # Hydra Engine (C++) when a Hydra renderer is selected, ties together the Scene Delegate and Render Delegate and executes the render. - Includes HDStorm (GL or Metal) render delegate which comes with Hydra. Users can install 3rd party render delegates via add-on preferences - Enables final and viewport rendering like other renderers in Blender. Users can select in render properties the render delegate to use. Users can select different render delegate for viewport vs final renders. - Blender Material Nodegraphs are automatically converted to USD just as exporting USD works now. Future changes (in a separate design) will use MaterialX for this - Lights and all geometry will be supported as well.

Added subscribers: @BrianSavery, @SonnyCampbell_Unity, @AlexeyAdamitsky, @DagerD, @Peine_Perdue, @Plyro, @EstebanCovo, @satishgoda1, @ianhsieh, @ahmed.hindy96, @JulianEisel, @Zhen-Dai, @paulgolter, @Scaredyfish, @JamesBarrie, @lichtwerk, @platerytter, @Yuro, @Roggii-4, @Alaska, @makowalski, @LazyDodo, @brecht, @dr.sybren

Added subscriber: @silex

From the user point of view I think it would be better not to have these two levels where you first select the Hydra render engine, and then select the delegate.

Another better approach could be to register each Hydra delegate as a render engine. With Python it should be possible to dynamically generate a subclass of a HydraRenderEngine class. Renderers that want to more closely integrate with Blender could have an add-on that subclasses HydraRenderEngine manually.

That would simplify the UI for users, make the -E command line option work, use the same mechanism for panel visibility as existing engines, etc.

One potential downside is that scanning the delegates may be expensive and negatively affect startup time. But I would hope that this involves USD just scanning a few plugInfo.json files without loading the actual shared libraries.

From the user point of view I think it would be better not to have these two levels where you first select the Hydra render engine, and then select the delegate. Another better approach could be to register each Hydra delegate as a render engine. With Python it should be possible to dynamically generate a subclass of a `HydraRenderEngine` class. Renderers that want to more closely integrate with Blender could have an add-on that subclasses `HydraRenderEngine` manually. That would simplify the UI for users, make the `-E` command line option work, use the same mechanism for panel visibility as existing engines, etc. One potential downside is that scanning the delegates may be expensive and negatively affect startup time. But I would hope that this involves USD just scanning a few `plugInfo.json` files without loading the actual shared libraries.

It's an interesting thought. Two potential issues I see:

Installing a Hydra render delegate is somewhat different than installing a "normal" Blender Addon in that where things go and what it manages is different. Right now we have it that in the Hydra add-on preferences you can install a render delegate with a dialog.
It's somewhat nice to allow one render engine for viewport and another for final rendering. with Hydra you should get similar results, but this is somewhat similar to the material preview mode in the current blender viewport vs rendered viewport mode.

Actually the way I could see this working would be that an Render Delegate "Add-on" could be similar to a normal blender add-on (but more minimal) and include in a zip file:

| - __init__.py - Register any classes for UI, render properties, and a subclass of HydraRenderEngine
| - engine.py  -   class MyHydraRenderEngine(HydraRenderEngine):

                                          - some property that tells Hydra where there the plugInfo.json and plugin directory is to install

                                    def get_usd_render_settings(self, context):
                                          """ Returns the USD render settings for this delegate, based on blender properties """

| - plugInfo.json
| - hydra_delegate_directory / - to be installed when MyHydraRenderEngine is registered.

The potential downside here is that you're making a new system for developers to add render engines, BUT there are two upsides:

They amount of code you'd have to write as a developer is minimal at most. Would open up some other renderers easily for Blender.
The render engine could then act like a "traditional" render add-on to blender, at least from the user perspective.

It's an interesting thought. Two potential issues I see: 1. Installing a Hydra render delegate is somewhat different than installing a "normal" Blender Addon in that where things go and what it manages is different. Right now we have it that in the Hydra add-on preferences you can install a render delegate with a dialog. 2. It's somewhat nice to allow one render engine for viewport and another for final rendering. with Hydra you should get similar results, but this is somewhat similar to the material preview mode in the current blender viewport vs rendered viewport mode. Actually the way I could see this working would be that an Render Delegate "Add-on" could be similar to a normal blender add-on (but more minimal) and include in a zip file: ``` HydraRenderDelegate.zip ``` | - __init__.py - Register any classes for UI, render properties, and a subclass of HydraRenderEngine | - engine.py - class MyHydraRenderEngine(HydraRenderEngine): ``` - some property that tells Hydra where there the plugInfo.json and plugin directory is to install ``` def get_usd_render_settings(self, context): """ Returns the USD render settings for this delegate, based on blender properties """ | - plugInfo.json | - hydra_delegate_directory / - to be installed when MyHydraRenderEngine is registered. ``` ``` The potential downside here is that you're making a new system for developers to add render engines, BUT there are two upsides: 1. They amount of code you'd have to write as a developer is minimal at most. Would open up some other renderers easily for Blender. 2. The render engine could then act like a "traditional" render add-on to blender, at least from the user perspective.

This would also answer the question @ianhsieh had about renderers wanting to handle UI of settings and put the settings into the USD stream.

Added subscribers: @fclem, @Jeroen-Bakker

I think the installation procedure being the same as any other add-on would be a benefit, especially if in the future we have an official add-ons repository, update mechanism, etc. It's something users are already familiar with. I think it would be useful if render delegates get registered automatically if they are available in the USD plugin paths, but for a nice user interface an actual add-on seems best.

More control over which engine to use for the viewport makes sense, I don't think it's tied to using add-ons or not. But the design for that is not obvious. Up to now render engines other than Workbench/Eevee are only used in the Rendered shading mode, aimed at the use case of shading and lighting. GLStorm would be useful for more use cases, I guess mainly in object mode setting up scenes, doing procedural modeling, etc. Modeling, sculpting and painting are probably not so practical. But even if we consider just object mode, you still need overlays and selection to work, which at the moment means loading the full scene into Blender.

So I'm not sure if those kinds of use cases are something you already want to address, or if it's just about having GLStorm used in Rendered shading mode

I think the installation procedure being the same as any other add-on would be a benefit, especially if in the future we have an official add-ons repository, update mechanism, etc. It's something users are already familiar with. I think it would be useful if render delegates get registered automatically if they are available in the USD plugin paths, but for a nice user interface an actual add-on seems best. More control over which engine to use for the viewport makes sense, I don't think it's tied to using add-ons or not. But the design for that is not obvious. Up to now render engines other than Workbench/Eevee are only used in the Rendered shading mode, aimed at the use case of shading and lighting. GLStorm would be useful for more use cases, I guess mainly in object mode setting up scenes, doing procedural modeling, etc. Modeling, sculpting and painting are probably not so practical. But even if we consider just object mode, you still need overlays and selection to work, which at the moment means loading the full scene into Blender. So I'm not sure if those kinds of use cases are something you already want to address, or if it's just about having GLStorm used in Rendered shading mode

Added subscriber: @GeorgiaPacific

@brecht and @Jeroen-Bakker updated the design description. Let me know if you have questions.

The design described in the task sounds good.

How exactly the scene delegate will work is not obvious to me. I assume that's using the code source/blender/io/usd module to avoid duplication, but some design changes are probably needed to make that work well? Particularly for doing incremental updates by tracking changes in the Blender scene depsgraph.

The Hydra Engine implementation I assume will be an implementation of a RenderEngineType, in render/intern/hydra_engine.cc or something like that?

The design described in the task sounds good. How exactly the scene delegate will work is not obvious to me. I assume that's using the code `source/blender/io/usd` module to avoid duplication, but some design changes are probably needed to make that work well? Particularly for doing incremental updates by tracking changes in the Blender scene depsgraph. The Hydra Engine implementation I assume will be an implementation of a `RenderEngineType`, in `render/intern/hydra_engine.cc` or something like that?

Added subscriber: @kevindietrich

Added subscriber: @pmoursnv

Added subscriber: @sarazinjean-francois

In #100892#1414151, @brecht wrote:
From the user point of view I think it would be better not to have these two levels where you first select the Hydra render engine, and then select the delegate.

Another better approach could be to register each Hydra delegate as a render engine. With Python it should be possible to dynamically generate a subclass of a HydraRenderEngine class. Renderers that want to more closely integrate with Blender could have an add-on that subclasses HydraRenderEngine manually.

That would simplify the UI for users, make the -E command line option work, use the same mechanism for panel visibility as existing engines, etc.

One potential downside is that scanning the delegates may be expensive and negatively affect startup time. But I would hope that this involves USD just scanning a few plugInfo.json files without loading the actual shared libraries.

I think hydra should be chosen as the render engine first, to avoid ambiguity where Cycles will be used without and with Hydra. From my past experiences, a render engine with Hydra could sometimes met issue and not be as up to date as a direct renderer. The user should be able to use Hydra or not. If your concern is more about Hydra not being really a render engine, we could have a "Hydra" checkbox, and then update the render engines list

> In #100892#1414151, @brecht wrote: > From the user point of view I think it would be better not to have these two levels where you first select the Hydra render engine, and then select the delegate. > > Another better approach could be to register each Hydra delegate as a render engine. With Python it should be possible to dynamically generate a subclass of a `HydraRenderEngine` class. Renderers that want to more closely integrate with Blender could have an add-on that subclasses `HydraRenderEngine` manually. > > That would simplify the UI for users, make the `-E` command line option work, use the same mechanism for panel visibility as existing engines, etc. > > One potential downside is that scanning the delegates may be expensive and negatively affect startup time. But I would hope that this involves USD just scanning a few `plugInfo.json` files without loading the actual shared libraries. > In #100892#1414151, @brecht wrote: > From the user point of view I think it would be better not to have these two levels where you first select the Hydra render engine, and then select the delegate. > > Another better approach could be to register each Hydra delegate as a render engine. With Python it should be possible to dynamically generate a subclass of a `HydraRenderEngine` class. Renderers that want to more closely integrate with Blender could have an add-on that subclasses `HydraRenderEngine` manually. > > That would simplify the UI for users, make the `-E` command line option work, use the same mechanism for panel visibility as existing engines, etc. > > One potential downside is that scanning the delegates may be expensive and negatively affect startup time. But I would hope that this involves USD just scanning a few `plugInfo.json` files without loading the actual shared libraries. I think hydra should be chosen as the render engine first, to avoid ambiguity where Cycles will be used without and with Hydra. From my past experiences, a render engine with Hydra could sometimes met issue and not be as up to date as a direct renderer. The user should be able to use Hydra or not. If your concern is more about Hydra not being really a render engine, we could have a "Hydra" checkbox, and then update the render engines list

I don't think it would be ambiguous to select Cycles, and then have an option to use Hydra or not? Not sure why it would have to be the other way around.

In my experience is better an more clear to first go to a Hydra context and then have a list of render delegates, it seems a bit unpractical to go render engine by render engine and tick a Hydra check if available.
Also as previously mentioned, the delegate might be a different version.
It seems just cleaner to have a different context for everything hydra related.

In my experience is better an more clear to first go to a Hydra context and then have a list of render delegates, it seems a bit unpractical to go render engine by render engine and tick a Hydra check if available. Also as previously mentioned, the delegate might be a different version. It seems just cleaner to have a different context for everything hydra related.

In #100892#1451251, @brecht wrote:
I don't think it would be ambiguous to select Cycles, and then have an option to use Hydra or not? Not sure why it would have to be the other way around.

This logic seems backwards to me.

I also support what Esteban suggests. First you should be able to choose whether you want to work with “classic” render engine or Hydra context. If you decide to work with Hydra then you should have an option to choose which delegate exactly you want since there will be more than one available to you.

> In #100892#1451251, @brecht wrote: > I don't think it would be ambiguous to select Cycles, and then have an option to use Hydra or not? Not sure why it would have to be the other way around. This logic seems backwards to me. I also support what Esteban suggests. First you should be able to choose whether you want to work with “classic” render engine or Hydra context. If you decide to work with Hydra then you should have an option to choose which delegate exactly you want since there will be more than one available to you.

I expect that over time, most engines add-ons will become either Hydra only, or not use Hydra. Engine support for both is likely to be temporary while Hydra support is incomplete and being stabilized, so making both prominently available is not needed, when only one is well supported.

When referring to "Hydra context", I guess you are referring to something like Solaris in Houdini? Where not only the render engine changes, but you have a different workspace with different editors, viewports and nodes.

But that is not being proposed here, nor is it the direction I think we should go in. One of the strengths of Blender is that you don't have to switch between applications or contexts to do different tasks. You can be sculpting or tweaking geometry nodes while an interactive Cycles preview is running, texture painting, using the viewport compositor, etc. And we are moving even further into this kind of thing with plans for the viewport compositor to be able to combine e.g. grease pencil and other renders, and proposals for interactively working with multiple scenes in the sequencer. And in that view a "Hydra context" does not seem like a good fit.

I expect that over time, most engines add-ons will become either Hydra only, or not use Hydra. Engine support for both is likely to be temporary while Hydra support is incomplete and being stabilized, so making both prominently available is not needed, when only one is well supported. When referring to "Hydra context", I guess you are referring to something like Solaris in Houdini? Where not only the render engine changes, but you have a different workspace with different editors, viewports and nodes. But that is not being proposed here, nor is it the direction I think we should go in. One of the strengths of Blender is that you don't have to switch between applications or contexts to do different tasks. You can be sculpting or tweaking geometry nodes while an interactive Cycles preview is running, texture painting, using the viewport compositor, etc. And we are moving even further into this kind of thing with plans for the viewport compositor to be able to combine e.g. grease pencil and other renders, and proposals for interactively working with multiple scenes in the sequencer. And in that view a "Hydra context" does not seem like a good fit.

I agree with you, Blender is a flexible and well designed tool, I appreciate the effort that goes into keeping it that way. I don't think that going as far as Solaris is where efforts should go, for that we have Solaris after all :), what I do think is that there are ways to expose, read and write USD data that is not only friendly to the artist but also robust in scene management and more important interoperability. This is something where Blender, as much as I love it, needs to improve.
One of such ways has already (as far as I can remember) being explored by Brian in his addon, which is a node based scene flow which would be amazing to read, combine, manage, maintain a live reference and finally write USD, this is not Solaris, I can't stress this enough, but it seems to adapt to blender's ethos very well, and if the geometry can be red natively by blender it can conform to everything you are describing, going as far as to maybe breaking reference, caching the geometry on file, sculpting on top and then writing back to USD, it would open up so many exciting possibilities and allow for multi file worflows with real references that can be integreted to many pipelines. At least modifying a pipeline to fit blender would be easy, and switching to blender on a pipeline that already has other Tool would be a breeze. Also It would ligthen some of the burden that blender and cycles have to do it all, since we can just cheaply switch delegatesto render massive scenes and use cycles for what it does best.

I want to use Blender more, my team loves the tool, every animator we bring is a Maya user that gets converted practically and philosolycally to blender in less than 2 weeks and I can certianly see why. I wish we had the tools to really integrate blender to a standarized workflow and not have to bend over backwards to adapt the workflow to blender which often is just not possible.

Regarding your first point and returning to the topic at hand, I see what you are saying, although I think the time frame for this to happen would be speculative at best, maybe we can have the option in each render to use it as hydra or native.
This might get weird rather quickly though, since all Hydra delegates are standarized in a way that natives don't have to conform to.
Maybe a better way is to flag renderers when a delegate is encountered, and maybe have a global hydra switch that sill only show delegates, but in practice averything looks and feels the same you just have a different set of renderers when hydra is activated and another when disabled.

Having said all that, let me thank you for all the effort and care you and the team put into the tool's development, it's a large community and it must be a nightmare to keep everyone satisfied.

I agree with you, Blender is a flexible and well designed tool, I appreciate the effort that goes into keeping it that way. I don't think that going as far as Solaris is where efforts should go, for that we have Solaris after all :), what I do think is that there are ways to expose, read and write USD data that is not only friendly to the artist but also robust in scene management and more important interoperability. This is something where Blender, as much as I love it, needs to improve. One of such ways has already (as far as I can remember) being explored by Brian in his addon, which is a node based scene flow which would be amazing to read, combine, manage, maintain a live reference and finally write USD, this is not Solaris, I can't stress this enough, but it seems to adapt to blender's ethos very well, and if the geometry can be red natively by blender it can conform to everything you are describing, going as far as to maybe breaking reference, caching the geometry on file, sculpting on top and then writing back to USD, it would open up so many exciting possibilities and allow for multi file worflows with real references that can be integreted to many pipelines. At least modifying a pipeline to fit blender would be easy, and switching to blender on a pipeline that already has other Tool would be a breeze. Also It would ligthen some of the burden that blender and cycles have to do it all, since we can just cheaply switch delegatesto render massive scenes and use cycles for what it does best. I want to use Blender more, my team loves the tool, every animator we bring is a Maya user that gets converted practically and philosolycally to blender in less than 2 weeks and I can certianly see why. I wish we had the tools to really integrate blender to a standarized workflow and not have to bend over backwards to adapt the workflow to blender which often is just not possible. Regarding your first point and returning to the topic at hand, I see what you are saying, although I think the time frame for this to happen would be speculative at best, maybe we can have the option in each render to use it as hydra or native. This might get weird rather quickly though, since all Hydra delegates are standarized in a way that natives don't have to conform to. Maybe a better way is to flag renderers when a delegate is encountered, and maybe have a global hydra switch that sill only show delegates, but in practice averything looks and feels the same you just have a different set of renderers when hydra is activated and another when disabled. Having said all that, let me thank you for all the effort and care you and the team put into the tool's development, it's a large community and it must be a nightmare to keep everyone satisfied.

Hi all, good feedback. Let me throw in a few thoughts:

When you select a different render engine in Blender other than Cycles (like any 3rd party engine, ProRender, RenderMan, etc) you do have the expectation that it does not function or give exactly the same picture as Cycles. I think users are pretty used to this.
Brecht gave the feedback that having Hydra engines work like any render engine in blender was useful and I agree with him here. You could imagine in the add-on preferences a render engine could label itself as being Hydra based, but that shouldn't change the users workflow.
There ARE advantages like @EstebanCovo mentioned we have explored with the Hydra workflow for referencing USD files etc. But, that is slightly extraneous to the render engine discussion, and can happen later. There is already a mechanism in Blender for marking / disabling features not working with certain render engines, and that could be used for all Hydra based engines. But ideally any things added will work as much as possible across Blender, regardless of engine.

Hi all, good feedback. Let me throw in a few thoughts: 1. When you select a different render engine in Blender other than Cycles (like any 3rd party engine, ProRender, RenderMan, etc) you do have the expectation that it does not function or give exactly the same picture as Cycles. I think users are pretty used to this. 2. Brecht gave the feedback that having Hydra engines work like any render engine in blender was useful and I agree with him here. You could imagine in the add-on preferences a render engine could label itself as being Hydra based, but that shouldn't change the users workflow. 3. There ARE advantages like @EstebanCovo mentioned we have explored with the Hydra workflow for referencing USD files etc. But, that is slightly extraneous to the render engine discussion, and can happen later. There is already a mechanism in Blender for marking / disabling features not working with certain render engines, and that could be used for all Hydra based engines. But ideally any things added will work as much as possible across Blender, regardless of engine.

Added subscriber: @matthewlow-dwa

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

USD Hydra render engine #100892

Hydra Engine (C++) when a Hydra renderer is selected, ties together the Scene Delegate and Render Delegate and executes the render.