blender/blender

Cycles: texture cache and mipmaps #68917

New Issue

Open

opened 2019-08-20 21:27:05 +02:00 by Dalai Felinto · 16 comments

Dalai Felinto commented

2019-08-20 21:27:05 +02:00

Owner

There are a few parts to this:

Use OIIO texture cache in SVM
Reduced quality lookups on indirect bounces
Improve ray differentials in BSDF to ensure higher MIP levels are used
Preserve ray differentials in SVM
.tx file workflow, including automated generation
Bundle maketx with Blender
Optimize texture lookups by randomly sampling one of two MIP levels instead of interpolating between them

For GPU rendering, the OptiX SDK has code that we could adapt. Potentially this could be used for both CPU and GPU rendering. Some of the work from Tangent Animation can also be used to implement this, though that implementation is CPU only.

There are a few parts to this: - [ ] Use OIIO texture cache in SVM - [ ] Reduced quality lookups on indirect bounces - [ ] Improve ray differentials in BSDF to ensure higher MIP levels are used - [ ] Preserve ray differentials in SVM - [ ] .tx file workflow, including automated generation - [ ] Bundle maketx with Blender - [ ] Optimize texture lookups by randomly sampling one of two MIP levels instead of interpolating between them For GPU rendering, the OptiX SDK has code that we could adapt. Potentially this could be used for both CPU and GPU rendering. Some of the work from [Tangent Animation](https://github.com/tangent-opensource/blender) can also be used to implement this, though that implementation is CPU only.

👍 10 👎 2 ❤️ 10 🚀 8

Brecht Van Lommel changed title from ~~Texture cache and mipmaps~~ to Cycles: texture cache and mipmaps

2021-10-28 15:10:44 +02:00

baoyu commented

2021-11-15 09:22:58 +01:00

Please do consider add a option in
preference->file Paths
to allow setting custom path for the generated .tx files!
I used to use arnold and have .tx files in my cloud drive which are all over the place and I hate that!

Please do consider add a option in ```preference->file Paths``` to allow setting custom path for the generated .tx files! I used to use arnold and have .tx files in my cloud drive which are all over the place and I hate that!

Brecht Van Lommel added this to the Render & Cycles project 2023-02-07 19:08:06 +01:00

Philipp Oeser removed the

Render & Cycles

label 2023-02-09 13:59:50 +01:00

Quentin commented

2023-06-09 17:03:34 +02:00

Hi guys, is there any new about this features ?
Is it plan to put someone on this soon ?

This could be a big change and optimisation for Blender and Cycles-X. A lot of people are waiting for this feature.
I know that in the past a guy work on it for the old cycles. Could it be possible to used a part of is job to advance more easily on this development ?
We really need .tx to give more details to our projet and drasticly lower RAM usage.

Cheers <3

Hi guys, is there any new about this features ? Is it plan to put someone on this soon ? This could be a big change and optimisation for Blender and Cycles-X. A lot of people are waiting for this feature. I know that in the past a guy work on it for the old cycles. Could it be possible to used a part of is job to advance more easily on this development ? We really need .tx to give more details to our projet and drasticly lower RAM usage. Cheers <3

Tristan commented

2023-07-26 07:31:45 +02:00

This would be one of the more important features for large productions. We’re currently working on a feature film and had to implement many workarounds to handle texture memory. This would make things so much more streamlined!

This would be one of the more important features for large productions. We’re currently working on a feature film and had to implement many workarounds to handle texture memory. This would make things so much more streamlined!

Fabian commented

2023-08-28 16:05:38 +02:00

The lack of .tx support is a huge time drain in complex scenes with a lot of foliage, Quixel and other photo scan assets. This is currently one of the biggest issues with Cycles I see. All render times with de-noise are fine but the amount of artist-time spent manually fixing textures to fit into GPU memory is not. Please look into this. Thank you.

The lack of .tx support is a huge time drain in complex scenes with a lot of foliage, Quixel and other photo scan assets. This is currently one of the biggest issues with Cycles I see. All render times with de-noise are fine but the amount of artist-time spent manually fixing textures to fit into GPU memory is not. Please look into this. Thank you.

DGruwier commented

2023-08-28 17:56:57 +02:00

.exr supports tiling/caching as well, it’d be handy if a Cycles implementation also supports .exr mipmaps.

At least a few larger VFX studios I know of store all of their textures across the pipeline as .exr for this purpose. It enables hero assets with tens if not hundreds of GB of UDIM textures, and for some render engines at least it can provide quite drastic speed ups for network rendering (though less for local rendering with modern SSD speeds).

.exr supports tiling/caching as well, it’d be handy if a Cycles implementation also supports .exr mipmaps. At least a few larger VFX studios I know of store all of their textures across the pipeline as .exr for this purpose. It enables hero assets with tens if not hundreds of GB of UDIM textures, and for some render engines at least it can provide quite drastic speed ups for network rendering (though less for local rendering with modern SSD speeds).

Quentin commented

2024-04-02 11:29:15 +02:00

Hi everyone ! Any news about that ?

The feature could be a huge time saver for rendering and batch rendering. As Oiio is implemented in C++ it could be possible to used it to easly generate .tx texture. Make it usefull in Python to could give access to dev to manually do it.
Wouldn’t a labour party already be partially implemented?

On a competitor software, use the .tx optimised the render time from 1h30 to 4 minutes by frame.
I also know that for many studios, freelance, the lack of this feature is a barrier to switch to using Blender/Cycles for rendering.

Shouldn't it be a high priority request ?

I have no knowledge about C++. But if it can help, I probably could help to dev the interface for this features.

Hi everyone ! Any news about that ? The feature could be a huge time saver for rendering and batch rendering. As Oiio is implemented in C++ it could be possible to used it to easly generate .tx texture. Make it usefull in Python to could give access to dev to manually do it. Wouldn’t a labour party already be partially implemented? On a competitor software, use the .tx optimised the render time from 1h30 to 4 minutes by frame. I also know that for many studios, freelance, the lack of this feature is a barrier to switch to using Blender/Cycles for rendering. Shouldn't it be a high priority request ? I have no knowledge about C++. But if it can help, I probably could help to dev the interface for this features.

👍 3 ❤️ 3

Robert Rioux commented

2024-04-02 14:01:09 +02:00

This issue was opened 5 years ago. I wonder which feature in the Cycles development is more important than this one. If it was my call, it would be on top of the list. This is such a big issue for us in VFX and for the Tiki movie. Sets are huge, so are the textures. Please @dfelinto, tell us what's going on with mip mapping. Cycles is the only professional renderer that doesn't support it. :-)

This issue was opened 5 years ago. I wonder which feature in the Cycles development is more important than this one. If it was my call, it would be on top of the list. This is such a big issue for us in VFX and for the Tiki movie. Sets are huge, so are the textures. Please @dfelinto, tell us what's going on with mip mapping. Cycles is the only professional renderer that doesn't support it. :-)

👍 14 ❤️ 8

Roland-Vyens commented

2024-05-07 16:42:57 +02:00

This issue was opened 5 years ago. I wonder which feature in the Cycles development is more important than this one. If it was my call, it would be on top of the list. This is such a big issue for us in VFX and for the Tiki movie. Sets are huge, so are the textures. Please @dfelinto, tell us what's going on with mip mapping. Cycles is the only professional renderer that doesn't support it. :-)

This is very true. Please blender make your way into high-end production. The chance is right here, every big studio is trying to acquire blender to pipeline these days.

> This issue was opened 5 years ago. I wonder which feature in the Cycles development is more important than this one. If it was my call, it would be on top of the list. This is such a big issue for us in VFX and for the Tiki movie. Sets are huge, so are the textures. Please @dfelinto, tell us what's going on with mip mapping. Cycles is the only professional renderer that doesn't support it. :-) This is very true. Please blender make your way into high-end production. The chance is right here, every big studio is trying to acquire blender to pipeline these days.

Roland-Vyens commented

2024-05-16 05:13:01 +02:00

This feature is very needed. We are talking about assets with 90+ UDIM tiles with 4K resolution each tile. If mipmap doesn't exist, it's impossible to render this kind of stuff.

This feature is very needed. We are talking about assets with 90+ UDIM tiles with 4K resolution each tile. If mipmap doesn't exist, it's impossible to render this kind of stuff.

Quentin commented

2024-05-16 09:31:09 +02:00

I can’t be more agree with you. That non existing feature cut the utility of gpu render in half. A 2 millions polygone head sculpt with 4K textures kill the 8Gb VRam of my CG. I don’t want to imagine the nightmare for a big projet with lots of mod, texture and shading.

Blender Foundation save is please ❤️

I can’t be more agree with you. That non existing feature cut the utility of gpu render in half. A 2 millions polygone head sculpt with 4K textures kill the 8Gb VRam of my CG. I don’t want to imagine the nightmare for a big projet with lots of mod, texture and shading. Blender Foundation save is please ❤️

Raimund Klink commented

2024-05-16 11:43:44 +02:00

Contributor

I asked Sergey on rocket chat.
"Mipmaps are on the roadmap. It is a matter of allocating a time slot to work on the implementation."

They didn't forget this task :)

I asked Sergey on rocket chat. "Mipmaps are on the roadmap. It is a matter of allocating a time slot to work on the implementation." They didn't forget this task :)

Petter commented

2024-05-16 12:51:55 +02:00

I asked Sergey on rocket chat.
"Mipmaps are on the roadmap. It is a matter of allocating a time slot to work on the implementation."

They didn't forget this task :)

Sounds promising, but I wouldn't hold my breath :)

> I asked Sergey on rocket chat. > "Mipmaps are on the roadmap. It is a matter of allocating a time slot to work on the implementation." > > They didn't forget this task :) Sounds promising, but I wouldn't hold my breath :)

Raymond Siewert commented

2024-05-16 20:37:41 +02:00

Coming from a game engine style cinematic pipelines, mipmapping w/ cuda shared memory support would be essential as it would allow us to use artist computers in a render farm with even our largest scenes.

I think this present limitation is what is giving me the most amount of trouble in fully utilizing the beautiful results from cycles.

Coming from a game engine style cinematic pipelines, mipmapping w/ cuda shared memory support would be essential as it would allow us to use artist computers in a render farm with even our largest scenes. I think this present limitation is what is giving me the most amount of trouble in fully utilizing the beautiful results from cycles.

Guido commented

2024-05-17 04:41:17 +02:00

How big is the task to implement mip maps this way into blender?
According to PythonGPT-4o::

Implementing mipmaps in Blender using the OpenImageIO (OIIO) texture cache and making various other improvements is a significant task. Here's a breakdown of each task, its complexity, and the considerations involved:

Breakdown of Tasks:

Use OIIO Texture Cache in SVM

Complexity: High
Description: Integrate OpenImageIO’s texture cache system within Blender’s shading system (SVM - Shader Virtual Machine). This would involve interfacing with the OIIO library to handle texture caching and lookups.
Considerations:
Deep integration with Blender's existing shading and texture systems.
Ensuring compatibility with Blender's internal data structures and render engines.
Performance optimizations to handle the cache efficiently during rendering.

Reduced Quality Lookups on Indirect Bounces

Complexity: Medium
Description: Implement a system to reduce texture quality for indirect bounces in ray tracing. This could involve using lower-resolution mipmap levels or other approximations.
Considerations:
Modifying the ray tracing logic to use different texture sampling strategies based on bounce depth.
Ensuring that visual quality degradation is minimized.

Improve Ray Differentials in BSDF to Ensure Higher MIP Levels are Used

Complexity: Medium to High
Description: Enhance the calculation of ray differentials in BSDFs (Bidirectional Scattering Distribution Functions) to improve the selection of appropriate mipmap levels.
Considerations:
Adjusting BSDF implementations to compute and preserve accurate ray differentials.
Ensuring that these changes improve mipmap usage without negatively impacting other aspects of shading.

Preserve Ray Differentials in SVM

Complexity: Medium to High
Description: Ensure that ray differentials are preserved throughout the SVM pipeline. This is crucial for accurate texture sampling at different mipmap levels.
Considerations:
Modifying the SVM infrastructure to handle and propagate ray differential information.
Maintaining compatibility with existing shaders and rendering workflows.

.tx File Workflow, Including Automated Generation

Complexity: Medium
Description: Implement support for .tx files (texture files optimized for rendering) and automate their generation. OIIO’s maketx tool can be used for this purpose.
Considerations:
Adding a pipeline for converting images to .tx files within Blender.
Ensuring that the .tx files are used seamlessly in the rendering process.

Bundle maketx with Blender

Complexity: Low to Medium
Description: Include the maketx tool as part of Blender’s distribution, ensuring users have access to texture optimization tools.
Considerations:
Ensuring licensing compatibility.
Integrating maketx into Blender’s build and distribution systems.

Optimize Texture Lookups by Randomly Sampling One of Two MIP Levels Instead of Interpolating Between Them

Complexity: Medium
Description: Implement a texture sampling optimization that randomly selects one of two mipmap levels instead of interpolating between them, potentially improving performance.
Considerations:
Adjusting the texture sampling algorithm in the render engine.
Testing to ensure that the visual quality remains acceptable with this optimization.

Overall Complexity and Effort

Integration with Existing Systems: Many of these tasks require deep integration with Blender’s existing rendering and shading systems. This involves understanding and modifying complex codebases.

Performance and Quality Trade-offs: Changes to texture caching, mipmap usage, and ray differentials must be carefully balanced to ensure performance improvements do not come at the cost of significant visual quality degradation.
Cross-Disciplinary Expertise: These tasks span multiple areas, including shader development, ray tracing, texture management, and software distribution, requiring a broad range of expertise.

Testing and Validation: Extensive testing is necessary to ensure that changes improve performance and visual quality as intended and do not introduce regressions.
Estimated Effort

Initial Research and Planning: 1-2 months to fully scope out the tasks, understand the current system, and develop a detailed implementation plan.

Implementation: 6-12 months depending on the availability of developers with the required expertise.

Testing and Optimization: 3-6 months of rigorous testing, performance benchmarking, and iterative optimization.

Documentation and User Support: 1-2 months to update documentation, create tutorials, and provide support for new workflows.

Conclusion
Implementing these tasks requires significant effort and coordination among developers familiar with Blender’s rendering engine, shader system, and texture management. The complexity is high, but the potential benefits in terms of performance and rendering quality make it a worthwhile investment for Blender’s development community. Collaboration with the Blender developer community and possibly seeking funding or grants for such an extensive project might be beneficial.

How big is the task to implement mip maps this way into blender? According to PythonGPT-4o:: Implementing mipmaps in Blender using the OpenImageIO (OIIO) texture cache and making various other improvements is a significant task. Here's a breakdown of each task, its complexity, and the considerations involved: Breakdown of Tasks: Use OIIO Texture Cache in SVM Complexity: High Description: Integrate OpenImageIO’s texture cache system within Blender’s shading system (SVM - Shader Virtual Machine). This would involve interfacing with the OIIO library to handle texture caching and lookups. Considerations: Deep integration with Blender's existing shading and texture systems. Ensuring compatibility with Blender's internal data structures and render engines. Performance optimizations to handle the cache efficiently during rendering. Reduced Quality Lookups on Indirect Bounces Complexity: Medium Description: Implement a system to reduce texture quality for indirect bounces in ray tracing. This could involve using lower-resolution mipmap levels or other approximations. Considerations: Modifying the ray tracing logic to use different texture sampling strategies based on bounce depth. Ensuring that visual quality degradation is minimized. Improve Ray Differentials in BSDF to Ensure Higher MIP Levels are Used Complexity: Medium to High Description: Enhance the calculation of ray differentials in BSDFs (Bidirectional Scattering Distribution Functions) to improve the selection of appropriate mipmap levels. Considerations: Adjusting BSDF implementations to compute and preserve accurate ray differentials. Ensuring that these changes improve mipmap usage without negatively impacting other aspects of shading. Preserve Ray Differentials in SVM Complexity: Medium to High Description: Ensure that ray differentials are preserved throughout the SVM pipeline. This is crucial for accurate texture sampling at different mipmap levels. Considerations: Modifying the SVM infrastructure to handle and propagate ray differential information. Maintaining compatibility with existing shaders and rendering workflows. .tx File Workflow, Including Automated Generation Complexity: Medium Description: Implement support for .tx files (texture files optimized for rendering) and automate their generation. OIIO’s maketx tool can be used for this purpose. Considerations: Adding a pipeline for converting images to .tx files within Blender. Ensuring that the .tx files are used seamlessly in the rendering process. Bundle maketx with Blender Complexity: Low to Medium Description: Include the maketx tool as part of Blender’s distribution, ensuring users have access to texture optimization tools. Considerations: Ensuring licensing compatibility. Integrating maketx into Blender’s build and distribution systems. Optimize Texture Lookups by Randomly Sampling One of Two MIP Levels Instead of Interpolating Between Them Complexity: Medium Description: Implement a texture sampling optimization that randomly selects one of two mipmap levels instead of interpolating between them, potentially improving performance. Considerations: Adjusting the texture sampling algorithm in the render engine. Testing to ensure that the visual quality remains acceptable with this optimization. Overall Complexity and Effort Integration with Existing Systems: Many of these tasks require deep integration with Blender’s existing rendering and shading systems. This involves understanding and modifying complex codebases. Performance and Quality Trade-offs: Changes to texture caching, mipmap usage, and ray differentials must be carefully balanced to ensure performance improvements do not come at the cost of significant visual quality degradation. Cross-Disciplinary Expertise: These tasks span multiple areas, including shader development, ray tracing, texture management, and software distribution, requiring a broad range of expertise. Testing and Validation: Extensive testing is necessary to ensure that changes improve performance and visual quality as intended and do not introduce regressions. Estimated Effort Initial Research and Planning: 1-2 months to fully scope out the tasks, understand the current system, and develop a detailed implementation plan. Implementation: 6-12 months depending on the availability of developers with the required expertise. Testing and Optimization: 3-6 months of rigorous testing, performance benchmarking, and iterative optimization. Documentation and User Support: 1-2 months to update documentation, create tutorials, and provide support for new workflows. Conclusion Implementing these tasks requires significant effort and coordination among developers familiar with Blender’s rendering engine, shader system, and texture management. The complexity is high, but the potential benefits in terms of performance and rendering quality make it a worthwhile investment for Blender’s development community. Collaboration with the Blender developer community and possibly seeking funding or grants for such an extensive project might be beneficial.

Quentin commented

2024-05-22 17:51:46 +02:00

I think all people who really know a little bit about mipmap, know that it's not a "simple" task to implement. Thank to you to ask to gpt a listing of the tasks.
But I think, all of this speech can be summarized by this simple sentence given by gpt itself in its own conclusion : " The complexity is high, but the potential benefits in terms of performance and rendering quality make it a worthwhile investment for Blender’s development community". So thanks to it and to you ^^ that confirm the importance of this feature for the pro, the indie and everyone.
BF, if we can help you in any way, it will be with pleasure for me :) <3

I think all people who really know a little bit about mipmap, know that it's not a "simple" task to implement. Thank to you to ask to gpt a listing of the tasks. But I think, all of this speech can be summarized by this simple sentence given by gpt itself in its own conclusion : " The complexity is high, but the potential benefits in terms of performance and rendering quality make it a worthwhile investment for Blender’s development community". So thanks to it and to you ^^ that confirm the importance of this feature for the pro, the indie and everyone. BF, if we can help you in any way, it will be with pleasure for me :) <3

Matt Curtis commented

2024-06-04 00:52:53 +02:00

Commenting to show more support for this feature. It's often possible to get around the texture memory bottleneck by splitting large scenes into multiple view layers, but this isn't possible when rendering VFX characters or other assets with dozens or hundreds of UDIMs on a single object.
Implementing caching/mipmapping support would be a huge improvement to the pipeline for heavy scenes like this.

Commenting to show more support for this feature. It's often possible to get around the texture memory bottleneck by splitting large scenes into multiple view layers, but this isn't possible when rendering VFX characters or other assets with dozens or hundreds of UDIMs on a single object. Implementing caching/mipmapping support would be a huge improvement to the pipeline for heavy scenes like this.

Sign in to join this conversation.

No Label

Animation & Rigging

Automated Testing

Blender Asset Bundle

Dependency Graph

Development Management

EEVEE & Viewport

Images & Movies

Motion Tracking

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Render Pipeline

Sculpt, Paint & Texture

Video Sequencer

Virtual Reality

Asset Browser Project

Blender 2.8 Project

Milestone 1: Basic, Local Asset Browser

Good First Issue

Animation & Rigging

Development Management

EEVEE & Viewport

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Sculpt, Paint & Texture

Needs Info from Developers

Needs Information from User

No Milestone

No project

No Assignees

13 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#68917

No description provided.