High Memory Usage when render animation with Persistant Images enabled, and using Fix (T77734) reproducible with Latest 2.90 Blender #78537

New Issue

shaun · 2020-07-02T16:37:47+02:00

shaun commented

2020-07-02 16:37:47 +02:00

System Information
Tested on

Threadripper 2970WX / 32GB DDR4 / Aorus X399 / 1KW PSU / GTX 1070
Ryzen 3900X / 32GB DDR4 / 850W PSU / 1070 GTX

Operating system:
Windows 10 PRO

Graphics card:
GTX 1070 8GB

Blender Version
Broken 2.90 - 2-JULY 2020
previous editions of blender can see the memory usage problem described in this bug report, but at risk of more frequent crash to desktops, the CTD seems to be fixed with 2.90

Short description of error
Memory usage increases at an insane rate progressively when render animation frames,
The recent fix #77734 has proven to make blender more stable, but an underlining issue is still here where memory usage is seen to be abused

Exact steps for others to reproduce the error

Download my file, https://drive.google.com/file/d/1vrl-kien3uePeQ6oexIHdCU35FSTdGA-/view?usp=sharing
open the blend file
file is setup to to enable the viewporn with materials, ensure that the materials load fully
render as animation
monitor the render frames information, specifically the memory usage in the rendering window.

for additional support I have created some videos
Please view the 3 videos for how the issue responds with 2.81a and 2.90, (2.81a crashes, where 2.90 stays alive to show the memory usage seems to increase way too much)

FAIL Attempt 1 (Blender 2.81a) - https://youtu.be/LXXpIUKanP4
FAIL Attempt 2 (Blender 2.81a) - https://youtu.be/fqIev-diTfU
you can see that blender just simply disapears when trying to render with persistent images enabled.

New Blender 2.90,
Improved, but issues Attempt 3 (Blender 2.90 2-July-2020 download) - https://youtu.be/5NoCXXHsLbw

I have 4 computers with similar spec, the Threadripper 2970WX being most powerful,
I have seen that on the AMD 3900X with 32GB ram and a GTX 1070, I see similar results but with 1 CTD, the threadripper was ran 5 times and never CTD, but says System is out of GPU Host shared memory when it stops at frame 13.

Hi.
I have seen improvements made with the latest 2.90 downloaded today,
Please take the time to view my videos which monitor my computers Main Memory, and CUDA memory.
Persistent images seems to be storing way too much information which clogs up blender and starts acting up.

you will see that with the blender 2.90 video, blender will render a few frames really really fast then there are massive delays before rendering frames.

I have noticed the following, computer system memory increases intensely during the rendering of each frame progressively from say 4-6GB at the start, and ends around 24GB+

When I use persistent images i see huge gains in time rendered for animations in both Windows and Linux, I am truly hoping that this issue where blender seems to be way to memory hungry can be optimized.

in the render preview window you can see that memory usage is approx 23GB at the frame it stops working.
hwinfo will show that the data for the video card become overwhelmed..

I have tested the same on 4 different class systems,
1st is a threadripper 2970WX 1070GTX
2nd is a Ryzen 3900X 32GB RAM 1070GTX
3rd is Intel i7 8600K 32GB RAM 1080TI
4th is an Intel i7 8750H 32GB 1060 GTX laptop

apart from system 2 which does a CTD with 2.90, the other 3 act similar to system 1 which the videos were recorded.

I can see that persistent images when DISABLED, blender will work fine. but I loose performance.

Enable Simplified to limit textures down to 1024 help, but over a long run of frame renders it will end up with same memory full issue.

Many thanks
Shaun

**System Information** Tested on 1. Threadripper 2970WX / 32GB DDR4 / Aorus X399 / 1KW PSU / GTX 1070 2. Ryzen 3900X / 32GB DDR4 / 850W PSU / 1070 GTX Operating system: Windows 10 PRO Graphics card: GTX 1070 8GB **Blender Version** Broken 2.90 - 2-JULY 2020 previous editions of blender can see the memory usage problem described in this bug report, but at risk of more frequent crash to desktops, the CTD seems to be fixed with 2.90 **Short description of error** Memory usage increases at an insane rate progressively when render animation frames, The recent fix #77734 has proven to make blender more stable, but an underlining issue is still here where memory usage is seen to be abused **Exact steps for others to reproduce the error** 1. Download my file, https://drive.google.com/file/d/1vrl-kien3uePeQ6oexIHdCU35FSTdGA-/view?usp=sharing 2. open the blend file 3. file is setup to to enable the viewporn with materials, ensure that the materials load fully 4. render as animation 5. monitor the render frames information, specifically the memory usage in the rendering window. for additional support I have created some videos Please view the 3 videos for how the issue responds with 2.81a and 2.90, (2.81a crashes, where 2.90 stays alive to show the memory usage seems to increase way too much) FAIL Attempt 1 (Blender 2.81a) - https://youtu.be/LXXpIUKanP4 FAIL Attempt 2 (Blender 2.81a) - https://youtu.be/fqIev-diTfU you can see that blender just simply disapears when trying to render with persistent images enabled. New Blender 2.90, Improved, but issues Attempt 3 (Blender 2.90 2-July-2020 download) - https://youtu.be/5NoCXXHsLbw I have 4 computers with similar spec, the Threadripper 2970WX being most powerful, I have seen that on the AMD 3900X with 32GB ram and a GTX 1070, I see similar results but with 1 CTD, the threadripper was ran 5 times and never CTD, but says System is out of GPU Host shared memory when it stops at frame 13. Hi. I have seen improvements made with the latest 2.90 downloaded today, Please take the time to view my videos which monitor my computers Main Memory, and CUDA memory. Persistent images seems to be storing way too much information which clogs up blender and starts acting up. you will see that with the blender 2.90 video, blender will render a few frames really really fast then there are massive delays before rendering frames. I have noticed the following, computer system memory increases intensely during the rendering of each frame progressively from say 4-6GB at the start, and ends around 24GB+ When I use persistent images i see huge gains in time rendered for animations in both Windows and Linux, I am truly hoping that this issue where blender seems to be way to memory hungry can be optimized. in the render preview window you can see that memory usage is approx 23GB at the frame it stops working. hwinfo will show that the data for the video card become overwhelmed.. I have tested the same on 4 different class systems, 1st is a threadripper 2970WX 1070GTX 2nd is a Ryzen 3900X 32GB RAM 1070GTX 3rd is Intel i7 8600K 32GB RAM 1080TI 4th is an Intel i7 8750H 32GB 1060 GTX laptop apart from system 2 which does a CTD with 2.90, the other 3 act similar to system 1 which the videos were recorded. I can see that persistent images when DISABLED, blender will work fine. but I loose performance. Enable Simplified to limit textures down to 1024 help, but over a long run of frame renders it will end up with same memory full issue. Many thanks Shaun

shaun commented

2020-07-02 16:37:48 +02:00

Added subscriber: @toxsickcity

byron commented

2020-07-02 18:33:35 +02:00

Added subscriber: @vyrwn-2

byron commented

2020-07-02 18:33:35 +02:00

I am not sure if this has been resolved or not, as I have found multiple entries.

I just wanted to add that I have the same error. I upgraded to 2.83 this morning and immediatly I started getting this issue, where texture images are loaded on every frame even when persistent images is selected and they seem to stack up on every frame. When memory reaches the available limit then the animation stops.
Blender and the machine do not crash but I have to restart blender to reset the memory before trying a new render.

My system settings (although I doubt they are useful for this issue):
Processor: AMD Ryzen 3700x 8-core 3.59GHz
RAM: 16GB
GPU: RTX 2070 Super

Disabling the persistend images of course resolves the issue.

thank you.
kind regards,
Byron

I am not sure if this has been resolved or not, as I have found multiple entries. I just wanted to add that I have the same error. I upgraded to 2.83 this morning and immediatly I started getting this issue, where texture images are loaded on every frame even when persistent images is selected and they seem to stack up on every frame. When memory reaches the available limit then the animation stops. Blender and the machine do not crash but I have to restart blender to reset the memory before trying a new render. My system settings (although I doubt they are useful for this issue): Processor: AMD Ryzen 3700x 8-core 3.59GHz RAM: 16GB GPU: RTX 2070 Super Disabling the persistend images of course resolves the issue. thank you. kind regards, Byron

Germano Cavalcante commented

2020-07-03 19:05:36 +02:00

Added subscriber: @mano-wii

Germano Cavalcante commented

2020-07-03 19:05:36 +02:00

Each frame takes a long time to render. So it is not very feasible for me to try to reproduce the problem.

The scenario described is too time consuming for us to track down, we require the bug reporter to narrow down the problem.

Normally .blend files can be simplified by removing most objects and disabling settings, until the problem reveals itself more clearly.

The frame I tested the memory usage by the system was actually smaller in blender 2.90 (11.2 GB in 2.83 vs 10.6 GB in 2.90).

Each frame takes a long time to render. So it is not very feasible for me to try to reproduce the problem. The scenario described is too time consuming for us to track down, we require the bug reporter to narrow down the problem. Normally .blend files can be simplified by removing most objects and disabling settings, until the problem reveals itself more clearly. The frame I tested the memory usage by the system was actually smaller in blender 2.90 (11.2 GB in 2.83 vs 10.6 GB in 2.90).

shaun commented

2020-07-03 23:00:58 +02:00

Hi Germano,

The .blend file I have included and similar
Blend files which are easy to cause this problem to blender usually have high res textures. It's the loading of the textures into memory is what I believe the cause of the issue is.
When persistent images is enabled.. I am not a Dev. But it would appear that the textures maybe being duplicated? I don't understand blenders cycles engine.. but for some reason enable persistent images seems to be adding stuff to the point the system memory is so full blender cannot work anymore. There might be room to tweak or limit how persistent images works?

If the time taken to render the frames is too long you can easily solve that by reducing the samples down to a much lower number try render samples at 5. It should render frames in just a few seconds.

It allows the frames to render much quicker. And you'll see the issues I am speaking about after blender renders approx 13 frames in animation mode. (Ctrl - F12)

The sample size is not important here. It is having persistent images on and watching blender after rendering perhaps 5 frames start using memory in a very loose way.

I also notice that if you render as just using CPU the problem still exsist which leads me to believe this isn't really something to do with cuda memory usage.

The issue described where memory usage is very high by the 13th frame (on my system in the render window with the render timer etc, memory is reported to be using 24,000MB) before it stops on my system.

Using simplified and forcing textures down to 1024 or 512.
Will help but eventually at say render animation maybe the 40th frame will be at 24,000MB..

Persistent images always seems to fill the memory in one way or another.

Difference between render GPU Hybrid vs CPU Only
GPU Hybrid
Lastly, when I mentioned in my original report. I suggest that the blender program slows down after rendering the 4th-5th frame. In my HWinfo window I notice that 8GB video card memory is full (D3D dedicated memory), then the data seems to populate the D3D dynamic memory) which is basically system memory.
CPU Only
Instead of using the memory from the video card, blender starts filling up system memory at a much faster rate!
My system configuration 32GB physical and 32GB virtual memory for total of 64GB commit
Rendering at a sample rate of 5 I reach frame 28 in approx 2 minutes where memory usage is 52,000MB
In this scenario blender CTD

If you have any questions or need help further diagnose the issue please let me know. I am very eager to have this issue explored by blender Dev/expert.

Hi Germano, The .blend file I have included and similar Blend files which are easy to cause this problem to blender usually have high res textures. It's the loading of the textures into memory is what I believe the cause of the issue is. When persistent images is enabled.. I am not a Dev. But it would appear that the textures maybe being duplicated? I don't understand blenders cycles engine.. but for some reason enable persistent images seems to be adding stuff to the point the system memory is so full blender cannot work anymore. There might be room to tweak or limit how persistent images works? If the time taken to render the frames is too long you can easily solve that by reducing the samples down to a much lower number try render samples at 5. It should render frames in just a few seconds. It allows the frames to render much quicker. And you'll see the issues I am speaking about after blender renders approx 13 frames in animation mode. (Ctrl - F12) The sample size is not important here. It is having persistent images on and watching blender after rendering perhaps 5 frames start using memory in a very loose way. I also notice that if you render as just using CPU the problem still exsist which leads me to believe this isn't really something to do with cuda memory usage. The issue described where memory usage is very high by the 13th frame (on my system in the render window with the render timer etc, memory is reported to be using 24,000MB) before it stops on my system. Using simplified and forcing textures down to 1024 or 512. Will help but eventually at say render animation maybe the 40th frame will be at 24,000MB.. Persistent images always seems to fill the memory in one way or another. Difference between render GPU Hybrid vs CPU Only GPU Hybrid Lastly, when I mentioned in my original report. I suggest that the blender program slows down after rendering the 4th-5th frame. In my HWinfo window I notice that 8GB video card memory is full (D3D dedicated memory), then the data seems to populate the D3D dynamic memory) which is basically system memory. CPU Only Instead of using the memory from the video card, blender starts filling up system memory at a much faster rate! My system configuration 32GB physical and 32GB virtual memory for total of 64GB commit Rendering at a sample rate of 5 I reach frame 28 in approx 2 minutes where memory usage is 52,000MB In this scenario blender CTD If you have any questions or need help further diagnose the issue please let me know. I am very eager to have this issue explored by blender Dev/expert.

Germano Cavalcante commented

2020-07-03 23:20:38 +02:00

I am not a cycles developer. But I can forward the report to the specialized team.

However, this report is not in the recommended style to be confirmed/forwarding.

The files are very large, it takes a long time to investigate and the problem is not well defined.

I'm tagging the module anyway

I am not a cycles developer. But I can forward the report to the specialized team. However, this report is not in the recommended style to be confirmed/forwarding. The files are very large, it takes a long time to investigate and the problem is not well defined. I'm tagging the module anyway

Vincent Blankfield commented

2020-07-04 04:05:29 +02:00

Added subscriber: @VincentBlankfield

Vincent Blankfield commented

2020-07-04 04:05:29 +02:00

Hi shaun,

I took a look into this issue, and it appears that something is off with your file. Indeed when persistent images is enabled, each frame would create more and more render slots for the images, allocating memory until it has it all. But when I dived a bit deeper I've noticed that the image_user objects for the textures in this file are set up as for 100 frame long animations. While I've no idea why this may be this way, and where did you get these materials (they look generated by 3rd party software to me), this does cause the described out-of-memory issue by increasing the memory usage by 100 times or so. While at it I've noticed a little possible issue with Blender that made the situation with animated textures and persistent images even worse, but this is a whole another topic. Anyway, if I'm not mistaken about this problem, here's how to repair the provided test file:

Make sure you have a backup of your work, I don't guarantee anything.
Open the blend file.
Go to the scripting tab.
Press New.
Paste the code from below in the script editor.
On the main menu click Window > Toggle System Console - here you'll see the script output when you run it.
You may run the script in "no fix" mode (explained below) to see all the issues without changing anything. The output will be in the system console.
To fix all the issues run the script in "fix" mode (explained below). You may run it a couple times. When everything is fixed there will be no output.

Save, and you may try rendering. Note that previous renders leave a lot of stuff in memory. So for the best experience reopen the file.

Also note that this script will mess up the real animated textures if there are such (I din't see any in your example). Though if you do use animated textures, I highly recommend against using "persistent images" with them currently.

The script:

lines=10
import bpy

- Set this to True to correct the situation by making all the textures
- used in the shader nodes 1 frame long.
- Note that every texture user will be made single frame, even ones
- that are meant to be animated.
# Set this to False for read only check.
fix = False


defaults = {
    "frame_offset": 0,
    "frame_start": 1,
    "frame_duration": 1,
    "frame_current": 1,
}


def checkNode(nd):
    if nd.type != "TEX_IMAGE":
        if nd.type == "GROUP":
            for subNd in nd.node_tree.nodes:
                checkNode(subNd)
        return

    isDefault = True
    for k in defaults:
        if getattr(nd.image_user, k) != defaults[k]:
            isDefault = False
            break
            
    if not isDefault:
        print()
        print(f"Material: '{mat.name}'")
        print(f"  Node: '{nd.name}'")
        print(f"    Using texture: {nd.image.name}")
        print(f"    from {nd.image.filepath}")
        for k in defaults:
            print(f"    Has {k}: {getattr(nd.image_user, k)}")
            if fix and getattr(nd.image_user, k) != defaults[k]:
                print(f"    Setting {k} to {defaults[k]}")
                setattr(nd.image_user, k, defaults[k])


print("Checking image users")

for mat in bpy.data.materials:
    if mat.node_tree is None:
        continue
    
    for nd in mat.node_tree.nodes:
        checkNode(nd)

for ng in bpy.data.node_groups:
    checkNode(ng)

print("Done")

To run it in "just check, don't fix" mode leave fix = False. To run it in "check and fix" mode, change it to fix = True.

If the problem proves to be in the file and not Blender code, than please close the issue as invalid so the devs don't waste time trying to reproduce it. Otherwise I have a simplified test file and a tip on what to look for (I've already said it - image_users with 100 frame duration and look for the slots allocation).

Anyways, it was great fun debugging this on an 8GB potato (:

Hi shaun, I took a look into this issue, and it appears that something is off with your file. Indeed when persistent images is enabled, each frame would create more and more render slots for the images, allocating memory until it has it all. But when I dived a bit deeper I've noticed that the `image_user` objects for the textures in this file are set up as for 100 frame long animations. While I've no idea why this may be this way, and where did you get these materials (they look generated by 3rd party software to me), this does cause the described out-of-memory issue by increasing the memory usage by 100 times or so. While at it I've noticed a little possible issue with Blender that made the situation with animated textures and persistent images even worse, but this is a whole another topic. Anyway, if I'm not mistaken about this problem, here's how to repair the provided test file: - Make sure you have a backup of your work, I don't guarantee anything. - Open the blend file. - Go to the scripting tab. - Press `New`. - Paste the code from below in the script editor. - On the main menu click `Window` > `Toggle System Console` - here you'll see the script output when you run it. - You may run the script in "no fix" mode (explained below) to see all the issues without changing anything. The output will be in the system console. - To fix all the issues run the script in "fix" mode (explained below). You may run it a couple times. When everything is fixed there will be no output. # Save, and you may try rendering. Note that previous renders leave a lot of stuff in memory. So for the best experience reopen the file. Also note that this script will mess up the real animated textures if there are such (I din't see any in your example). Though if you do use animated textures, I highly recommend against using "persistent images" with them currently. The script: ``` lines=10 import bpy - Set this to True to correct the situation by making all the textures - used in the shader nodes 1 frame long. - Note that every texture user will be made single frame, even ones - that are meant to be animated. # Set this to False for read only check. fix = False defaults = { "frame_offset": 0, "frame_start": 1, "frame_duration": 1, "frame_current": 1, } def checkNode(nd): if nd.type != "TEX_IMAGE": if nd.type == "GROUP": for subNd in nd.node_tree.nodes: checkNode(subNd) return isDefault = True for k in defaults: if getattr(nd.image_user, k) != defaults[k]: isDefault = False break if not isDefault: print() print(f"Material: '{mat.name}'") print(f" Node: '{nd.name}'") print(f" Using texture: {nd.image.name}") print(f" from {nd.image.filepath}") for k in defaults: print(f" Has {k}: {getattr(nd.image_user, k)}") if fix and getattr(nd.image_user, k) != defaults[k]: print(f" Setting {k} to {defaults[k]}") setattr(nd.image_user, k, defaults[k]) print("Checking image users") for mat in bpy.data.materials: if mat.node_tree is None: continue for nd in mat.node_tree.nodes: checkNode(nd) for ng in bpy.data.node_groups: checkNode(ng) print("Done") ``` To run it in "just check, don't fix" mode leave `fix = False`. To run it in "check and fix" mode, change it to `fix = True`. If the problem proves to be in the file and not Blender code, than please close the issue as invalid so the devs don't waste time trying to reproduce it. Otherwise I have a simplified test file and a tip on what to look for (I've already said it - `image_user`s with 100 frame duration and look for the slots allocation). *Anyways, it was great fun debugging this on an 8GB potato (:*

shaun commented

2020-07-04 14:22:48 +02:00

Hi Vincent,

I am glad to say that your script has reconfigured my file and has solved the problems described in my report, blender no longer goes memory crazy, it rendered 150 frames in a matter of minutes,
thank-you so much for your deep dive. would you kindly provide me a few minutes to explain the problem more so I can understand it.

I am actually wanting to know more about this problem that my file was causing to blender?
I dont even know where to find the setting inside the image texture where it was set at 100 frames? is this somewhere in node setup? or only adjustable via script?

I did use a plugin which imports DAZ Projects. I will write to the script developer to see if they can look to see if they can improve/fix or if its intended!

are you able to elaborate on a scenario where I would want to use 100 frames of texture?

thanks again Vincent, I understand the my issue has crossed the bounds of user support, and I do appreciate your time.
I will change the status to completed.

Thank-You
Shaun

Hi Vincent, I am glad to say that your script has reconfigured my file and has solved the problems described in my report, blender no longer goes memory crazy, it rendered 150 frames in a matter of minutes, thank-you so much for your deep dive. would you kindly provide me a few minutes to explain the problem more so I can understand it. I am actually wanting to know more about this problem that my file was causing to blender? I dont even know where to find the setting inside the image texture where it was set at 100 frames? is this somewhere in node setup? or only adjustable via script? I did use a plugin which imports DAZ Projects. I will write to the script developer to see if they can look to see if they can improve/fix or if its intended! are you able to elaborate on a scenario where I would want to use 100 frames of texture? thanks again Vincent, I understand the my issue has crossed the bounds of user support, and I do appreciate your time. I will change the status to completed. Thank-You Shaun

Vincent Blankfield commented

2020-07-04 19:18:52 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Vincent Blankfield closed this issue

2020-07-04 19:18:52 +02:00

Vincent Blankfield self-assigned this 2020-07-04 19:18:52 +02:00

Vincent Blankfield commented

2020-07-04 19:18:52 +02:00

The problem was that the Image Texture nodes in the material graphs had their image_user objects misconfigured. The image_user (https://docs.blender.org/api/current/bpy.types.ImageUser.html) is an internal object, that lets renderer know how to use a texture. It's not meant to be directly accessed by the user, so it has no direct representation in the UI. It is indirectly accessed when you create a texture. Try adding a texture node to some material, and then set it to some video (or change its source type to Movie or Image Sequence) - you'll see the options like Frames, Start Frame, Offset, etc. So this is used when creating animated textures (here's your use case for a 100 frame texture). But it has no sense with texture type set to Singe Image, as was in the file. There's also an advanced mode of the Outliner, meant for the scripters use, that can show all the internal data structures of a blend file. It's called "Data API" (second dropdown on the left in the outliner header). There you can drill down to the objects in question. The path would be Materials>MATERIAL_NAME>Node Tree>Nodes>TEXTURE_NODE_NAME>Image User. I'll remind that changing stuff directly in the Data API outliner mode will most likely break something if you don't know what you're doing.

So in your case the importer script created those object with wrong parameters. It has set the texture type to Single Image, but duration, start frame and current frame were set up as if it were an animation. Which makes no sense (at least to me). You can see what was set to what values when you run my script on the misconfigured file in fix = False mode. The output (in the system console) will show something like this for each misconfigured texture node:

Material: 'wraps-1'
  Node: '4be_Charmer_gloves_bump_01'
    Using texture: 4be_Charmer_gloves_bump_01
    from D:\Games\DAZ 3D\MyLibrary\Runtime\textures\4blueyes\Charmer\4be_Charmer_gloves_bump_01.jpg
    Has frame_offset: 0
    Has frame_start: 1
    Has frame_duration: 100
    Has frame_current: 0

With this image_user configuration and the Persistent Images option checked the renderer will advance the texture current frame on each animation frame rendered, allocate new slots for the texture images frame, render the frame, don't free the render slots (because persistent images means it). Then on the next frame it will check if all the needed images are loaded - it checks that the image and it's frame are the same. The image frame will differ, so once again it will allocate new render slots, an so on. The memory usage will go through the roof very quickly. Considering that the "single image but multi frame" configuration isn't legal, it can't be said that Blender did something wrong here.

As I've already invested into this investigation, I'm now curious to know how exactly did it happen. Can you drop a link to the exact version of the importer plugin (and maybe the data it was importing) so I can take a look at it? Of course if such sharing isn't restricted by it's license. Just don't upload the files here. A drive link, as in your report, would be perfect. Not sure I will be able to hunt the problem down, as I have no clue what DAZ is, still I'm curious to check it out.

I hope the Blender team will forgive this little off topic conversation here in the name of finding the truth and making the infrastructure around blender more robust.

The problem was that the Image Texture nodes in the material graphs had their `image_user` objects misconfigured. The `image_user` (https://docs.blender.org/api/current/bpy.types.ImageUser.html) is an internal object, that lets renderer know how to use a texture. It's not meant to be directly accessed by the user, so it has no direct representation in the UI. It is indirectly accessed when you create a texture. Try adding a texture node to some material, and then set it to some video (or change its source type to Movie or Image Sequence) - you'll see the options like Frames, Start Frame, Offset, etc. So this is used when creating animated textures (here's your use case for a 100 frame texture). But it has no sense with texture type set to Singe Image, as was in the file. There's also an advanced mode of the Outliner, meant for the scripters use, that can show all the internal data structures of a blend file. It's called "Data API" (second dropdown on the left in the outliner header). There you can drill down to the objects in question. The path would be `Materials`>`MATERIAL_NAME`>`Node Tree`>`Nodes`>`TEXTURE_NODE_NAME`>`Image User`. I'll remind that changing stuff directly in the Data API outliner mode will most likely break something if you don't know what you're doing. So in your case the importer script created those object with wrong parameters. It has set the texture type to Single Image, but duration, start frame and current frame were set up as if it were an animation. Which makes no sense (at least to me). You can see what was set to what values when you run my script on the misconfigured file in `fix = False` mode. The output (in the system console) will show something like this for each misconfigured texture node: ``` Material: 'wraps-1' Node: '4be_Charmer_gloves_bump_01' Using texture: 4be_Charmer_gloves_bump_01 from D:\Games\DAZ 3D\MyLibrary\Runtime\textures\4blueyes\Charmer\4be_Charmer_gloves_bump_01.jpg Has frame_offset: 0 Has frame_start: 1 Has frame_duration: 100 Has frame_current: 0 ``` With this `image_user` configuration and the Persistent Images option checked the renderer will advance the texture current frame on each animation frame rendered, allocate new slots for the texture images frame, render the frame, don't free the render slots (because persistent images means it). Then on the next frame it will check if all the needed images are loaded - it checks that the image and it's frame are the same. The image frame will differ, so once again it will allocate new render slots, an so on. The memory usage will go through the roof very quickly. Considering that the "single image but multi frame" configuration isn't legal, it can't be said that Blender did something wrong here. As I've already invested into this investigation, I'm now curious to know how exactly did it happen. Can you drop a link to the exact version of the importer plugin (and maybe the data it was importing) so I can take a look at it? Of course if such sharing isn't restricted by it's license. Just don't upload the files here. A drive link, as in your report, would be perfect. Not sure I will be able to hunt the problem down, as I have no clue what DAZ is, still I'm curious to check it out. I hope the Blender team will forgive this little off topic conversation here in the name of finding the truth and making the infrastructure around blender more robust.

Vincent Blankfield removed their assignment 2020-07-04 19:19:34 +02:00

shaun commented

2020-07-05 18:40:46 +02:00

Hi Vincent.

Thanks again so much for follow up. I totally understand that bug tracker is not meant for user support which this is kind of turning into. Thankyou none the less as this issue has plagued me for several versions of blender. Possibly others too so it's good this exsist for someone may search this and get help..

Thomas has joined in and created a bug report here and linked to my report. Not sure why tho. I asked him about it.
Anyways I opened a case for him to look into this at his Bitbucket

https://bitbucket.org/Diffeomorphic/import_daz/issues/122/image_user-objects-for-the-textures-in

Anyway Daz is a 3d program with much free content and lots of paid content. It's no way as powerful as blender as to why many will use Thomas's plugin to directly import a Daz project.

Thomas's plugin is out of this world in terms of powerful features to do what it does. The length and time and work required to bring in Daz content use to be painful he has simplified this to the extent where it's basically a single click! so I thank him also! He has scripted ways for the skeleton to convert to rigify.. seriously he has performed magic..

I know of a site to get the Daz content.. :P I have some 600GB of stuff on my disks and some really sweet models that cycles renders beautifully

Anyways give me a little bit of time to compile up a free use scenario for you to able to play with daz studio

It's all free so if you have spare time and wanted to look at it. It's available free. Go grab it, install it let the download manager download all the free models. I think genesis is there

The content is where money is made I guess. They do give you a few freebies via download manager tho. But the free stuff I don't know of the level texture sizes etc.. most stuff I have are 4k sometimes more.

I will see if I can replicate issue using default content and share a project file.

None the less Thomas seems to have stated he rectified the issue but had no time to test it.

Thanks Vincent.

Shaun.

EDIT: I think what threw me to think this was a blender issue, was I have downloaded .blend files from few websites.. and not only my project files have the issue. It might be lots more people use Daz then I thought. Or maybe in some scenario blender might add image as movie? This is unknown to me.. yeah so that's why I pushed this problem to hard..

Hi Vincent. Thanks again so much for follow up. I totally understand that bug tracker is not meant for user support which this is kind of turning into. Thankyou none the less as this issue has plagued me for several versions of blender. Possibly others too so it's good this exsist for someone may search this and get help.. Thomas has joined in and created a bug report here and linked to my report. Not sure why tho. I asked him about it. Anyways I opened a case for him to look into this at his Bitbucket https://bitbucket.org/Diffeomorphic/import_daz/issues/122/image_user-objects-for-the-textures-in Anyway Daz is a 3d program with much free content and lots of paid content. It's no way as powerful as blender as to why many will use Thomas's plugin to directly import a Daz project. Thomas's plugin is out of this world in terms of powerful features to do what it does. The length and time and work required to bring in Daz content use to be painful he has simplified this to the extent where it's basically a single click! so I thank him also! He has scripted ways for the skeleton to convert to rigify.. seriously he has performed magic.. I know of a site to get the Daz content.. :P I have some 600GB of stuff on my disks and some really sweet models that cycles renders beautifully Anyways give me a little bit of time to compile up a free use scenario for you to able to play with daz studio It's all free so if you have spare time and wanted to look at it. It's available free. Go grab it, install it let the download manager download all the free models. I think genesis is there The content is where money is made I guess. They do give you a few freebies via download manager tho. But the free stuff I don't know of the level texture sizes etc.. most stuff I have are 4k sometimes more. I will see if I can replicate issue using default content and share a project file. None the less Thomas seems to have stated he rectified the issue but had no time to test it. Thanks Vincent. Shaun. EDIT: I think what threw me to think this was a blender issue, was I have downloaded .blend files from few websites.. and not only my project files have the issue. It might be lots more people use Daz then I thought. Or maybe in some scenario blender might add image as movie? This is unknown to me.. yeah so that's why I pushed this problem to hard..

Vincent Blankfield commented

2020-07-05 19:45:42 +02:00

Added subscriber: @thomasl-3

Vincent Blankfield commented

2020-07-05 19:45:42 +02:00

Nevermind the request for content. A bug report @thomasl-3 created explains the situation perfectly. Looks like there is a weird default hardcoded value of 100 frame duration. I've already found where it is, changed, tested - works fine in this usecase. Will post a patch after updating the repos, rebuilding and testing. Will need some devs to look at it, as there may have been some reason for such value I don't know of.

blender-admin commented

2020-07-16 14:54:45 +02:00

This issue was referenced by a71490c4b2

This issue was referenced by a71490c4b2fc08d677c3c20403c847753aa52bf4

blender-admin commented

2020-07-16 14:54:45 +02:00

This issue was referenced by c7aa0f9d74

This issue was referenced by c7aa0f9d743ee9cafe91b4a51d9f1ecc5e437d71

Brecht Van Lommel commented

2020-07-16 15:51:25 +02:00

Changed status from 'Archived' to: 'Resolved'

shaun commented

2020-07-18 08:54:01 +02:00

Hi Brecht,

Can you advise me how I can track the progress of this fix?

Has this been applied already to 2.83.2 or 2.90 daily?
Is this the right link to see the actual fixes applied to blendeR?
https://developer.blender.org/T77348

many thanks

Hi Brecht, Can you advise me how I can track the progress of this fix? Has this been applied already to 2.83.2 or 2.90 daily? Is this the right link to see the actual fixes applied to blendeR? https://developer.blender.org/T77348 many thanks

Peter B commented

2021-08-30 03:38:07 +02:00

Added subscriber: @PJB

Peter B commented

2021-08-30 03:38:07 +02:00

Hi Guys,

This problem still exits in Blender 2.93.

Using Cycles, I can minimise the memory problem by selecting Persistent Data - Checked. In this case, if I a render of the scene without the Daz figure set to Show In Renders - Unchecked, it takes about 5 secs. If I then set the Daz figure to Show in Renders - Checked On, the render takes several minutes to complete.

If I render using Eevee I get an immediate pause and eventually a GPU memory exceeded (NVidia Quadro RTX 4000) at the frame where Show in Renders - Checked On is set.

The figure I am using is generated in DAZ Studio 4.15 Pro using Genesis 8.1 Male with a Tactical Assault Outfit for Genesis 8 Male(s) and Female(s). The wardrobe is Armour, Tactical Assault Male Outfit Complete Day.

Export to Blender is through the Daz to Blender Bridge.

Thanks for your help

Peter

Hi Guys, This problem still exits in Blender 2.93. Using Cycles, I can minimise the memory problem by selecting Persistent Data - Checked. In this case, if I a render of the scene without the Daz figure set to Show In Renders - Unchecked, it takes about 5 secs. If I then set the Daz figure to Show in Renders - Checked On, the render takes several minutes to complete. If I render using Eevee I get an immediate pause and eventually a GPU memory exceeded (NVidia Quadro RTX 4000) at the frame where Show in Renders - Checked On is set. The figure I am using is generated in DAZ Studio 4.15 Pro using Genesis 8.1 Male with a Tactical Assault Outfit for Genesis 8 Male(s) and Female(s). The wardrobe is Armour, Tactical Assault Male Outfit Complete Day. Export to Blender is through the Daz to Blender Bridge. Thanks for your help Peter

Vincent Blankfield commented

2021-08-30 14:36:04 +02:00

Hi Peter,

I've tested the original issue with the original .blend file - can't reproduce it.

Just to clarify things, the original issue was that for the models imported with some addons, texture memory was doubling every frame rendered with Persistent Images = True. The textures were loaded each frame and never discarded. This lead to memory usage going overboard while rendering animation. Normal behaviour for persistent images is to load the images once and reuse them. With persistent images unchecked, the textures are loaded and discarded each frame. The issue was that they were re-loaded each frame but never discarded.

From your description I understand that checking the persistent data setting helps to mitigate the issue. In the original case the issue appeared with the persistent images checked, which is opposite. Overall, it seems to me that what you have described is a typical out of memory situation. In general, being unable to render a scene with insufficient memory isn't considered to be a bug, unless Blender is doing something unexpected that leads to OOM. Can't say more about your issue without an example file and a test case (steps to do, expected result, actual result).

Pro tip: It's usually better to create a new bug report than posting a comment on an old one. Comments like this may slip through unnoticed, because only the task subscribers will see them.

Hi Peter, I've tested the original issue with the original .blend file - can't reproduce it. Just to clarify things, the original issue was that for the models imported with some addons, texture memory was doubling every frame rendered with Persistent Images = True. The textures were loaded each frame and never discarded. This lead to memory usage going overboard while rendering animation. Normal behaviour for persistent images is to load the images once and reuse them. With persistent images unchecked, the textures are loaded and discarded each frame. The issue was that they were re-loaded each frame but never discarded. From your description I understand that checking the persistent data setting helps to mitigate the issue. In the original case the issue appeared with the persistent images checked, which is opposite. Overall, it seems to me that what you have described is a typical out of memory situation. In general, being unable to render a scene with insufficient memory isn't considered to be a bug, unless Blender is doing something unexpected that leads to OOM. Can't say more about your issue without an example file and a test case (steps to do, expected result, actual result). Pro tip: It's usually better to create a new bug report than posting a comment on an old one. Comments like this may slip through unnoticed, because only the task subscribers will see them.

Peter B commented

2021-09-02 12:02:17 +02:00

Thanks for the rapid response Vincent.

Your clarification corrected my misunderstanding of your fix. I interpreted what you have said in your original response back to front - got it now!

Needless to say, after you said you could not replicate the error I went back to my original exports from Daz to refine the workflow for further error checking. I had also done some additional reading on OBJ and FBX imports.

What I discovered was that an animation run after a Daz to Blender import took as much as 10 minutes per png in Cycles and completely stopped with a Graphics card out of memory error under Eevee. At that rate it would take days to complete.

When I did the same animation from an FBX export the animation rendered each png in about 5 seconds. WOW!!!

Apologies for not reading your fix correctly but for information of others running into a Graphics card out of memory error I would recommend doing a Daz export using FBX in lieu of the Daz to Blender Bridge.

Cheers

Peter

Thanks for the rapid response Vincent. Your clarification corrected my misunderstanding of your fix. I interpreted what you have said in your original response back to front - got it now! Needless to say, after you said you could not replicate the error I went back to my original exports from Daz to refine the workflow for further error checking. I had also done some additional reading on OBJ and FBX imports. What I discovered was that an animation run after a Daz to Blender import took as much as 10 minutes per png in Cycles and completely stopped with a Graphics card out of memory error under Eevee. At that rate it would take days to complete. When I did the same animation from an FBX export the animation rendered each png in about 5 seconds. WOW!!! Apologies for not reading your fix correctly but for information of others running into a Graphics card out of memory error I would recommend doing a Daz export using FBX in lieu of the Daz to Blender Bridge. Cheers Peter

shaun commented

2021-09-02 15:13:06 +02:00

I can recall this problem is Daz model related.
Something about the way the materials import as movies or something
If you take a look at a material you should see that it's configured to image sequence and breaks blender or something. I forgot exactly.

This is a good find as I recently tried to use the persistence mode and keep crashing on my sys hahaha..

I remember a script can remove this from all the materials.
Let me see if I can find it. I have another post or ticket here where the bloke that made diffomorphic helped to solve the issue.

His latest version of the blender importer of Daz files no longer generates materials that way.

Cheers

Edit: link to script to clean up older Daz models with the dodgy materials

https://developer.blender.org/T78646

I can recall this problem is Daz model related. Something about the way the materials import as movies or something If you take a look at a material you should see that it's configured to image sequence and breaks blender or something. I forgot exactly. This is a good find as I recently tried to use the persistence mode and keep crashing on my sys hahaha.. I remember a script can remove this from all the materials. Let me see if I can find it. I have another post or ticket here where the bloke that made diffomorphic helped to solve the issue. His latest version of the blender importer of Daz files no longer generates materials that way. Cheers Edit: link to script to clean up older Daz models with the dodgy materials https://developer.blender.org/T78646

Vincent Blankfield commented

2021-09-02 16:10:18 +02:00

@PJB, Nice find. I suspect it may be related to this issue, as @toxsickcity has mentioned. I will look into it sometime later. In the meantime, it would be great if someone provided a simple example file.

@toxsickcity, The script you have linked is a demonstration of the issue. I believe you meant my clean-up demo script here: https://developer.blender.org/T78537#972624 . Though it isn't supposed to be used in production. It was just a simple demo to make sure I understood the problem.

I do suspect the issue may be related to the image_user objects. Though I haven't looked into it. Installing Daz and the bridge, familiarizing myself with it and recreating the issue will take time, which I currently lack. ~~What's curious is iirc, the bridge addon got a patch that fixed the image_user issue on their end by overriding the defaults~~ (edit: that was Diffeomorphic Daz Importer, not the bundled bridge). As for Blender, I proposed 3 patches for this issue: one made Cycles not bloat-up memory when image_user objects had incorrect values (D8242), and the other two (D8213 and D8228) fixed the incorrect default values in the first place. The latter two have been rejected.

Edit: Installed DAZ and the bridge, tried importing some ready made models, couldn't reproduce the issue. Saw no significant difference in rendering time or memory usage with Cycles/EEVEE, FBX/Bridge import, Persistent Data checked/unchecked. A proper bug report with an example is needed to further investigate this issue.

@PJB, Nice find. I suspect it may be related to this issue, as @toxsickcity has mentioned. I will look into it sometime later. In the meantime, it would be great if someone provided a simple example file. @toxsickcity, The script you have linked is a demonstration of the issue. I believe you meant my clean-up demo script here: https://developer.blender.org/T78537#972624 . Though it isn't supposed to be used in production. It was just a simple demo to make sure I understood the problem. I do suspect the issue may be related to the `image_user` objects. Though I haven't looked into it. Installing Daz and the bridge, familiarizing myself with it and recreating the issue will take time, which I currently lack. ~~What's curious is iirc, the bridge addon got a patch that fixed the `image_user` issue on their end by overriding the defaults~~ (edit: that was Diffeomorphic Daz Importer, not the bundled bridge). As for Blender, I proposed 3 patches for this issue: one made Cycles not bloat-up memory when `image_user` objects had incorrect values ([D8242](https://archive.blender.org/developer/D8242)), and the other two ([D8213](https://archive.blender.org/developer/D8213) and [D8228](https://archive.blender.org/developer/D8228)) fixed the incorrect default values in the first place. The latter two have been rejected. Edit: Installed DAZ and the bridge, tried importing some ready made models, couldn't reproduce the issue. Saw no significant difference in rendering time or memory usage with Cycles/EEVEE, FBX/Bridge import, Persistent Data checked/unchecked. A proper bug report with an example is needed to further investigate this issue.

Peter B commented

2021-09-04 06:40:54 +02:00

Soldier Seated 6.blend1

Soldier Seated 7 FBX Import.blend

Soldier Seated 6.blend

Soldier Seated 6.blend1

Hi Guys,

I have striped the files so there is only a single actor. The FBX imports are in the first two files and the Daz to Blender Bridge import are in the second two.. When you run them you will see the significant performance difference.

Cheers

[Soldier Seated 6.blend1](https://archive.blender.org/developer/F10375335/Soldier_Seated_6.blend1) [Soldier Seated 7 FBX Import.blend](https://archive.blender.org/developer/F10375340/Soldier_Seated_7_FBX_Import.blend) [Soldier Seated 6.blend](https://archive.blender.org/developer/F10375448/Soldier_Seated_6.blend) [Soldier Seated 6.blend1](https://archive.blender.org/developer/F10375473/Soldier_Seated_6.blend1) Hi Guys, I have striped the files so there is only a single actor. The FBX imports are in the first two files and the Daz to Blender Bridge import are in the second two.. When you run them you will see the significant performance difference. Cheers

Peter B commented

2021-09-04 07:22:02 +02:00

Hmm, no FBX... .blend1. Let me know if you need one.

Vincent Blankfield commented

2021-09-04 09:52:54 +02:00

The .blend1 files aren't needed. Those are just backups.

The Soldier Seated 6.blend file is missing textures (the actual image files are in your Daz 3D folder). You can pack all external data into the .blend file using File>External Data>Pack Resources. But in that case the file may get quite huge. And would be a bad idea to upload it here.

Anyway, I've replaced the missing textures in the Bridge-imported file with a dummy 4k image. On my machine the Bridge-imported file renders a frame in 2 to 15 seconds with Eevee, using around 17GB of system and 6GB of GPU memory. The FBX-imported one gives about 2FPS, using 1.5GB GPU and 1.8 system memory.

Can confirm that the Bridge-imported file renders significantly slower than the FBX-imported one. But that should be expected since the Bridge-imported one uses 10 times more textures. The FBX-imported file contains 12 images and 50 image users, the Bridge-imported one has 132 images and 141 users. Image users are configured the same way in both files, so this issue isn't related to the previous bug. Also "fixing" the image users' settings doesn't affect the rendering time or memory usage. Frankly, I don't see this as a bug at all. The Bridge-imported file simply has more complex materials, and more textures (judging by the names, some of them seem to be the same texture duplicated multiple times). See the Chest Plate Male_13422.Shape mesh material for example: In the Bridge-imported file (called Chest Plate Male_13422_Chest Plate) it has 6 textures connected to the IrayUber node group with some logic inside. FBX-imported file has just one texture, directly connected to Principled BSDF (material is called Chest Plate).

The `.blend1` files aren't needed. Those are just backups. The `Soldier Seated 6.blend` file is missing textures (the actual image files are in your Daz 3D folder). You can pack all external data into the `.blend` file using `File`>`External Data`>`Pack Resources`. But in that case the file may get quite huge. And would be a bad idea to upload it here. Anyway, I've replaced the missing textures in the Bridge-imported file with a dummy 4k image. On my machine the Bridge-imported file renders a frame in 2 to 15 seconds with Eevee, using around 17GB of system and 6GB of GPU memory. The FBX-imported one gives about 2FPS, using 1.5GB GPU and 1.8 system memory. Can confirm that the Bridge-imported file renders significantly slower than the FBX-imported one. But that should be expected since the Bridge-imported one uses 10 times more textures. The FBX-imported file contains 12 images and 50 image users, the Bridge-imported one has 132 images and 141 users. Image users are configured the same way in both files, so this issue isn't related to the previous bug. Also "fixing" the image users' settings doesn't affect the rendering time or memory usage. Frankly, I don't see this as a bug at all. The Bridge-imported file simply has more complex materials, and more textures (judging by the names, some of them seem to be the same texture duplicated multiple times). See the `Chest Plate Male_13422.Shape` mesh material for example: In the Bridge-imported file (called `Chest Plate Male_13422_Chest Plate`) it has 6 textures connected to the `IrayUber` node group with some logic inside. FBX-imported file has just one texture, directly connected to `Principled BSDF` (material is called `Chest Plate`).

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

High Memory Usage when render animation with Persistant Images enabled, and using Fix (T77734) reproducible with Latest 2.90 Blender #78537

Save, and you may try rendering. Note that previous renders leave a lot of stuff in memory. So for the best experience reopen the file.