Memory leak when exporting to gltf (using blender as a python module) #116319

New Issue

Julie-Fabre · 2023-12-19T02:22:31+01:00

Julie-Fabre commented

2023-12-19 02:22:31 +01:00

System Information
Operating system: MacOS Ventura
Operating system: Linux-6.6.8-200.fc39.x86_64-x86_64-with-glibc2.38 64 Bits, X11 UI

Blender Version
bpy 4.0.0
bpy version: 4.1.0 Alpha, branch: main, commit date: 2024-01-21 18:49, hash: 3df7939eec96

Short description of error
Memory leak when exporting to gltf (using blender as a python module)

Exact steps for others to reproduce the error

install bpy as a python module (following https://developer.blender.org/docs/handbook/building_blender/python_module/ or through other sources such as pip)
save the following script and call it with an appropriate output filename
(it just exports the startup file as gltf [or FBX or alembic])
when exporting to gltf, a memory leak is reported

import bpy

bpy.ops.wm.read_factory_settings()

bpy.ops.export_scene.gltf(filepath = sys.argv[1]) # Error: Not freed memory blocks: 27936, total unfreed memory 9.345375 MB
#bpy.ops.export_scene.fbx(filepath = sys.argv[1]) # no leaks reported
#bpy.ops.wm.alembic_export(filepath = sys.argv[1]) # no leaks reported

Original Report

I'm using bpy (4.0.0) to convert USDZ to GLB and my script is showing errors like this when exiting:

Error: Not freed memory blocks: 25605, total unfreed memory 8.550426 MB

I'm new to Python and Blender so i'm assuming i need to free memory somewhere, but i'm not sure what's causing it, I've tried resetting Blender to factory settings before the program exits as well as setting iFile to None and deleting it.

I've also tried to call gc.collect() explicitly but that didn't help, I checked gc.garbage but it is empty ([]).

I'm thinking maybe there is an issue with usd_import or maybe export_scene?

The script

import bpy
import os
import sys
import gc
import tempfile
import shutil
import zipfile

# method definitions

#--------------------------------------------------------------------------------------------
#find the USDC file inside the specified directory
def findUSDC(dirpath):

  files = os.listdir(dirpath)
  dirs = []

  for file in files:
    parts = file.split('.')
    filepath = os.path.join(dirpath, file)

    if os.path.isdir(filepath):
      dirs.append(filepath)
    elif len(parts) > 0 and parts[-1] == 'usdc':
      return filepath

  for dir in dirs:
    file = findUSDC(dir)
    if file != '':
      return file
  return ''

#--------------------------------------------------------------------------------------------
#unzip the specified USDZ file to a temp directory, look for the USDC file and return it
def extractUSDC(filepath):

  try:
    if os.path.exists(filepath) == False:
      raise Exception(filepath + " could not be found")

    filePath, fileName = os.path.split(filepath)
    fileName, fileType = fileName.split('.')
  except Exception as e:
    print(e, "unprocessable input")
    return None

  if fileType == 'usdz':
    with zipfile.ZipFile(filepath, 'r') as zf:

      try:
        print("---> temp path:"+tempPath)
        zf.extractall(tempPath)
      except Exception as e:
        print(e)

      zf.close()

      #get the usdc file
      usdcFile = findUSDC(tempPath)

      #recursively process the file
      return extractUSDC(usdcFile)

  elif fileType == 'usdc':
    print("--> USDC Found: "+filepath)
    return filepath
  else:
    print("unprocessable input")
    return None

#--------------------------------------------------------------------------------------------
# import the usdz, export the glb
def convert(usdzPath, glbPath):

  #clear possibly previously loaded data by starting with a blank scene
  bpy.ops.wm.read_factory_settings(use_empty = True)

  print("---> Loading File: ", usdzPath, os.path.getsize(usdzPath), "Bytes")

  #import the USD
  #see: https://docs.blender.org/api/current/bpy.ops.wm.html?highlight=usd#bpy.ops.wm.usd_import
  iFile = extractUSDC(usdzPath)

  if iFile == None:
      print("---> USDC extraction failed")
      return None

  bpy.ops.wm.usd_import(filepath = iFile, import_usd_preview = True)

  #export the GLB
  #see: https://docs.blender.org/api/current/bpy.ops.export_scene.html?highlight=glb#bpy.ops.export_scene.gltf
  bpy.ops.export_scene.gltf(filepath = glbPath)

  #clear all data in memory to avoid memory leaks
  bpy.ops.wm.read_factory_settings(use_empty = True)
  iFile = None
  del iFile

  print("---> Converted file: ", glbPath, os.path.getsize(glbPath), "Bytes")

#------------------------#
# main program execution #
#------------------------#

#make sure we have all the arguments
if len(sys.argv) != 3:
  print("Missing arguments.")
  print("Usage: usdz_to_glb.py file_to_convert.usdz file_converted.glb")
  exit(1)

gc.enable()

#Create a temp directory to extract to
#global so that after all the recusions we can clean it up
tempPath = tempfile.mkdtemp()

convert(sys.argv[1], sys.argv[2])

shutil.rmtree(tempPath)

gc.collect()

**System Information** Operating system: MacOS Ventura Operating system: Linux-6.6.8-200.fc39.x86_64-x86_64-with-glibc2.38 64 Bits, X11 UI **Blender Version** `bpy 4.0.0` bpy version: 4.1.0 Alpha, branch: main, commit date: 2024-01-21 18:49, hash: `3df7939eec96` **Short description of error** Memory leak when exporting to gltf (using blender as a python module) **Exact steps for others to reproduce the error** - install `bpy` as a python module (following https://developer.blender.org/docs/handbook/building_blender/python_module/ or through other sources such as pip) - save the following script and call it with an appropriate output filename - (it just exports the startup file as gltf [or FBX or alembic]) - when exporting to gltf, a memory leak is reported ```python import bpy bpy.ops.wm.read_factory_settings() bpy.ops.export_scene.gltf(filepath = sys.argv[1]) # Error: Not freed memory blocks: 27936, total unfreed memory 9.345375 MB #bpy.ops.export_scene.fbx(filepath = sys.argv[1]) # no leaks reported #bpy.ops.wm.alembic_export(filepath = sys.argv[1]) # no leaks reported ``` **Original Report** I'm using `bpy` (`4.0.0`) to convert USDZ to GLB and my script is showing errors like this when exiting: ``` Error: Not freed memory blocks: 25605, total unfreed memory 8.550426 MB ``` I'm new to Python and Blender so i'm assuming i need to free memory somewhere, but i'm not sure what's causing it, I've tried resetting Blender to factory settings before the program exits as well as setting `iFile` to `None` and deleting it. I've also tried to call `gc.collect()` explicitly but that didn't help, I checked `gc.garbage` but it is empty (`[]`). I'm thinking maybe there is an issue with `usd_import` or maybe `export_scene`? **The script** ```Py import bpy import os import sys import gc import tempfile import shutil import zipfile # method definitions #-------------------------------------------------------------------------------------------- #find the USDC file inside the specified directory def findUSDC(dirpath): files = os.listdir(dirpath) dirs = [] for file in files: parts = file.split('.') filepath = os.path.join(dirpath, file) if os.path.isdir(filepath): dirs.append(filepath) elif len(parts) > 0 and parts[-1] == 'usdc': return filepath for dir in dirs: file = findUSDC(dir) if file != '': return file return '' #-------------------------------------------------------------------------------------------- #unzip the specified USDZ file to a temp directory, look for the USDC file and return it def extractUSDC(filepath): try: if os.path.exists(filepath) == False: raise Exception(filepath + " could not be found") filePath, fileName = os.path.split(filepath) fileName, fileType = fileName.split('.') except Exception as e: print(e, "unprocessable input") return None if fileType == 'usdz': with zipfile.ZipFile(filepath, 'r') as zf: try: print("---> temp path:"+tempPath) zf.extractall(tempPath) except Exception as e: print(e) zf.close() #get the usdc file usdcFile = findUSDC(tempPath) #recursively process the file return extractUSDC(usdcFile) elif fileType == 'usdc': print("--> USDC Found: "+filepath) return filepath else: print("unprocessable input") return None #-------------------------------------------------------------------------------------------- # import the usdz, export the glb def convert(usdzPath, glbPath): #clear possibly previously loaded data by starting with a blank scene bpy.ops.wm.read_factory_settings(use_empty = True) print("---> Loading File: ", usdzPath, os.path.getsize(usdzPath), "Bytes") #import the USD #see: https://docs.blender.org/api/current/bpy.ops.wm.html?highlight=usd#bpy.ops.wm.usd_import iFile = extractUSDC(usdzPath) if iFile == None: print("---> USDC extraction failed") return None bpy.ops.wm.usd_import(filepath = iFile, import_usd_preview = True) #export the GLB #see: https://docs.blender.org/api/current/bpy.ops.export_scene.html?highlight=glb#bpy.ops.export_scene.gltf bpy.ops.export_scene.gltf(filepath = glbPath) #clear all data in memory to avoid memory leaks bpy.ops.wm.read_factory_settings(use_empty = True) iFile = None del iFile print("---> Converted file: ", glbPath, os.path.getsize(glbPath), "Bytes") #------------------------# # main program execution # #------------------------# #make sure we have all the arguments if len(sys.argv) != 3: print("Missing arguments.") print("Usage: usdz_to_glb.py file_to_convert.usdz file_converted.glb") exit(1) gc.enable() #Create a temp directory to extract to #global so that after all the recusions we can clean it up tempPath = tempfile.mkdtemp() convert(sys.argv[1], sys.argv[2]) shutil.rmtree(tempPath) gc.collect() ```

Julie-Fabre added the

labels 2023-12-19 02:22:32 +01:00

Philipp Oeser commented

2023-12-22 11:26:22 +01:00

Error: Not freed memory blocks: 25605, total unfreed memory 8.550426 MB

Do you get the same leaks reported when you do the steps manually?
If so, at which step does this occur?
the usd_import?
the export_scene.gltf?

Might also help if you attach an example USDZ (since I cannot reproduce with a simple test file here)

> Error: Not freed memory blocks: 25605, total unfreed memory 8.550426 MB Do you get the same leaks reported when you do the steps manually? If so, at which step does this occur? the `usd_import`? the `export_scene.gltf`? Might also help if you attach an example USDZ (since I cannot reproduce with a simple test file here)

Philipp Oeser added

Status

Needs Information from User

and removed

Status

Needs Triage

labels 2023-12-22 11:26:32 +01:00

Julie-Fabre commented

2023-12-22 16:20:51 +01:00

Error: Not freed memory blocks: 25605, total unfreed memory 8.550426 MB

Do you get the same leaks reported when you do the steps manually?
If so, at which step does this occur?
the usd_import?
the export_scene.gltf?

Might also help if you attach an example USDZ (since I cannot reproduce with a simple test file here)

If i only run usd_import there is no error, it only happens if i call the export_scene.gltf

I've attached both filed that i've tried this with, both give me the memory error.
Note that the converted gecko has the eyes texture floating way above where they are supposed to be and the alberto looks much darker, not sure how this could be fixed but i guess thats a different discussion.

> > Error: Not freed memory blocks: 25605, total unfreed memory 8.550426 MB > > Do you get the same leaks reported when you do the steps manually? > If so, at which step does this occur? > the `usd_import`? > the `export_scene.gltf`? > > Might also help if you attach an example USDZ (since I cannot reproduce with a simple test file here) If i only run `usd_import` there is no error, it only happens if i call the `export_scene.gltf` I've attached both filed that i've tried this with, both give me the memory error. Note that the converted `gecko` has the eyes texture floating way above where they are supposed to be and the `alberto` looks much darker, not sure how this could be fixed but i guess thats a different discussion.

gecko.usdz

4.6 MiB

alberto.usdz

8.9 MiB

Philipp Oeser added

Status

Needs Triage

and removed

Status

Needs Information from User

labels 2023-12-27 11:50:30 +01:00

Philipp Oeser commented

2023-12-28 14:06:03 +01:00

I cant spot leaks using the provided files when doing this through blenders UI.

@Julie-Fabre : do you also get leaks reported when you do this from inside blender?

I'm using bpy (4.0.0)

So you only get the memory leaks when using blender as a python module?

I cant spot leaks using the provided files when doing this through blenders UI. @Julie-Fabre : do you also get leaks reported when you do this from inside blender? > I'm using bpy (4.0.0) So you only get the memory leaks when using blender as a python module?

Philipp Oeser added

Status

Needs Information from User

and removed

Status

Needs Triage

labels 2023-12-28 14:06:14 +01:00

Julie-Fabre commented

2023-12-29 04:47:01 +01:00

I haven't tried through the UI I don't have that installed I don't know how to use it I just need to use it through scripting since it's for a web app, but I guess I can give it a try and figure it out just to rule things out, I'll report back.

Julie-Fabre commented

2023-12-30 16:57:01 +01:00

I cant spot leaks using the provided files when doing this through blenders UI.

@Julie-Fabre : do you also get leaks reported when you do this from inside blender?

I'm using bpy (4.0.0)

So you only get the memory leaks when using blender as a python module?

So it seems inside blender no errors pop up so i guess this just happens using blender as a python module.
Any ideas how to solve this?

> I cant spot leaks using the provided files when doing this through blenders UI. > > @Julie-Fabre : do you also get leaks reported when you do this from inside blender? > > > I'm using bpy (4.0.0) > > So you only get the memory leaks when using blender as a python module? So it seems inside blender no errors pop up so i guess this just happens using blender as a python module. Any ideas how to solve this?

Philipp Oeser changed title from ~~Memory leak when working with USDZ files~~ to Memory leak when working with USDZ files (when using blender as a python module)

2024-01-18 11:15:09 +01:00

Philipp Oeser commented

2024-01-18 11:15:46 +01:00

Will have to check again with blenderas a python module (sorry this has been lying around for a bit...)

Philipp Oeser added

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Memory leak when exporting to gltf (using blender as a python module) #116319