OpenVDB cache files have extra "_00" at end. #47655

New Issue

System Information
MacBook Pro (15-inch, Mid 2009)
Processor: 2.8 GHz Intel Core 2 Duo
Memory: 4 GB 1067 MHz DDR3
Graphics: NVIDIA GeForce 9600/9400M 256 MB

Blender Version
2.77 RC
date: 2016-02-26 12:32
Hash: b594b25
Branch: Head

Short description of error
When Smoke cache files are written, there is an extra "_00" at the end, just before the extension. So a files are usually called something like, SmokeCache_000010_00.vdb. Up till now this hasn't been a problem. But when using OpenVDB, it causes many problems when trying to load them into other programs. At this point you must manually rename the files by hand in order to get them to load. I have tested this with Modo and Vray in 3DSMax.

Exact steps for others to reproduce the error
No need for steps. It's Pretty obvious.

**System Information** MacBook Pro (15-inch, Mid 2009) Processor: 2.8 GHz Intel Core 2 Duo Memory: 4 GB 1067 MHz DDR3 Graphics: NVIDIA GeForce 9600/9400M 256 MB **Blender Version** 2.77 RC date: 2016-02-26 12:32 Hash: b594b25 Branch: Head **Short description of error** When Smoke cache files are written, there is an extra "_00" at the end, just before the extension. So a files are usually called something like, SmokeCache_000010_00.vdb. Up till now this hasn't been a problem. But when using OpenVDB, it causes many problems when trying to load them into other programs. At this point you must manually rename the files by hand in order to get them to load. I have tested this with Modo and Vray in 3DSMax. **Exact steps for others to reproduce the error** No need for steps. It's Pretty obvious.

Galen Beals commented

Changed status to: 'Open'

Galen Beals commented

Added subscriber: @galenbeals

blender-admin commented

#55151 was marked as duplicate of this issue

Kévin Dietrich commented

Added subscriber: @kevindietrich

Kévin Dietrich commented

Changed status from 'Open' to: 'Archived'

Kévin Dietrich closed this issue

Kévin Dietrich self-assigned this 2016-03-02 11:07:17 +01:00

Kévin Dietrich commented

2016-03-05 06:15:28 +01:00

This is not a bug but the way file names are generated by the point cache system. The "_00" is actually the index of the cache, so you can also have "_01", "_02", etc... It's annoying indeed, but changing that would require a complete rewrite of the overall cache system, which is kinda planned for Blender 2.8.

Galen Beals commented

Hi Kévin, I have a lot of respect for you and your work and I don't want to get on your bad side but this really isn't an acceptable answer. What was the point of using OpenVDB if you can't exchange cache files between apps? It certainly isn't a "Better" replacement for the current cacheing system. It actually hinders playback speed. So, if you can't easily use these caches in other applications, what is the point of having it. I know I'm coming off as confrontational but I have waited sooo long to get this feature just to be told, that's the way it's supposed to work. There must be a better solution. I'm on OS X and there's no simple way to batch rename files. In windows it might be simple to "rename" with wild cards in the command line, but in Unix the standard way rename files is with 'mv' which does't handle batches of files. I could write a shell script but is every user going to need to do this if they want to use the files in other programs? Not to mention that renaming the files invalidates them for use in Blender.

Also, while I know I don't understand the intricacies of the cacheing system, wouldn't it be trivial to just swap the order of how it reads and writes the file names; [CacheName][FrameNumber][Index].vdb to -> [CacheName][Index][FrameNumber].vdb? I can't imagine that would cause you to have to rewrite the cacheing system.

Hi Kévin, I have a lot of respect for you and your work and I don't want to get on your bad side but this really isn't an acceptable answer. What was the point of using OpenVDB if you can't exchange cache files between apps? It certainly isn't a "Better" replacement for the current cacheing system. It actually hinders playback speed. So, if you can't easily use these caches in other applications, what is the point of having it. I know I'm coming off as confrontational but I have waited sooo long to get this feature just to be told, that's the way it's supposed to work. There must be a better solution. I'm on OS X and there's no simple way to batch rename files. In windows it might be simple to "rename" with wild cards in the command line, but in Unix the standard way rename files is with 'mv' which does't handle batches of files. I could write a shell script but is every user going to need to do this if they want to use the files in other programs? Not to mention that renaming the files invalidates them for use in Blender. Also, while I know I don't understand the intricacies of the cacheing system, wouldn't it be trivial to just swap the order of how it reads and writes the file names; [CacheName]_[FrameNumber]_[Index].vdb to -> [CacheName]_[Index]_[FrameNumber].vdb? I can't imagine that would cause you to have to rewrite the cacheing system.

Erick Tukuniata commented

2016-03-05 13:31:49 +01:00

Added subscriber: @ErickNyanduKabongo

Erick Tukuniata commented

2016-03-05 13:31:49 +01:00

"What was the point of using OpenVDB if you can't exchange cache files between apps?" cache is using less disc space.
Convert yourself fully to Blender you will be saved. LOL

"What was the point of using OpenVDB if you can't exchange cache files between apps?" cache is using less disc space. Convert yourself fully to Blender you will be saved. LOL

Ton Roosendaal commented

2016-03-05 14:02:17 +01:00

Added subscriber: @Ton

Ton Roosendaal commented

2016-03-05 14:02:17 +01:00

Galen: when we add new features in Blender we should carefully try to be compatible with other programs. Having OpenVDB files to be exchanged in a mixed pipeline is a logical feature to implement.

What Kevin probably points at is an old convention in Blender for caches. We have to fix that, but it's not trivial. He didn't purposely design it to not work.

Someone mentioned that mac finder now supports batch rename. Select files, RMB menu.

Galen: when we add new features in Blender we should carefully try to be compatible with other programs. Having OpenVDB files to be exchanged in a mixed pipeline is a logical feature to implement. What Kevin probably points at is an old convention in Blender for caches. We have to fix that, but it's not trivial. He didn't purposely design it to not work. Someone mentioned that mac finder now supports batch rename. Select files, RMB menu.

Kévin Dietrich commented

2016-03-05 14:45:03 +01:00

As far as I know some people already managed to get .vdb files out of Blender working in other software (Octane and Houdini, iirc), with the only issue being that the grids are Z-up (but that's not much of a problem). So they are compatible with other software (and I open them in other software too).

Now the only real problem seems to be the file names (though, this is the first time I hear about this issue). Lukas kinda pointed at the problems the way names are generated by the point cache in his Alembic design task (#37578), and the index is pretty much required since you can have multiple caches in the same ID block, so it's not really a convention, more like a design issue. Maybe we could reorder the elements in the name (to .), but that will automatically break every existing caches, therefore some weird versioning would need to be done. I say "weird" because it's not in the DNA, so not straightforward (or is it?), and might cause some complication as far as as maintenance is concerned.

Now to comment on @galenbeals's comment:

the primary goal was smaller cache files, not exchange format, though that can work too.
it's not a replacement of the current cache system, but an extra file format for the current cache system.
it hinders on playback because you need to convert between VDB grids and dense buffers (which is single threaded currently).
FYI, on Linux there is the 'rename' command to do batch renaming.

As far as I know some people already managed to get .vdb files out of Blender working in other software (Octane and Houdini, iirc), with the only issue being that the grids are Z-up (but that's not much of a problem). So they are compatible with other software (and I open them in other software too). Now the only real problem seems to be the file names (though, this is the first time I hear about this issue). Lukas kinda pointed at the problems the way names are generated by the point cache in his Alembic design task (#37578), and the index is pretty much required since you can have multiple caches in the same ID block, so it's not really a convention, more like a design issue. Maybe we could reorder the elements in the name (to <name>_<index>_<frame>.<ext>), but that will automatically break every existing caches, therefore some weird versioning would need to be done. I say "weird" because it's not in the DNA, so not straightforward (or is it?), and might cause some complication as far as as maintenance is concerned. Now to comment on @galenbeals's comment: - the primary goal was smaller cache files, not exchange format, though that can work too. - it's not a replacement of the current cache system, but an extra file format for the current cache system. - it hinders on playback because you need to convert between VDB grids and dense buffers (which is single threaded currently). - FYI, on Linux there is the 'rename' command to do batch renaming.

Galen Beals commented

2016-03-05 20:36:27 +01:00

Ton: thanks, I'll have a look at the finder option.
kévin: thanks, I apriciate you keeping a level head while I lost mine. ;-) I think I understand now. It's more about backward compatibility than anything else. By the way, I understood that the cache files were designed like this before you even worked on it. I was just surprised that there wasn't a solution at this point. Thanks again.

Ton: thanks, I'll have a look at the finder option. kévin: thanks, I apriciate you keeping a level head while I lost mine. ;-) I think I understand now. It's more about backward compatibility than anything else. By the way, I understood that the cache files were designed like this before you even worked on it. I was just surprised that there wasn't a solution at this point. Thanks again.

Bastien Montagne commented

2018-07-22 13:07:51 +02:00

Added subscribers: @Rawalanche, @lichtwerk, @brecht, @dr.sybren

Thomas Volkmann commented

Added subscriber: @knekke

Thomas Volkmann commented

Changed status from 'Archived' to: 'Open'

Thomas Volkmann reopened this issue

Thomas Volkmann commented