Vertex group (memory access?) change after applying modifier #93896

New Issue

MACHIN3 · 2021-12-09T12:04:01+01:00

MACHIN3 commented

2021-12-09 12:04:01 +01:00

System Information
Operating system: Linux-4.15.0-159-generic-x86_64-with-glibc2.27 64 Bits
Graphics card: GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.91.03

Blender Version
Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557
Worked: 2,93

Looks like this came with 263fa406cd: Fix #90087: Assigning object data doesn't copy vertex groups / 99738fbfdc: Fix memory leak from 263fa406cd

Short description of error
I have an object with an existing vertex group.
I then run a script, that creates a new vertex group and stores it in a variable accordingly.
The script then creates a modifier - any kind I think - and applies it using the object.modifier_apply() op.

In 3.0 the vgroup variable now references the existing vertex group instead of the newly created one, that was assigned to it.
The memory address remains unchanged however - at least on linux.
Removing the vgroup referenced by the variable will then remove the existing one, instead of the new one.
I think on windows, it may cause an exception but I need confirmation of that from a Windows user.

In 2.93, everything works as expected, the vgroup assigned to the variable never changes and is removed properly

See this video demo .

vgroup_changs_after_applying_modifier3.blend

Exact steps for others to reproduce the error

open the attached blend file in 2.93 and 3.0
go to the Scripting workspace and run the script
note how in 3.0 the print out of the vgroup variable changes after the modifier is applied

**System Information** Operating system: Linux-4.15.0-159-generic-x86_64-with-glibc2.27 64 Bits Graphics card: GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.91.03 **Blender Version** Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: `f1cca30557` Worked: 2,93 Looks like this came with 263fa406cd: Fix #90087: Assigning object data doesn't copy vertex groups / 99738fbfdc: Fix memory leak from 263fa406cd **Short description of error** I have an object with an existing vertex group. I then run a script, that creates a new vertex group and stores it in a variable accordingly. The script then creates a modifier - any kind I think - and applies it using the `object.modifier_apply()` op. In 3.0 the vgroup variable now references the existing vertex group instead of the newly created one, that was assigned to it. The memory address remains unchanged however - at least on linux. Removing the vgroup referenced by the variable will then remove the existing one, instead of the new one. I think on windows, it may cause an exception but I need confirmation of that from a Windows user. In 2.93, everything works as expected, the vgroup assigned to the variable never changes and is removed properly See this [video demo ](https://youtu.be/WMGjQKEgNco). [vgroup_changs_after_applying_modifier3.blend](https://archive.blender.org/developer/F12719190/vgroup_changs_after_applying_modifier3.blend) **Exact steps for others to reproduce the error** * open the attached blend file in 2.93 and 3.0 * go to the Scripting workspace and run the script * note how in 3.0 the print out of the vgroup variable changes after the modifier is applied

MACHIN3 commented

2021-12-09 12:04:01 +01:00

Added subscriber: @MACHIN3

blender-admin commented

2021-12-09 12:04:01 +01:00

#98751 was marked as duplicate of this issue

MACHIN3 commented

2021-12-09 13:25:01 +01:00

I think I should clarify some more.

On Linux, this is the output I get in 3.0:

<bpy_struct, VertexGroup("new_vgroup") at 0x7fe6da812968>
<bpy_struct, VertexGroup("existing_vgroup") at 0x7fe6da812968>

Note the different vgroups even though its the same variable and the memory address is the same. There is no exception when removing the vgroup. However the existing vgroup is removed of course, instead of the new one.

While in 2.93 it's this

<bpy_struct, VertexGroup("new_vgroup") at 0x7fea6ae871e8>
<bpy_struct, VertexGroup("new_vgroup") at 0x7fea6ae871e8>

Same vgroup before and after applying the mod. The new vgroup is removed as intended.

I think I should clarify some more. On Linux, this is the output I get in 3.0: > <bpy_struct, VertexGroup("new_vgroup") at 0x7fe6da812968> > <bpy_struct, VertexGroup("existing_vgroup") at 0x7fe6da812968> Note the different vgroups even though its the same variable and the memory address is the same. There is no exception when removing the vgroup. However the existing vgroup is removed of course, instead of the new one. While in 2.93 it's this > <bpy_struct, VertexGroup("new_vgroup") at 0x7fea6ae871e8> > <bpy_struct, VertexGroup("new_vgroup") at 0x7fea6ae871e8> Same vgroup before and after applying the mod. The new vgroup is removed as intended.

MACHIN3 commented

2021-12-09 13:41:45 +01:00

This is the result on Windows

version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557, type: release
build date: 2021-12-03, 00:44:02
platform: 'Windows-10-10.0.19042-SP0'
renderer: 'NVIDIA GeForce RTX 2060 SUPER/PCIe/SSE2'
vendor: 'NVIDIA Corporation'
version: '4.5.0 NVIDIA 472.47'

Note how here in 3.0, the vgroup seems unchanged after applying the modifier. Yet removing the new vgroup throws an exception.

Traceback (most recetn call last):
File "C:\Users\K\Desktop\vgroup_changs_after_applying_modifier.blend\Text", line 22, in
RuntimeError: Error: DeformGroup 'new_vgroup' not in object 'Cube'

In 2.93 the behavior is identical as on Linux. The new group is removed without any issues.

This is the result on Windows > version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca3055776, type: release > build date: 2021-12-03, 00:44:02 > platform: 'Windows-10-10.0.19042-SP0' > renderer: 'NVIDIA GeForce RTX 2060 SUPER/PCIe/SSE2' > vendor: 'NVIDIA Corporation' > version: '4.5.0 NVIDIA 472.47' ![windows_3.0.jpg](https://archive.blender.org/developer/F12719881/windows_3.0.jpg) Note how here in 3.0, the vgroup seems unchanged after applying the modifier. Yet removing the new vgroup throws an exception. > Traceback (most recetn call last): > File "C:\Users\K\Desktop\vgroup_changs_after_applying_modifier.blend\Text", line 22, in <module> > RuntimeError: Error: DeformGroup 'new_vgroup' not in object 'Cube' ![windows_2.93.jpg](https://archive.blender.org/developer/F12719895/windows_2.93.jpg) In 2.93 the behavior is identical as on Linux. The new group is removed without any issues.

Bohdan Lvov commented

2021-12-09 13:50:34 +01:00

Added subscriber: @ostapblender

Bohdan Lvov commented

2021-12-09 13:50:34 +01:00

Yup, I can see exactly the same.
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 970/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 472.12
Worked: version: 2.93.0, branch: master, commit date: 2021-06-02 11:21, hash: 84da05a8b8
Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557

2.93

3.0

Yup, I can see exactly the same. Operating system: Windows-10-10.0.19041-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 970/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 472.12 Worked: version: 2.93.0, branch: master, commit date: 2021-06-02 11:21, hash: 84da05a8b8 Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557 2.93 ![image.png](https://archive.blender.org/developer/F12719950/image.png) 3.0 ![image.png](https://archive.blender.org/developer/F12719940/image.png)

MACHIN3 commented

2021-12-09 13:53:54 +01:00

You didn't get an exception on Windows? It get's more interesting by the minute.

Guido commented

2021-12-09 14:16:40 +01:00

Added subscriber: @Nurb2Kea

Guido commented

2021-12-09 14:16:40 +01:00

On macOS 12..0.1
blender 2.93.5 keeps existing vgroup
blender 3.0 stable & last beta keeps both vgroups
blender 3.1 from blender.org works as intended
blender 3.1 self compiled & works as intended (as of 9-december 12o'clock updated and compiled)

On macOS 12..0.1 blender 2.93.5 keeps existing vgroup blender 3.0 stable & last beta keeps both vgroups blender 3.1 from blender.org works as intended blender 3.1 self compiled & works as intended (as of 9-december 12o'clock updated and compiled)

Philipp Oeser commented

2021-12-09 14:38:05 +01:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2021-12-09 14:38:05 +01:00

Can confirm the behavior on Linux, will bisect to see what changed this.
(I can imagine operations such as applying a modifier can screw with memory, not sure if references from python are garuanteed to be valid after this -- but I dont see this particularly mentioned in https://docs.blender.org/api/3.0/info_gotcha.html either)

Can confirm the behavior on Linux, will bisect to see what changed this. (I can imagine operations such as applying a modifier can screw with memory, not sure if references from python are garuanteed to be valid after this -- but I dont see this particularly mentioned in https://docs.blender.org/api/3.0/info_gotcha.html either)

zender commented

2021-12-09 15:06:35 +01:00

Added subscriber: @aschellekens09

zender commented

2021-12-09 15:06:35 +01:00

Can confirm on win10, latest stable 3.0

First attempt no exception raised, on a second attempt however:

RuntimeError: Error: DeformGroup 'new_vgroup' not in object 'Cube'

third attempt no error.

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1650/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 472.47

Blender Version
Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557

Can confirm on win10, latest stable 3.0 First attempt no exception raised, on a second attempt however: RuntimeError: Error: DeformGroup 'new_vgroup' not in object 'Cube' third attempt no error. **System Information** Operating system: Windows-10-10.0.19044-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1650/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 472.47 **Blender Version** Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: `f1cca30557`

Guido commented

2021-12-09 15:27:32 +01:00

I tested it again several times with 3.0 stable and beta.
One time everything works, the next time not. It works one/off.
Totally wired. Not kidding!

Downloaded the testfile also several times to be save. Worked 3 times and then failes.!?

PS: Same with 3.1
2.93.5 keeps the existing no new one
I'm a bit confused. Once again tested it with all versions + new compiled one 3.1, no clue.
You can open this file several times, and getting always another result. (also redownloaded the file)

I tested it again several times with 3.0 stable and beta. One time everything works, the next time not. It works one/off. Totally wired. Not kidding! Downloaded the testfile also several times to be save. Worked 3 times and then failes.!? PS: Same with 3.1 2.93.5 keeps the existing no new one I'm a bit confused. Once again tested it with all versions + new compiled one 3.1, no clue. You can open this file several times, and getting always another result. (also redownloaded the file)

Philipp Oeser commented

2021-12-09 16:19:17 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser commented

2021-12-09 16:19:17 +01:00

Added subscribers: @HooglyBoogly, @ideasman42

Philipp Oeser commented

2021-12-09 16:19:17 +01:00

So looks like this came with 263fa406cd / 99738fbfdc

Since BKE_mesh_nomain_to_mesh is called for applying modifiers too, this now duplicates [as in new memory allocated an memcopied] over vertexgroups (even though it seems unneccessary in this case).
@HooglyBoogly , @ideasman42 : I wonder if this can somehow be made conditionally? Like so?

P2653: T93896_snippet



diff --git a/source/blender/blenkernel/intern/mesh_convert.cc b/source/blender/blenkernel/intern/mesh_convert.cc
index e8054884f26..47fd2d0e0c2 100644
--- a/source/blender/blenkernel/intern/mesh_convert.cc
+++ b/source/blender/blenkernel/intern/mesh_convert.cc
@@ -1586,9 +1586,16 @@ void BKE_mesh_nomain_to_mesh(Mesh *mesh_src,
   /* skip the listbase */
   MEMCPY_STRUCT_AFTER(mesh_dst, &tmp, id.prev);
 
-  BLI_freelistN(&mesh_dst->vertex_group_names);
-  BKE_defgroup_copy_list(&mesh_dst->vertex_group_names, &mesh_src->vertex_group_names);
-  mesh_dst->vertex_group_active_index = mesh_src->vertex_group_active_index;
+  /* Hm, how can we tell if the need to duplicate vertexgroups over?
+   * This is also done when applying modifiers, guess we dont want to change py references to existing ones?
+   * If we come here via py new_from_object(), then we want to duplicate over for sure...
+   * This will make both #90087 & #93896 work. */
+  if (BLI_listbase_is_empty(&mesh_dst->vertex_group_names)) {
+    BLI_freelistN(&mesh_dst->vertex_group_names);
+    BKE_defgroup_copy_list(&mesh_dst->vertex_group_names, &mesh_src->vertex_group_names);
+    mesh_dst->vertex_group_active_index = mesh_src->vertex_group_active_index;
+  }
+
 
   if (take_ownership) {
     if (alloctype == CD_ASSIGN) {

Quite possible this covers not all cases, so maybe this would need to be more granular (comparing each existing vertexgroup with the new ones?)

So looks like this came with 263fa406cd / 99738fbfdc Since `BKE_mesh_nomain_to_mesh` is called for applying modifiers too, this now duplicates [as in new memory allocated an memcopied] over vertexgroups (even though it seems unneccessary in this case). @HooglyBoogly , @ideasman42 : I wonder if this can somehow be made conditionally? Like so? [P2653: T93896_snippet](https://archive.blender.org/developer/P2653.txt) ``` diff --git a/source/blender/blenkernel/intern/mesh_convert.cc b/source/blender/blenkernel/intern/mesh_convert.cc index e8054884f26..47fd2d0e0c2 100644 --- a/source/blender/blenkernel/intern/mesh_convert.cc +++ b/source/blender/blenkernel/intern/mesh_convert.cc @@ -1586,9 +1586,16 @@ void BKE_mesh_nomain_to_mesh(Mesh *mesh_src, /* skip the listbase */ MEMCPY_STRUCT_AFTER(mesh_dst, &tmp, id.prev); - BLI_freelistN(&mesh_dst->vertex_group_names); - BKE_defgroup_copy_list(&mesh_dst->vertex_group_names, &mesh_src->vertex_group_names); - mesh_dst->vertex_group_active_index = mesh_src->vertex_group_active_index; + /* Hm, how can we tell if the need to duplicate vertexgroups over? + * This is also done when applying modifiers, guess we dont want to change py references to existing ones? + * If we come here via py new_from_object(), then we want to duplicate over for sure... + * This will make both #90087 & #93896 work. */ + if (BLI_listbase_is_empty(&mesh_dst->vertex_group_names)) { + BLI_freelistN(&mesh_dst->vertex_group_names); + BKE_defgroup_copy_list(&mesh_dst->vertex_group_names, &mesh_src->vertex_group_names); + mesh_dst->vertex_group_active_index = mesh_src->vertex_group_active_index; + } + if (take_ownership) { if (alloctype == CD_ASSIGN) { ``` Quite possible this covers not all cases, so maybe this would need to be more granular (comparing each existing vertexgroup with the new ones?)

Philipp Oeser changed title from ~~vertex group (memory access?) change after applying modifier~~ to Vertex group (memory access?) change after applying modifier

2021-12-09 16:20:12 +01:00

Philipp Oeser commented

2021-12-09 16:24:06 +01:00

Added subscribers: @Sergey, @dr.sybren

Philipp Oeser commented

2021-12-09 16:24:06 +01:00

@dr.sybren / @Sergey might be interested as well

Bohdan Lvov commented

2021-12-09 18:27:22 +01:00

In #93896#1269065, @MACHIN3 wrote:
You didn't get an exception on Windows? It get's more interesting by the minute.

Well, when you've mentioned it I did run script for a dozen of times and it is tend to return different results indeed.

Sometimes its returning "existing_group", sometimes it's "new_group" both times, rarely an exception and mostly Blender just freezes for a few seconds while performing script.

> In #93896#1269065, @MACHIN3 wrote: > You didn't get an exception on Windows? It get's more interesting by the minute. Well, when you've mentioned it I did run script for a dozen of times and it is tend to return different results indeed. ![image.png](https://archive.blender.org/developer/F12721829/image.png) Sometimes its returning "existing_group", sometimes it's "new_group" both times, rarely an exception and mostly Blender just freezes for a few seconds while performing script.

Barry Libenson commented

2021-12-10 17:56:32 +01:00

Added subscriber: @riprazor

Hans Goudey commented

2021-12-13 22:57:18 +01:00

Thanks for the investigation Philipp. I don't really have enough background in this area to know whether that's expected or not. It does seem like something that would happen pretty frequently with other data.

Sergey Sharybin commented

2022-01-04 12:24:46 +01:00

This is a more generic issues: you can not store pointer to a data as python variable and rely on it being valid after underlying object modification. Is just similar to how you can not reference a vertex and hope that it will be valid after doing a topology change operation on a mesh.

If the issue is only that the python variable points to an invalid memory but the object itself in a consistent state then to me it seems more a known issue than a bug in Blender.

This is a more generic issues: you can not store pointer to a data as python variable and rely on it being valid after underlying object modification. Is just similar to how you can not reference a vertex and hope that it will be valid after doing a topology change operation on a mesh. If the issue is only that the python variable points to an invalid memory but the object itself in a consistent state then to me it seems more a known issue than a bug in Blender.

MACHIN3 commented

2022-01-04 12:27:50 +01:00

That's pretty inconvenient though, especially since it used for work perfectly in the past. What would you recommend? Keeping track of the vgroup names and use that to fetch them?

Sergey Sharybin commented

2022-01-05 14:45:55 +01:00

It is inconvenient indeed, but that's just outcome of how Python integration is done (at least from my understanding unless something did change).
Using names will work reliably for fetching vgroup of interest. Additionally, this will allow you to keep track of vgroup from an evaluated object, where the underlying pointer of vgroup changes with every object evaluation. So name is a portable solution for all cases I'd say.

It is inconvenient indeed, but that's just outcome of how Python integration is done (at least from my understanding unless something did change). Using names will work reliably for fetching vgroup of interest. Additionally, this will allow you to keep track of vgroup from an evaluated object, where the underlying pointer of vgroup changes with every object evaluation. So name is a portable solution for all cases I'd say.

MACHIN3 commented

2022-01-05 15:07:46 +01:00

Understood, thank you. Much appreciated! Feel free to close.

Omar Emara commented

2022-06-10 14:19:42 +02:00

Added subscribers: @Mysteryem-2, @OmarEmaraDev

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Vertex group (memory access?) change after applying modifier #93896