Vertex group (memory access?) change after applying modifier #93896

System Information
Operating system: Linux-4.15.0-159-generic-x86_64-with-glibc2.27 64 Bits
Graphics card: GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 460.91.03

Blender Version
Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557
Worked: 2,93

Looks like this came with 263fa406cd: Fix #90087: Assigning object data doesn't copy vertex groups / 99738fbfdc: Fix memory leak from 263fa406cd

Short description of error
I have an object with an existing vertex group.
I then run a script, that creates a new vertex group and stores it in a variable accordingly.
The script then creates a modifier - any kind I think - and applies it using the object.modifier_apply() op.

In 3.0 the vgroup variable now references the existing vertex group instead of the newly created one, that was assigned to it.
The memory address remains unchanged however - at least on linux.
Removing the vgroup referenced by the variable will then remove the existing one, instead of the new one.
I think on windows, it may cause an exception but I need confirmation of that from a Windows user.

In 2.93, everything works as expected, the vgroup assigned to the variable never changes and is removed properly

See this video demo .

vgroup_changs_after_applying_modifier3.blend

Exact steps for others to reproduce the error

open the attached blend file in 2.93 and 3.0
go to the Scripting workspace and run the script
note how in 3.0 the print out of the vgroup variable changes after the modifier is applied

Added subscriber: @MACHIN3

#98751 was marked as duplicate of this issue

I think I should clarify some more.

On Linux, this is the output I get in 3.0:

<bpy_struct, VertexGroup("new_vgroup") at 0x7fe6da812968>
<bpy_struct, VertexGroup("existing_vgroup") at 0x7fe6da812968>

Note the different vgroups even though its the same variable and the memory address is the same. There is no exception when removing the vgroup. However the existing vgroup is removed of course, instead of the new one.

While in 2.93 it's this

<bpy_struct, VertexGroup("new_vgroup") at 0x7fea6ae871e8>
<bpy_struct, VertexGroup("new_vgroup") at 0x7fea6ae871e8>

Same vgroup before and after applying the mod. The new vgroup is removed as intended.

This is the result on Windows

version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557, type: release
build date: 2021-12-03, 00:44:02
platform: 'Windows-10-10.0.19042-SP0'
renderer: 'NVIDIA GeForce RTX 2060 SUPER/PCIe/SSE2'
vendor: 'NVIDIA Corporation'
version: '4.5.0 NVIDIA 472.47'

Note how here in 3.0, the vgroup seems unchanged after applying the modifier. Yet removing the new vgroup throws an exception.

Traceback (most recetn call last):
File "C:\Users\K\Desktop\vgroup_changs_after_applying_modifier.blend\Text", line 22, in
RuntimeError: Error: DeformGroup 'new_vgroup' not in object 'Cube'

In 2.93 the behavior is identical as on Linux. The new group is removed without any issues.

Added subscriber: @ostapblender

Yup, I can see exactly the same.
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 970/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 472.12
Worked: version: 2.93.0, branch: master, commit date: 2021-06-02 11:21, hash: 84da05a8b8
Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557

2.93

3.0

You didn't get an exception on Windows? It get's more interesting by the minute.

Added subscriber: @Nurb2Kea

On macOS 12..0.1
blender 2.93.5 keeps existing vgroup
blender 3.0 stable & last beta keeps both vgroups
blender 3.1 from blender.org works as intended
blender 3.1 self compiled & works as intended (as of 9-december 12o'clock updated and compiled)

Added subscriber: @lichtwerk

Can confirm the behavior on Linux, will bisect to see what changed this.
(I can imagine operations such as applying a modifier can screw with memory, not sure if references from python are garuanteed to be valid after this -- but I dont see this particularly mentioned in https://docs.blender.org/api/3.0/info_gotcha.html either)

Added subscriber: @aschellekens09

Can confirm on win10, latest stable 3.0

First attempt no exception raised, on a second attempt however:

RuntimeError: Error: DeformGroup 'new_vgroup' not in object 'Cube'

third attempt no error.

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1650/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 472.47

Blender Version
Broken: version: 3.0.0, branch: master, commit date: 2021-12-02 18:35, hash: f1cca30557

I tested it again several times with 3.0 stable and beta.
One time everything works, the next time not. It works one/off.
Totally wired. Not kidding!

Downloaded the testfile also several times to be save. Worked 3 times and then failes.!?

PS: Same with 3.1
2.93.5 keeps the existing no new one
I'm a bit confused. Once again tested it with all versions + new compiled one 3.1, no clue.
You can open this file several times, and getting always another result. (also redownloaded the file)

Changed status from 'Needs Triage' to: 'Confirmed'

Added subscribers: @HooglyBoogly, @ideasman42

So looks like this came with 263fa406cd / 99738fbfdc

Since BKE_mesh_nomain_to_mesh is called for applying modifiers too, this now duplicates [as in new memory allocated an memcopied] over vertexgroups (even though it seems unneccessary in this case).
@HooglyBoogly , @ideasman42 : I wonder if this can somehow be made conditionally? Like so?

P2653: T93896_snippet



diff --git a/source/blender/blenkernel/intern/mesh_convert.cc b/source/blender/blenkernel/intern/mesh_convert.cc
index e8054884f26..47fd2d0e0c2 100644
--- a/source/blender/blenkernel/intern/mesh_convert.cc
+++ b/source/blender/blenkernel/intern/mesh_convert.cc
@@ -1586,9 +1586,16 @@ void BKE_mesh_nomain_to_mesh(Mesh *mesh_src,
   /* skip the listbase */
   MEMCPY_STRUCT_AFTER(mesh_dst, &tmp, id.prev);
 
-  BLI_freelistN(&mesh_dst->vertex_group_names);
-  BKE_defgroup_copy_list(&mesh_dst->vertex_group_names, &mesh_src->vertex_group_names);
-  mesh_dst->vertex_group_active_index = mesh_src->vertex_group_active_index;
+  /* Hm, how can we tell if the need to duplicate vertexgroups over?
+   * This is also done when applying modifiers, guess we dont want to change py references to existing ones?
+   * If we come here via py new_from_object(), then we want to duplicate over for sure...
+   * This will make both #90087 & #93896 work. */
+  if (BLI_listbase_is_empty(&mesh_dst->vertex_group_names)) {
+    BLI_freelistN(&mesh_dst->vertex_group_names);
+    BKE_defgroup_copy_list(&mesh_dst->vertex_group_names, &mesh_src->vertex_group_names);
+    mesh_dst->vertex_group_active_index = mesh_src->vertex_group_active_index;
+  }
+
 
   if (take_ownership) {
     if (alloctype == CD_ASSIGN) {

Quite possible this covers not all cases, so maybe this would need to be more granular (comparing each existing vertexgroup with the new ones?)

Added subscribers: @Sergey, @dr.sybren

@dr.sybren / @Sergey might be interested as well

In #93896#1269065, @MACHIN3 wrote:
You didn't get an exception on Windows? It get's more interesting by the minute.

Well, when you've mentioned it I did run script for a dozen of times and it is tend to return different results indeed.

Sometimes its returning "existing_group", sometimes it's "new_group" both times, rarely an exception and mostly Blender just freezes for a few seconds while performing script.

Added subscriber: @riprazor

Thanks for the investigation Philipp. I don't really have enough background in this area to know whether that's expected or not. It does seem like something that would happen pretty frequently with other data.

This is a more generic issues: you can not store pointer to a data as python variable and rely on it being valid after underlying object modification. Is just similar to how you can not reference a vertex and hope that it will be valid after doing a topology change operation on a mesh.

If the issue is only that the python variable points to an invalid memory but the object itself in a consistent state then to me it seems more a known issue than a bug in Blender.

That's pretty inconvenient though, especially since it used for work perfectly in the past. What would you recommend? Keeping track of the vgroup names and use that to fetch them?

It is inconvenient indeed, but that's just outcome of how Python integration is done (at least from my understanding unless something did change).
Using names will work reliably for fetching vgroup of interest. Additionally, this will allow you to keep track of vgroup from an evaluated object, where the underlying pointer of vgroup changes with every object evaluation. So name is a portable solution for all cases I'd say.

Understood, thank you. Much appreciated! Feel free to close.

Added subscribers: @Mysteryem-2, @OmarEmaraDev

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Vertex group (memory access?) change after applying modifier #93896