Race condition during fluid simulation with IK-affected effectors #115636

New Issue

Eugene-Kuznetsov · 2023-11-30T22:57:35+01:00

Eugene-Kuznetsov commented

2023-11-30 22:57:35 +01:00

I'm observing a crash when trying to perform fluid simulation when some of the effectors are affected by an armature with IK.

The immediate cause of the crash is that two different threads attempt to execute iksolver_execute_tree (blender/source/blender/ikplugin/intern/iksolver_plugin.cc) for the same object and pchan_root, at the same time. They both attempt to allocate and free sub-objects inside pchan_root, resulting in a conflict and a segfault.

Stacks for the two calls look like so:
1.

#0  0x0000000007edf600 in iksolver_execute_tree ()
#1  0x000000000217e8e6 in BKE_pose_where_is(Depsgraph*, Scene*, Object*) ()
#2  0x000000000114aa49 in BKE_object_modifier_update_subframe(Depsgraph*, Scene*, Object*, bool, int, float, int) [clone .localalias] ()
#3  0x000000000114a5ff in BKE_object_modifier_update_subframe(Depsgraph*, Scene*, Object*, bool, int, float, int) [clone .localalias] ()
#4  0x0000000002407276 in update_flowsfluids(Depsgraph*, Scene*, Object*, FluidDomainSettings*, float, float, int, float) ()
#5  0x000000000241aa84 in BKE_fluid_modifier_do ()
#6  0x000000000392380f in fluid_modifier_do_isolated(void*) ()
#7  0x00007ffff7f54a55 in tbb::interface7::internal::isolate_within_arena(tbb::interface7::internal::delegate_base&, long) ()
#8  0x0000000002b591b5 in BLI_task_isolate ()

#0  0x0000000007edf600 in iksolver_execute_tree ()
#1  0x0000000002fb675c in blender::deg::(anonymous namespace)::evaluate_node(blender::deg::(anonymous namespace)::DepsgraphEvalState const*, blender::deg::OperationNode*) [clone .isra.0] ()
#2  0x0000000002fb70f5 in blender::deg::(anonymous namespace)::deg_task_run_func(TaskPool*, void*) ()
#3  0x0000000002b4fac1 in tbb::internal::function_task<Task>::execute() ()

I can work around the issue (very crudely) by slapping a global mutex on the entire body of iksolver_execute_tree(). I'm not sure how to do it cleaner.

I'm observing a crash when trying to perform fluid simulation when some of the effectors are affected by an armature with IK. The immediate cause of the crash is that two different threads attempt to execute iksolver_execute_tree (blender/source/blender/ikplugin/intern/iksolver_plugin.cc) for the same object and pchan_root, at the same time. They both attempt to allocate and free sub-objects inside pchan_root, resulting in a conflict and a segfault. Stacks for the two calls look like so: 1. ``` #0 0x0000000007edf600 in iksolver_execute_tree () #1 0x000000000217e8e6 in BKE_pose_where_is(Depsgraph*, Scene*, Object*) () #2 0x000000000114aa49 in BKE_object_modifier_update_subframe(Depsgraph*, Scene*, Object*, bool, int, float, int) [clone .localalias] () #3 0x000000000114a5ff in BKE_object_modifier_update_subframe(Depsgraph*, Scene*, Object*, bool, int, float, int) [clone .localalias] () #4 0x0000000002407276 in update_flowsfluids(Depsgraph*, Scene*, Object*, FluidDomainSettings*, float, float, int, float) () #5 0x000000000241aa84 in BKE_fluid_modifier_do () #6 0x000000000392380f in fluid_modifier_do_isolated(void*) () #7 0x00007ffff7f54a55 in tbb::interface7::internal::isolate_within_arena(tbb::interface7::internal::delegate_base&, long) () #8 0x0000000002b591b5 in BLI_task_isolate () ``` 2. ``` #0 0x0000000007edf600 in iksolver_execute_tree () #1 0x0000000002fb675c in blender::deg::(anonymous namespace)::evaluate_node(blender::deg::(anonymous namespace)::DepsgraphEvalState const*, blender::deg::OperationNode*) [clone .isra.0] () #2 0x0000000002fb70f5 in blender::deg::(anonymous namespace)::deg_task_run_func(TaskPool*, void*) () #3 0x0000000002b4fac1 in tbb::internal::function_task<Task>::execute() () ``` I can work around the issue (very crudely) by slapping a global mutex on the entire body of iksolver_execute_tree(). I'm not sure how to do it cleaner.

Eugene-Kuznetsov added the

labels 2023-11-30 22:57:36 +01:00

Iliya Katushenock commented

2023-12-01 09:10:25 +01:00

This report does not contain all the requested information, which is required for us to investigate the issue.

Please submit a new report and carefully follow the instructions. Be sure to provide system information, Blender version, the last Blender version which worked, and a .blend file with exact steps to reproduce the problem.

A guideline for making a good bug report can be found at https://wiki.blender.org/wiki/Process/Bug_Reports

This report does not contain all the requested information, which is required for us to investigate the issue. Please submit a new report and carefully follow the instructions. Be sure to provide system information, Blender version, the last Blender version which worked, and a .blend file with exact steps to reproduce the problem. A guideline for making a good bug report can be found at https://wiki.blender.org/wiki/Process/Bug_Reports

Iliya Katushenock closed this issue

2023-12-01 09:10:27 +01:00

Blender Bot added

and removed

labels 2023-12-01 09:10:29 +01:00

Eugene-Kuznetsov commented

2023-12-01 11:35:36 +01:00

Attaching a blend file that reproduces the issue. Go to "Fluid Domain" -> "Physics" and click on "Bake All". It may take a few tries but it will eventually crash.

I've observed the issue with head of trunk on Linux. I am now checking other tags and operating systems, will update the ticket with results.

Attaching a blend file that reproduces the issue. Go to "Fluid Domain" -> "Physics" and click on "Bake All". It may take a few tries but it will eventually crash. I've observed the issue with head of trunk on Linux. I am now checking other tags and operating systems, will update the ticket with results.

fluid_ik_crash.blend

749 KiB

Eugene-Kuznetsov reopened this issue

2023-12-01 11:35:39 +01:00

Blender Bot added

and removed

labels 2023-12-01 11:35:41 +01:00

Iliya Katushenock commented

2023-12-01 11:44:11 +01:00

Not able to confirm.

Eugene-Kuznetsov commented

2023-12-01 18:06:19 +01:00

Try this one

fluid_ik_crash_v2.blend

798 KiB

Pratik Borhade commented

2023-12-07 11:32:05 +01:00

Hi, thanks for the report. Unable to confirm either. Entire simulation is baked and played without problem. If threading is involved here, could you verify crash on single thread -t 1?
I'm not sure which blender version you're using. Could you share that? Also, are you building from source? Would be good if you check in release build.

Hi, thanks for the report. Unable to confirm either. Entire simulation is baked and played without problem. If threading is involved here, could you verify crash on single thread `-t 1`? I'm not sure which blender version you're using. Could you share that? Also, are you building from source? Would be good if you check in release build.

Pratik Borhade added

Status

Needs Information from User

and removed

Status

Needs Triage

labels 2023-12-07 11:32:12 +01:00

Eugene-Kuznetsov commented

2023-12-07 16:02:00 +01:00

Threading is definitely involved. iksolver_execute_tree is executed twice per frame per IK bone from different threads, and the crash occurs if the two calls for the same bone happen to overlap.

It crashes during baking. Sometimes it takes a while for the right thread alignment to occur. Try "Bake All" -> "Free All" a few times.

I'm building it from source, I've tested it and reproduced the crash with both 4.0.0 and head of trunk. I've also been able to reproduce the crash with the "official" 4.0.2 https://www.blender.org/download/release/Blender4.0/blender-4.0.2-linux-x64.tar.xz/

It does not crash with -t 1.

Threading is definitely involved. iksolver_execute_tree is executed twice per frame per IK bone from different threads, and the crash occurs if the two calls for the same bone happen to overlap. It crashes during baking. Sometimes it takes a while for the right thread alignment to occur. Try "Bake All" -> "Free All" a few times. I'm building it from source, I've tested it and reproduced the crash with both 4.0.0 and head of trunk. I've also been able to reproduce the crash with the "official" 4.0.2 https://www.blender.org/download/release/Blender4.0/blender-4.0.2-linux-x64.tar.xz/ It does not crash with -t 1.

Iliya Katushenock added the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Race condition during fluid simulation with IK-affected effectors #115636