Animating the speed setting in Rigid Body World causes strange behavior with constraints #38391

New Issue

gandalf3 · 2014-01-29T03:04:27+01:00

gandalf3 commented

2014-01-29 03:04:27 +01:00

System Information
Archlinux
Nvidia GTX 460

Blender Version
Broken: 2.69, 2.69.10 (c7ac0ed)

Short description of error
Animating the speed value causes constraints to behave weirdly.

Exact steps for others to reproduce the error
Based on rigid-body-speed.blend

Open attached file
Play animation. When the ball hits the stack of cubes, the /Speed// value gets set to .01. For some reason this makes the constraints behave in a very odd way.
Remove the keyframes and play the animation again. The motion is as expected.

Changing the speed works fine without constraints though, see the other scene in the attached file.

**System Information** Archlinux Nvidia GTX 460 **Blender Version** Broken: 2.69, 2.69.10 (c7ac0ed) **Short description of error** Animating the speed value causes constraints to behave weirdly. **Exact steps for others to reproduce the error** Based on [rigid-body-speed.blend](https://archive.blender.org/developer/F75558/rigid-body-speed.blend) 1. Open attached file 2. Play animation. When the ball hits the stack of cubes, the /Speed// value gets set to .01. For some reason this makes the constraints behave in a very odd way. 3. Remove the keyframes and play the animation again. The motion is as expected. Changing the speed works fine without constraints though, see the other scene in the attached file.

gandalf3 commented

2014-01-29 03:04:27 +01:00

Changed status to: 'Open'

gandalf3 commented

2014-01-29 03:04:27 +01:00

Added subscriber: @gandalf3

Matias S commented

2014-01-29 08:29:20 +01:00

Added subscriber: @CodeMatias

Matias S commented

2014-01-29 08:29:20 +01:00

It looks like a feature/bug where the simulation framerate is not dependent on the speed<-should be real framerate, which results in non-standard behavior. If you bake using 1000hz (or 600Hz) simulation instead of 60hz you get a reasonable result.

edit Upon further testing (and looking closely at both impact point and chain), it's not so much an issue with blender or bullet as it is poor simulation setup. The chain was made with point constraints that are under high displacement before impact, which was over-corrected when frame-rate was increased (ERP is dependent on frame-rate rather than simulation time). As expected using constraint over-ride or higher simulation iterations solves the issue, and applying the same ratios as speed to other parameters results in very similar joint displacement.Perhaps it's time to allow other solvers to be exposed? The current bullet2 code includes the 2.82 updates for MLCP solvers, which should provide hard constraints without time resolution dependence.

It looks like a feature/bug where the simulation framerate is not dependent on the *speed*<-should be real framerate, which results in non-standard behavior. If you bake using 1000hz (or 600Hz) simulation instead of 60hz you get a reasonable result. *edit* Upon further testing (and looking closely at both impact point and chain), it's not so much an issue with blender or bullet as it is poor simulation setup. The chain was made with point constraints that are under high displacement before impact, which was over-corrected when frame-rate was increased (ERP is dependent on frame-rate rather than simulation time). As expected using constraint over-ride or higher simulation iterations solves the issue, and applying the same ratios as speed to other parameters results in very similar joint displacement.Perhaps it's time to allow other solvers to be exposed? The current bullet2 code includes the 2.82 updates for MLCP solvers, which should provide hard constraints without time resolution dependence.

Lukas Tönne commented

2014-01-29 11:37:50 +01:00

Added subscriber: @LukasTonne

Lukas Tönne commented

2014-01-29 11:37:50 +01:00

I think this is a problem with the way our rigid body system defines substeps for Bullet currently:
https://developer.blender.org/diffusion/B/browse/master/source/blender/blenkernel/intern/rigidbody.c;39eb314cb922b805e9126d5f0352f31c2f84f151$1367

Quoting from Bullet wiki pages:
"Bullet maintains an internal clock, in order to keep the actual length of ticks constant. This is pivotally important for framerate independence."

What happens with the time_scale value is that both the overall time step for a single frame as well as the "fixed" time step for Bullet are scaled. This is supposed to ensure a consistent resolution of ticks for every frame, but it looks like that throws off the Bullet solver, especially with such extreme differences (2.0 vs. 0.01).

I've managed to get the expected result by ignoring time scale for the fixed substeps, but this leads to very choppy motion since Bullet then only does a real tick every 30 frames or so ...

I think this is a problem with the way our rigid body system defines substeps for Bullet currently: https://developer.blender.org/diffusion/B/browse/master/source/blender/blenkernel/intern/rigidbody.c;39eb314cb922b805e9126d5f0352f31c2f84f151$1367 Quoting from [Bullet wiki pages](http://bulletphysics.org/mediawiki-1.5.8/index.php/Stepping_The_World): "Bullet maintains an internal clock, in order to keep the actual length of ticks constant. This is pivotally important for framerate independence." What happens with the time_scale value is that both the overall time step for a single frame *as well as the "fixed" time step* for Bullet are scaled. This is supposed to ensure a consistent resolution of ticks for every frame, but it looks like that throws off the Bullet solver, especially with such extreme differences (2.0 vs. 0.01). I've managed to get the expected result by ignoring time scale for the fixed substeps, but this leads to very choppy motion since Bullet then only does a real tick every 30 frames or so ...

Sergej Reich commented

2014-01-29 12:12:34 +01:00

Added subscriber: @sreich

Sergej Reich commented

2014-01-29 12:12:34 +01:00

Yeah, it's a side effect of the way we do time scaling. Maybe it would be better to have it not change the steps per second (I tried that as well but you have to manually adjust the steps per seconds then, which is often not obvious).
From what I can tell, the problem doesn't necessarily have to do with not having a fixed time step though. It's jsut that the constraints get really stiff, when you do so many steps. You can get similar problems even without using the speed property.

You can try to play with the steps per second and see if it makes a difference (in this case increasing it even further seems to help).

Yeah, it's a side effect of the way we do time scaling. Maybe it would be better to have it not change the steps per second (I tried that as well but you have to manually adjust the steps per seconds then, which is often not obvious). From what I can tell, the problem doesn't necessarily have to do with not having a fixed time step though. It's jsut that the constraints get really stiff, when you do so many steps. You can get similar problems even without using the speed property. You can try to play with the steps per second and see if it makes a difference (in this case increasing it even further seems to help).

Matias S commented

2014-01-31 07:03:09 +01:00

"From what I can tell, the problem doesn't necessarily have to do with not having a fixed time step though. It's jsut that the constraints get really stiff, when you do so many steps. You can get similar problems even without using the speed property."

Yes, it's caused by a sudden shift of any parameter that changes total constraint error correction per frame (speed, fps, iteration count, which determine joint error correction), and it's not limited to blender alone (did quick test in 2.82). Only real ways to fix it are:

Document and tell users to keep iterations-per-frame changes small
Apply some sort of filter to keep iteration-per-frame changes small (i.e. clamp step size * iteration changes to ~25% between frames), perhaps as an option
Use hard solvers like the MLCP (Dantzing/ Lemke) ones (doesn't solve everything since penetration issues would also cause a similar issue). Hard solver implementation should be the easiest of the bunch for those willing to build a modified version of the physics engine, it literally involves only replacing https://developer.blender.org/diffusion/B/browse/master/intern/rigidbody/rb_bullet_api.cpp;39eb314cb922b805e9126d5f0352f31c2f84f151$160 with the MLCP version (MLCP has auto-fallback to PGS if it fails, Building solver option into the system will be more complicated though). I can't really say if it will play nice with Blender in all cases, but this one is simple enough that it should be fine.

Currently Blender doesn't have proper joint motor implementations (i.e. hinge joint with internal motor), but if those are added the SequentialImpulseSolver will show motor strength variance depending on both iterations per frame and fps due to the same issue. Motors with extremely high fps can also be unstable especially if the speed is changed and it over-corrects for an error. It's not necessarily a bug, rather a limitation of the approximation used and settings for the simulation. Again, MLCP code fixes a lot of that at the cost of performance in some cases.

"From what I can tell, the problem doesn't necessarily have to do with not having a fixed time step though. It's jsut that the constraints get really stiff, when you do so many steps. You can get similar problems even without using the speed property." Yes, it's caused by a sudden shift of any parameter that changes total constraint error correction per frame (speed, fps, iteration count, which determine joint error correction), and it's not limited to blender alone (did quick test in 2.82). Only real ways to fix it are: - Document and tell users to keep iterations-per-frame changes small - Apply some sort of filter to keep iteration-per-frame changes small (i.e. clamp step size * iteration changes to ~25% between frames), perhaps as an option - Use hard solvers like the MLCP (Dantzing/ Lemke) ones (doesn't solve everything since penetration issues would also cause a similar issue). Hard solver implementation should be the easiest of the bunch for those willing to build a modified version of the physics engine, it literally involves only replacing https://developer.blender.org/diffusion/B/browse/master/intern/rigidbody/rb_bullet_api.cpp;39eb314cb922b805e9126d5f0352f31c2f84f151$160 with the MLCP version (MLCP has auto-fallback to PGS if it fails, Building solver option into the system will be more complicated though). I can't really say if it will play nice with Blender in all cases, but this one is simple enough that it should be fine. Currently Blender doesn't have proper joint motor implementations (i.e. hinge joint with internal motor), but if those are added the SequentialImpulseSolver will show motor strength variance depending on both iterations per frame and fps due to the same issue. Motors with extremely high fps can also be unstable especially if the speed is changed and it over-corrects for an error. It's not necessarily a bug, rather a limitation of the approximation used and settings for the simulation. Again, MLCP code fixes a lot of that at the cost of performance in some cases.

Sergej Reich commented

2014-01-31 08:47:14 +01:00

As far as I can see the best fix for now would be to either completely disallow animating the properties that intfuence timestep or just warn that bigger changes will have side effects.

gandalf3 commented

2014-01-31 08:53:14 +01:00

@sreich Animating the timestep is quite useful for slow motion effects, etc. As a user I would rather it not be disabled...
And it works fine in cases with no constraints.

BTW, I want to say thanks for implementing this feature :)

@sreich Animating the timestep is quite useful for slow motion effects, etc. As a user I would rather it not be disabled... And it works fine in cases with no constraints. BTW, I want to say thanks for implementing this feature :)

Sergej Reich commented

2014-01-31 12:18:48 +01:00

@gandalf3 it's not just a problem with constraints, if you change the speed of the simulation or the steps per second during collisions you'll have the same problem. So be careful.

After looking into it for a bit more, there might be a way of fixing it. That will have to wait for 2.71 though, I wouldn't want to risk introducing regression so close to release.

@CodeMatias: Using the MLCP solvers in bullet is not really an option for us unfortunately, they're just too slow. Also they don't fix this particular problem.

@gandalf3 it's not just a problem with constraints, if you change the speed of the simulation or the steps per second during collisions you'll have the same problem. So be careful. After looking into it for a bit more, there might be a way of fixing it. That will have to wait for 2.71 though, I wouldn't want to risk introducing regression so close to release. @CodeMatias: Using the MLCP solvers in bullet is not really an option for us unfortunately, they're just too slow. Also they don't fix this particular problem.

Matias S commented

2014-01-31 16:37:17 +01:00

@sreich : The MLCP solvers are not perfect for linear displacements like this one, but they would help substantially in most common cases. In my Bullet simulations I generally find Dantzig MLCP to be about as effective as 100-200 iteration PGS with all other settings equal for minimizing joint error (the "bug"'s root cause). The .blend posted here works just fine if the iterations is cranked up to 200, so perhaps the MLCP solvers could have helped as well (though certainly not a 100% thing).

I can't talk for all cases, as I generally focus on robotic simulations with a low number of simulated items (3~18 motors in a chain), but the Dantzig solver tends to be only about three times slower than the standard PGS settings (10 iteration). In fact, it can be faster in some situations where you would otherwise need high iteration counts, especially when it doesn't have to fall back to PGS. The 200 iteration case I normally deal with actually makes Dantzig up to twice as fast as PGS. Of course the Lemke implementation, as it currently is, makes anything seem blazing fast; that algorithm would need serious baking time for even simple scenes.

But yes, it's not really necessary for Blender, though would be nice as an option.

Perhaps a realistic target for the next sprint would be making ERP dependent on Blender frame rate rather than bullet frame-rate? i.e. making framerate and iteration values fixed for a simulation (unlike now) and varying ERP based on the change in speed , perhaps using ERP = h kp / (h kp + kd) where kp and kd remain constant [ which should be ERP_corrected= ERP_base/ ( (1- ERP_base ) / Speed + ERP_base) ] . You should end up with joint error for a simulation dependent on the base "steps per second" and correction amount is the same for every Blender frame (no discontinuous stiffness). This would affect both joints and penetration too, which can be both good and bad depending on what you consider to be more necessary.
Since h is step size, you could possibly even make framerate variable and just replace Speed with a function of Speed and "steps per second", though it would require you to expose the ERP_base to the user and expect them to figure it out (because higher "steps per second" would no longer mean higher joint accuracy).

Unfortunately that goes beyond the simple stuff I could possibly submit as even a proof of concept patch, since it would involve exposing btContactSolverInfo()->m_erp and using it before the call linked to by lukastoenne (and so much more).

edit And I'm sure you probably have something better planned (which I noticed after reading the comment again)

@sreich : The MLCP solvers are not perfect for linear displacements like this one, but they would help substantially in most common cases. In my Bullet simulations I generally find Dantzig MLCP to be about as effective as 100-200 iteration PGS with all other settings equal for minimizing joint error (the "bug"'s root cause). The .blend posted here works just fine if the iterations is cranked up to 200, so perhaps the MLCP solvers could have helped as well (though certainly not a 100% thing). I can't talk for all cases, as I generally focus on robotic simulations with a low number of simulated items (3~18 motors in a chain), but the Dantzig solver tends to be only about three times slower than the standard PGS settings (10 iteration). In fact, it can be faster in some situations where you would otherwise need high iteration counts, especially when it doesn't have to fall back to PGS. The 200 iteration case I normally deal with actually makes Dantzig up to twice as fast as PGS. Of course the Lemke implementation, as it currently is, makes anything seem blazing fast; that algorithm would need serious baking time for even simple scenes. *But yes, it's not really necessary for Blender, though would be nice as an option.* Perhaps a realistic target for the next sprint would be making ERP dependent on Blender frame rate rather than bullet frame-rate? i.e. making framerate and iteration values fixed for a simulation (unlike now) and varying ERP based on the change in speed , perhaps using ERP = h kp / (h kp + kd) where kp and kd remain constant [ which should be ERP_corrected= ERP_base/ ( (1- ERP_base ) / Speed + ERP_base) ] . You should end up with joint error for a simulation dependent on the base "steps per second" and correction amount is the same for every Blender frame (no discontinuous stiffness). This would affect both joints and penetration too, which can be both good and bad depending on what you consider to be more necessary. Since h is step size, you could possibly even make framerate variable and just replace Speed with a function of Speed and "steps per second", though it would require you to expose the ERP_base to the user and expect them to figure it out (because higher "steps per second" would no longer mean higher joint accuracy). Unfortunately that goes beyond the simple stuff I could possibly submit as even a proof of concept patch, since it would involve exposing btContactSolverInfo()->m_erp and using it before the call linked to by lukastoenne (and so much more). *edit* And I'm sure you probably have something better planned (which I noticed after reading the comment again)

Sergej Reich was assigned by Sergey Sharybin

2014-02-03 12:56:49 +01:00

Sergey Sharybin commented

2014-02-03 12:56:49 +01:00

Added subscriber: @Sergey

Sergey Sharybin commented

2014-02-03 12:56:49 +01:00

@sreich, you're the rigid body boss, so assigning to you to take care of the issue. Feel free to either fix, re-assign or move to TODO :)

Sergey Sharybin commented

2014-06-03 10:10:03 +02:00

@sreich, any chance looking into the report? Disabling animation of time-related things seem fine to me. It's rather tricky to support anyway..

gandalf3 commented

2014-06-03 10:17:33 +02:00

I would personally rather it be left as is than be disabled entirely, as it's very useful for neat slo-mo effects, even with the current limitations..

I would personally rather it be left as is than be disabled entirely, as it's *very* useful for neat slo-mo effects, even with the current limitations..

Sergej Reich commented

2014-08-27 20:03:33 +02:00

Ok, I think messing with bullet's main loop is not a good idea after all.
Closing this, as a limitation.

Ok, I think messing with bullet's main loop is not a good idea after all. Closing this, as a limitation.

Sergej Reich commented

2014-08-27 20:04:11 +02:00

Changed status from 'Open' to: 'Archived'

Sergej Reich closed this issue

2014-08-27 20:04:11 +02:00

Joseph Poole commented

2014-10-01 02:27:55 +02:00

Added subscriber: @JosephPoole

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Animating the speed setting in Rigid Body World causes strange behavior with constraints #38391