Fix #119909: Unkeyable custom properties receive keyframes #119914

Merged
Christoph Lendenfeld merged 13 commits from ChrisLend/blender:fix_more_selective_when_keying_custom_props into main 2024-04-12 14:48:20 +02:00

In the case that "Custom Properties" was enabled in the user preferences,
the keyframing code would key all custom properties, regardless of that
property's type.
This can cause issues since it is keying e.g. the custom property that
cycles adds.

With this PR this is now limited to only Boolean, Int, Float, Double and Array.
Custom properties that have been defined via an addon are also not keyed.


Unit tests to cover this are added in this PR: #120422: WIP: Anim: Add unit tests for keying custom properties

In the case that "Custom Properties" was enabled in the user preferences, the keyframing code would key all custom properties, regardless of that property's type. This can cause issues since it is keying e.g. the custom property that cycles adds. With this PR this is now limited to only Boolean, Int, Float, Double and Array. Custom properties that have been defined via an addon are also not keyed. ------ Unit tests to cover this are added in this PR: [#120422: WIP: Anim: Add unit tests for keying custom properties](https://projects.blender.org/blender/blender/pulls/120422)
Christoph Lendenfeld added the
Module
Animation & Rigging
label 2024-03-26 12:12:39 +01:00
Christoph Lendenfeld added 1 commit 2024-03-26 12:12:44 +01:00
Christoph Lendenfeld requested review from Sybren A. Stüvel 2024-03-26 12:13:05 +01:00
Contributor

Fix for #119909

Fix for #119909
Christoph Lendenfeld added 1 commit 2024-03-26 12:44:19 +01:00
Christoph Lendenfeld changed title from Fix: Unkeyable custom properties receive keyframes to Fix #115518: Unkeyable custom properties receive keyframes 2024-03-26 12:45:55 +01:00
Christoph Lendenfeld changed title from Fix #115518: Unkeyable custom properties receive keyframes to Fix #119909: Unkeyable custom properties receive keyframes 2024-03-26 12:46:15 +01:00
Sybren A. Stüvel requested changes 2024-03-26 15:16:32 +01:00
@ -301,0 +303,4 @@
eIDPropertyType::IDP_INT,
eIDPropertyType::IDP_FLOAT,
eIDPropertyType::IDP_DOUBLE,
eIDPropertyType::IDP_ARRAY))

The IDP_ARRAY check is likely not sufficient. It might be an array of unkeyable types, and thus shouldn't be keyed.
I think it's better to keep the array check separate, as that would need to check the type of the array elements as well.

Probably best to move the ELEM(prop->type, eIDPropertyType::IDP_BOOLEAN, eIDPropertyType::IDP_INT, eIDPropertyType::IDP_FLOAT, eIDPropertyType::IDP_DOUBLE into a is_keyable_type() like function, and call that on the array elements as well.

The `IDP_ARRAY` check is likely not sufficient. It might be an array of unkeyable types, and thus shouldn't be keyed. I think it's better to keep the array check separate, as that would need to check the type of the array elements as well. Probably best to move the `ELEM(prop->type, eIDPropertyType::IDP_BOOLEAN, eIDPropertyType::IDP_INT, eIDPropertyType::IDP_FLOAT, eIDPropertyType::IDP_DOUBLE` into a `is_keyable_type()` like function, and call that on the array elements as well.
Author
Member

thanks for spotting that, I've made the changes, but I am not sure how I would create an Array with subtype Group.
I assume that can only be done using python?

thanks for spotting that, I've made the changes, but I am not sure how I would create an Array with subtype Group. I assume that can only be done using python?

I've done some more digging, and the code as it is seems to be safe. In source/blender/blenkernel/intern/idprop_create.cc there are three functions to create IDP_ARRAY properties, for int, float, and double, and so these are indeed always safe to key. Creating a custom property like object["name"] = ["a", "b", "c"] will create a property of a type that's shown as "Python" in the UI, not as "Array". If someone finds a way to create an array of groups, we'll get a bug report and handle it then, when we have a concrete counter-example.

I've done some more digging, and the code as it is seems to be safe. In `source/blender/blenkernel/intern/idprop_create.cc` there are three functions to create `IDP_ARRAY` properties, for `int`, `float`, and `double`, and so these are indeed always safe to key. Creating a custom property like `object["name"] = ["a", "b", "c"]` will create a property of a type that's shown as "Python" in the UI, not as "Array". If someone finds a way to create an array of groups, we'll get a bug report and handle it then, when we have a concrete counter-example.
@ -301,2 +309,4 @@
continue;
}
std::string name = prop->name;
std::string rna_path = "[\"" + name + "\"]";

Not part of this PR, but this will go wrong: property names can contain quotes. This should be using BLI_str_escape() to ensure the name can be safely quoted.

Not part of this PR, but this will go wrong: property names can contain quotes. This should be using `BLI_str_escape()` to ensure the name can be safely quoted.
Author
Member

thanks for pointing that out, made a PR for that.
#119922: Fix: Escape property name when keying

thanks for pointing that out, made a PR for that. [#119922: Fix: Escape property name when keying](https://projects.blender.org/blender/blender/pulls/119922)
Christoph Lendenfeld added 1 commit 2024-03-26 15:46:08 +01:00
Member

Is it a problem here that it keyframes all IDProperties regardless of whether they are "internal" (i.e. used as storage for some addon's data). This is distinguished in the custom properties UI with the "API Defined" text. I'd expect some similar "is RNA" check to be required here.

Is it a problem here that it keyframes all IDProperties regardless of whether they are "internal" (i.e. used as storage for some addon's data). This is distinguished in the custom properties UI with the `"API Defined"` text. I'd expect some similar "is RNA" check to be required here.
Author
Member

@HooglyBoogly could do, but I am not sure how. IDProperty doesn't have anything to indicate if it is API defined or not

@HooglyBoogly could do, but I am not sure how. `IDProperty` doesn't have anything to indicate if it is API defined or not
Member

You should be able to replicate the is_rna check from rna_prop_ui.py in C++ fairly easily.

You should be able to replicate the `is_rna` check from `rna_prop_ui.py` in C++ fairly easily.
Author
Member

@HooglyBoogly did some digging and this is done via PropertyRNA->flag_internal & PROP_INTERN_RUNTIME for which I'd need to include "RNA_internal_types.hh".
It really feels like trying to do that is a hack.

@HooglyBoogly did some digging and this is done via `PropertyRNA->flag_internal & PROP_INTERN_RUNTIME` for which I'd need to include `"RNA_internal_types.hh"`. It really feels like trying to do that is a hack.
Christoph Lendenfeld requested review from Sybren A. Stüvel 2024-03-28 16:29:38 +01:00

I think it would be fine to create a new function bool RNA_property_is_runtime(prop) that can live next to RNA_property_editable() and the likes in RNA_access.hh. That way you can abstract away the direct flag check.

I think it would be fine to create a new function `bool RNA_property_is_runtime(prop)` that can live next to `RNA_property_editable()` and the likes in `RNA_access.hh`. That way you can abstract away the direct flag check.
Christoph Lendenfeld added 3 commits 2024-04-02 16:47:56 +02:00
Author
Member

@dr.sybren implemented that function. Thanks, I was unsure about adding anything to RNA. Given that I've done that now who else should be reviewing this PR?

@dr.sybren implemented that function. Thanks, I was unsure about adding anything to RNA. Given that I've done that now who else should be reviewing this PR?
Member

What does PROP_INTERN_RUNTIME mean? I didn't think that was the same thing as an addon using an IDProperty as the backing storage for one if its RNA properties.

What does `PROP_INTERN_RUNTIME` mean? I didn't think that was the same thing as an addon using an `IDProperty` as the backing storage for one if its RNA properties.
Author
Member

@HooglyBoogly I am not 100% sure how that flag relates to addons using IDProperty but here are the breadcrumbs from rna_prop_ui.py

  • API Defined is added when is_rna is true (rna_prop_ui.py/233)
  • is_rna is true if the key is in rna_properties (rna_prop_ui.py/188)
  • rna_properties is a list of prop.identifier where prop.is_runtime is true (rna_prop_ui.py/183)
  • is_runtime is defined on the C side at rna_rna.cc/3212 and has a boolean function rna_Property_is_runtime_get
  • this function checks the PROP_INTERN_RUNTIME flag (rna_rna.cc/843)

The only place where I can see this flag being set is rna_define.cc/1493
I am not sure what DefRNA.preprocess means though

@HooglyBoogly I am not 100% sure how that flag relates to addons using `IDProperty` but here are the breadcrumbs from `rna_prop_ui.py` * `API Defined` is added when `is_rna` is true (rna_prop_ui.py/233) * `is_rna` is true if the key is in `rna_properties` (rna_prop_ui.py/188) * `rna_properties` is a list of `prop.identifier` where `prop.is_runtime` is true (rna_prop_ui.py/183) * `is_runtime` is defined on the C side at `rna_rna.cc`/3212 and has a boolean function `rna_Property_is_runtime_get` * this function checks the `PROP_INTERN_RUNTIME` flag (rna_rna.cc/843) The only place where I can see this flag being set is `rna_define.cc`/1493 I am not sure what `DefRNA.preprocess` means though
Hans Goudey reviewed 2024-04-04 18:22:46 +02:00
Hans Goudey left a comment
Member

Thanks for that, it's helpful! AFAIK DefRNA.preprocess means the RNA system is being used for makesrna, AKA the compile time RNA registration. So that all adds up, even if it's super convoluted.

I'm still worried that this isn't exactly the right check though. I think what we need here is to avoid keying any IDProperty that is just the storage for an addon-registered RNA property. I think then that it would be simpler to just check if there is any RNA property with the same identifier as the custom property.

Thanks for that, it's helpful! AFAIK `DefRNA.preprocess` means the RNA system is being used for `makesrna`, AKA the compile time RNA registration. So that all adds up, even if it's super convoluted. I'm still worried that this isn't exactly the right check though. I think what we need here is to avoid keying any `IDProperty` that is just the storage for an addon-registered RNA property. I think then that it would be simpler to just check if there is any RNA property with the same identifier as the custom property.
@ -290,1 +290,4 @@
/**
* A property is a runtime property if the PROP_INTERN_RUNTIME flag is set on it.
* */
Member

* */ -> */

`* */` -> `*/`
mont29 marked this conversation as resolved
Christoph Lendenfeld added 2 commits 2024-04-05 09:18:53 +02:00
Author
Member

I'm still worried that this isn't exactly the right check though.

I am not sure I understand.

> I'm still worried that this isn't exactly the right check though. I am not sure I understand.
Sybren A. Stüvel added this to the 4.1 milestone 2024-04-05 10:58:37 +02:00

I am not sure I understand.

When you use bpy.types.Object.prop = bpy.props.FloatProperty() to register a new property, its value will actually be stored in an IDProperty with the same name, so the_object["prop"]. This introduces two RNA paths to the value: the_object.prop and the_object["prop"]. We should avoid having two different F-Curves that still animate the same property. Since Blender in this case will not show ["prop"] in the Custom Properties panel, it makes sense to exclude it from anything that keys "custom properties", and simply treat it as any other non-custom, non-transform property.

On top of this, the bpy.types.Object.prop = bpy.props.FloatProperty() call can also add getter/setter functions, and an on-update callback. These are not called when accessing ob["prop"] directly, and writing to that can even set it to the wrong type and make the stored value incompatible with the actual registered property:

>>> def u(a, b): print(f"{a=}, {b=}")
... 
>>> bpy.types.Object.prop = bpy.props.FloatProperty(update=u)
>>> C.object.prop
0.0

>>> C.object.prop = 3  # This calls the update function.
a=bpy.data.objects['Suzanne'], b=<bpy_struct, Context at 0x7fe4ecc251e8>

>>> C.object["prop"]
3.0

>>> C.object["prop"] = 4  # Does not call the update function, and changes type to int.
>>> C.object.prop  # Property value no longer interpreted correctly.
0.0

So I fully agree with @HooglyBoogly that we should not be keying/animating these "storage type" custom properties. I discussed this briefly with @nathanvegdahl and he agrees.

> I am not sure I understand. When you use `bpy.types.Object.prop = bpy.props.FloatProperty()` to register a new property, its value will actually be stored in an `IDProperty` with the same name, so `the_object["prop"]`. This introduces two RNA paths to the value: `the_object.prop` and `the_object["prop"]`. We should avoid having two different F-Curves that still animate the same property. Since Blender in this case will not show `["prop"]` in the Custom Properties panel, it makes sense to exclude it from anything that keys "custom properties", and simply treat it as any other non-custom, non-transform property. On top of this, the `bpy.types.Object.prop = bpy.props.FloatProperty()` call can also add getter/setter functions, and an on-update callback. These are not called when accessing `ob["prop"]` directly, and writing to that can even set it to the wrong type and make the stored value incompatible with the actual registered property: ```python >>> def u(a, b): print(f"{a=}, {b=}") ... >>> bpy.types.Object.prop = bpy.props.FloatProperty(update=u) >>> C.object.prop 0.0 >>> C.object.prop = 3 # This calls the update function. a=bpy.data.objects['Suzanne'], b=<bpy_struct, Context at 0x7fe4ecc251e8> >>> C.object["prop"] 3.0 >>> C.object["prop"] = 4 # Does not call the update function, and changes type to int. >>> C.object.prop # Property value no longer interpreted correctly. 0.0 ``` So I fully agree with @HooglyBoogly that we should not be keying/animating these "storage type" custom properties. I discussed this briefly with @nathanvegdahl and he agrees.
Author
Member

@dr.sybren thanks for the example code

with that I was able to verify that those properties come up as is_runtime = True so with this PR they are no longer keyed.
To verify for yourself open the attached blend file
You can execute this python snippet to get the props that are runtime
print([prop.name for prop in C.active_object.bl_rna.properties if prop.is_runtime])

and with this you can see that the property gets no keys
print([fcu.data_path for fcu in bpy.data.actions["CubeAction"].fcurves])

@dr.sybren thanks for the example code with that I was able to verify that those properties come up as `is_runtime = True` so with this PR they are no longer keyed. To verify for yourself open the attached blend file You can execute this python snippet to get the props that are runtime `print([prop.name for prop in C.active_object.bl_rna.properties if prop.is_runtime])` and with this you can see that the property gets no keys `print([fcu.data_path for fcu in bpy.data.actions["CubeAction"].fcurves])`
Christoph Lendenfeld requested review from Bastien Montagne 2024-04-05 16:28:03 +02:00
Author
Member

@mont29 Adding you as a reviewer because this touches the RNA system.

TLDR
We don't want to key properties that have been created by addons by e.g.
bpy.types.Object.prop = bpy.props.FloatProperty()

As far as I can tell this is indicated by the PROP_INTERN_RUNTIME on the flag_internal of PropertyRNA.
To access that I've made a new function RNA_property_is_runtime in RNA_access.

Q1: Is the assumption correct that properties created this way always have that flag set?
Q2: Is it ok to add that function

@mont29 Adding you as a reviewer because this touches the RNA system. **TLDR** We don't want to key properties that have been created by addons by e.g. `bpy.types.Object.prop = bpy.props.FloatProperty()` As far as I can tell this is indicated by the `PROP_INTERN_RUNTIME` on the `flag_internal` of `PropertyRNA`. To access that I've made a new function `RNA_property_is_runtime` in `RNA_access`. Q1: Is the assumption correct that properties created this way always have that flag set? Q2: Is it ok to add that function
Bastien Montagne requested changes 2024-04-12 11:35:06 +02:00
Dismissed
Bastien Montagne left a comment
Owner

Generally LGTM besides a few minor comments below.

Q1: Is the assumption correct that properties created this way always have that flag set?

Yes.

Q2: Is it ok to add that function

yes.

Generally LGTM besides a few minor comments below. > Q1: Is the assumption correct that properties created this way always have that flag set? Yes. > Q2: Is it ok to add that function yes.
@ -247,6 +247,35 @@ static int insert_key_with_keyingset(bContext *C, wmOperator *op, KeyingSet *ks)
return OPERATOR_FINISHED;
}
static bool is_keyable_type(IDProperty *prop, const PropertyRNA *property_rna)

This name is not specific enough. Should be something like is_idproperty_keyable or so (the _type part can be removed I think, since it's not only checking the IDP type anymore).

This name is not specific enough. Should be something like `is_idproperty_keyable` or so (the `_type` part can be removed I think, since it's not only checking the IDP type anymore).
mont29 marked this conversation as resolved
@ -291,0 +291,4 @@
/**
* A property is a runtime property if the PROP_INTERN_RUNTIME flag is set on it.
*/
bool RNA_property_is_runtime(const PropertyRNA *prop);

This should also be called by rna_Property_is_runtime_get then, to avoid code duplication.

This should also be called by `rna_Property_is_runtime_get` then, to avoid code duplication.
mont29 marked this conversation as resolved
Christoph Lendenfeld added 4 commits 2024-04-12 12:02:24 +02:00
Author
Member

@mont29 thanks for the review
I used your suggestion of is_idproperty_keyable.

@mont29 thanks for the review I used your suggestion of `is_idproperty_keyable`.
Christoph Lendenfeld requested review from Bastien Montagne 2024-04-12 12:03:36 +02:00
Sybren A. Stüvel approved these changes 2024-04-12 12:10:28 +02:00
Sybren A. Stüvel left a comment
Member

Code LGTM, I left two minor notes that you can handle while landing (or not, they're that minor).

Code LGTM, I left two minor notes that you can handle while landing (or not, they're that minor).
@ -250,0 +263,4 @@
}
if (prop->type == eIDPropertyType::IDP_ARRAY) {
if (ELEM(prop->subtype,

This could be a return ELEM(...);

No strong feelings, because on one hand it has the nice advantage of being a clear point of return (instead of a nested one + a fallthrough), yet it looses a bit of the symmetry with the check above. Feel free to land either style.

This could be a `return ELEM(...);` No strong feelings, because on one hand it has the nice advantage of being a clear point of return (instead of a nested one + a fallthrough), yet it looses a bit of the symmetry with the check above. Feel free to land either style.
@ -306,0 +334,4 @@
std::string path = fmt::format("[\"{}\"]", name_escaped);
PointerRNA resolved_ptr;
PropertyRNA *resolved_prop;
const bool resolved_path = RNA_path_resolve_property(

I think is_resolved (or simply ok) could be a better name than resolved_path, as this variable does not contain "the resolved path".

I think `is_resolved` (or simply `ok`) could be a better name than `resolved_path`, as this variable does not contain "the resolved path".
mont29 marked this conversation as resolved
Christoph Lendenfeld added 1 commit 2024-04-12 12:21:11 +02:00
change var name
All checks were successful
buildbot/vexp-code-patch-lint Build done.
buildbot/vexp-code-patch-linux-x86_64 Build done.
buildbot/vexp-code-patch-darwin-arm64 Build done.
buildbot/vexp-code-patch-darwin-x86_64 Build done.
buildbot/vexp-code-patch-windows-amd64 Build done.
buildbot/vexp-code-patch-coordinator Build done.
73d2fc08d9
Author
Member

@blender-bot build

@blender-bot build
Bastien Montagne approved these changes 2024-04-12 14:35:39 +02:00
Christoph Lendenfeld merged commit 3fda0d5f8f into main 2024-04-12 14:48:20 +02:00
Christoph Lendenfeld deleted branch fix_more_selective_when_keying_custom_props 2024-04-12 14:48:22 +02:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#119914
No description provided.