Python API, changes to type registration in 2.8 #52599

New Issue

Campbell Barton · 2017-08-31T08:48:07+02:00

Campbell Barton commented

2017-08-31 08:48:07 +02:00

Recently D2774 was committed to master which replaces list lookups with a hash.

This exposed naming collisions in add-ons, at first I thought it reasonable to rename classes to account for this however considering how many 3rd party addons there are, it's not practical.

This raises the question of why we have a global name-space for registered classes at all (currently in bpy.types) at all.

This tasks outlines changes to registration that avoid problems with add-on naming collisions, and keeps class registration manageable.

Note: none of these changes apply to 2.7x releases.

General Proposal

Only built-in (non STRUCT_RUNTIME) types are added to bpy.types.

Scripts that need to access registered classes can:

For known types - access them from the module that defines them (as you would in any other kind of class in Python).
For introspection/scanning for all types use bpy.types.*.__subclasses__() (needed for generating docs for eg).
No checks for naming collisions are performed but...
Each type has a unique identifier that can collide. (for example, two operators can't have the same bl_idname, two RenderEngine's ... UIList's etc)
On collision with dynamic types with matching ID's. The new types will overwrite the old ones.

While it might be good to change this to prevent accidents, it's been working for years, it's handy for Python developers who want to re-run their scripts to try modified behavior, we could make this more strict so scripts don't accidentally clobber eachother's ID's, this would be better to handle separately.

Option 1) Exception for Compatibility (now in master)

From searching over add-ons I only found only one case of an add-on referencing it's own class via bpy.types
so thats easily resolved.

The issue is there are classes that are accessed to extend Blender, mainly menu's but also panels and headers.

We could update scripts to replace bpy.types.INFO_MT_file_export with bl_ui.space_info.INFO_MT_file_export,
but this is going to break a lot of scripts and exposes Blender's internal module layout.

Currently register-able types are:

AddonPreferences
Header
KeyingSetInfo
Menu
NodeSocket
NodeSocketInterface
NodeTree
Operator
Panel
PropertyGroup
RenderEngine
UIList

I'm proposing [Menu, Panel, Header, UIList, Operator] continue to be accessible from bpy.types under their bl_idname (not Python class-name).
so scripts can use them to manipulate the interface.

We'll have to ensure bl_idname conventions are followed *_MT_*, *_PT_*, *_HT_*, *_UL_*, *_OT_*.
(this is going to break some scripts, but means using a global name-space won't collide with different types).

Option 2) Expose general types via `bpy.types.*.find(...)`

Instead of keeping some classes in bpy.types, we could use an function. eg:

bpy.types.INFO_MT_file_export would be accessed as bpy.types.Menu.find("INFO_MT_file_export")

This has the advantage that we don't need to be strict about naming, it also simplifies the code (not having to track public/private structs).

The main disadvantage is scripts will need to be updated, however it will be quite straightforward and nearly all scripts will need some updates for 2.8x anyway.

Update: submitted patch: D2816 (option 1)

Recently [D2774](https://archive.blender.org/developer/D2774) was committed to master which replaces list lookups with a hash. This exposed naming collisions in add-ons, at first I thought it reasonable to rename classes to account for this however considering how many 3rd party addons there are, it's not practical. This raises the question of why we have a global name-space for registered classes at all (currently in `bpy.types`) at all. This tasks outlines changes to registration that avoid problems with add-on naming collisions, and keeps class registration manageable. Note: none of these changes apply to 2.7x releases. General Proposal **** - Only built-in (non `STRUCT_RUNTIME`) types are added to `bpy.types`. ``` Scripts that need to access registered classes can: ``` - For known types - access them from the module that defines them *(as you would in any other kind of class in Python).* - For introspection/scanning for all types use `bpy.types.*.__subclasses__()` *(needed for generating docs for eg).* - No checks for naming collisions are performed **but...** - Each type has a unique identifier that can collide. *(for example, two operators can't have the same `bl_idname`, two RenderEngine's ... UIList's etc)* - On collision with dynamic types with matching ID's. The new types will overwrite the old ones. *While it might be good to change this to prevent accidents, it's been working for years, it's handy for Python developers who want to re-run their scripts to try modified behavior, we could make this more strict so scripts don't accidentally clobber eachother's ID's, this would be better to handle separately.* Option 1) Exception for Compatibility *(now in master)* ---- From searching over add-ons I only found only one case of an add-on referencing it's own class via `bpy.types` so thats easily resolved. The issue is there are classes that are accessed to extend Blender, mainly menu's but also panels and headers. We *could* update scripts to replace `bpy.types.INFO_MT_file_export` with `bl_ui.space_info.INFO_MT_file_export`, but this is going to break a lot of scripts and exposes Blender's internal module layout. Currently register-able types are: - AddonPreferences - Header - KeyingSetInfo - Menu - NodeSocket - NodeSocketInterface - NodeTree - Operator - Panel - PropertyGroup - RenderEngine - UIList I'm proposing `[Menu, Panel, Header, UIList, Operator]` continue to be accessible from `bpy.types` under their `bl_idname` (not Python class-name). so scripts can use them to manipulate the interface. We'll have to ensure `bl_idname` conventions are followed `*_MT_*`, `*_PT_*`, `*_HT_*`, `*_UL_*`, `*_OT_*`. *(this is going to break some scripts, but means using a global name-space won't collide with different types).* Option 2) Expose general types via `bpy.types.*.find(...)` ------ Instead of keeping some classes in `bpy.types`, we could use an function. eg: `bpy.types.INFO_MT_file_export` would be accessed as `bpy.types.Menu.find("INFO_MT_file_export")` This has the advantage that we don't need to be strict about naming, it also simplifies the code (not having to track public/private structs). The main disadvantage is scripts will need to be updated, however it will be quite straightforward and nearly all scripts will need some updates for 2.8x anyway. ---- Update: submitted patch: [D2816](https://archive.blender.org/developer/D2816) (option 1)

Bastien Montagne was assigned by Campbell Barton

2017-08-31 08:48:07 +02:00

Campbell Barton commented

2017-08-31 08:48:07 +02:00

Changed status to: 'Open'

Campbell Barton commented

2017-08-31 08:48:07 +02:00

Added subscribers: @ideasman42, @mont29, @Sergey

Campbell Barton changed title from ~~Python API, changes to type registration~~ to Python API, changes to type registration in 2.8

2017-08-31 08:50:26 +02:00

Jacques Lucke commented

2017-08-31 10:39:20 +02:00

Added subscriber: @JacquesLucke

Jacques Lucke commented

2017-08-31 10:39:20 +02:00

Here are a few ideas that I have on the topic:

For every register-able type Blender could store a hash table (bl_idname -> cls).
So whenever you call bpy.utils.register_class(MyClass), the type is inserted into the corresponding hash table.

Registered types can than be accessed like this: bpy.types.TYPE.get(bl_idname)
This would eliminate the need for strange bl_idname conventions which I personally do not like at all.
The only naming rule that should be enforced (imo) is that bl_idnames should be valid Python identifiers. This also makes the system more future proof if we decide to use a different access pattern later on.
I'm not sure if it is good that Operators are handled in a special way (the bl_idname needs exactly one . somewhere in the middle, while on both sides are valid Python identifiers) but I guess that can't really be changed easily.
We could also have bpy.types.TYPE.get_all() to get all registered subclasses of a type. The problem with TYPE.__subclasses__() is that there can be subclasses that are not registered.

Just wanted to share a few different ideas on the reloading issue as well although I'm not fully convinced by them.

Print a warning in the console whenever a type is overwritten (this is quite essential I guess)
Make replacing an existing type an exception if Blender is not in a "developer mode". This allows the users of addons which don't have this enabled to report name-collisions early. (They might have addons installed the original developer does not have)
Change the signature of the registration function to register_class(cls, group = ""). This group string (which does not have to be set by the developer) is stored in the hash table next to the class. The hash table would then look like this: bl_idname -> (cls, group). Whenever a new type is registered, the group is checked. If it is equal, cls is replaced, otherwise an exception is raised. Types that are not part of a group (group = "") cannot be reloaded at all. The group should usually be the addon name, but can also be just a random string. It only has to stay constant within one execution of Blender.

(btw: Also Node is register-able)

Here are a few ideas that I have on the topic: For every register-able type Blender could store a hash table (`bl_idname -> cls`). So whenever you call `bpy.utils.register_class(MyClass)`, the type is inserted into the corresponding hash table. Registered types can than be accessed like this: `bpy.types.TYPE.get(bl_idname)` This would eliminate the need for strange `bl_idname` conventions which I personally do not like at all. The only naming rule that should be enforced (imo) is that `bl_idname`s should be valid Python identifiers. This also makes the system more future proof if we decide to use a different access pattern later on. I'm not sure if it is good that `Operator`s are handled in a special way (the `bl_idname` needs exactly one `.` somewhere in the middle, while on both sides are valid Python identifiers) but I guess that can't really be changed easily. We could also have `bpy.types.TYPE.get_all()` to get all registered subclasses of a type. The problem with `TYPE.__subclasses__()` is that there can be subclasses that are not registered. Just wanted to share a few different ideas on the reloading issue as well although I'm not fully convinced by them. - Print a warning in the console whenever a type is overwritten (this is quite essential I guess) - Make replacing an existing type an exception if Blender is not in a "developer mode". This allows the users of addons which don't have this enabled to report name-collisions early. (They might have addons installed the original developer does not have) - Change the signature of the registration function to `register_class(cls, group = "")`. This group string (which does not have to be set by the developer) is stored in the hash table next to the class. The hash table would then look like this: `bl_idname -> (cls, group)`. Whenever a new type is registered, the group is checked. If it is equal, `cls` is replaced, otherwise an exception is raised. Types that are not part of a group (`group = ""`) cannot be reloaded at all. The `group` should usually be the addon name, but can also be just a random string. It only has to stay constant within one execution of Blender. (btw: Also `Node` is register-able)

Satish Goda commented

2017-08-31 11:34:32 +02:00

Added subscriber: @satishgoda1

Campbell Barton commented

2017-08-31 11:41:05 +02:00

In #52599#456054, @JacquesLucke wrote:
Here are a few ideas that I have on the topic:

For every register-able type Blender could store a hash table (bl_idname -> cls).
So whenever you call bpy.utils.register_class(MyClass), the type is inserted into the corresponding hash table.

We have per type hash maps for almost all registerable types already (except the new PropertyGroup).

While writing the proposal I considered supporting:

  mt = bpy.types.Menu.find("SOME_MT_menu")

The main reason against this is it would break a lot of scripts, (anything that touches the file-menu - so import/exporters).

For 2.8x perhaps this is OK, I'd rather avoid large breakages for minimal gains.

Registered types can than be accessed like this: bpy.types.TYPE.get(bl_idname)
This would eliminate the need for strange bl_idname conventions which I personally do not like at all.
The only naming rule that should be enforced (imo) is that bl_idnames should be valid Python identifiers. This also makes the system more future proof if we decide to use a different access pattern later on.
I'm not sure if it is good that Operators are handled in a special way (the bl_idname needs exactly one . somewhere in the middle, while on both sides are valid Python identifiers) but I guess that can't really be changed easily.

We need operators names the way they are so they can be accessed as bpy.ops.xxx.xxx().

We could also have bpy.types.TYPE.get_all() to get all registered subclasses of a type. The problem with TYPE.__subclasses__() is that there can be subclasses that are not registered.

Scripts can check the subclass is registered by calling cls.is_registered().

Just wanted to share a few different ideas on the reloading issue as well although I'm not fully convinced by them.

Print a warning in the console whenever a type is overwritten (this is quite essential I guess)

Make replacing an existing type an exception if Blender is not in a "developer mode". This allows the users of addons which don't have this enabled to report name-collisions early. (They might have addons installed the original developer does not have)

Change the signature of the registration function to register_class(cls, group = ""). This group string (which does not have to be set by the developer) is stored in the hash table next to the class. The hash table would then look like this: bl_idname -> (cls, group). Whenever a new type is registered, the group is checked. If it is equal, cls is replaced, otherwise an exception is raised. Types that are not part of a group (group = "") cannot be reloaded at all. The group should usually be the addon name, but can also be just a random string. It only has to stay constant within one execution of Blender.

(btw: Also Node is register-able)

Not sure about each type needing a group, in general though Im not against something like what you're suggesting. I just rather not make that change part of this proposal.

> In #52599#456054, @JacquesLucke wrote: > Here are a few ideas that I have on the topic: > > For every register-able type Blender could store a hash table (`bl_idname -> cls`). > So whenever you call `bpy.utils.register_class(MyClass)`, the type is inserted into the corresponding hash table. We have per type hash maps for almost all registerable types already (except the new PropertyGroup). While writing the proposal I considered supporting: ``` mt = bpy.types.Menu.find("SOME_MT_menu") ``` The main reason against this is it would break a lot of scripts, (anything that touches the file-menu - so import/exporters). For 2.8x perhaps this is OK, I'd rather avoid large breakages for minimal gains. > Registered types can than be accessed like this: `bpy.types.TYPE.get(bl_idname)` > This would eliminate the need for strange `bl_idname` conventions which I personally do not like at all. > The only naming rule that should be enforced (imo) is that `bl_idname`s should be valid Python identifiers. This also makes the system more future proof if we decide to use a different access pattern later on. > I'm not sure if it is good that `Operator`s are handled in a special way (the `bl_idname` needs exactly one `.` somewhere in the middle, while on both sides are valid Python identifiers) but I guess that can't really be changed easily. We need operators names the way they are so they can be accessed as `bpy.ops.xxx.xxx()`. > We could also have `bpy.types.TYPE.get_all()` to get all registered subclasses of a type. The problem with `TYPE.__subclasses__()` is that there can be subclasses that are not registered. Scripts can check the subclass is registered by calling `cls.is_registered()`. > Just wanted to share a few different ideas on the reloading issue as well although I'm not fully convinced by them. > > - Print a warning in the console whenever a type is overwritten (this is quite essential I guess) > - Make replacing an existing type an exception if Blender is not in a "developer mode". This allows the users of addons which don't have this enabled to report name-collisions early. (They might have addons installed the original developer does not have) > - Change the signature of the registration function to `register_class(cls, group = "")`. This group string (which does not have to be set by the developer) is stored in the hash table next to the class. The hash table would then look like this: `bl_idname -> (cls, group)`. Whenever a new type is registered, the group is checked. If it is equal, `cls` is replaced, otherwise an exception is raised. Types that are not part of a group (`group = ""`) cannot be reloaded at all. The `group` should usually be the addon name, but can also be just a random string. It only has to stay constant within one execution of Blender. > > (btw: Also `Node` is register-able) Not sure about each type needing a group, in general though Im not against something like what you're suggesting. I just rather not make that change part of this proposal.

Jacques Lucke commented

2017-08-31 12:02:33 +02:00

While writing the proposal I considered supporting:

mt = bpy.types.Menu.find("SOME_MT_menu")
The main reason against this is it would break a lot of scripts, (anything that touches the file-menu - so import/exporters).

For 2.8x perhaps this is OK, I'd rather avoid large breakages for minimal gains.

The benefit of using bpy.types.Menu.find(NAME) over bl_ui.space_info.NAME is that the first already contains the type information. That means that NAME does not have to contain any type information anymore. In my opinion this is highly preferable (works well with having one bl_idname namespace per type instead of a global namespace).
I think updating existing scripts can mostly be automated here as well. By using bl_ui.space_info.NAME NAME stil has to be unique across multiple types. Maybe I also understood that wrong, I didn't know about the bl_ui module before.

Scripts can check the subclass is registered by calling cls.is_registered().

Ah good to know. I haven't found that in the api docs.

Not sure about each type needing a group, in general though Im not against something like what you're suggesting. I just rather not make that change part of this proposal.

Alright, that can be discussed later.

My main incentive here is that I don't want to use these __MT__, ... conventions. :D

> While writing the proposal I considered supporting: > > ```mt = bpy.types.Menu.find("SOME_MT_menu")``` > The main reason against this is it would break a lot of scripts, (anything that touches the file-menu - so import/exporters). > > For 2.8x perhaps this is OK, I'd rather avoid large breakages for minimal gains. The benefit of using `bpy.types.Menu.find(NAME)` over `bl_ui.space_info.NAME` is that the first already contains the type information. That means that `NAME` does not have to contain any type information anymore. In my opinion this is highly preferable (works well with having one `bl_idname` namespace per type instead of a global namespace). I think updating existing scripts can mostly be automated here as well. By using `bl_ui.space_info.NAME` `NAME` stil has to be unique across multiple types. Maybe I also understood that wrong, I didn't know about the `bl_ui` module before. > Scripts can check the subclass is registered by calling `cls.is_registered()`. Ah good to know. I haven't found that in the api docs. > Not sure about each type needing a group, in general though Im not against something like what you're suggesting. I just rather not make that change part of this proposal. Alright, that can be discussed later. My main incentive here is that I don't want to use these `__MT__`, ... conventions. :D

Campbell Barton commented

2017-08-31 12:20:25 +02:00

@JacquesLucke, While bl_ui is a regular Python module it's currently not documented as part of the API, I'd rather not expose it publicly.

Added the option for an accessor function as Option 2 in the proposal.

@JacquesLucke, While `bl_ui` is a regular Python module it's currently not documented as part of the API, I'd rather not expose it publicly. Added the option for an accessor function as `Option 2` in the proposal.

Bastien Montagne commented

2017-08-31 14:22:06 +02:00

I’d go for option 1 (“I'd rather avoid large breakages for minimal gains.” -> 100% agree!).

And strict (naming) conventions are actually a big +1 from my side, if we had had them in the first place we would not have hit those collisions issues at all. It’s pretty stupid to write two public classes with same name, at best it's confusing, and unless you have a real strict and good control over your scopes it will bite you at one point or the other. Python coders tend to be sloppy, see quality of code in many existing addons, losing conventions will only makes things worse.

@ideasman42 scripts like i18n messages extraction etc. are already using bpy.types.*.__subclasses__() trick anyway, at least due to stupid operators (iirc, their definition generates two classes with same name, one for operator, one for its properties, and in bpy.types.foo you only have access to the propertygroup one, or something like that).

I’d go for option 1 (“*I'd rather avoid large breakages for minimal gains.*” -> 100% agree!). And strict (naming) conventions are actually a big +1 from my side, if we had had them in the first place we would not have hit those collisions issues at all. It’s pretty stupid to write two public classes with same name, at best it's confusing, and unless you have a real strict and good control over your scopes it will bite you at one point or the other. Python coders tend to be sloppy, see quality of code in many existing addons, losing conventions will only makes things worse. @ideasman42 scripts like i18n messages extraction etc. are already using `bpy.types.*.__subclasses__()` trick anyway, at least due to stupid operators (iirc, their definition generates two classes with same name, one for operator, one for its properties, and in `bpy.types.foo` you only have access to the propertygroup one, or something like that).

Vuk Gardašević commented

2017-08-31 15:02:18 +02:00

Added subscriber: @VukGardasevic

Vuk Gardašević commented

2017-08-31 15:02:18 +02:00

I'm supporting conventions in naming. bl_idname and the registered class name should follow the same pattern, i.e. the bl_idname and the class name can be guessed correctly from each other. Plus the class name should reflect the internal structure one.

Currently while editing the an add-on, in the class overview, there can be different naming conventions inside the same script. This makes comprehension, debugging difficult, not to mention accessing them from search.

Example from contrib (one add-on that had a naming collision)

In that example class RemuevePropiedades is an Operator, View3DMCPanel is a stub class for inheritance.

The problem is as something is not an clear simple rule that is mandatory, people don't think it is important and they spent very little time thinking about the naming conventions. The result is a mess (for bl_idname some add-ons use context like object.some_operator, some use an name of add-on or an abbreviation of it like terrain.some_operator , opr.some_operator , sky.dyn or something remotely related to the functionality like add.some_operator).

Still, everything depends on extensiveness of other API changes. If most of the add-ons end up breaking ( for instance accessing selection, layers are changed, add-ons relying on Blender Internal etc.) then it makes sense to impose naming rules during the refactor/rewrite of them. If the changes only affect a smaller number of add-ons, then some other solutions are also legitimate (allowing similar naming as today as an exemption to the rule for third party add-ons).

Also +1 for strict naming conventions. :)

I'm supporting conventions in naming. `bl_idname` and the registered class name should follow the same pattern, i.e. the bl_idname and the class name can be guessed correctly from each other. Plus the class name should reflect the internal structure one. Currently while editing the an add-on, in the class overview, there can be different naming conventions inside the same script. This makes comprehension, debugging difficult, not to mention accessing them from search. Example from contrib (one add-on that had a naming collision) ![example_classes.jpg](https://archive.blender.org/developer/F750149/example_classes.jpg) In that example class `RemuevePropiedades` is an Operator, `View3DMCPanel` is a stub class for inheritance. The problem is as something is not an clear simple rule that is mandatory, people don't think it is important and they spent very little time thinking about the naming conventions. The result is a mess (for bl_idname some add-ons use context like `object.some_operator`, some use an name of add-on or an abbreviation of it like `terrain.some_operator `, `opr.some_operator` , `sky.dyn` or something remotely related to the functionality like `add.some_operator`). Still, everything depends on extensiveness of other API changes. If most of the add-ons end up breaking ( for instance accessing selection, layers are changed, add-ons relying on Blender Internal etc.) then it makes sense to impose naming rules during the refactor/rewrite of them. If the changes only affect a smaller number of add-ons, then some other solutions are also legitimate (allowing similar naming as today as an exemption to the rule for third party add-ons). Also +1 for strict naming conventions. :)

Jacques Lucke commented

2017-08-31 15:47:46 +02:00

I'm not against naming conventions. However I'm against knowledge duplication in code. Eg: when I already say that a specific class is a Panel, then I don't need to put that information into the identifier.

If you want that the identifier and class name can be guessed from each other, there just should not be both but only one string in the first place.

I agree that the operator categories are a mess in many addons. I think there are multiple reasons for that:

There is no clear definition of what kind of operator go into which category. So you would have to go to the manual and see all the categories that exist, then you go inside and see that it is also a mess.
For low level operators it might be easy to find a specific catogory. However, once you create larger macros that touch many areas of Blender, it is nearly impossible to select only one category to put the operator in.
People, at least me, fear that they run into collisions with other addons. It is not very unlikely that this happens when everyone puts their operators into the correct categories. So you start to create your own category -> your own namespace, to feel safer. Many large addons (Animation Nodes, Retopoflow, and possibly more) take this approach as it is pretty much impossible to make sure that no one else uses one of your operator names. However it is much easier to check that no one uses the same operator category that you are using.

I'm not saying that the current operator naming "rules" are bad but that they don't work well for larger addons. I'm not sure yet what the best way is to solve this.

It’s pretty stupid to write two public classes with same name

True, if you are the only person who writes addons. As an addon developer you don't have control over other addon developers.

We could update scripts to replace bpy.types.INFO_MT_file_export with bl_ui.space_info.INFO_MT_file_export,
but this is going to break a lot of scripts and exposes Blender's internal module layout.

While bl_ui is a regular Python module it's currently not documented as part of the API, I'd rather not expose it publicly.

Addons also need access to that data, so the two statements don't work well together. (or not?)

Speaking from an addon developer perspective (I don't know anything about the internals):
Blender is responsible for making sure that addon developers can feel save and that naming collisions with other addons are impossible. And by that I mean that it should not be enforced by convention but by data structure.

This would be the ideal goal we should strive for. None of the proposals above archieve this goal yet. I don't know if it is possible to archieve this goal for Blender 2.8, if not, we should try to get close IMO.

I'm not against naming conventions. However I'm against knowledge duplication in code. Eg: when I already say that a specific class is a Panel, then I don't need to put that information into the identifier. If you want that the identifier and class name can be guessed from each other, there just should not be both but only one string in the first place. I agree that the operator categories are a mess in many addons. I think there are multiple reasons for that: - There is no clear definition of what kind of operator go into which category. So you would have to go to the manual and see all the categories that exist, then you go inside and see that it is also a mess. - For low level operators it might be easy to find a specific catogory. However, once you create larger macros that touch many areas of Blender, it is nearly impossible to select only one category to put the operator in. - People, at least me, fear that they run into collisions with other addons. It is not very unlikely that this happens when everyone puts their operators into the correct categories. So you start to create your own category -> your own namespace, to feel safer. Many large addons (Animation Nodes, Retopoflow, and possibly more) take this approach as it is pretty much impossible to make sure that no one else uses one of your operator names. However it is much easier to check that no one uses the same operator category that you are using. I'm not saying that the current operator naming "rules" are bad but that they don't work well for larger addons. I'm not sure yet what the best way is to solve this. > It’s pretty stupid to write two public classes with same name True, if you are the only person who writes addons. As an addon developer you don't have control over other addon developers. > We could update scripts to replace `bpy.types.INFO_MT_file_export` with `bl_ui.space_info.INFO_MT_file_export`, > but this is going to break a lot of scripts and exposes Blender's internal module layout. > While `bl_ui` is a regular Python module it's currently not documented as part of the API, I'd rather not expose it publicly. Addons also need access to that data, so the two statements don't work well together. (or not?) Speaking from an addon developer perspective (I don't know anything about the internals): *Blender **is responsible** for making sure that addon developers can feel save and that naming collisions with other addons are impossible. And by that I mean that it should not be enforced by convention but by data structure.* This would be the ideal goal we should strive for. None of the proposals above archieve this goal yet. I don't know if it is possible to archieve this goal for Blender 2.8, if not, we should try to get close IMO.

Campbell Barton commented

2017-08-31 16:57:03 +02:00

Changed status from 'Open' to: 'Resolved'

Campbell Barton closed this issue

2017-08-31 16:57:03 +02:00

Campbell Barton commented

2017-08-31 16:57:03 +02:00

Committed (Option 1) 636baa598a & 0bbae3f3f6

Committed (Option 1) 636baa598a & 0bbae3f3f6

Campbell Barton commented

2017-09-01 10:17:14 +02:00

In #52599#456103, @JacquesLucke wrote:
I'm not against naming conventions. However I'm against knowledge duplication in code. Eg: when I already say that a specific class is a Panel, then I don't need to put that information into the identifier.

Currently you do, because it's accessed via bpy.types, moving elsewhere could work but we opted for Option 1.

If you want that the identifier and class name can be guessed from each other, there just should not be both but only one string in the first place.

This is the case, Python developers don't need to define an bl_idname in this case the class name will be used.

I agree that the operator categories are a mess in many addons. I think there are multiple reasons for that:

There is no clear definition of what kind of operator go into which category. So you would have to go to the manual and see all the categories that exist, then you go inside and see that it is also a mess.

For low level operators it might be easy to find a specific catogory. However, once you create larger macros that touch many areas of Blender, it is nearly impossible to select only one category to put the operator in.

People, at least me, fear that they run into collisions with other addons. It is not very unlikely that this happens when everyone puts their operators into the correct categories. So you start to create your own category -> your own namespace, to feel safer. Many large addons (Animation Nodes, Retopoflow, and possibly more) take this approach as it is pretty much impossible to make sure that no one else uses one of your operator names. However it is much easier to check that no one uses the same operator category that you are using.

I'm not saying that the current operator naming "rules" are bad but that they don't work well for larger addons. I'm not sure yet what the best way is to solve this.

We could have a convention that all addons use ADDON_ID_##_own_name,
effectively a (category, name) tuple.

It’s pretty stupid to write two public classes with same name

True, if you are the only person who writes addons. As an addon developer you don't have control over other addon developers.

Right, so a convention would be helpful.

We could update scripts to replace bpy.types.INFO_MT_file_export with bl_ui.space_info.INFO_MT_file_export,
but this is going to break a lot of scripts and exposes Blender's internal module layout.

While bl_ui is a regular Python module it's currently not documented as part of the API, I'd rather not expose it publicly.

Addons also need access to that data, so the two statements don't work well together. (or not?)

Not sure what you mean. Addon have access to it via bpy.types.

Speaking from an addon developer perspective (I don't know anything about the internals):
Blender is responsible for making sure that addon developers can feel save and that naming collisions with other addons are impossible. And by that I mean that it should not be enforced by convention but by data structure.

There are many areas in computing where avoiding collisions is done by conventions - commands in your $PATH, Python module names, key bindings between your OS and applications... etc.

I see what you're saying, but blender is not just a python library, all the logic has C internals and is defined in C. This means in some cases Python API follows C conventions.

While its possible to have an extra level of categories - this is a big change - then every reference to an operator would need to be "category.subcategory.toolname" or so.

This would be the ideal goal we should strive for. None of the proposals above archieve this goal yet. I don't know if it is possible to archieve this goal for Blender 2.8, if not, we should try to get close IMO.

At the moment I think its not very practical, and we can manage the problem using conventions.

> In #52599#456103, @JacquesLucke wrote: > I'm not against naming conventions. However I'm against knowledge duplication in code. Eg: when I already say that a specific class is a Panel, then I don't need to put that information into the identifier. Currently you do, because it's accessed via `bpy.types`, moving elsewhere could work but we opted for *Option 1*. > If you want that the identifier and class name can be guessed from each other, there just should not be both but only one string in the first place. This is the case, Python developers don't need to define an `bl_idname` in this case the class name will be used. > I agree that the operator categories are a mess in many addons. I think there are multiple reasons for that: > - There is no clear definition of what kind of operator go into which category. So you would have to go to the manual and see all the categories that exist, then you go inside and see that it is also a mess. > - For low level operators it might be easy to find a specific catogory. However, once you create larger macros that touch many areas of Blender, it is nearly impossible to select only one category to put the operator in. > - People, at least me, fear that they run into collisions with other addons. It is not very unlikely that this happens when everyone puts their operators into the correct categories. So you start to create your own category -> your own namespace, to feel safer. Many large addons (Animation Nodes, Retopoflow, and possibly more) take this approach as it is pretty much impossible to make sure that no one else uses one of your operator names. However it is much easier to check that no one uses the same operator category that you are using. > > I'm not saying that the current operator naming "rules" are bad but that they don't work well for larger addons. I'm not sure yet what the best way is to solve this. We could have a convention that all addons use `ADDON_ID_##_own_name`, effectively a (category, name) tuple. >> It’s pretty stupid to write two public classes with same name > > True, if you are the only person who writes addons. As an addon developer you don't have control over other addon developers. Right, so a convention would be helpful. > >> We could update scripts to replace `bpy.types.INFO_MT_file_export` with `bl_ui.space_info.INFO_MT_file_export`, >> but this is going to break a lot of scripts and exposes Blender's internal module layout. > >> While `bl_ui` is a regular Python module it's currently not documented as part of the API, I'd rather not expose it publicly. > > Addons also need access to that data, so the two statements don't work well together. (or not?) Not sure what you mean. Addon have access to it via `bpy.types`. > > Speaking from an addon developer perspective (I don't know anything about the internals): > *Blender **is responsible** for making sure that addon developers can feel save and that naming collisions with other addons are impossible. And by that I mean that it should not be enforced by convention but by data structure.* There are many areas in computing where avoiding collisions is done by conventions - commands in your $PATH, Python module names, key bindings between your OS and applications... etc. I see what you're saying, but blender is not just a python library, all the logic has C internals and is defined in C. This means in some cases Python API follows C conventions. While its possible to have an extra level of categories - this is a big change - then every reference to an operator would need to be "category.subcategory.toolname" or so. > This would be the ideal goal we should strive for. None of the proposals above archieve this goal yet. I don't know if it is possible to archieve this goal for Blender 2.8, if not, we should try to get close IMO. At the moment I think its not very practical, and we can manage the problem using conventions.

Jacques Lucke commented

2017-09-04 20:14:13 +02:00

I was wondering, how are the types selected that require _MT_, ... in the bl_idname? Because atm this convention only exists for some types, right? Most of the types you mention in your initial post don't require this (yet), correct?

It’s pretty stupid to write two public classes with same name

True, if you are the only person who writes addons. As an addon developer you don't have control over other addon developers.

Right, so a convention would be helpful.

I'm not sure how a convention helps here. It might even increase the likelyhood that two developers name different operators the same.

I was wondering, how are the types selected that require `_MT_`, ... in the `bl_idname`? Because atm this convention only exists for some types, right? Most of the types you mention in your initial post don't require this (yet), correct? >>> It’s pretty stupid to write two public classes with same name >>True, if you are the only person who writes addons. As an addon developer you don't have control over other addon developers. >Right, so a convention would be helpful. I'm not sure how a convention helps here. It might even increase the likelyhood that two developers name different operators the same.

Vuk Gardašević commented

2017-09-04 21:29:40 +02:00

If the add-on module name is included in the name not so much since it has to be unique for it to be activated properly (without the duplicates warning).
Also on the other side, having a prefix like that, it'll make search and replace much easier. :)
Having to think only about an unique name of the module to be used inside the prefix is less of a possible surface area for collisions.

Of course bl_idname complicates things a bit.

Let's say the the prefix is :
ADDON_ID_MATERIALS_UTILS_add_material what the bl_idname would be? Maybe materials_utils.add_material ?

In cases like this the module name is used as the first word which should be unique. However with context of the operator will not be clear when accessing from search.
The module/script name can be long.
Still there could be a name collision.

Some of the concerns can be addressed by developer tool / add-on where it can be easily enabled by the user and do a one button check for collisions.
That could be part of the package manager - like an option for checking for conflicts > output to a file like system info.

If the add-on module name is included in the name not so much since it has to be unique for it to be activated properly (without the duplicates warning). Also on the other side, having a prefix like that, it'll make search and replace much easier. :) Having to think only about an unique name of the module to be used inside the prefix is less of a possible surface area for collisions. Of course bl_idname complicates things a bit. Let's say the the prefix is : `ADDON_ID_MATERIALS_UTILS_add_material` what the bl_idname would be? Maybe `materials_utils.add_material `? - In cases like this the module name is used as the first word which should be unique. However with context of the operator will not be clear when accessing from search. - The module/script name can be long. - Still there could be a name collision. Some of the concerns can be addressed by developer tool / add-on where it can be easily enabled by the user and do a one button check for collisions. That could be part of the package manager - like an option for checking for conflicts > output to a file like system info.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Python API, changes to type registration in 2.8 #52599

Option 1) Exception for Compatibility (now in master)

Option 2) Expose general types via bpy.types.*.find(...)

Option 2) Expose general types via `bpy.types.*.find(...)`