IO: Add initial support for File Handlers registration #112466

Merged
Jesse Yurkovich merged 15 commits from guishe/blender:file-handler into main 2023-12-09 05:06:24 +01:00
Contributor

Adds initial support for File Handlers registration with a python apy.
This file handlers will allow developers to associate file extensions
with operators and give them extended UI capabilities as drag & drop support.

Currently this changes would not have any user visible changes, but
this initial state will help create a way to provide file drag & drop
capabilities to operators that can manage files (#111242).

As proposed in #68935 this file handlers also can be extended to be
able to show dynamic import windows with dynamic import options
depending on selected files, or automatically extend file import/export
menus.

Not resolved in the PR:

  1. Method to create import / export settings
  2. Method to make UI layout of import / export settings
  3. Method to export a file
  4. Method to provide metadata about file

Usage

WM_OT_gpencil_import_svg operator import .svg files as gpencil objects, we can register a FileHandler as follows:

class WM_FH_gpencil(bpy.types.FileHandler):
    bl_idname = "WM_FH_gpencil"
    bl_label = "gpencil"
    bl_import_operator = "WM_OT_gpencil_import_svg"
    bl_file_extensions = ".svg"

    @classmethod
    def poll_drop(cls, context):
        return context.area.type == "VIEW_3D"

Adds initial support for File Handlers registration with a python apy. This file handlers will allow developers to associate file extensions with operators and give them extended UI capabilities as drag & drop support. Currently this changes would not have any user visible changes, but this initial state will help create a way to provide file drag & drop capabilities to operators that can manage files (#111242). As proposed in #68935 this file handlers also can be extended to be able to show dynamic import windows with dynamic import options depending on selected files, or automatically extend file import/export menus. Not resolved in the PR: 1. Method to create import / export settings 2. Method to make UI layout of import / export settings 3. Method to export a file 5. Method to provide metadata about file Usage --- `WM_OT_gpencil_import_svg` operator import `.svg` files as gpencil objects, we can register a `FileHandler` as follows: ``` python class WM_FH_gpencil(bpy.types.FileHandler): bl_idname = "WM_FH_gpencil" bl_label = "gpencil" bl_import_operator = "WM_OT_gpencil_import_svg" bl_file_extensions = ".svg" @classmethod def poll_drop(cls, context): return context.area.type == "VIEW_3D" ```
Guillermo Venegas added 1 commit 2023-09-17 05:44:48 +02:00
Guillermo Venegas added 1 commit 2023-09-17 06:24:33 +02:00
Iliya Katushenock added this to the Pipeline, Assets & IO project 2023-09-17 10:23:41 +02:00
Iliya Katushenock added the
Module
User Interface
Interest
Python API
labels 2023-09-17 10:23:51 +02:00
Guillermo Venegas requested review from Dalai Felinto 2023-09-18 19:09:21 +02:00
Guillermo Venegas added 2 commits 2023-10-26 21:16:01 +02:00
Brecht Van Lommel requested changes 2023-11-03 18:43:14 +01:00
@ -0,0 +25,4 @@
bool (*poll_drop)(const struct bContext *C, FileHandlerType *file_handle_type);
/* List of file extensions supported by the file handler. */
blender::Vector<bFileExtension> file_extensions;

bFileExtension can just be a std::string?

`bFileExtension` can just be a `std::string`?
guishe marked this conversation as resolved
@ -0,0 +2,4 @@
#include "BKE_file_handler.hh"
static blender::Vector<FileHandlerType *> *file_handlers = nullptr;

I think this can be simplified to static blender::Vector<FileHandlerType> file_handlers. For new C++ we should try to avoid this kind of manual memory allocation.

The only thing is that it should be defined inside a function, so that it is lazily initialized, after the memory allocator is initialized.

I think this can be simplified to `static blender::Vector<FileHandlerType> file_handlers`. For new C++ we should try to avoid this kind of manual memory allocation. The only thing is that it should be defined inside a function, so that it is lazily initialized, after the memory allocator is initialized.
Author
Contributor

I had to use static blender::RawVector<std::unique_ptr<FileHandlerType>> file_handlers;
Vector will not be released on closing
And had to use a unique_ptr, the RNAStruc pointer can be invalid if the array is rearranged (by removing or adding new elements) with just RawVector<FileHandlerType>

I had to use `static blender::RawVector<std::unique_ptr<FileHandlerType>> file_handlers;` `Vector` will not be released on closing And had to use a `unique_ptr`, the RNAStruc pointer can be invalid if the array is rearranged (by removing or adding new elements) with just `RawVector<FileHandlerType>`

Ok, that makes sense. Calling clear_and_shrink on Vector would have worked as well.

Ok, that makes sense. Calling `clear_and_shrink` on `Vector` would have worked as well.
brecht marked this conversation as resolved
@ -1455,0 +1563,4 @@
}
char_begin = char_end[0] ? char_end + 1 : char_end;
char_end = BLI_strchr_or_end(char_begin, char_separator);
}

Can this parsing be moved into file_handler.cc?

Can this parsing be moved into `file_handler.cc`?
guishe marked this conversation as resolved
Brecht Van Lommel requested review from Bastien Montagne 2023-11-03 18:45:16 +01:00
Brecht Van Lommel removed review request for Dalai Felinto 2023-11-03 18:45:20 +01:00
Guillermo Venegas added 1 commit 2023-11-07 18:46:41 +01:00
Guillermo Venegas added 1 commit 2023-11-07 18:47:44 +01:00
Brecht Van Lommel requested changes 2023-11-07 22:06:50 +01:00
@ -0,0 +10,4 @@
return file_handlers;
}
const blender::Vector<FileHandlerType *> BKE_file_handlers()

I don't think it's necessary to allocate another vector, can't this just return a const RawVector directly? It's less convenient, but I don't think we should be making copies just to iterate over this.

I don't think it's necessary to allocate another vector, can't this just return a `const RawVector` directly? It's less convenient, but I don't think we should be making copies just to iterate over this.
Author
Contributor

My concern was more about using unique_ptr, but returning the RawVector as a const disables the data ownership transfer, so it really shouldn't be a problem.

My concern was more about using `unique_ptr`, but returning the RawVector as a const disables the data ownership transfer, so it really shouldn't be a problem.
brecht marked this conversation as resolved
Guillermo Venegas added 2 commits 2023-11-08 15:21:27 +01:00
Brecht Van Lommel approved these changes 2023-11-08 15:23:10 +01:00
Bastien Montagne requested changes 2023-11-08 18:18:33 +01:00
Bastien Montagne left a comment
Owner

Besides notes below, this LGTM in general.

There are no tests for this new code though, would be good to add some in the BKE code to validate the adding, finding and removal of file handlers.

Besides notes below, this LGTM in general. There are no tests for this new code though, would be good to add some in the BKE code to validate the adding, finding and removal of file handlers.
@ -0,0 +8,4 @@
struct FileHandlerType {
char idname[OP_MAX_TYPENAME]; /* Unique name. */

Use /** */ doxygen syntax, put all comments on top.

See also https://wiki.blender.org/wiki/Style_Guide/C_Cpp#Comments

Use `/** */` doxygen syntax, put all comments on top. See also https://wiki.blender.org/wiki/Style_Guide/C_Cpp#Comments
guishe marked this conversation as resolved
@ -0,0 +14,4 @@
char import_operator[OP_MAX_TYPENAME]; /* Import operator name. */
/* String list of file extensions supported by the file handler. */

Should also document the expected format of this string (separator, with or without dot, etc.)

Should also document the expected format of this string (separator, with or without dot, etc.)
guishe marked this conversation as resolved
@ -0,0 +15,4 @@
char import_operator[OP_MAX_TYPENAME]; /* Import operator name. */
/* String list of file extensions supported by the file handler. */
char file_extensions_str[OP_MAX_TYPENAME];

I don't think 64 chars is enough here? That's max 12 3-chars extensions... E.g. a handler for image files may support tens of them...

Think it needs its own define, probably at 256 or 512 chars?

I don't think 64 chars is enough here? That's max 12 3-chars extensions... E.g. a handler for image files may support tens of them... Think it needs its own define, probably at 256 or 512 chars?
guishe marked this conversation as resolved
@ -0,0 +27,4 @@
ExtensionRNA rna_ext;
};
void BKE_file_handler_add(std::unique_ptr<FileHandlerType> file_handler);

All of these functions need to be properly documented.

All of these functions need to be properly documented.
guishe marked this conversation as resolved
@ -0,0 +30,4 @@
void BKE_file_handler_add(std::unique_ptr<FileHandlerType> file_handler)
{

Useless empty line

Useless empty line
guishe marked this conversation as resolved
@ -2201,0 +2321,4 @@
PropertyRNA *prop;
srna = RNA_def_struct(brna, "FileHandler", nullptr);
RNA_def_struct_ui_text(srna, "File Handler Type", "I/O File handler");

That description is not really useful, just rewording the struct label.

Better to get a short explanation of what is a file handler here.

That description is not really useful, just rewording the struct label. Better to get a short explanation of what is a file handler here.
Author
Contributor

As it is currently it will only add drag and drop support, I think this would fit:

Extends functionality to operators that manages files, such as adding drag and drop support.

As it is currently it will only add drag and drop support, I think this would fit: `Extends functionality to operators that manages files, such as adding drag and drop support.`
guishe marked this conversation as resolved
@ -2201,0 +2348,4 @@
RNA_def_property_ui_text(
prop,
"Operator",
"Operator that can handle import files with extension in bl_file_extensions");

with the extensions given in bl_file_extensions

`with the extensions given in bl_file_extensions`
guishe marked this conversation as resolved
@ -2201,0 +2368,4 @@
func = RNA_def_function(srna, "poll_drop", nullptr);
RNA_def_function_ui_description(
func, "If this method returns a non-null output, then the file hanlder can be used");

typo: handler

a non-null output -> True

Would also add precision: can be used to handle the drop of a drag-and-drop action. At some point I would expect we add more poll callbacks for other types of file handling.

typo: `handler` `a non-null output` -> `True` Would also add precision: `can be used to handle the drop of a drag-and-drop action`. At some point I would expect we add more poll callbacks for other types of file handling.
guishe marked this conversation as resolved
@ -2201,0 +2370,4 @@
RNA_def_function_ui_description(
func, "If this method returns a non-null output, then the file hanlder can be used");
RNA_def_function_flag(func, FUNC_NO_SELF | FUNC_REGISTER_OPTIONAL);
RNA_def_function_return(func, RNA_def_boolean(func, "visible", true, "", ""));

visible is very confusing name here? Also for booleans, we recommend using is_ or use_ prefixes, e.g. here could be is_usable...

See also https://wiki.blender.org/wiki/Source/Architecture/RNA#Defining_Structs_and_Properties

`visible` is very confusing name here? Also for booleans, we recommend using `is_` or `use_` prefixes, e.g. here could be `is_usable`... See also https://wiki.blender.org/wiki/Source/Architecture/RNA#Defining_Structs_and_Properties
guishe marked this conversation as resolved
Guillermo Venegas added 1 commit 2023-11-09 15:51:38 +01:00
Guillermo Venegas added 1 commit 2023-11-09 16:03:38 +01:00
buildbot/vexp-code-patch-coordinator Build done. Details
0654c78038
Add comment to tests and remove full stop in ui texts
Bastien Montagne approved these changes 2023-11-10 11:14:09 +01:00
Bastien Montagne left a comment
Owner

Generally LGTM now.

Details noted below can be addressed before commit and should not require another review.

Generally LGTM now. Details noted below can be addressed before commit and should not require another review.
@ -0,0 +16,4 @@
/** Import operator name. */
char import_operator[OP_MAX_TYPENAME];
/** Formatted string of file extensions supported by the file handler, each extension should
* start with a `.` and separated by a `;`. For Example: `".blend;.ble"`. */

picky and be separated by

*picky* `and be separated by`
guishe marked this conversation as resolved
@ -0,0 +7,4 @@
namespace blender::tests {
static FileHandlerType *file_handlers[8];

Would recommend rather using a #define here, easier to manage if this number needs to be changed later. 'Magic numbers' should be avoided as much as possible.

Would recommend rather using a `#define` here, easier to manage if this number needs to be changed later. 'Magic numbers' should be avoided as much as possible.
guishe marked this conversation as resolved
@ -0,0 +16,4 @@
blender::Vector<std::string> expected_file_extensions)
{
EXPECT_EQ(BKE_file_handlers().size(), test_number - 1);

Would add a check here that test_number <= <size of file_handlers> (using the #defined value for the size of this static array, as suggested above).

Would add a check here that `test_number <= <size of file_handlers>` (using the `#define`d value for the size of this static array, as suggested above).
guishe marked this conversation as resolved
@ -0,0 +38,4 @@
"File Handler Test 1",
".blender;.blend;.ble",
{".blender", ".blend", ".ble"});
file_handler_add_test(2, "Test_FH_blender2", "File Handler Test 1", ".ble", {".ble"});

This label and the next two ones seem to be off-by-one, is this expected? or copy-paste glitch?

This label and the next two ones seem to be off-by-one, is this expected? or copy-paste glitch?
guishe marked this conversation as resolved
@ -0,0 +49,4 @@
TEST(file_handler, find)
{

Useless empty line ;)

Useless empty line ;)
guishe marked this conversation as resolved

@blender-bot build

@blender-bot build
Guillermo Venegas added 1 commit 2023-11-10 16:29:50 +01:00
Guillermo Venegas added 1 commit 2023-11-10 16:36:34 +01:00
buildbot/vexp-code-patch-lint Build done. Details
buildbot/vexp-code-patch-linux-x86_64 Build done. Details
buildbot/vexp-code-patch-darwin-x86_64 Build done. Details
buildbot/vexp-code-patch-darwin-arm64 Build done. Details
buildbot/vexp-code-patch-windows-amd64 Build done. Details
buildbot/vexp-code-patch-coordinator Build done. Details
b1b3dddae7
fix license header

@blender-bot build

@blender-bot build
Guillermo Venegas added 2 commits 2023-12-08 22:34:40 +01:00
buildbot/vexp-code-patch-lint Build done. Details
buildbot/vexp-code-patch-linux-x86_64 Build done. Details
buildbot/vexp-code-patch-darwin-x86_64 Build done. Details
buildbot/vexp-code-patch-darwin-arm64 Build done. Details
buildbot/vexp-code-patch-windows-amd64 Build done. Details
buildbot/vexp-code-patch-coordinator Build done. Details
431afecc77
Merge remote-tracking branch 'origin/main' into file-handler

@blender-bot build

@blender-bot build
Author
Contributor

Make format don't fixes this for me, i will adding this manually after builds ends

image

Make format don't fixes this for me, i will adding this manually after builds ends ![image](/attachments/292a27b8-a062-437b-93d9-12311889fbd1)
Guillermo Venegas added 1 commit 2023-12-08 23:35:04 +01:00
Author
Contributor

Lint issue was fixed in last commit,the builds are still fine
image

Lint issue was fixed in last commit,the builds are still fine ![image](/attachments/172e5245-4d82-40ae-b8ad-3f579535c8b9)
Jesse Yurkovich merged commit 7c5fa8bf6c into main 2023-12-09 05:06:24 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#112466
No description provided.