Blenloader Decentralization #76372

New Issue

Jacques Lucke · 2020-05-03T16:51:12+02:00

Jacques Lucke commented

2020-05-03 16:51:12 +02:00

This is part of a larger refactoring towards a more extensible architecture in Blender (#75724).

Add new blenloader API that allows decentralizing the .blend file I/O code (48075b2c05).
Use new API in readfile.c and writefile.c.
Add new callbacks to ModifierTypeInfo (b6981d9e48).
Add new callbacks to IDTypeInfo (a443287908).
Move remaining id type specific code from readfile.c and writefile.c to IDTypeInfo callbacks.

This is part of a larger refactoring towards a more extensible architecture in Blender (#75724). - [x] Add new blenloader API that allows decentralizing the .blend file I/O code (48075b2c05). - [x] Use new API in `readfile.c` and `writefile.c`. - [x] Add new callbacks to `ModifierTypeInfo` (b6981d9e48). - [x] Add new callbacks to `IDTypeInfo` (a443287908). - [x] Move remaining id type specific code from `readfile.c` and `writefile.c` to `IDTypeInfo` callbacks.

Jacques Lucke self-assigned this 2020-05-03 16:51:12 +02:00

Jacques Lucke commented

2020-05-03 16:51:12 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Jacques Lucke commented

2020-05-03 16:51:12 +02:00

Added subscriber: @JacquesLucke

Jacques Lucke commented

2020-05-03 16:51:45 +02:00

Added subscribers: @brecht, @mont29

Bastien Montagne commented

2020-05-19 18:15:20 +02:00

Generally fine with this proposal.

I’m missing a more/better structured planning though, with some clear milestones etc.

Another question worse investigating is regarding the lib linking and expand processes. Those two could be entirely abstracted and performed through generic lib_query “foreach_id” system, which would save us adding two of the three callbacks currently needed for read code. Main issue/work here would be to add handling of deprecated DNA ID Pointers to lib_query, but that should not be a huge work (we already support conditional handling of some pointers for UI data-block pointers, doing the same thing for deprecated ones should be fairly straightforward).

Generally fine with this proposal. I’m missing a more/better structured planning though, with some clear milestones etc. Another question worse investigating is regarding the lib linking and expand processes. Those two could be entirely abstracted and performed through generic lib_query “foreach_id” system, which would save us adding two of the three callbacks currently needed for read code. Main issue/work here would be to add handling of deprecated DNA ID Pointers to lib_query, but that should not be a huge work (we already support conditional handling of some pointers for UI data-block pointers, doing the same thing for deprecated ones should be fairly straightforward).

Jacques Lucke commented

2020-05-20 12:40:05 +02:00

I agree that lib linking and expanding can and should be done using the generic foreach_id system. However, I don't think that we should do both refactorings at the same time. If we decide to do both refactorings separately, the order probably does not matter too much. The difference is that I could start with the decentralization immediately, while you'd probably need to spend some time to make lib linking and expand work with the foreach_id system.

We can also decide that we only decentralize file writing and direct data reading at first.

As for milestones, I'd say that a first milestone would be to get all modifier specific code out of readfile.c.

I agree that lib linking and expanding can and should be done using the generic `foreach_id` system. However, I don't think that we should do both refactorings at the same time. If we decide to do both refactorings separately, the order probably does not matter too much. The difference is that I could start with the decentralization immediately, while you'd probably need to spend some time to make lib linking and expand work with the `foreach_id` system. We can also decide that we only decentralize file writing and direct data reading at first. As for milestones, I'd say that a first milestone would be to get all modifier specific code out of `readfile.c`.

Jacques Lucke commented

2020-05-25 12:56:20 +02:00

Once the API is committed, I could also start using it in readfile.c and writefile.c directly. This would simplify the actual decentralization later on.

If we agree on the API, this should be fairly straight forward to do.

Once the API is committed, I could also start using it in `readfile.c` and `writefile.c` directly. This would simplify the actual decentralization later on. If we agree on the API, this should be fairly straight forward to do.

Bastien Montagne commented

2020-05-25 13:30:01 +02:00

My main point is that if we use libquery for linking and expanding, we do not need those two callbacks in IDTypeInfo then. Feels a bit useless and very noisy to add, then remove two callbacks from such a struct.

I do not know how urgent it is for you to move forward on this topic, if needed i can spend a few hours to check how feasible using libquery for this is…

My main point is that if we use libquery for linking and expanding, we do not need those two callbacks in `IDTypeInfo` then. Feels a bit useless and very noisy to add, then remove two callbacks from such a struct. I do not know how urgent it is for you to move forward on this topic, if needed i can spend a few hours to check how feasible using libquery for this is…

Jacques Lucke commented

2020-06-03 10:39:16 +02:00

For the next code quality day, I'd like to follow the checklist I added to the main post. Is that ok?

I think updating lib linking and expand to use the new API is a code quality improvement by itself, independent on how we decide to implement it in the future.

I read in the weekly report that @mont29 did some initial investigation to check if it is possible to use libquery for lib linking and expand. It seems to be not straight forward (which is also what I found). Let's discuss how to proceed with this topic, when I'm done with the tasks listed in the main post.

For the next code quality day, I'd like to follow the checklist I added to the main post. Is that ok? I think updating lib linking and expand to use the new API is a code quality improvement by itself, independent on how we decide to implement it in the future. I read in the weekly report that @mont29 did some initial investigation to check if it is possible to use libquery for lib linking and expand. It seems to be not straight forward (which is also what I found). Let's discuss how to proceed with this topic, when I'm done with the tasks listed in the main post.

Bastien Montagne commented

2020-06-03 11:43:22 +02:00

Yeah, am not super happy to have to go through those intermediate steps (since the IDTypeInfo liblink/expand callbacks will likely be removed soon-ish), but I don’t really plan to work on this usage of libquery for those in immediate future, so I'm fine with your plan.

Brecht Van Lommel commented

2020-06-03 18:46:08 +02:00

Going ahead with this for the next code quality day sounds good then.

Kenzie commented

2020-08-07 11:32:55 +02:00

Added subscriber: @KenzieMac130

blender-admin commented

2020-09-10 10:33:05 +02:00

This issue was referenced by 538817d9a8

This issue was referenced by 538817d9a86c446cdab7b6b2e84378e1f9b99f22

blender-admin commented

2020-09-10 11:17:35 +02:00

This issue was referenced by b443f9b54c

This issue was referenced by b443f9b54c38570ca6b7cdc25a8eb1f3c97b735e

blender-admin commented

2020-10-30 14:44:03 +01:00

This issue was referenced by bcb612e361

This issue was referenced by bcb612e3615e216f6defb2d631ef0f8c3c6d6c82

blender-admin commented

2020-10-30 15:27:57 +01:00

This issue was referenced by fc9ec1b9d8

This issue was referenced by fc9ec1b9d80fb17d3cc5d3c63f8e4bb356fa7ebc

blender-admin commented

2020-11-06 13:16:17 +01:00

This issue was referenced by 9ad2965921

This issue was referenced by 9ad2965921f064250830a8647de3dce68bcb2e4d

blender-admin commented

2020-11-06 13:22:10 +01:00

This issue was referenced by da96389dcb

This issue was referenced by da96389dcba7b07506fd241616e7b1db3ba7a27b

blender-admin commented

2020-11-06 13:35:19 +01:00

This issue was referenced by 41db8f2fce

This issue was referenced by 41db8f2fce1763c312a7fa47ed3aa338e6d77124

blender-admin commented

2020-11-06 14:05:09 +01:00

This issue was referenced by 62223e9851

This issue was referenced by 62223e98510f2c0d735cb7084134c5213c1cb804

blender-admin commented

2020-11-06 14:09:58 +01:00

This issue was referenced by c32a5ce17c

This issue was referenced by c32a5ce17c7609c29c832a2d229c60e12d0ccbb5

blender-admin commented

2020-11-06 14:27:51 +01:00

This issue was referenced by dfa027fe09

This issue was referenced by dfa027fe09b53485802cb0d6224ffc6bbc058b87

blender-admin commented

2020-11-06 14:40:49 +01:00

This issue was referenced by 7dd76329e1

This issue was referenced by 7dd76329e11cd5dc13b5804670bee0efe4d51fa6

blender-admin commented

2020-11-06 15:58:27 +01:00

This issue was referenced by f6524aaa80

This issue was referenced by f6524aaa8093bfc88c658107b8077cd09ac3e989

blender-admin commented

2020-11-06 16:42:37 +01:00

This issue was referenced by af24532612

This issue was referenced by af245326125aed919bc7560d20f6152e55d1478f

blender-admin commented

2020-11-06 17:13:04 +01:00

This issue was referenced by 019407810b

This issue was referenced by 019407810b9eeebce0e7fdaf5fa89906a40841ed

blender-admin commented

2020-11-06 17:22:20 +01:00

This issue was referenced by 57414e6d1d

This issue was referenced by 57414e6d1de53c1c0c32304252a2e790d2fc4786

blender-admin commented

2020-11-06 17:33:00 +01:00

This issue was referenced by 37ef37711d

This issue was referenced by 37ef37711d997899041d84e8c7a17e1fbb4efab9

blender-admin commented

2020-11-06 17:45:29 +01:00

This issue was referenced by 58e9b51f95

This issue was referenced by 58e9b51f9551bc68011b85554bad4dd6487576ce

blender-admin commented

2020-11-06 17:58:12 +01:00

This issue was referenced by 2acf01ec62

This issue was referenced by 2acf01ec62158229d62944a3adac96fe370e040a

blender-admin commented

2020-11-06 18:37:52 +01:00

This issue was referenced by 992b8f6f5f

This issue was referenced by 992b8f6f5fce7f34139ad4acf4a2b74c0cf49185

Jacques Lucke commented

2021-03-09 12:48:12 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Jacques Lucke closed this issue

2021-03-09 12:48:12 +01:00

Jacques Lucke commented

2021-03-09 12:48:12 +01:00

Closing because this task is done from a core-module perspective. Other modules now have the capability to structure their .blend I/O code however they wish. The API for that is available in BLO_read_write.h.

Closing because this task is done from a core-module perspective. Other modules now have the capability to structure their .blend I/O code however they wish. The API for that is available in `BLO_read_write.h`.

Thomas Dinges added this to the 2.93 LTS milestone 2023-02-07 18:46:39 +01:00

Brecht Van Lommel referenced this issue

2023-07-21 13:48:52 +02:00

Extensible Architecture Refactoring #75724

Sign in to join this conversation.