Re-design of submodules used in blender.git #104573
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
4 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#104573
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This is a proposal to improve the situation with the usage of submodules in Blender.
Introduction
The submodules are designed to be used to track dependencies which do not change that often: for example, an external library. However, our usage requires behavior closer to
svn:external
where the latest state of an external is being tracked, without state stored in the main repository.The behavior of Git tooling around submodules and their
ignore
setting also changed since the initial setup of the submodules done back in 2013.Back in that time we did not have "smart"
make update
tool which could have taken care of certain aspects of setup.There was an attempt to improve the situation
aab707ab70
, but the initial though of making it more clear what's going on behind the scenes caused even more confusion. People are confused what they need to do with the submodules, and some tools still stage those automatically.One more thing to mention before jumping to the proposal to help understanding some of the parts of the proposal and the timeline. There is planned project for the year to introduce the Blender Extensions platform, which eventually will take care of community maintained addons.
Proposal
The proposal is divided into 3 different actions.
Addons
Remove
addons
andaddons_contrib
submodules, replace them with a Git checkout to a directory within a local working tree, which is ignored by.gitignore
.To avoid conflict the proposal is to check them out to a top level
scripts
directory. The addons are not part of the release process anyway, and such move will solve the problam of possible conflicts when doing bisect between two points in time before and after the change.The
make update
script will take care of initializing the checkout ofaddons
andaddons_contrib
, and keeping those up-to-date.The proposal is to not automatically de-initialize the submodule, to avoid possible loss of local branches. If there is a way to automatically safely unregister submodule we can do that instead.
Evenetually, those folders will be removed from the
blender.git
once the extension platform is in a production state. Additionally, the tricky logic for such checkout will be removed from themake update
script.The downside of this change is that all release builders would need to use
make update
pipeline for the time being. It is not a problem for us as we already do so, but it is something to clearly communicate to the external projects like Linux distribution.Locales & Developer tools
The proposal is to move them to the
blender.git
. They will go to the top level folrderslocales
andtools
respectively. Their submodules will be removed (deinitialized).The initial idea of the tools being applicable for all releases does not seem to work well, and they are already tagged and packaged at a per-release basis anyway.
The locales are because they are maintained by just Bastien, and they are not too big to warrant the dedicated repo.
Timeline
aab707ab70
.make update
script changes will be made/finished soon after that, and the change to the repository will be implemented.git pull && make update
. If it is not, it will be written and shared as well.I like most of the idea.
My only remark is that core functionality that happens to be implemented as add-ons (I/O mostly, but also OpenXR and Grease Pencil features) should be brought to inside Blender at some point.
Perhaps it would be better to align how the addon directories are handled with the proposed bundled assets #104474?
That is, instead of hosting the addon repositories as "ghost" repos inside of the blender git folder like this:
We have something like this:
With this I think we are more consistent in how we place optional/non-vital addons and data. IE, you can still build and run Blender without the three directories in the
lib
folder. However you will miss out on features.It should also make it very clear that these folders are not part of the main blender repo. If we create "ghost" repos inside the main repo with the update script, I think it might be easy for people to forget sometimes and make changes in them expecting them to be picked up when commiting to the main repo.
Mostly I don't have issues with this proposal although I don't personally have such a problem with submodules either (it seems others do, PR's are often including submodule changes accidentally)
I would prefer to instead have a small number of core add-ons in Blender's main repository, as there can still be advantages in keeping functionality modular & self contained - regarding maintenance, deprecating & upgrading functionality. There may be some exceptions to this where it makes sense to make an add-on built-in, I don't see the need to automatically make all of these core add-ons built-in.
I am not sure what is the advantage of using
lib
folder would be, but the disadvantages are:lib
folder can be seen as a "sparse" checkout of SVN.svn:external
in a Git way.pyproject.toml
is used for both built-in Blender scripts and add-ons. Sure, this is not something we can enforce with the add-ons moved to the extensions platform, but until this is done we need to keep things working in a consistent way.For Campbell and Dalai:
Indeed i forgot to state in the proposal that the built-in scripts will stay within the
scripts
directory in the main repository. This includes add-ons maintained by BF. At least in the short term.In a longer term I think those should become more of a built-in scripts like the
bl_operators
. It is still possible to keep code organized in a modular manner, but often there is extra maintenance for both us and add-on developers to check that certain other addon is enabled. But, baby steps!The PR #104755 which implements this design has been merged to
blender-v3.5-release
andmain
branches.Brecht Van Lommel referenced this issue2023-06-27 10:15:20 +02:00