Proposal: Switch Blender Studio to Pull Pipeline #85

New Issue

Nick Alberelli · 2023-06-16T21:19:06+02:00

Nick Alberelli commented

2023-06-16 21:19:06 +02:00

Issue

Currently the blender studio uses a push pipeline, which means all files are 'online' and linked to each other. This means only a single version of an asset can be linked to production files, alternative version for things like character rigs need to be created as separate assets. If a file update has a breaking changes this update will hit all production files as soon as the change is saved. This is known as a "Push Pipeline"

Example of Push Pipeline

└── pro/
    ├── shots/
    │   └── 010_intro/
    │       └── 010_0010/
    │           ├── 010_0010.anim.blend 
    │           └── 010_0010.lighting.blend
    └── lib/
        └── char/
            └── dog/
                ├── dog.modeling.blend
                ├── dog.rigging.blend
                ├── dog.shading.blend
                └── publish/
                    └── dog.v001.blend # Linked to all shots

Solution

Switching to a Pull Pipeline, would reduce the cascading errors during production, as it requires a user to update their production files to use another version of a published asset. The beauty of this system is that it allows users to decided when to create new published versions, in theory this system should also be able to PUSH as well by saving new changes on existing files. The choice of when to cerate new versions is determined on a pre-production basis.

Production Files have a Work folder and a Publish folder
Users always edit the Work files, When invoking a "Publish Operator" an addon will copy file to publish if they pass Convention Checks
Production Files are saved with Version Numbers indicting work/publish version
Addon would inform users when opening a file that there is a newer version of a linked asset available in publish folder
Updating linked files will be done via Relocating the library to new new .blend file
Users are responsible for reviewing their file when updating asset in cases where asset has breaking changes
Solution is task type agnostic, will work on all linked data for example from Asset->Animation or from Animation->Lighting
Solution is Version Control agnostic, can be deployed via SVN, GITLFS or on a simple NAS for a local team.

Example of Pull Pipeline

└── pro/
    ├── shots/
    │   └── 010_intro/
    │       └── 010_0010/
    │           ├── work/
    │           │   ├── 010_0010_v001.anim.blend
    │           │   ├── 010_0010_v002.anim.blend # user decided not to publish this version
    │           │   └── 010_0010_v003.anim.blend
    │           └── publsh/
    │               ├── 010_0010_v001.anim.blend # linked to dog_v002
    │               └── 010_0010_v003.anim.blend # linked to dog_v003
    └── lib/
        └── char/
            └── dog/
                ├── work/
                │   ├── dog.modeling.blend
                │   ├── dog.rigging.blend
                │   └── dog.shading.blend
                └── publish/ 
                    ├── dog_v001.blend 
                    ├── dog_v002.blend # legacy publish
                    └── dog_v003.blend # latest publish

# Issue Currently the blender studio uses a push pipeline, which means all files are 'online' and linked to each other. This means only a single version of an asset can be linked to production files, alternative version for things like character rigs need to be created as separate assets. If a file update has a breaking changes this update will hit all production files as soon as the change is saved. This is known as a "Push Pipeline" ## Example of Push Pipeline ```bash └── pro/ ├── shots/ │ └── 010_intro/ │ └── 010_0010/ │ ├── 010_0010.anim.blend │ └── 010_0010.lighting.blend └── lib/ └── char/ └── dog/ ├── dog.modeling.blend ├── dog.rigging.blend ├── dog.shading.blend └── publish/ └── dog.v001.blend # Linked to all shots ``` # Solution Switching to a Pull Pipeline, would reduce the cascading errors during production, as it requires a user to update their production files to use another version of a published asset. The beauty of this system is that it allows users to decided when to create new published versions, in theory this system should also be able to PUSH as well by saving new changes on existing files. The choice of when to cerate new versions is determined on a pre-production basis. - Production Files have a Work folder and a Publish folder - Users always edit the Work files, When invoking a "Publish Operator" an addon will copy file to publish if they pass [Convention Checks](https://projects.blender.org/studio/blender-studio-pipeline/issues/84) - Production Files are saved with Version Numbers indicting work/publish version - Addon would inform users when opening a file that there is a newer version of a linked asset available in publish folder - Updating linked files will be done via [Relocating](https://docs.blender.org/manual/en/latest/files/linked_libraries/link_append.html#library-reload-relocate) the library to new new .blend file - Users are responsible for reviewing their file when updating asset in cases where asset has breaking changes - Solution is task type agnostic, will work on all linked data for example from Asset->Animation or from Animation->Lighting - Solution is Version Control agnostic, can be deployed via SVN, GITLFS or on a simple NAS for a local team. ## Example of Pull Pipeline ```bash └── pro/ ├── shots/ │ └── 010_intro/ │ └── 010_0010/ │ ├── work/ │ │ ├── 010_0010_v001.anim.blend │ │ ├── 010_0010_v002.anim.blend # user decided not to publish this version │ │ └── 010_0010_v003.anim.blend │ └── publsh/ │ ├── 010_0010_v001.anim.blend # linked to dog_v002 │ └── 010_0010_v003.anim.blend # linked to dog_v003 └── lib/ └── char/ └── dog/ ├── work/ │ ├── dog.modeling.blend │ ├── dog.rigging.blend │ └── dog.shading.blend └── publish/ ├── dog_v001.blend ├── dog_v002.blend # legacy publish └── dog_v003.blend # latest publish ```

👍 1

Nick Alberelli added the

Kind

Proposal

label 2023-06-16 21:19:07 +02:00

Nick Alberelli referenced this issue

2023-06-16 21:20:51 +02:00

Proposal: Create 'Convention Check' System #84

Nick Alberelli commented

2023-06-21 18:04:51 +02:00

@fsiddi has some concerns about this system as it may cause conflicts because we use a rolling release of blender. So we should probably stick to a Push pipeline for now.

@FoxGlove @ZedDB ^

@fsiddi has some concerns about this system as it may cause conflicts because we use a rolling release of blender. So we should probably stick to a Push pipeline for now. @FoxGlove @ZedDB ^

Nick Alberelli closed this issue

2023-06-21 18:04:51 +02:00

Sign in to join this conversation.

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: studio/blender-studio-tools#85

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Proposal: Switch Blender Studio to Pull Pipeline #85

Issue

Example of Push Pipeline

Solution

Example of Pull Pipeline