Core: new blenlib library for implicit-sharing #105994

Merged
Jacques Lucke merged 17 commits from JacquesLucke/blender:implicit-sharing into main 2023-03-28 13:58:02 +02:00
Member

This was originally part of #104478, but has been split out for easier review.

The overall goal is to use implicit-sharing in many places in Blender that currently do unnecessary copies. See #95845 for more details.

This commit only adds the base data structures in blenlib and uses those in GeometrySet and AnonymousAttributeID, which used a more ad-hoc version of implicit sharing already.

The plan is to follow this up with implicit-sharing integrations in places in Blender:

  • CustomData: Allows sharing attributes between multiple data blocks, significantly reducing the cost of copying a geometry data-block.
  • Undo: Allows avoiding a full copy of attributes in the undo step and comparing the old attribute values to the new ones.
  • Cycles Attribute: Avoids copying attributes into cycles for rendering.
  • GPUVertBuf; Avoids copying attributes for viewport/eevee rendering.
This was originally part of #104478, but has been split out for easier review. The overall goal is to use implicit-sharing in many places in Blender that currently do unnecessary copies. See #95845 for more details. This commit only adds the base data structures in blenlib and uses those in `GeometrySet` and `AnonymousAttributeID`, which used a more ad-hoc version of implicit sharing already. The plan is to follow this up with implicit-sharing integrations in places in Blender: * `CustomData`: Allows sharing attributes between multiple data blocks, significantly reducing the cost of copying a geometry data-block. * Undo: Allows avoiding a full copy of attributes in the undo step and comparing the old attribute values to the new ones. * Cycles `Attribute`: Avoids copying attributes into cycles for rendering. * `GPUVertBuf`; Avoids copying attributes for viewport/eevee rendering.
Jacques Lucke added 6 commits 2023-03-22 14:08:49 +01:00
Jacques Lucke added 1 commit 2023-03-22 14:10:03 +01:00
Jacques Lucke added 1 commit 2023-03-22 14:11:26 +01:00
Jacques Lucke added 1 commit 2023-03-22 14:14:27 +01:00
Jacques Lucke requested review from Hans Goudey 2023-03-22 14:23:42 +01:00
Hans Goudey added 2 commits 2023-03-22 15:29:16 +01:00
Hans Goudey approved these changes 2023-03-22 16:02:50 +01:00
Hans Goudey left a comment
Member

This looks good to me. I added some comments inline but they're more like questions so I'll accept this now. The naming seems fine to me, though I don't think Implicit is adding much besides making the name a bit longer and more unique.


Do you think it would be reasonable for ImplicitSharingPtr to have a function like make_mutable(), essentially to handle the "copy" part of copy-on-write.

That would be useful for get_component_for_write and potentially when eagerly updating mesh caches (if they were using ImplicitSharingPtr instead of shared_ptr. It would also have to know how to copy the data, so maybe it's fine to do that on another level, I'm just curious honestly.

This looks good to me. I added some comments inline but they're more like questions so I'll accept this now. The naming seems fine to me, though I don't think `Implicit` is adding much besides making the name a bit longer and more unique. --- Do you think it would be reasonable for `ImplicitSharingPtr` to have a function like `make_mutable()`, essentially to handle the "copy" part of copy-on-write. That would be useful for `get_component_for_write` and potentially when eagerly updating mesh caches (if they were using `ImplicitSharingPtr` instead of `shared_ptr`. It would also have to know how to copy the data, so maybe it's fine to do that on another level, I'm just curious honestly.
@ -0,0 +52,4 @@
bool is_shared() const
{
return users_.load(std::memory_order_relaxed) >= 2;
}
Member

This could be over-documenting and redundant, I wonder what you think, but possibly some comments like this could help people answer their questions faster:

  • /** True if there are other const references to the resource, meaning it cannot be modified. */
  • /** Whether the resource can be modified without a copy. */
  • /** Call when creating another const reference/pointer to the data. */
    etc.
This could be over-documenting and redundant, I wonder what you think, but possibly some comments like this could help people answer their questions faster: - `/** True if there are other const references to the resource, meaning it cannot be modified. */` - `/** Whether the resource can be modified without a copy. */` - `/** Call when creating another const reference/pointer to the data. */` etc.
JacquesLucke marked this conversation as resolved
@ -0,0 +80,4 @@
};
/**
* Makes it easy to embed implicit-sharing behavior into a struct. This also allows the subclass to
Member

This comment is a bit hard to follow, maybe replacing the subclass with something more specific would help (like saying the ptr class can be used with a class that derives from the mixin class).

This comment is a bit hard to follow, maybe replacing `the subclass` with something more specific would help (like saying the ptr class can be used with a class that derives from the mixin class).
JacquesLucke marked this conversation as resolved
@ -0,0 +13,4 @@
/**
* #ImplicitSharingPtr is a smart pointer that manages implicit sharing. It's designed to work with
* types that derive from #ImplicitSharingMixin. It is fairly similar to #std::shared_ptr but
* expects the reference count to be embedded in the data.
Member

expects -> requires, right? std::make_shared will allocate the use count with the object so it's helpful to clarify this a bit.

`expects` -> `requires`, right? `std::make_shared` will allocate the use count with the object so it's helpful to clarify this a bit.
JacquesLucke marked this conversation as resolved
Author
Member

This looks good to me. I added some comments inline but they're more like questions so I'll accept this now. The naming seems fine to me, though I don't think Implicit is adding much besides making the name a bit longer and more unique.

The reason for using Implicit is that this is a standard name for what we are doing here, think that's good. I don't care much about the slightly longer name here. In practice that name should show up in code super often, since its use is implicit ;D

Do you think it would be reasonable for ImplicitSharingPtr to have a function like make_mutable(), essentially to handle the "copy" part of copy-on-write.

Currently, I think no, because then the ImplicitSharingInfo would also have to know how to copy the data. It's possible that different users want to copy the data in different ways though (e.g. copy an attribute into a buffer allocated by another allocator, not that we need that now, but it's a possibility). Besides that, I think especially for attributes it probably often makes sense to just replace an attribute buffer instead of copying and then modifying it.

> This looks good to me. I added some comments inline but they're more like questions so I'll accept this now. The naming seems fine to me, though I don't think `Implicit` is adding much besides making the name a bit longer and more unique. The reason for using `Implicit` is that this is a standard name for what we are doing here, think that's good. I don't care much about the slightly longer name here. In practice that name should show up in code super often, since its use is implicit ;D > Do you think it would be reasonable for `ImplicitSharingPtr` to have a function like `make_mutable()`, essentially to handle the "copy" part of copy-on-write. Currently, I think no, because then the `ImplicitSharingInfo` would also have to know how to copy the data. It's possible that different users want to copy the data in different ways though (e.g. copy an attribute into a buffer allocated by another allocator, not that we need that now, but it's a possibility). Besides that, I think especially for attributes it probably often makes sense to just replace an attribute buffer instead of copying and then modifying it.
Jacques Lucke added 2 commits 2023-03-22 18:14:38 +01:00
Author
Member

@blender-bot build

@blender-bot build
Jacques Lucke requested review from Brecht Van Lommel 2023-03-22 19:02:45 +01:00
Jacques Lucke requested review from Sergey Sharybin 2023-03-22 19:02:45 +01:00
Brecht Van Lommel requested changes 2023-03-24 13:05:45 +01:00
@ -0,0 +73,4 @@
*/
void remove_user_and_delete_if_last() const
{
const int old_user_count = users_.fetch_sub(1, std::memory_order_relaxed);

My understanding is that user count decrement requires std::memory_order_acq_rel, while increment can use std::memory_order_relaxed.

For example libc++ does this for shared_ptr. More details:
https://stackoverflow.com/questions/48124031/stdmemory-order-relaxed-atomicity-with-respect-to-the-same-atomic-variable

My understanding is that user count decrement requires `std::memory_order_acq_rel`, while increment can use `std::memory_order_relaxed`. For example libc++ does this for shared_ptr. More details: https://stackoverflow.com/questions/48124031/stdmemory-order-relaxed-atomicity-with-respect-to-the-same-atomic-variable
brecht marked this conversation as resolved
Jacques Lucke added 2 commits 2023-03-27 13:37:57 +02:00
Jacques Lucke requested review from Brecht Van Lommel 2023-03-27 13:38:05 +02:00
Brecht Van Lommel approved these changes 2023-03-27 13:42:21 +02:00
Jacques Lucke added 1 commit 2023-03-28 12:44:51 +02:00
Sergey Sharybin approved these changes 2023-03-28 12:55:05 +02:00
Sergey Sharybin left a comment
Owner

Would be nice to have the BLI types tested. Even if currently the tests might look trivial, it does help in the future nevertheless.

Since it is not really a new library, but more like a refactor of existing code is probably fine to move forward with this patch.

Would be nice to have the BLI types tested. Even if currently the tests might look trivial, it does help in the future nevertheless. Since it is not really a new library, but more like a refactor of existing code is probably fine to move forward with this patch.
Jacques Lucke added 1 commit 2023-03-28 13:24:34 +02:00
buildbot/vexp-code-patch-coordinator Build done. Details
aff843b7f1
add unit test
Author
Member

@blender-bot build

@blender-bot build
Jacques Lucke merged commit fbcddfcd68 into main 2023-03-28 13:58:02 +02:00
Jacques Lucke deleted branch implicit-sharing 2023-03-28 13:58:03 +02:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#105994
No description provided.