GPUTexture partial update #92613

New Issue

Jeroen Bakker · 2021-10-29T15:42:07+02:00

Jeroen Bakker commented

2021-10-29 15:42:07 +02:00

Current state: WIP - Still want to perform some tests and discuss with Brecht and Clement.

Partial Updates

Blender supports partial updating of images. When a user draws a stroke on an image or in the 3d view during texture painting only the area that is changed is send to the GPU. This can be done directly or post-poned until the next time any GPU texture is requested from the image.

BKE_image_update_gputexture: called during painting and will update all known GPU textures of the given image.
BKE_image_update_gputexture_delayed: called during rendering and will update GPU textures when the first GPU texture is requested.

Currently this works as an image keeps track of each GPU texture and its usage/texture format.
Limitation of this solution is that GPU textures are updated, even when not used (anymore).
Different texture formats would be implemented as part of the image. The gputexture attribute of Image might get a new dimension and add gaps.

typedef struct Image{
  ...
  /** Not written in file 3 = TEXTARGET_COUNT, 2 = stereo eyes, 2 = IMA_TEXTURE_RESOLUTION_LEN. */
  struct GPUTexture *gputexture[3][2][2];

Approach

Record the changes in partial_updates inside the image. A circular ring with a fix size. When a partial update is recorded it will store the area inside the list. When an area wants to perform a partial update of an GPU texture it can retrieve the combined area from its last partial_update_id to the last recorded transaction. When the last partial_update_id isn't known anymore (was removed from the partial update list) a full update should be done.
When done the last partial_update_id is updated to the latest recorded partial_update_id.

NOTE: The area of partial updates are recorded in tiles. (see IMA_PARTIAL_REFRESH_TILE_SIZE default size (256*256)

NOTE: this is an C API, an C++ API could be different.

/*
 * Calculate the rect to update.
 * \param image: The image to request the rect from.
 * \param from_parial_update_id: last known parial_update_id  from the caller. When -1 is requested the full list is requested.
 * \param r_area_to_update: area that contains the area from all updates between `from_partial_update_id+1` to the last_recorded_partial_update_id known by the image. When from_partial_update is the last recorded partial update id it will contain no tiles.
 * \param r_last_recorded_partial_update_id: received the last known recorded partial_update id of the image.
 *     caller should use this the next time when invoking this function.
 * \return: false - couldn't determine the rect, caller should perform a full update.
 *   true: determined area, caller can perform a partial update.
 */
bool BKE_image_partial_update_calc_area(&const Image *image, int from_transaction_id, ImagePartialUpdateArea &r_area_to_update, int &r_last_recorded_transaction_id)

The ImagePartialUpdateArea is a data structure that stores the tiles (tile coordinates) that have changes. We need to experiment with the best data structure.

TileMerging: does it make a difference when we merge neighboring tiles into a single bigger tile. It saves some allocations. The data structure
Reduce circle list by moving to the next element in the circle when an area is requested. Would reduce the amount of allocations and reduce evaluating same data multiple times.
Keep track of the area to update per listener. least amount of evaluation, but how to setup a clean API for this?

I think TileMerging and reducing elements in the circle would benefit the most. The reduction would almost be the same as keeping tract of listeners, except in a clean way for the API user.

BKE_image_update_gputexture will be removed. as all updated needs to happen delayed.
BKE_image_update_gputexture_delayed will add a rect to the partial_update circular list. partial_update id will be increased. Note that it isn't decided if atomics are needed. Could be that everything works from the main thread.

Use case

Space Image will manage their own GPU textures to draw huge images, without allocating/transferring too much towards GPU.
Cycles partial texture updates?

Alternatives

During design some alternatives had been research:

Use wmMsgBus to communicate the areas that needed to be uploaded: wmMsgBus is designed for DNA/RNA UI communication.
Use of wmNotify

Current state: `WIP` - Still want to perform some tests and discuss with Brecht and Clement. ## Partial Updates Blender supports partial updating of images. When a user draws a stroke on an image or in the 3d view during texture painting only the area that is changed is send to the GPU. This can be done directly or post-poned until the next time any GPU texture is requested from the image. `BKE_image_update_gputexture`: called during painting and will update all known GPU textures of the given image. `BKE_image_update_gputexture_delayed`: called during rendering and will update GPU textures when the first GPU texture is requested. Currently this works as an image keeps track of each GPU texture and its usage/texture format. Limitation of this solution is that GPU textures are updated, even when not used (anymore). Different texture formats would be implemented as part of the image. The gputexture attribute of Image might get a new dimension and add gaps. ``` typedef struct Image{ ... /** Not written in file 3 = TEXTARGET_COUNT, 2 = stereo eyes, 2 = IMA_TEXTURE_RESOLUTION_LEN. */ struct GPUTexture *gputexture[3][2][2]; ``` ## Approach Record the changes in partial_updates inside the image. A circular ring with a fix size. When a partial update is recorded it will store the area inside the list. When an area wants to perform a partial update of an GPU texture it can retrieve the combined area from its last partial_update_id to the last recorded transaction. When the last partial_update_id isn't known anymore (was removed from the partial update list) a full update should be done. When done the last partial_update_id is updated to the latest recorded partial_update_id. NOTE: The area of partial updates are recorded in tiles. (see `IMA_PARTIAL_REFRESH_TILE_SIZE` default size (256*256) NOTE: this is an C API, an C++ API could be different. ``` /* * Calculate the rect to update. * \param image: The image to request the rect from. * \param from_parial_update_id: last known parial_update_id from the caller. When -1 is requested the full list is requested. * \param r_area_to_update: area that contains the area from all updates between `from_partial_update_id+1` to the last_recorded_partial_update_id known by the image. When from_partial_update is the last recorded partial update id it will contain no tiles. * \param r_last_recorded_partial_update_id: received the last known recorded partial_update id of the image. * caller should use this the next time when invoking this function. * \return: false - couldn't determine the rect, caller should perform a full update. * true: determined area, caller can perform a partial update. */ bool BKE_image_partial_update_calc_area(&const Image *image, int from_transaction_id, ImagePartialUpdateArea &r_area_to_update, int &r_last_recorded_transaction_id) ``` The `ImagePartialUpdateArea` is a data structure that stores the tiles (tile coordinates) that have changes. We need to experiment with the best data structure. * TileMerging: does it make a difference when we merge neighboring tiles into a single bigger tile. It saves some allocations. The data structure * Reduce circle list by moving to the next element in the circle when an area is requested. Would reduce the amount of allocations and reduce evaluating same data multiple times. * Keep track of the area to update per listener. least amount of evaluation, but how to setup a clean API for this? I think TileMerging and reducing elements in the circle would benefit the most. The reduction would almost be the same as keeping tract of listeners, except in a clean way for the API user. `BKE_image_update_gputexture` will be removed. as all updated needs to happen delayed. `BKE_image_update_gputexture_delayed` will add a rect to the partial_update circular list. partial_update id will be increased. Note that it isn't decided if atomics are needed. Could be that everything works from the main thread. ## Use case * Space Image will manage their own GPU textures to draw huge images, without allocating/transferring too much towards GPU. * Cycles partial texture updates? ## Alternatives During design some alternatives had been research: * Use wmMsgBus to communicate the areas that needed to be uploaded: wmMsgBus is designed for DNA/RNA UI communication. * Use of wmNotify

Jeroen Bakker self-assigned this 2021-10-29 15:42:07 +02:00

Jeroen Bakker commented

2021-10-29 15:42:07 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'