Inconsistent behavior of bpy.types.Image.pixels depending on image precision #117830

Open
opened 2024-02-05 11:04:59 +01:00 by Eugene-Kuznetsov · 4 comments
Contributor

Attached file contains two textures 'tex_test1' and 'tex_test2'. Both are set to 'sRGB'.

Trying to pull colors out of these textures with a script produces:

>>> bpy.data.images["tex_test1.png"].pixels[0:3]
(0.2635093033313751, 0.18732529878616333, 0.1290532946586609)

>>> bpy.data.images["tex_test2.png"].pixels[0:3]
(0.501960813999176, 0.3607843220233917, 0.2823529541492462)

The same colors are shown by the color picker tool in the image editor.

However, it's obvious that tex_test1 is actually lighter than tex_test2. With a "separate color" + "greater than" shader node combo, you can establish that these texture nodes output R=0.26 and R=0.22 respectively.

The discrepancy occurs because tex_test1 is float (RGBA16F) and tex_test2 is byte (RGBA8). As far as I can tell, what happens is this:

  • If the texture is byte, image editor and 'pixels' both return whatever is actually stored in the texture file, before colorspace conversion. The shader receives data after conversion from whatever is specified in the texture node (in this case, sRGB) to scene linear.
  • If the texture is float, contents of the file is converted from sRGB into scene linear right away, and then everyone (the image editor, 'pixels', and the shader) get the same data.

This becomes a problem because it inteferes with the ability to use 'pixels' in scripts (e.g. for color matching/blending). If all my textures are byte and I need to determine the color that corresponds to a particular pixel, I can write
mathutils.Color(bpy.data.images["tex_test1.png"].pixels[0:3]).from_srgb_to_scene_linear()

But this stops working for float textures and it's not at all obvious why (or even what makes those textures different, since nothing in the documentation gives any hint that float textures behave differently).

Attached file contains two textures 'tex_test1' and 'tex_test2'. Both are set to 'sRGB'. Trying to pull colors out of these textures with a script produces: ``` >>> bpy.data.images["tex_test1.png"].pixels[0:3] (0.2635093033313751, 0.18732529878616333, 0.1290532946586609) >>> bpy.data.images["tex_test2.png"].pixels[0:3] (0.501960813999176, 0.3607843220233917, 0.2823529541492462) ``` The same colors are shown by the color picker tool in the image editor. However, it's obvious that tex_test1 is actually lighter than tex_test2. With a "separate color" + "greater than" shader node combo, you can establish that these texture nodes output R=0.26 and R=0.22 respectively. The discrepancy occurs because tex_test1 is float (RGBA16F) and tex_test2 is byte (RGBA8). As far as I can tell, what happens is this: * If the texture is byte, image editor and 'pixels' both return whatever is actually stored in the texture file, before colorspace conversion. The shader receives data after conversion from whatever is specified in the texture node (in this case, sRGB) to scene linear. * If the texture is float, contents of the file is converted from sRGB into scene linear right away, and then everyone (the image editor, 'pixels', and the shader) get the same data. This becomes a problem because it inteferes with the ability to use 'pixels' in scripts (e.g. for color matching/blending). If all my textures are byte and I need to determine the color that corresponds to a particular pixel, I can write ```mathutils.Color(bpy.data.images["tex_test1.png"].pixels[0:3]).from_srgb_to_scene_linear()``` But this stops working for float textures and it's not at all obvious why (or even what makes those textures different, since nothing in the documentation gives any hint that float textures behave differently).
Eugene-Kuznetsov added the
Priority
Normal
Type
Report
Status
Needs Triage
labels 2024-02-05 11:05:00 +01:00
Iliya Katushenock added the
Interest
Images & Movies
label 2024-02-05 15:51:52 +01:00
Iliya Katushenock added
Module
Core
Status
Confirmed
and removed
Status
Needs Triage
labels 2024-02-05 15:54:19 +01:00
@aras_p /

@mod_moder not sure why I'm tagged, but wild guess: isn't that because within Blender, byte textures are (typically) sRGB, non-premultiplied, whereas float textures are in scene linear color space, and with alpha premultiplied?

(as in, while possibly confusing, that would land on a "by design" area that perhaps should be documented somewhere, if it's not already)

@mod_moder not sure why I'm tagged, but wild guess: isn't that because within Blender, byte textures are (typically) sRGB, non-premultiplied, whereas float textures are in scene linear color space, _and_ with alpha premultiplied? (as in, while possibly confusing, that would land on a "by design" area that perhaps should be documented somewhere, if it's not already)

@aras_p Thanks for reply, i saw that you been working in this area recently (mainly in context of speedup) to decide to ping you in order to see if you have thoughts about this issue. Maybe i also need to ping someone from eevee module since this is just inconsistency between cpu and gpu color treatments.

@aras_p Thanks for reply, i saw that you been working in this area recently (mainly in context of speedup) to decide to ping you in order to see if you have thoughts about this issue. Maybe i also need to ping someone from eevee module since this is just inconsistency between cpu and gpu color treatments.
Iliya Katushenock added the
Interest
EEVEE & Viewport
label 2024-02-05 18:30:20 +01:00
Author
Contributor

@mod_moder not sure why I'm tagged, but wild guess: isn't that because within Blender, byte textures are (typically) sRGB, non-premultiplied, whereas float textures are in scene linear color space, and with alpha premultiplied?

(as in, while possibly confusing, that would land on a "by design" area that perhaps should be documented somewhere, if it's not already)

The inconsistency is between treatment of 8-bit-per-component and 16-bit-per-component images. 'tex_test1' is loaded from a 16-bit PNG. 'tex_test2' is loaded from a 8-bit PNG. 8-bit images are loaded as byte textures, and higher precision images are loaded as float textures:

https://projects.blender.org/blender/blender/src/branch/main/source/blender/imbuf/IMB_imbuf_types.hh#L195-L211

As far as what should actually be done about it, it seems to me that there are two options. The easy option is to update the documentation, so that here https://docs.blender.org/api/current/bpy.types.Image.html#bpy.types.Image.pixels it says "for an image file, returns original pixel values as stored in the file if it's a 8-bit file, and returns pixel values after conversion to scene-linear colorspace if it's a 16-bit file."

The hard option is to add colorspace conversions here https://projects.blender.org/blender/blender/src/branch/main/source/blender/makesrna/intern/rna_image.cc#L622 (for Image.pixels), here https://projects.blender.org/blender/blender/src/branch/main/source/blender/editors/space_node/node_view.cc#L596 and maybe in some other places where this could arise.

This is all very old code and no one may be familiar with it any more. I can write a PR if we can reach some agreement on what should be happening. I'll ping @sergey on account of #107609.

> @mod_moder not sure why I'm tagged, but wild guess: isn't that because within Blender, byte textures are (typically) sRGB, non-premultiplied, whereas float textures are in scene linear color space, _and_ with alpha premultiplied? > > (as in, while possibly confusing, that would land on a "by design" area that perhaps should be documented somewhere, if it's not already) The inconsistency is between treatment of 8-bit-per-component and 16-bit-per-component images. 'tex_test1' is loaded from a 16-bit PNG. 'tex_test2' is loaded from a 8-bit PNG. 8-bit images are loaded as byte textures, and higher precision images are loaded as float textures: https://projects.blender.org/blender/blender/src/branch/main/source/blender/imbuf/IMB_imbuf_types.hh#L195-L211 As far as what should actually be done about it, it seems to me that there are two options. The easy option is to update the documentation, so that here https://docs.blender.org/api/current/bpy.types.Image.html#bpy.types.Image.pixels it says "for an image file, returns original pixel values as stored in the file if it's a 8-bit file, and returns pixel values after conversion to scene-linear colorspace if it's a 16-bit file." The hard option is to add colorspace conversions here https://projects.blender.org/blender/blender/src/branch/main/source/blender/makesrna/intern/rna_image.cc#L622 (for Image.pixels), here https://projects.blender.org/blender/blender/src/branch/main/source/blender/editors/space_node/node_view.cc#L596 and maybe in some other places where this could arise. This is all very old code and no one may be familiar with it any more. I can write a PR if we can reach some agreement on what should be happening. I'll ping @sergey on account of https://projects.blender.org/blender/blender/pulls/107609.
Eugene-Kuznetsov changed title from Discrepancy between float and byte texture operation to Inconsistent behavior of bpy.types.Image.pixels depending on image precision 2024-02-06 01:04:32 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#117830
No description provided.