Color picker colors drift value for some OCIO configurations #101000

New Issue

System Information
Operating system: Windows-10-10.0.22000-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3050 Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.59

Blender Version
Broken: version: 3.4.0 Alpha, branch: master, commit date: 2022-09-04 04:25, hash: 0ff920b777
Worked: never

Short description of error
The OCIO config's color_picking role seems to only accept two situations: 1. Colorspace with Linear Transfer Function or 2. Colorspace with Blender's sRGB LUT. Note OCIOv2's builtin sRGB transform also doesn't work, it seems to only work if we use Blender's sRGB LUT.

Here is the demostration of the error:
Color Picker Error.mp4

Sept 19 EDIT:
Color Picker Bug 2.mp4

See how the max(RGB)/HSV's V keeps getting higher and higher.

Exact steps for others to reproduce the error
To demonstrate the point, we will use a colorspace that doesn't make sense to use the sRGB LUT, BT.2020.
Download and unzip this testing OCIO config and put it in colormanagement folder:
color picker bug.7z

This testing config has the working space (scene_linear role) set to Linear BT.2020, and the color_picking role set to the BT.2020 with this colorspace standard's native EOTF ITU-R BT.1886 2.4.

Install the test config and launch Blender
Click on any color input like in the shader nodes, drag the dot in the color wheel and keep moving, you should see the V value goes up over time. Then close Blender
Open the test config with any text editor, change the color_picking role at line 11 to BT.2020 with sRGB LUT, then reopen Blender
Do step two again, this time the V is unchanged.

It doesn't make sense for BT.2020 to use the sRGB LUT, ideally we should be able to use the colorspace with its native EOTF (or any transfer function you want to use it with), instead of hardcode it for a specific LUT.

This is problematic for the development of the Filmic upgrade (AgX) config, as we will not include the sRGB LUT in the new config at all.

**System Information** Operating system: Windows-10-10.0.22000-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3050 Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.59 **Blender Version** Broken: version: 3.4.0 Alpha, branch: master, commit date: 2022-09-04 04:25, hash: `0ff920b777` Worked: never **Short description of error** The OCIO config's `color_picking` role seems to only accept two situations: 1. Colorspace with Linear Transfer Function or 2. Colorspace with Blender's sRGB LUT. Note OCIOv2's builtin sRGB transform also doesn't work, it seems to only work if we use Blender's sRGB LUT. Here is the demostration of the error: [Color Picker Error.mp4](https://archive.blender.org/developer/F13483262/Color_Picker_Error.mp4) Sept 19 EDIT: [Color Picker Bug 2.mp4](https://archive.blender.org/developer/F13512544/Color_Picker_Bug_2.mp4) See how the max(RGB)/HSV's V keeps getting higher and higher. **Exact steps for others to reproduce the error** To demonstrate the point, we will use a colorspace that doesn't make sense to use the sRGB LUT, BT.2020. Download and unzip this testing OCIO config and put it in colormanagement folder: [color picker bug.7z](https://archive.blender.org/developer/F13483277/color_picker_bug.7z) This testing config has the working space (`scene_linear` role) set to Linear BT.2020, and the `color_picking` role set to the BT.2020 with this colorspace standard's native EOTF `ITU-R BT.1886 2.4`. 1. Install the test config and launch Blender 2. Click on any color input like in the shader nodes, drag the dot in the color wheel and keep moving, you should see the `V` value goes up over time. Then close Blender 3. Open the test config with any text editor, change the `color_picking` role at line 11 to `BT.2020 with sRGB LUT`, then reopen Blender 4. Do step two again, this time the `V` is unchanged. It doesn't make sense for BT.2020 to use the sRGB LUT, ideally we should be able to use the colorspace with its native EOTF (or any transfer function you want to use it with), instead of hardcode it for a specific LUT. This is problematic for the development of the Filmic upgrade (AgX) config, as we will not include the sRGB LUT in the new config at all.

Added subscriber: @Eary

I believe color management is under Render & Cycles module

I believe color management is under `Render & Cycles` module

Added subscriber: @brecht

I didn't check the bug yet, but I hope the plan is not to use BT.2020 as the scene linear color space in a config intended for users? The industry is standardizing on ACEScg for that purpose, using something else is going to create a compatibility mess and I would very strongly advise against it.

The plan is to use Linear DCI-[P3](https://archive.blender.org/developer/P3.txt). The BT.2020 here is just for demonstration.

The plan is to use `Linear DCI-[P3](https://archive.blender.org/developer/P3.txt)`. The BT.2020 here is just for demonstration.

That's not any better, still going to be different than what everyone else uses.

I don't understand how is using a different working space is going to create compatibility issue. Compatibility should be fine with import and export options. The internal working space shouldn't really matter, if I understand it correctly.

Conversion between color spaces is imperfect for arbitrary shading nodes manipulating those colors, smaller gamut means ACEScg colors imported into Blender will be clipped, and from a user point of view it will just be confusing to see different scene linear values in different applications and make interop harder.

In #101000#1415577, @brecht wrote:
smaller gamut means ACEScg colors imported into Blender will be clipped

From my tests out-of-scope values seems to be present as negative values, the negatives seems to go all the way into the end compositor, where some operations like denoising would clip those negative values. If you don't apply any operation in compositor to clip those negative values, we are developing some negative value handling in the view transform at the moment to bring some values back in range for image formation.

But note we will go spectral in the future. We can easily render values outside of even ACEScg with spectral rendering. Setting the working space to ACEScg won't really solve the problem. And ACEScg involves parts outside of the spectral locus, it will create problems for 1. The image formation (what should a "color" that is not actually a "color" looks like? e.g. ACEScg (0, 1, 0) ) , 2. the spectral reconstruction for spectral Cycles.

It is important to realize that people with Mac etc. have Display P3 monitors, so a reasonable amount of users can actually see those P3 values as they are rendered, on their screens. ACEScg, however, the AP1 primaries are just math results of the CIE XYZ projection. There will never be a display device for it. BT.2020 currently only has laser projectors, but it still exists. So BT.2020 is protentially acceptable given some of its problems with image formation etc. will hopefully be solved in the future.

> In #101000#1415577, @brecht wrote: > smaller gamut means ACEScg colors imported into Blender will be clipped From my tests out-of-scope values seems to be present as negative values, the negatives seems to go all the way into the end compositor, where some operations like denoising would clip those negative values. If you don't apply any operation in compositor to clip those negative values, we are developing some negative value handling in the view transform at the moment to bring some values back in range for image formation. But note we will go spectral in the future. We can easily render values outside of even ACEScg with spectral rendering. Setting the working space to ACEScg won't really solve the problem. And ACEScg involves parts outside of the spectral locus, it will create problems for 1. The image formation (what should a "color" that is not actually a "color" looks like? e.g. ACEScg (0, 1, 0) ) , 2. the spectral reconstruction for spectral Cycles. It is important to realize that people with Mac etc. have Display [P3](https://archive.blender.org/developer/P3.txt) monitors, so a reasonable amount of users can actually see those [P3](https://archive.blender.org/developer/P3.txt) values as they are rendered, on their screens. ACEScg, however, the AP1 primaries are just math results of the CIE XYZ projection. There will never be a display device for it. BT.2020 currently only has laser projectors, but it still exists. So BT.2020 is protentially acceptable given some of its problems with image formation etc. will hopefully be solved in the future.

In #101000#1415595, @Eary wrote:
From my tests out-of-scope values seems to be present as negative values, the negatives seems to go all the way into the end compositor, where some operations like denoising would clip those negative values. If you don't apply any operation in compositor to clip those negative values, we are developing some negative value handling in the view transform at the moment to bring some values back in range for image formation.

Regardless if it clips or gives negative values, that's an interop issues users now have to deal with if they bring in EXRs rendered in another application that uses ACEScg that they wouldn't have to deal with otherwise.

But note we will go spectral in the future. We can easily render values outside of even ACEScg with spectral rendering. Setting the working space to ACEScg won't really solve the problem.

As a user, no you should not be able to easily get values outside of ACEScg in a render buffer, the renderer should clip that for you.

In general, I don't agree with the apparent argument that because X or Y would be a problem regardless, we may as well accept dozens of other interop issues that can be avoided.

> In #101000#1415595, @Eary wrote: > From my tests out-of-scope values seems to be present as negative values, the negatives seems to go all the way into the end compositor, where some operations like denoising would clip those negative values. If you don't apply any operation in compositor to clip those negative values, we are developing some negative value handling in the view transform at the moment to bring some values back in range for image formation. Regardless if it clips or gives negative values, that's an interop issues users now have to deal with if they bring in EXRs rendered in another application that uses ACEScg that they wouldn't have to deal with otherwise. > But note we will go spectral in the future. We can easily render values outside of even ACEScg with spectral rendering. Setting the working space to ACEScg won't really solve the problem. As a user, no you should not be able to easily get values outside of ACEScg in a render buffer, the renderer should clip that for you. In general, I don't agree with the apparent argument that because X or Y would be a problem regardless, we may as well accept dozens of other interop issues that can be avoided.

Added subscriber: @kram1032

The big issue I see with ACEScg as a working space is, that you can get unphysical values not just as initial colors (which, ok, if you happen to know that you can just not make use of those values I guess), but also as bounce light which would be very tricky to account for.
I think with ACEScg that's going to happen for the red and green region. - In those regions, there simply is no good choice for what to display for those colors. They do not exist.

Meanwhile, a large chunk of green and purple values are also excluded by it, so the renderer would be unable to produce them (before spectral rendering and without negative light sources) - but that may be a lesser problem. The only way to fix that with a tristimulus gamut would be to exacerbate the previous problem, going beyond spectral even more.

IMO Color pickers and such ought to have a colorspace dropdown menu, defaulting to whatever the working space is. The ideal implementation would also let people pick what that default is, perhaps in some sort of advanced options menu in the color management menu. That way, people can decide for themselves whether ACEScg is what they want.

But just getting all of that through OCIO configs would also work to the same effect. Either way, it should not be hardcoded.
If you decide, that ACEScg should indeed be the new default, at least making it easily possible to switch to whatever config is desired for a given project by a given user.

An advanced method might display, in an "Advanced Colormanagement" subpanel, what the currently chosen OCIO config gives you for various spaces, plus the ability to overwrite pieces of that config if you so desire.

I.e. it could show this section of OCIO configs (or if the config doesn't specify, use the default Blender configs)

(there are example settings and may not be sensible as such - I'm honestly not sure what some of these mean, although most are pretty self-explanatory I think)

roles:
  color_picking: sRGB 2.2 EOTF I-E
  color_timing: sRGB 2.2 EOTF
  compositing_log: sRGB 2.2 EOTF
  data: Generic Data
  default: sRGB 2.2 EOTF
  default_byte: sRGB 2.2 EOTF
  default_float: Linear BT.709 I-E 
  default_sequencer: sRGB 2.2 EOTF
  matte_paint: sRGB 2.2 EOTF
  reference: Linear CIE-XYZ I-E
  scene_linear: Linear DCI-P3 I-E
  texture_paint: sRGB 2.2 EOTF
  XYZ: Linear CIE-XYZ I-D65

and add an overwrite-toggle next to each, in which case the corresponding line would turn into a dropdown menu with all color spaces Blender currently knows about from various configs.
This would also future-proof Blender in case the aforementioned industry standard gets changed yet again.

But anyway, even if not that, at least a per-material or per Color-node dropdown menu to that effect would be great to have, and could certainly fix the shown issue for any situation. The color picker would simply convert whatever color it picks from what space it currently is into the space specified by the dropdown menu.
Probably should also show some clear UI indicator if the chosen space in the dropdown menu doesn't match the working space to avoid confusion, and the default should be either the working space or, if the config specifies, the color_picking space.

The Sequencer already has a special menu overwrite, presumably defaulting to default_sequencer but overwriting that if need be, and imported images (as textures or in the compositor) also have their overwritable defaults, so it makes sense to expand that to more places in Blender where specifying the way color is interpreted might make a difference.

This might be me getting ahead of myself but it would also be nice if configs could actually be passed in as a special socket type in the various node editors. Imagine a huge node tree which, once everything is connected up as desired, you can set to work for a different specific other color space in a single click. If no noodle goes in, it's a regular dropdown menu, but otherwise the colorspace can be passed around freely, including implicit conversions and such.
This would also be quite useful for the Convert Colorspace node in the Compositor, such that you don't have to think about how to interpret the color, you could just go "convert from whatever it currently is (by plugging in the relevant noodle) to this specific one I now want (which also gets stored in an output for yet another noodle)"

The big issue I see with ACEScg as a working space is, that you can get unphysical values not just as initial colors (which, ok, if you happen to know that you can just not make use of those values I guess), but also as bounce light which would be very tricky to account for. I think with ACEScg that's going to happen for the red and green region. - In those regions, there simply is no good choice for what to display for those colors. They do not exist. Meanwhile, a large chunk of green and purple values are also excluded by it, so the renderer would be unable to produce them (before spectral rendering and without negative light sources) - but that may be a lesser problem. The only way to fix *that* with a tristimulus gamut would be to exacerbate the previous problem, going beyond spectral even more. --- IMO Color pickers and such ought to have a colorspace dropdown menu, defaulting to whatever the working space is. The ideal implementation would also let people pick what that default is, perhaps in some sort of advanced options menu in the color management menu. That way, people can decide for themselves whether ACEScg is what they want. But just getting all of that through OCIO configs would also work to the same effect. Either way, it should not be hardcoded. If you decide, that ACEScg should indeed be the new default, at least making it easily possible to switch to whatever config is desired for a given project by a given user. An advanced method might display, in an "Advanced Colormanagement" subpanel, what the currently chosen OCIO config gives you for various spaces, plus the ability to overwrite pieces of that config if you so desire. I.e. it could show this section of OCIO configs (or if the config doesn't specify, use the default Blender configs) (there are example settings and may not be sensible as such - I'm honestly not sure what some of these mean, although most are pretty self-explanatory I think) ``` roles: color_picking: sRGB 2.2 EOTF I-E color_timing: sRGB 2.2 EOTF compositing_log: sRGB 2.2 EOTF data: Generic Data default: sRGB 2.2 EOTF default_byte: sRGB 2.2 EOTF default_float: Linear BT.709 I-E default_sequencer: sRGB 2.2 EOTF matte_paint: sRGB 2.2 EOTF reference: Linear CIE-XYZ I-E scene_linear: Linear DCI-P3 I-E texture_paint: sRGB 2.2 EOTF XYZ: Linear CIE-XYZ I-D65 ``` and add an overwrite-toggle next to each, in which case the corresponding line would turn into a dropdown menu with all color spaces Blender currently knows about from various configs. This would also future-proof Blender in case the aforementioned industry standard gets changed yet again. But anyway, even if not that, at least a per-material or per Color-node dropdown menu to that effect would be great to have, and could certainly fix the shown issue for any situation. The color picker would simply convert whatever color it picks from what space it currently is into the space specified by the dropdown menu. Probably should also show some clear UI indicator if the chosen space in the dropdown menu doesn't match the working space to avoid confusion, and the default should be either the working space or, if the config specifies, the `color_picking` space. The Sequencer already has a special menu overwrite, presumably defaulting to `default_sequencer` but overwriting that if need be, and imported images (as textures or in the compositor) also have their overwritable defaults, so it makes sense to expand that to more places in Blender where specifying the way color is interpreted might make a difference. --- This might be me getting ahead of myself but it would also be nice if configs could actually be passed in as a special socket type in the various node editors. Imagine a huge node tree which, once everything is connected up as desired, you can set to work for a different specific other color space in a single click. If no noodle goes in, it's a regular dropdown menu, but otherwise the colorspace can be passed around freely, including implicit conversions and such. This would also be quite useful for the Convert Colorspace node in the Compositor, such that you don't have to think about how to interpret the color, you could just go "convert from whatever it currently is (by plugging in the relevant noodle) to this specific one I now want (which also gets stored in an output for yet another noodle)"

In #101000#1415622, @brecht wrote:
Regardless if it clips or gives negative values, that's an interop issues users now have to deal with if they bring in EXRs rendered in another application that uses ACEScg that they wouldn't have to deal with otherwise.

No the renderer should make sure it doesn't clip the negatives so the end view transform can utilize the values. If you clip it upstream in the renderer, the gamut clipping and posterization will be a done deal. But with negative values preserved, the view transform will be able to convert those values into XYZ and do stuff to pull more data back in, during the image formation chain.

In #101000#1415622, @brecht wrote:
In general, I don't agree with the apparent argument that because X or Y would be a problem regardless, we may as well accept dozens of other interop issues that can be avoided.

We need to stop pretending setting ACEScg as working space will avoid any of the stated issue. People would use Adobe RGB, ProPhoto RGB etc. in other software, setting the working space to ACEScg is not going to avoid any interop issue. People have been and will be seeing different working space values in different software, it is more or less expected.

The solution I see, is 1. make sure the renderers don't clip the negatives, so the view transform can do its job. 2. support convert colorspace node in the shader editor. 3. support easy on-the-fly change of config file they want to use

BTW P3 is actually an industrial standard working space, P3 is the standard for end cinema delivery, so it's natural to use that as working space. For instance, the Lego movie used Linear P3 I-D60 working space for the rendering, and used Baselight for downstream process like view transform etc. We are not "doing something the industry isn't", since P3 is also widely used in "the industry", if you will.

> In #101000#1415622, @brecht wrote: > Regardless if it clips or gives negative values, that's an interop issues users now have to deal with if they bring in EXRs rendered in another application that uses ACEScg that they wouldn't have to deal with otherwise. No the renderer should make sure it doesn't clip the negatives so the end view transform can utilize the values. If you clip it upstream in the renderer, the gamut clipping and posterization will be a done deal. But with negative values preserved, the view transform will be able to convert those values into XYZ and do stuff to pull more data back in, during the image formation chain. > In #101000#1415622, @brecht wrote: > In general, I don't agree with the apparent argument that because X or Y would be a problem regardless, we may as well accept dozens of other interop issues that can be avoided. We need to stop pretending setting ACEScg as working space will avoid any of the stated issue. People would use Adobe RGB, ProPhoto RGB etc. in other software, setting the working space to ACEScg is not going to avoid any interop issue. People have been and will be seeing different working space values in different software, it is more or less expected. The solution I see, is 1. make sure the renderers don't clip the negatives, so the view transform can do its job. 2. support `convert colorspace` node in the shader editor. 3. support easy on-the-fly change of config file they want to use BTW [P3](https://archive.blender.org/developer/P3.txt) is actually an industrial standard working space, [P3](https://archive.blender.org/developer/P3.txt) is the standard for end cinema delivery, so it's natural to use that as working space. For instance, the Lego movie used Linear [P3](https://archive.blender.org/developer/P3.txt) I-[D60](https://archive.blender.org/developer/D60) working space for the rendering, and used Baselight for downstream process like view transform etc. We are not "doing something the industry isn't", since [P3](https://archive.blender.org/developer/P3.txt) is also widely used in "the industry", if you will.

In #101000#1416570, @Eary wrote:
No the renderer should make sure it doesn't clip the negatives so the end view transform can utilize the values. If you clip it upstream in the renderer, the gamut clipping and posterization will be a done deal. But with negative values preserved, the view transform will be able to convert those values into XYZ and do stuff to pull more data back in, during the image formation chain.

The values will be need to be clipped, you can't correctly accumulate negative samples in a render buffer or EXR file.

We need to stop pretending setting ACEScg as working space will avoid any of the stated issue. People would use Adobe RGB, ProPhoto RGB etc. in other software, setting the working space to ACEScg is not going to avoid any interop issue.People have been and will be seeing different working space values in different software, it is more or less expected.

When it comes to 3D apps, I think the only people who will be doing this is those who already know what they are doing with color spaces. Most users will use the default configs that use either Rec.709 or ACEScg linear spaces. And so assets interchanged between apps, USD files, are not going to be AdobeRGB or something.

For instance, the Lego movie used Linear P3 I-D60 working space for the rendering, and Baselight for downstream process like view transform etc.

And then the studio switched to ACEScg once that was established as the standard.

> In #101000#1416570, @Eary wrote: > No the renderer should make sure it doesn't clip the negatives so the end view transform can utilize the values. If you clip it upstream in the renderer, the gamut clipping and posterization will be a done deal. But with negative values preserved, the view transform will be able to convert those values into XYZ and do stuff to pull more data back in, during the image formation chain. The values will be need to be clipped, you can't correctly accumulate negative samples in a render buffer or EXR file. > We need to stop pretending setting ACEScg as working space will avoid any of the stated issue. People would use Adobe RGB, ProPhoto RGB etc. in other software, setting the working space to ACEScg is not going to avoid any interop issue.People have been and will be seeing different working space values in different software, it is more or less expected. When it comes to 3D apps, I think the only people who will be doing this is those who already know what they are doing with color spaces. Most users will use the default configs that use either Rec.709 or ACEScg linear spaces. And so assets interchanged between apps, USD files, are not going to be AdobeRGB or something. > For instance, the Lego movie used Linear [P3](https://archive.blender.org/developer/P3.txt) I-[D60](https://archive.blender.org/developer/D60) working space for the rendering, and Baselight for downstream process like view transform etc. And then the studio switched to ACEScg once that was established as the standard.

In #101000#1416576, @brecht wrote:
And then the studio switched to ACEScg once that was established as the standard.

It has been very clear that ACES has been used as file interchange space. "The industry is using ACES" is more or less an illusion.

Again, P3 is industrial standard, we are not doing things very different from what "the industry" is doing.

> In #101000#1416576, @brecht wrote: > And then the studio switched to ACEScg once that was established as the standard. It has been very clear that ACES has been used as file interchange space. "The industry is using ACES" is more or less an illusion. Again, [P3](https://archive.blender.org/developer/P3.txt) is industrial standard, we are not doing things very different from what "the industry" is doing.

In #101000#1416576, @brecht wrote:
The values will be need to be clipped, you can't correctly accumulate negative samples in a render buffer or EXR file.

Then I guess it will be solved by spectral rendering, where the accumulated samples are spectral based, and it will go through the observer model into RGB after the sampling process. I wish there is a way to avoid clamping negatives upstream for RGB rendering though. One thing I am sure is that EXR files can correctly encode negative vlaues, since the EXRs we have been testing against have a lot of negatives in them.

> In #101000#1416576, @brecht wrote: > The values will be need to be clipped, you can't correctly accumulate negative samples in a render buffer or EXR file. Then I guess it will be solved by spectral rendering, where the accumulated samples are spectral based, and it will go through the observer model into RGB after the sampling process. I wish there is a way to avoid clamping negatives upstream for RGB rendering though. One thing I am sure is that EXR files can correctly encode negative vlaues, since the EXRs we have been testing against have a lot of negatives in them.

In #101000#1416584, @Eary wrote:
It has been very clear that ACES has been used as file interchange space. "The industry is using ACES" is more or less an illusion.

Well, it's not an illusion in the Animal Logic case I think, given the assets in the ALab scene. And also not an illusion for all the Maya users that have it as their default config.

Again, P3 is industrial standard, we are not doing things very different from what "the industry" is doing.

P3 is a standard display color space, not a standard working space in 3D apps.

Then I guess it will be solved by spectral rendering, where the accumulated samples are spectral based, and it will go through the observer model into RGB after the sampling process.

For pipelines that are already using ACEScg this may not be a real problem so it remains to be seen if a lot of applications will even try to solving it. I certainly would not prioritize it in Cycles development.

> In #101000#1416584, @Eary wrote: > It has been very clear that ACES has been used as file interchange space. "The industry is using ACES" is more or less an illusion. Well, it's not an illusion in the Animal Logic case I think, given the assets in the ALab scene. And also not an illusion for all the Maya users that have it as their default config. > Again, [P3](https://archive.blender.org/developer/P3.txt) is industrial standard, we are not doing things very different from what "the industry" is doing. [P3](https://archive.blender.org/developer/P3.txt) is a standard display color space, not a standard working space in 3D apps. > Then I guess it will be solved by spectral rendering, where the accumulated samples are spectral based, and it will go through the observer model into RGB after the sampling process. For pipelines that are already using ACEScg this may not be a real problem so it remains to be seen if a lot of applications will even try to solving it. I certainly would not prioritize it in Cycles development.

In #101000#1416600, @brecht wrote:
P3 is a standard display color space, not a standard working space in 3D apps.

It's also a standard practice to use P3 working space in "the industry".

For pipelines that are already using ACEScg this may not be a real problem so it remains to be seen if a lot of applications will even try to solving it. I certainly would not prioritize it in Cycles development.

Again ACEScg doesn't solve the problem since it doesn't cover the entire spectral locus like spectral rendering does. Negative value handling is essential to prevent clipping and posterization. I can already tell you the current in-development AgX can already avoid some posterization present in "the ACES workflow" by having the in-development negative value handling. But clamping it upstream will make this effort meaningless, since the data will not make it to the end view transform.

In #101000#1416600, @brecht wrote:
Well, it's not an illusion in the Animal Logic case I think, given the assets in the ALab scene. And also not an illusion for all the Maya users that have it as their default config.

Here:
https://devtalk.blender.org/t/filmic-and-aces/22408/38?u=eary

Someone who actually worked in Animal Logic said:

ACES is mainly used for exchanging files in AP0 to avoid ambiguity.

The thing about Maya etc. was those parties reaffirming the illusion.

RE-EDIT: Ok they did mention using ACEScg for rendering, I admit my selection bias here

I would deliver ACEScg exr files as the client requested. That is the delivery format. But I promise you : I would most certainly render in ACEScg and display with something else, as most of the industry actually does.

> In #101000#1416600, @brecht wrote: > [P3](https://archive.blender.org/developer/P3.txt) is a standard display color space, not a standard working space in 3D apps. It's also a standard practice to use [P3](https://archive.blender.org/developer/P3.txt) working space in "the industry". > For pipelines that are already using ACEScg this may not be a real problem so it remains to be seen if a lot of applications will even try to solving it. I certainly would not prioritize it in Cycles development. Again ACEScg doesn't solve the problem since it doesn't cover the entire spectral locus like spectral rendering does. Negative value handling is essential to prevent clipping and posterization. I can already tell you the current in-development AgX can already avoid some posterization present in "the ACES workflow" by having the in-development negative value handling. But clamping it upstream will make this effort meaningless, since the data will not make it to the end view transform. > In #101000#1416600, @brecht wrote: > Well, it's not an illusion in the Animal Logic case I think, given the assets in the ALab scene. And also not an illusion for all the Maya users that have it as their default config. Here: https://devtalk.blender.org/t/filmic-and-aces/22408/38?u=eary Someone who actually worked in Animal Logic said: > ACES is mainly used for exchanging files in AP0 to avoid ambiguity. The thing about Maya etc. was those parties reaffirming the illusion. RE-EDIT: Ok they did mention using ACEScg for rendering, I admit my selection bias here > I would deliver ACEScg exr files as the client requested. That is the delivery format. But I promise you : I would most certainly render in ACEScg and display with something else, as most of the industry actually does.

Whether P3 or ACES is to be the default (Personally I'm on team P3), the current color picker does not work for anything other than Rec.709, right? Something needs to change, and I think a config-based approach is going to be the most flexible, most future-proof method.
Even if ACES really were the de-facto industry standard right now, there is no reason to expect that it's going to keep that distinction forever.

Whether [P3](https://archive.blender.org/developer/P3.txt) or ACES is to be the default (Personally I'm on team [P3](https://archive.blender.org/developer/P3.txt)), the current color picker does not work for anything other than Rec.709, right? Something needs to change, and I think a config-based approach is going to be the most flexible, most future-proof method. Even if ACES really were the de-facto industry standard right now, there is no reason to expect that it's going to keep that distinction forever.

In #101000#1416607, @Eary wrote:
https://devtalk.blender.org/t/filmic-and-aces/22408/38?u=eary

So they did confirm ACEScg is used for rendering, not sure why you edited your comment to remove that info.

> In #101000#1416607, @Eary wrote: > https://devtalk.blender.org/t/filmic-and-aces/22408/38?u=eary So they did confirm ACEScg is used for rendering, not sure why you edited your comment to remove that info.

@kram1032, yes, there is most likely a bug here that needs to be fixed. I highly doubt we'll ship any configuration in Blender that is not Rec.709 or ACEScg, but with OCIO config files you can do whatever.

In #101000#1416612, @brecht wrote:
So they did confirm ACEScg is used for rendering, not sure why you edited your comment to remove that info.

Ok I admit that's a selection bias on my part. But I still believe ACEScg is just a extended-to-outside-of-the-locus version of Linear BT.2020 with a non-standard white point, which means BT.2020 would be the more reasonable choice.

> In #101000#1416612, @brecht wrote: > So they did confirm ACEScg is used for rendering, not sure why you edited your comment to remove that info. Ok I admit that's a selection bias on my part. But I still believe ACEScg is just a extended-to-outside-of-the-locus version of Linear BT.2020 with a non-standard white point, which means BT.2020 would be the more reasonable choice.

In #101000#1416614, @brecht wrote:
I highly doubt we'll ship any configuration in Blender that is not Rec.709 or ACEScg

I thought properly at least supporting P3 was one of the goals of https://developer.blender.org/T68926 ?
At least that's how I understand the meaning of

Support more displays like Apple P3 and native display transform from the OS. Could be done using the new built-in transforms in 2.0.

That's of course a Display setting, but if you're gonna be able to use P3 displays, I'd hope you could also specify colors with respect to such a display. It'd be weird to only be able to either select from Rec.709 so you can't even reach the limits of P3, or select from all of ACEScg so you can easily escape from P3 on accident

> In #101000#1416614, @brecht wrote: > I highly doubt we'll ship any configuration in Blender that is not Rec.709 or ACEScg I thought properly at least supporting [P3](https://archive.blender.org/developer/P3.txt) was one of the goals of https://developer.blender.org/T68926 ? At least that's how I understand the meaning of > Support more displays like Apple [P3](https://archive.blender.org/developer/P3.txt) and native display transform from the OS. Could be done using the new built-in transforms in 2.0. That's of course a Display setting, but if you're gonna be able to use [P3](https://archive.blender.org/developer/P3.txt) displays, I'd hope you could also specify colors with respect to such a display. It'd be weird to only be able to either select from Rec.709 so you can't even reach the limits of [P3](https://archive.blender.org/developer/P3.txt), or select from all of ACEScg so you can easily escape from [P3](https://archive.blender.org/developer/P3.txt) on accident

It means only what it says, there's no intent to ship with a P3 working color space.

It means only what it says, there's no intent to ship with a [P3](https://archive.blender.org/developer/P3.txt) working color space.

Added subscriber: @OcularEvolution

@brecht So then will a proper config based method be implemented? Because currently, having to manually replace config files between Filmic, Aces, and AgX, based on either client or personal work is such a pain. :(

In #101000#1415577, @brecht wrote:
from a user point of view it will just be confusing to see different scene linear values in different applications and make interop harder.

I think the perfect solution is to have colorspace dropdown in the color picker wheel UI. Similar to units setting, when unit scale is at default 1.0, Blender internally uses meter unit, even when the user set the unit setting to centimeter. Similar things can be done with color picker UI. We can either have a per-color-wheel setting or a global setting, to have the color picker expose values in a certain colorspace that can be different from the working space, like internally meter, but UI displays centimeter.

This will also solve the problem with backward and forward compatibility when you change the working space. Older .blend files pre-3.x (which ever version AgX ends up merging to) can be detected and the setting will just fall back to Linear BT.709 I-D65. Newer .blend files can have that setting memorized, so in the future if we change the working space again, that setting will remain, making the RGB inputs consistent despite the change in the working space in future versions.

Working space choice should prioritize the need of the software itself, not "use the same values as other software". For example, in preparation for spectral Cycles, the working space will need to have illuminant E white point. So BT.709 or P3 or anything, it will need to be I-E based. If we keep the existing color picker UI, I imagine you will disagree with I-E as well. But with a UI only setting similar to the unit setting, I think you can accept P3.

> In #101000#1415577, @brecht wrote: > from a user point of view it will just be confusing to see different scene linear values in different applications and make interop harder. I think the perfect solution is to have colorspace dropdown in the color picker wheel UI. Similar to units setting, when unit scale is at default 1.0, Blender internally uses meter unit, even when the user set the unit setting to centimeter. Similar things can be done with color picker UI. We can either have a per-color-wheel setting or a global setting, to have the color picker expose values in a certain colorspace that can be different from the working space, like internally meter, but UI displays centimeter. This will also solve the problem with backward and forward compatibility when you change the working space. Older .blend files pre-3.x (which ever version AgX ends up merging to) can be detected and the setting will just fall back to Linear BT.709 I-[D65](https://archive.blender.org/developer/D65). Newer .blend files can have that setting memorized, so in the future if we change the working space again, that setting will remain, making the RGB inputs consistent despite the change in the working space in future versions. Working space choice should prioritize the need of the software itself, not "use the same values as other software". For example, in preparation for spectral Cycles, the working space will need to have illuminant E white point. So BT.709 or [P3](https://archive.blender.org/developer/P3.txt) or anything, it will need to be I-E based. If we keep the existing color picker UI, I imagine you will disagree with I-E as well. But with a UI only setting similar to the unit setting, I think you can accept [P3](https://archive.blender.org/developer/P3.txt).

I think the sane option is to have ACEScg as a working space and store the colors as such, and then provide UI options to edit colors in other spaces if desired. I don't think that spectral rendering requires illuminant E white point in the working space, why would it?

UI settings can only hide so much. Once you start doing things procedurally with shading nodes, geometry nodes, drivers, then the real values start to matter. Converting .blend files between different working spaces is something we plan to implement, but it will be imperfect since we can't convert procedural aspects. Interop between applications becomes a mess as colors become negative or get clipped, USD files and textures will have no or wrong metadata requiring users to manually specify color spaces, custom scripts will forget to do proper conversions, etc. The reality is not an ideal world where every part of the pipeline perfectly handles color spaces, it's a patchwork of many different things and using as few working spaces as possible prevents issues.

I think the sane option is to have ACEScg as a working space and store the colors as such, and then provide UI options to edit colors in other spaces if desired. I don't think that spectral rendering requires illuminant E white point in the working space, why would it? UI settings can only hide so much. Once you start doing things procedurally with shading nodes, geometry nodes, drivers, then the real values start to matter. Converting .blend files between different working spaces is something we plan to implement, but it will be imperfect since we can't convert procedural aspects. Interop between applications becomes a mess as colors become negative or get clipped, USD files and textures will have no or wrong metadata requiring users to manually specify color spaces, custom scripts will forget to do proper conversions, etc. The reality is not an ideal world where every part of the pipeline perfectly handles color spaces, it's a patchwork of many different things and using as few working spaces as possible prevents issues.

In #101000#1417936, @brecht wrote:
I don't think that spectral rendering requires illuminant E white point in the working space, why would it?

It's for the spectral reconstruction/upsampling from RGB textures. In users standpoint, RGB (1, 1, 1) means white, meaning they reflect the incoming light in 100%. But when you turn that RGB texture into spectral data, it only makes sense to have the "white" be illuminant E, otherwise the color mixing behavior would be biased. The spectral branch has been shipping a custom config with I-E based working space. It is essential to have I-E based working space looking ahead.

In #101000#1417936, @brecht wrote:
UI settings can only hide so much. Once you start doing things procedurally with shading nodes, geometry nodes, drivers, then the real values start to matter.

In those case we need to have Convert Colorspace node in the shader editor. The node can be auto-added as versioning solution. Like Geometry Nodes auto-adds Realize Instances node as versioning, I think auto add Convert Colorspace node should also work.

In #101000#1417936, @brecht wrote:
Interop between applications becomes a mess as colors become negative or get clipped

P3 is reasonably large already. It shouldn't matter as long as the view transform holds up and not producing a broken image. I think AgX is doing a good job at those areas. Just tested again and hard clipping in P3 is causing significantly less problems than hard clip in BT.709. As long as the end image holds up, I think hard clipping is fine.

Negative vlaues are fine as well since the view transform will transform them to XYZ and do stuff (it's actually better to leave those negatives for the view transform to handle). Again it's fine as long as the end image holds up, and the view transform is the key here.

> In #101000#1417936, @brecht wrote: > I don't think that spectral rendering requires illuminant E white point in the working space, why would it? It's for the spectral reconstruction/upsampling from RGB textures. In users standpoint, RGB (1, 1, 1) means white, meaning they reflect the incoming light in 100%. But when you turn that RGB texture into spectral data, it only makes sense to have the "white" be illuminant E, otherwise the color mixing behavior would be biased. The spectral branch has been shipping a custom config with I-E based working space. It is essential to have I-E based working space looking ahead. > In #101000#1417936, @brecht wrote: > UI settings can only hide so much. Once you start doing things procedurally with shading nodes, geometry nodes, drivers, then the real values start to matter. In those case we need to have `Convert Colorspace` node in the shader editor. The node can be auto-added as versioning solution. Like Geometry Nodes auto-adds `Realize Instances` node as versioning, I think auto add `Convert Colorspace` node should also work. > In #101000#1417936, @brecht wrote: > Interop between applications becomes a mess as colors become negative or get clipped [P3](https://archive.blender.org/developer/P3.txt) is reasonably large already. It shouldn't matter as long as the view transform holds up and not producing a broken image. I think AgX is doing a good job at those areas. Just tested again and hard clipping in [P3](https://archive.blender.org/developer/P3.txt) is causing significantly less problems than hard clip in BT.709. As long as the end image holds up, I think hard clipping is fine. Negative vlaues are fine as well since the view transform will transform them to XYZ and do stuff (it's actually better to leave those negatives for the view transform to handle). Again it's fine as long as the end image holds up, and the view transform is the key here.

In #101000#1417942, @Eary wrote:
It's for the spectral reconstruction/upsampling from RGB textures. In users standpoint, RGB (1, 1, 1) means white, meaning they reflect the incoming light in 100%. But when you turn that RGB texture into spectral data, it only makes sense to have the "white" be illuminant E, otherwise the color mixing behavior would be biased. The spectral branch has been shipping a custom config with I-E based working space. It is essential to have I-E based working space looking ahead.

I have not looked at this closely enough, but I imagine applications/studios using ACEScg somehow don't impose this requirement.

In those case we need to have Convert Colorspace node in the shader editor. The node can be auto-added as versioning solution. Like Geometry Nodes auto-adds Realize Instances node as versioning, I think auto add Convert Colorspace node should also work.

A shader or compositing setup where every other node is a convert colorspace node is hardly usable and will have poor performance. It would also only solve part of the problem. I don't see any application implementing this kind of thing.

Negative vlaues are fine as well since the view transform will transform them to XYZ and do stuff (it's actually better to leave those negatives for the view transform to handle). Again it's fine as long as the end image holds up, and the view transform is the key here.

The negative values are not going straight to the view transform, they're going to cause problems in shading and compositing before that.

> In #101000#1417942, @Eary wrote: > It's for the spectral reconstruction/upsampling from RGB textures. In users standpoint, RGB (1, 1, 1) means white, meaning they reflect the incoming light in 100%. But when you turn that RGB texture into spectral data, it only makes sense to have the "white" be illuminant E, otherwise the color mixing behavior would be biased. The spectral branch has been shipping a custom config with I-E based working space. It is essential to have I-E based working space looking ahead. I have not looked at this closely enough, but I imagine applications/studios using ACEScg somehow don't impose this requirement. > In those case we need to have `Convert Colorspace` node in the shader editor. The node can be auto-added as versioning solution. Like Geometry Nodes auto-adds `Realize Instances` node as versioning, I think auto add `Convert Colorspace` node should also work. A shader or compositing setup where every other node is a convert colorspace node is hardly usable and will have poor performance. It would also only solve part of the problem. I don't see any application implementing this kind of thing. > Negative vlaues are fine as well since the view transform will transform them to XYZ and do stuff (it's actually better to leave those negatives for the view transform to handle). Again it's fine as long as the end image holds up, and the view transform is the key here. The negative values are not going straight to the view transform, they're going to cause problems in shading and compositing before that.

Getting back to the bug report, I'm actually unable to reproduce this so far with either color_picking space. Tested on macOS Arm.`

It looks like some precision issue where the values drift away from 1.0.

Getting back to the bug report, I'm actually unable to reproduce this so far with either `color_picking` space. Tested on macOS Arm.` It looks like some precision issue where the values drift away from 1.0.

Changed status from 'Needs Triage' to: 'Resolved'

Fixed in 2eb19ee.

There's indeed some imprecision going back and forth between color spaces:

color_picking: 0.50000334 0.00000000 1.00000000
to scene_linear: 0.18947117 0.00000000 1.00002408
back to color_picking: 0.50000703 0.00000000 1.00001645

There's nothing hardcoded to Blender's sRGB LUT, that one just maps 1 exactly to 1.

Fixed in 2eb19ee. There's indeed some imprecision going back and forth between color spaces: ``` color_picking: 0.50000334 0.00000000 1.00000000 to scene_linear: 0.18947117 0.00000000 1.00002408 back to color_picking: 0.50000703 0.00000000 1.00001645 ``` There's nothing hardcoded to Blender's sRGB LUT, that one just maps 1 exactly to 1.

In #101000#1418166, @brecht wrote:
I have not looked at this closely enough, but I imagine applications/studios using ACEScg somehow don't impose this requirement.

Not sure how other spectral renderers handle this, but this is indeed what spectral Cycles requires. Note spectral Cycles also require XYZ role to be I-E based (well, the proper XYZ standard has always been I-E based anyways), meaning the sky texture, blackbody nodes etc. need to stop assuming XYZ I-D65, and instead properly expect the XYZ I-E standard. Keep in mind and be ready, an I-E based system will be more future proof, so AgX will come with an I-E based config.

In #101000#1418268, @brecht wrote:
There's nothing hardcoded to Blender's sRGB LUT, that one just maps 1 exactly to 1.

Good to know, glad it's fixed. Just not sure why the sRGB LUT didn't suffer from the precision issue.

> In #101000#1418166, @brecht wrote: > I have not looked at this closely enough, but I imagine applications/studios using ACEScg somehow don't impose this requirement. Not sure how other spectral renderers handle this, but this is indeed what spectral Cycles requires. Note spectral Cycles also require XYZ role to be I-E based (well, the proper XYZ standard has always been I-E based anyways), meaning the sky texture, blackbody nodes etc. need to stop assuming XYZ I-[D65](https://archive.blender.org/developer/D65), and instead properly expect the XYZ I-E standard. Keep in mind and be ready, an I-E based system will be more future proof, so AgX will come with an I-E based config. > In #101000#1418268, @brecht wrote: > There's nothing hardcoded to Blender's sRGB LUT, that one just maps 1 exactly to 1. Good to know, glad it's fixed. Just not sure why the sRGB LUT didn't suffer from the precision issue.

Just tested in the latest daily build, it's not fixed. If you set the V value to 1 in advance, it keeps 1 unchanged, but if you set V to 0.8 in advance, it starts drifting to 0.801, then 0.802 again. The precision issue seems to be still there.

Just tested in the latest daily build, it's not fixed. If you set the `V` value to 1 in advance, it keeps 1 unchanged, but if you set `V` to 0.8 in advance, it starts drifting to 0.801, then 0.802 again. The precision issue seems to be still there.

Updated the description with updated video

I mean some imprecision is gonna be unavoidable. However, the level of imprecision here probably comes down to how many digits after the decimal point the transformation matrices use...
If they only use like four digits, they are only accurate to like four digits, and every time the conversion happens, you add more and more errors on top.

Maybe the behavior could somehow be improved by locking in some coordinates so they can't change at all?
Specifically, when dragging the dot in the color circle, the value would by definition be fixed, right? So as long as the dragging happens, the conversion should ignore the updated value and keep the old value.
Meanwhile, if you drag the value slider, the hue and saturation ought to be locked.
I'm not sure if this is completely feasible, but that would certainly fix this issue, right?
Not sure if that would also automatically fix the issue in RGB mode but I'd kinda think so. A value of 1 means a maximum RGB value of 1, so it should be fine, right?

It's very clear now that the colorpicker itself does indeed change color spaces: You can tell by the new video, that it struggles to show the full space. For instance, there is a visible line between yellow and red. (To be clear, this is expected behavior, not a bug!)

I mean some imprecision is gonna be unavoidable. However, the level of imprecision here probably comes down to how many digits after the decimal point the transformation matrices use... If they only use like four digits, they are only accurate to like four digits, and every time the conversion happens, you add more and more errors on top. Maybe the behavior could somehow be improved by locking in some coordinates so they can't change at all? Specifically, when dragging the dot in the color circle, the value would by definition be fixed, right? So as long as the dragging happens, the conversion should ignore the updated value and keep the old value. Meanwhile, if you drag the value slider, the hue and saturation ought to be locked. I'm not sure if this is completely feasible, but that would certainly fix this issue, right? Not sure if that would also automatically fix the issue in RGB mode but I'd kinda think so. A value of 1 means a maximum RGB value of 1, so it should be fine, right? It's very clear now that the colorpicker itself does indeed change color spaces: You can tell by the new video, that it struggles to show the full space. For instance, there is a visible line between yellow and red. (To be clear, this is expected behavior, not a bug!)

In #101000#1419594, @kram1032 wrote:
It's very clear now that the colorpicker itself does indeed change color spaces: You can tell by the new video, that it struggles to show the full space. For instance, there is a visible line between yellow and red. (To be clear, this is expected behavior, not a bug!)

The display of the color wheel is different from the values, the wheel I believe uses the first view transform in the config. Meaning it can potentially be displayed with AgX if the order of the views change, but I think we can keep display's native as the first view.

> In #101000#1419594, @kram1032 wrote: > It's very clear now that the colorpicker itself does indeed change color spaces: You can tell by the new video, that it struggles to show the full space. For instance, there is a visible line between yellow and red. (To be clear, this is expected behavior, not a bug!) The display of the color wheel is different from the values, the wheel I believe uses the first view transform in the config. Meaning it can potentially be displayed with AgX if the order of the views change, but I think we can keep display's native as the first view.

At any rate, this task needs to be reopened, the problem is still there, just with a different behavior at 1, still the same as before at 0.8. @brecht

At any rate, this task needs to be reopened, the problem is still there, just with a different behavior at `1`, still the same as before at `0.8`. @brecht

Changed status from 'Resolved' to: 'Confirmed'

Added subscriber: @Accelero

To the actual Problem: From my own testing with OCIO configs, my first guess, without looking at the code would be, that the Picker is not limited to the visible color wheel and allows picking values outside the circle. When choosing sRGB to display colors in the picker and using a much larger scene referred color space, the selection can be outside the visible wheel anywhere in the scene referred space. This makes sRGB values above 1 possible and the picker's behavior is strangely for that. The issue should therefore only occur after touching the outer edge of the color wheel (V doesnt have to be 1, just one component above 1).
Ways to prevent this are:

Clamp the RGB values to the range 0..1 in the color space with the color picker role. This is was the case with the old LUT color config, but using matrix transforms doesnt automatically clamp the values (there is the RangeTransform in OCIO for that). This prevents you however from picking colors outside the sRGB space with the wheel.
Use the scene referred color space for the color picking. This lets you choose all colors of the scene referred space, but makes the colors in the wheel render shifted, which is legitimate behavior, since the primaries are different and with a smaller display space the saturation on screen will also be clamped towards the outside edge.

The behavior with values above 1 sould be looked at, but for now one of the 2 solutions above should prevent the problem.

To the actual Problem: From my own testing with OCIO configs, my first guess, without looking at the code would be, that the Picker is not limited to the visible color wheel and allows picking values outside the circle. When choosing sRGB to display colors in the picker and using a much larger scene referred color space, the selection can be outside the visible wheel anywhere in the scene referred space. This makes sRGB values above 1 possible and the picker's behavior is strangely for that. The issue should therefore only occur after touching the outer edge of the color wheel (V doesnt have to be 1, just one component above 1). Ways to prevent this are: - Clamp the RGB values to the range 0..1 in the color space with the color picker role. This is was the case with the old LUT color config, but using matrix transforms doesnt automatically clamp the values (there is the RangeTransform in OCIO for that). This prevents you however from picking colors outside the sRGB space with the wheel. - Use the scene referred color space for the color picking. This lets you choose all colors of the scene referred space, but makes the colors in the wheel render shifted, which is legitimate behavior, since the primaries are different and with a smaller display space the saturation on screen will also be clamped towards the outside edge. The behavior with values above 1 sould be looked at, but for now one of the 2 solutions above should prevent the problem.

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Color picker colors drift value for some OCIO configurations #101000