UI: Linux & Mac Support for Eyedropper Outside Blender #111303

New Issue

Harley Acheson · 2023-08-19T20:44:04+02:00

Harley Acheson commented

2023-08-19 20:44:04 +02:00

As of the merge of #105324 Windows users can now use the Color Picker Eyedropper to select colors from outside of the Blender window, e.g. from other applications or the desktop:

This is currently only for the Windows platform, but it should be quite trivial to add this functionality to other platforms. If you examine #105324 you will see many changed files, but almost all of them can be ignored as they were just adding general support for this feature. You can add this feature to other platforms by changing two files, mostly by adding a single small function.

For Mac, for example, you can open up intern/ghost/intern/GHOST_SystemCocoa.mm. There is a getCapabilities() function that is setting flags for what a platform is capable of. You can just remove GHOST_kCapabilityDesktopSample from that list of things not supported. Then add a new function to the file, getPixelAtCursor that matches the others. Just ask the operating system where the mouse cursor is now, find the color at that point, and then return that as three floats (0-1) in sRGB color space. Don't forget to add a function declaration to GHOST_SystemCocoa.hh.

Looking at https://stackoverflow.com/questions/11746471/detect-color-under-macos-mouse-cursor and https://stackoverflow.com/questions/4395420/get-the-color-a-pixel-on-the-screen-in-objective-c-cocoa-app this appears to need just a handful of lines.

As of the merge of #105324 Windows users can now use the Color Picker Eyedropper to select colors from outside of the Blender window, e.g. from other applications or the desktop: ![Eyedropper.gif](/attachments/6cc25041-fc12-40f0-b2f6-e51d8ca251c7) This is currently only for the Windows platform, but it should be quite trivial to add this functionality to other platforms. If you examine #105324 you will see many changed files, but almost all of them can be ignored as they were just adding general support for this feature. You can add this feature to other platforms by changing two files, mostly by adding a single small function. For Mac, for example, you can open up `intern/ghost/intern/GHOST_SystemCocoa.mm`. There is a `getCapabilities()` function that is setting flags for what a platform is capable of. You can just remove GHOST_kCapabilityDesktopSample from that list of things not supported. Then add a new function to the file, `getPixelAtCursor` that matches the others. Just ask the operating system where the mouse cursor is now, find the color at that point, and then return that as three floats (0-1) in sRGB color space. Don't forget to add a function declaration to `GHOST_SystemCocoa.hh`. Looking at https://stackoverflow.com/questions/11746471/detect-color-under-macos-mouse-cursor and https://stackoverflow.com/questions/4395420/get-the-color-a-pixel-on-the-screen-in-objective-c-cocoa-app this appears to need just a handful of lines.

Eyedropper.gif

107 KiB

Harley Acheson added the

Type

To Do

label 2023-08-19 20:44:04 +02:00

Harley Acheson added this to the User Interface project 2023-08-19 20:44:06 +02:00

Harley Acheson added the

Meta

Good First Issue

label 2023-08-19 20:44:28 +02:00

YimingWu commented

2023-08-20 04:48:06 +02:00

Oh here's already a todo 😅

Might want to also solve #111295 since it broke the color picker in preference window (or I guess all the temporary editors?).

Oh here's already a todo 😅 Might want to also solve #111295 since it broke the color picker in preference window (or I guess all the temporary editors?).

Caleb-Cleavinger commented

2023-08-21 19:00:06 +02:00

I'd love to take this issue! I'm currently pretty busy with school and work but should finish this (for mac at least) by end of week!

👍 1

Harley Acheson commented

2023-08-22 04:13:12 +02:00

@Caleb-Cleavinger - I'd love to take this issue! I'm currently pretty busy with school and work but should finish this (for mac at least) by end of week!

That would be awesome! Don't hesitate to ask for help if you need it.

> @Caleb-Cleavinger - I'd love to take this issue! I'm currently pretty busy with school and work but should finish this (for mac at least) by end of week! That would be awesome! Don't hesitate to ask for help if you need it.

Caleb-Cleavinger commented

2023-08-22 04:20:00 +02:00

I'm pretty sure I have a working function I just have no idea if it will run bug-free on MacOS. I don't own any apple products. Is there a VM or tooling for windows to test if it is working?

Harley Acheson commented

2023-08-22 04:52:28 +02:00

I'm pretty sure I have a working function I just have no idea if it will run bug-free on MacOS.

Actually we can test for that. If you following the following instructions to create a pull request... https://wiki.blender.org/wiki/Tools/Pull_Requests

Once it is on the system we can have the buildbots test it. They will report errors or warnings.

I don't own any apple products. Is there a VM or tooling for windows to test if it is working?

If you get a pull request that looks like it should work and passes on the buildbots you are most of the way there. There are quite a few of us that can take a look and give some guesses and feedback. And the buildbots can even make installable builds that Mac users can download and test.

> I'm pretty sure I have a working function I just have no idea if it will run bug-free on MacOS. Actually we can test for that. If you following the following instructions to create a pull request... https://wiki.blender.org/wiki/Tools/Pull_Requests Once it is on the system we can have the buildbots test it. They will report errors or warnings. > I don't own any apple products. Is there a VM or tooling for windows to test if it is working? If you get a pull request that looks like it should work and passes on the buildbots you are most of the way there. There are quite a few of us that can take a look and give some guesses and feedback. And the buildbots can even make installable builds that Mac users can download and test.

Caleb-Cleavinger referenced this issue

2023-08-22 17:17:43 +02:00

Draft: Added getPixelUnderCursor for MacOS_X #111397

Caleb-Cleavinger commented

2023-08-22 17:19:45 +02:00

Created pull request for MacOS here: #111397. I plan to follow up with Wayland getPixelUnderCursor by end of week.

Vitor Boschi commented

2023-08-24 16:58:23 +02:00

I'll give it a shot on the X11 backend

👍 1

Vitor Boschi commented

2023-08-24 19:07:21 +02:00

So it looks like things are a bit more complicated on X11 (and likely on other systems as well): the problem is that we don't receive any click event from outside of the Blender window, so getPixelAtCursor() is never called. I'll keep trying to find a solution, but it's going to take longer than I expected to get this ready.

@ideasman42 Any suggestion on how to detect a click outside of the Blender window on X11? It doesn't look like this is currently supported by Ghost.

So it looks like things are a bit more complicated on X11 (and likely on other systems as well): the problem is that we don't receive any click event from outside of the Blender window, so `getPixelAtCursor()` is never called. I'll keep trying to find a solution, but it's going to take longer than I expected to get this ready. @ideasman42 Any suggestion on how to detect a click outside of the Blender window on X11? It doesn't look like this is currently supported by Ghost.

Caleb-Cleavinger commented

2023-08-24 19:15:24 +02:00

So it looks like things are a bit more complicated on X11 (and likely on other systems as well): the problem is that we don't receive any click

Yeah that appears to be happening on Mac too. Guessing same with Wayland.
This seems like a bigger issue that could impact other future features. Maybe create new issue to address this?

> So it looks like things are a bit more complicated on X11 (and likely on other systems as well): the problem is that we don't receive any click Yeah that appears to be happening on Mac too. Guessing same with Wayland. This seems like a bigger issue that could impact other future features. Maybe create new issue to address this?

Alaska commented

2023-08-24 23:33:21 +02:00

We don't receive any click event from outside of the Blender window, so getPixelAtCursor() is never called. I'll keep trying to find a solution.

Just for testing, I've found that on macOS (and it may be the same for Linux with X11 and Wayland), Blender will detect keyboard inputs outside Blender. So you can press Enter or Return on your keyboard to sample the pixel under the cursor while you're waiting for a fix to the clicking issue.

> We don't receive any click event from outside of the Blender window, so `getPixelAtCursor()` is never called. I'll keep trying to find a solution. Just for testing, I've found that on macOS (and it may be the same for Linux with X11 and Wayland), Blender will detect keyboard inputs outside Blender. So you can press `Enter` or `Return` on your keyboard to sample the pixel under the cursor while you're waiting for a fix to the clicking issue.

Alaska referenced this issue

2023-08-25 00:16:41 +02:00

Draft: Added getPixelUnderCursor for MacOS_X #111397

Vitor Boschi referenced this issue

2023-08-25 00:19:31 +02:00

UI: Allow eyedropper outside of Blender (X11) #111493

Vitor Boschi commented

2023-08-25 00:21:53 +02:00

Just for testing, I've found that on macOS (and it may be the same for Linux with X11 and Wayland), Blender will detect keyboard inputs outside Blender. So you can press Enter or Return on your keyboard

Good idea. That allowed me to test and ensure the PR is working. I guess handling external mouse events can be implemented in another PR, separate from this.

> Just for testing, I've found that on macOS (and it may be the same for Linux with X11 and Wayland), Blender will detect keyboard inputs outside Blender. So you can press Enter or Return on your keyboard Good idea. That allowed me to test and ensure the PR is working. I guess handling external mouse events can be implemented in another PR, separate from this.

Caleb-Cleavinger commented

2023-08-25 04:07:44 +02:00

I've stepped down from the MacOS getPixelUnderCursor. I still plan on working on the Wayland version b/c it's in a language I've already messed with. Wish the best of luck on whoever chooses to tackle the Mac OS version of getPixelUnderCursor!

I've stepped down from the MacOS `getPixelUnderCursor`. I still plan on working on the Wayland version b/c it's in a language I've already messed with. Wish the best of luck on whoever chooses to tackle the Mac OS version of `getPixelUnderCursor`!

Alaska commented

2023-08-25 04:27:54 +02:00

Just for reference, @Caleb-Cleavinger is stepping down from the macOS version of getPixelUnderCursor. Their progress so far can be found here: 111397

Just for reference, @Caleb-Cleavinger is stepping down from the macOS version of `getPixelUnderCursor`. Their progress so far can be found here: 111397

Caleb-Cleavinger commented

2023-08-25 19:58:11 +02:00

I've been looking into what it'll take for a Wayland getPixelUnderCursor. Right now the 2 best options I've found is either running grim or a similar shell command or getting the framebuffer from dev/fb0 which would require root permissions. Either one of the solutions I'm mentioning is incredibly slow. Basically what I'm asking is if Blender has root access or how I would add grim, run, and retrieve the pixel information.

I've been looking into what it'll take for a Wayland `getPixelUnderCursor`. Right now the 2 best options I've found is either running grim or a similar shell command or getting the framebuffer from dev/fb0 which would require root permissions. Either one of the solutions I'm mentioning is incredibly slow. Basically what I'm asking is if Blender has root access or how I would add grim, run, and retrieve the pixel information.

RokenPrice commented

2023-08-31 09:16:45 +02:00

I'm not a developer, but on X11 it may be worth looking at the source for the xfce4-screenshooter (or similar) application, which certainly registers mouse events outside of the application window (to allow, for example, grabbing a region of the screen), which may help here.

Pablo Guarneros commented

2023-10-09 05:01:48 +02:00

Hi @Harley, would it be all right to implement the getPixelUnderCursor() for MacOS? @Caleb-Cleavinger and @Alaska, I'm reading over your notes on #111397 and will follow up on there. Thanks for doing all the work. I'm 😬 excited!

YimingWu commented

2023-10-09 05:15:40 +02:00

I'm not sure about the situation with #112444 and #111295 since apparently on linux I can't pick up colors in pop up windows at all. I'm calling eye dropper in the main window and trying to pick something in the user preference window, is it expected?

Pablo Guarneros commented

2023-10-17 00:33:46 +02:00

For picking a color, there is an Objective-C object that might be helpful to use called NSColorSampler.

I've been trying to use it this way:

GHOST_TSuccess GHOST_SystemCocoa::getPixelAtCursor(float r_color[3]) const
@autoreleasepool {
NSColorSampler *sampler = [NSColorSampler new];
[sampler showSamplerWithSelectionHandler:^(NSColor * _Nullable color) {
      selectedColor = color;
      if (selectedColor)
      {
        CGFloat red, green, blue, alpha;
        [selectedColor getRed:&red green:&green blue:&blue alpha:&alpha];
        
        NSLog(@"Red: %.2f, Green: %.2f, Blue: %.2f, Alpha: %.2f", red, green, blue, alpha);
      }}]
  }
}

but haven't figured out how to update the r_color array properly.

For picking a color, there is an Objective-C object that might be helpful to use called [NSColorSampler](https://developer.apple.com/documentation/appkit/nscolorsampler?language=objc). I've been trying to use it this way: ``` GHOST_TSuccess GHOST_SystemCocoa::getPixelAtCursor(float r_color[3]) const @autoreleasepool { NSColorSampler *sampler = [NSColorSampler new]; [sampler showSamplerWithSelectionHandler:^(NSColor * _Nullable color) { selectedColor = color; if (selectedColor) { CGFloat red, green, blue, alpha; [selectedColor getRed:&red green:&green blue:&blue alpha:&alpha]; NSLog(@"Red: %.2f, Green: %.2f, Blue: %.2f, Alpha: %.2f", red, green, blue, alpha); }}] } } ``` but haven't figured out how to update the r_color array properly.

Harley Acheson commented

2023-10-23 18:24:52 +02:00

@Pablo-Guarneros - there is an Objective-C object that might be helpful to use called [NSColorSampler]

Keep in mind that I don't compile for Mac, so my understanding here could be wildly incorrect. But looking at that object it is primarily designed to work asynchronously, as in you call it and it returns immediately, only returning your color at some future time. That sounds like a difficult thing to use here.

Except that their docs do mention calling this synchronously. They have a note about on the following page: https://developer.apple.com/documentation/appkit/nscolorsampler/3237187-show and also here: https://developer.apple.com/documentation/swift/calling-objective-c-apis-asynchronously

So I honestly can't help here. It looks like you want to call the version of that show with a closure that updates our color, but say much more.

> @Pablo-Guarneros - there is an Objective-C object that might be helpful to use called [NSColorSampler] Keep in mind that I don't compile for Mac, so my understanding here could be wildly incorrect. But looking at that object it is primarily designed to work asynchronously, as in you call it and it returns immediately, only returning your color at some future time. That sounds like a difficult thing to use here. Except that their docs do mention calling this synchronously. They have a note about on the following page: https://developer.apple.com/documentation/appkit/nscolorsampler/3237187-show and also here: https://developer.apple.com/documentation/swift/calling-objective-c-apis-asynchronously So I honestly can't help here. It looks like you want to call the version of that `show` with a closure that updates our color, but say much more.

Rajesh Malviya referenced this issue

2023-11-20 15:10:54 +01:00

Fix eyedropper outside blender on mac #115187

Rajesh Malviya commented

2023-11-20 15:21:11 +01:00

Hello, I have created #115187 which adds support for macOS, though it has similar limitations as X11 impl.
NSColorSampler can't be used unless we change/add new GHOST api, so for now it is implemented via ScreenCapture.

Hello, I have created #115187 which adds support for macOS, though it has similar limitations as X11 impl. `NSColorSampler` can't be used unless we change/add new GHOST api, so for now it is implemented via ScreenCapture.

Harley Acheson commented

2023-11-20 18:09:58 +01:00

@rajveermalviya - Hello, I have created #115187 which adds support for macOS, even though it has similar limitations as X11 impl.

Awesome! Will take a look very soon.

NSColorSampler can't be used unless we change/add new GHOST api, so for now it is implemented via ScreenCapture.

I can make you new GHOST api functions if that helps; just ask. When I looked at the docs for NSColorSampler it looked like it only worked asynchronously, and I couldn't see how we could make that work. But I don't compile for Mac so am only guessing.

> @rajveermalviya - Hello, I have created #115187 which adds support for macOS, even though it has similar limitations as X11 impl. Awesome! Will take a look very soon. > `NSColorSampler` can't be used unless we change/add new GHOST api, so for now it is implemented via ScreenCapture. I can make you new GHOST api functions if that helps; just ask. When I looked at the docs for `NSColorSampler` it looked like it only worked asynchronously, and I couldn't see how we could make that work. But I don't compile for Mac so am only guessing.

Rajesh Malviya commented

2023-11-20 19:41:55 +01:00

I can make you new GHOST api functions if that helps; just ask. When I looked at the docs for NSColorSampler it looked like it only worked asynchronously, and I couldn't see how we could make that work. But I don't compile for Mac so am only guessing.

If we do need to go via the NSColorSampler route, then GHOST would need to expose that callback based API, and somehow make that work in eyedropper_color_sample_fl. Also AFAIK NSColorSampler picker will override the current custom behaviour of eyedropper for when it's inside the Blender window - may/may not be desirable.

In any case here's how NSColorSampler's UI would look like: (see attached video)

> I can make you new GHOST api functions if that helps; just ask. When I looked at the docs for NSColorSampler it looked like it only worked asynchronously, and I couldn't see how we could make that work. But I don't compile for Mac so am only guessing. If we do need to go via the `NSColorSampler` route, then GHOST would need to expose that callback based API, and somehow make that work in `eyedropper_color_sample_fl`. Also AFAIK `NSColorSampler` picker will override the current custom behaviour of eyedropper for when it's inside the Blender window - may/may not be desirable. In any case here's how `NSColorSampler`'s UI would look like: (see attached video)

output_video.mp4

880 KiB

Rajesh Malviya referenced this issue

2023-11-27 21:11:09 +01:00

Fix eyedropper outside blender on mac #115187

Sign in to join this conversation.