GHOST: Process full mouse move resolution in Win32 #125949
No reviewers
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
FBX
Interest
Freestyle
Interest
Geometry Nodes
Interest
glTF
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Asset System
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
4 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#125949
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "ChengduLittleA/blender:ghost-win32-mousemove"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
In Win32,
WM_MOUSEMOVE
don't usually report all the mouse move events.So when using a non-pen device to draw stuff, like grease pencil or texture paint, you will very easily see jagged lines.
This is just how
WM_MOUSEMOVE
event behaves. It doesn't give you more granulated events unlike on X11.This patch attempts to get all the mouse events and send them in sequence so a smoother input can be achieved, and
any generic pointer device would be benefited.
Update:
There's likely a bug with
GetMouseMovePointsEx
in Win32 that prevents this from working properly. So probably can't implement this since if the event doesn't work you aren't even able to manipulate user interface.WIP: GHOST: Process full mouse move resolution in Win32to GHOST: Process full mouse move resolution in Win32I haven't actually tested this yet, or looked at it much, so the following comments could be all wrong and/or dumb...
I think you can pass your timestamp in the input MOUSEMOVEPOINT's time. That way if there are multiple positions with the same values it will use the timestamp to get the right one. "If your application supplies a time stamp, the GetMouseMovePointsEx function will use it to differentiate between two equal points that were recorded at different times."
The docs on this API function are a bit vague about issues with multiple monitors. It mentions that in the case of multiple monitors it can give bad data if you give it negative values or if returned coordinates are negative, which is says "can occur if multiple monitors are present". But then it goes on to give an example of how to correct for this yet it is only for the case of using GMMP_USE_HIGH_RESOLUTION_POINTS, not GMMP_USE_DISPLAY_POINTS. But the remarks don't explicitly say that you can get erroneous values in those situations only if using GMMP_USE_DISPLAY_POINTS. Nor does it say that you should use GMMP_USE_HIGH_RESOLUTION_POINTS for multiple monitor support.
Not trying to second-guess Microsoft, but their example code looks dodgy to me in the case of monitors that differ in DPI or user scale.
Their example is here: https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getmousemovepointsex
The above confusion about multiple monitors led me to this document about how Paint.net removed their use of GetMouseMovePointsEx specifically because it worked badly with multiple monitors and/or with differing DPI or scale.
https://blog.getpaint.net/2019/11/14/paint-net-4-2-6-alpha-build-7258/
So in total this makes me a bit nervous that MS added this function a long time ago (Windows 2000) and haven't update it or the docs to consider differing scale or dpi. Multiple monitors were a thing back then but not with different scale or dpi.
@Harley I actually have tested this with my own painting program with dual screen setup, if it's using
GMMP_USE_DISPLAY_POINTS
it will be correct in this logic regardless of screen configurations (and in their conversion code example they also just pass that along). I haven't tried to use it with DPI scale, I will be ensuring that works (or if it doesn't I'll try find a way to do it lol)Also I think Blender always use full screen resolution (not scaled by windows), so DPI scale shouldn't really be a problem...
Hi @Harley ! Actually it's not really the problem of DPI scale, but rather the placement of the second screen. (I'm on win11 atm)
I can have events working as long as screens are configured in layout 1 regardless of DPI scale (small screen is main screen). if in layout 2 then blender would only register the events below half of the screen (below origin of the main screen).
aaaand
GMMP_USE_HIGH_RESOLUTION_POINTS
seems to be broken as well, So this does seem to be a windows bug unfortunately :(Maybe one way to get pas this is to check whether we have more than 1 monitors plugged in (and also detect every time that configuration changes), sounds less optimal but would generally lead to a nicer mouse experience overall under windows. e.g. I typically use mouse to sculpt
if available maybe ping @PrototypeNM1 he likely walked this minefield before and may have insights.
Is it just that it can screw up in some circumstances for a window on a particular monitor if that monitor is set to a high dpi or scale? If that is the case we could just test for that and not run your code in that case. We are informed as our windows are moved between monitors (so we can adjust for the changes in the sizes of the title bar, etc).
Or can it also screw up on a regular monitor if there is a high-dpi one between it and the origin? That would pretty hard to deal with.
Or is it something more specific, like a high-dpi monitor that has parts below the origin point? We could test for that.
Yes, I have been having lots of Déjà vu about this particular API call. I think we've tried this before.
According to my tests, this always works:
This won't work:
I think if I print and look at the coordinates I could find out how it works (that is, if calls like "client to screen" always worked)...
Oh, so if that thing can't handle negative values then we could probably just skip your code if the window rect (relative to the combined desktop) contains negative values?
I'm not sure what the exact value is. I have to check tomorrow. The problems is that none of the coordinates are supposed to be negative in the first place, the framebuffer origin should be as low as you go so I'm not entirely sure about the mechanism at work here.
GetMouseMovePointsEx
returns corrupt data for quadrants 2 through 4 of the screen when the input is in absolute coordinates. This most commonly occurs with tablets, remote desktop, and Win32 calls to set the mouse position. This is likely due to Q2-4 having a leading 1 when reporting position, which combines with the aforementioned "negative" values issue due to bit casting in Windows mouse handling code (outside of our control).I believe I had previously chased down that we can't tell if the source of mouse movement is absolute or relative without using the RawInput API, which itself would provide values that are not usable with
GetMouseMovePointsEx
due to not factoring in mouse acceleration. To the best of my knowledge, the unfortunate situation on Windows is that there is no way to handle all mouse input without this breakage. Also unfortunate is that we have learned this lesson several times, last time I put a warning against usingGetMouseMovePointsEx
but that seems to have been removed at some point so I anticipate we'll continue relearning this lesson into the future. :(thanks @PrototypeNM1 ! I wonder if reporting this to microsoft would do any good... I guess since pen input works, this could be like... not a high priority but it kinda sucks this way compared to X11 🤔
Is it an option to use RAW mouse pointer input? I know that means we don't get pointer acceleration, but perhaps we could do that ourselves? https://stackoverflow.com/questions/36862013/raw-input-and-cursor-acceleration
@Harley doesn't this mean that the user might experience different acceleration than what would be through the OS?
Trying to match an undisclosed algorithm for mouse pointer ballistics pretty much guarantees we're gonna screw it up and not match the OS which will lead to "the mouse in blender in weird compared to other apps" tickets, which will likely be much more numerous than the number of the mouse resolution is low tickets we currently get.
Given there is no perfect solution, maybe like tablet input, we could offer a few backends and give the user the choice between low res with ballistics using the current method and a DirectInput or Raw HID input that'll have better resolution but have other drawbacks.
If we use raw mouse input then we'll also need to handle drawing a virtual cursor so that we don't have visual de-syncs from raw mouse and cursor having different accelerations. This is doable, but I'm not sure we want to do it given the ballistics inconsistency @LazyDodo mentioned.
It might be worth studying the
dwExtraInfo
field fromMOUSEMOVEPOINT
to determine if there's a pattern we can use to differentiate relative from absolute input, and disregard the latter if present. We would need to inspect that for mouse, tablet,SendInput
,SetCursorPos
, remote desktop. If there's a reliable identifier for absolute position input I believe we could useGetMouseMovePointsEx
without issue. Note, afaik there is not public documentation (from Microsoft or elsewhere) on thedwExtraInfo
field.No, that sounds like the showstopper for that idea. Thanks!
Checkout
From your project repository, check out a new branch and test the changes.