PyAPI: Object.ray_cast returns wrong face indices with some modifier combinations #72113

New Issue

System Information
Operating system: Windows-7-6.1.7601-SP1 64 Bits
Graphics card: AMD Radeon HD 6900 Series ATI Technologies Inc. 4.5.13399 Core Profile Context 15.200.1062.1003

Blender Version
Broken: version: 2.82 (sub 3), branch: master, commit date: 2019-11-30 17:33, hash: f1ac64921b
Worked: (optional)

Short description of error
With some modifier combinations, Object.ray_cast returns face indices that don't match the indices the object would have if the modifiers were applied or when using Object.evaluated_get. I tested this also in 2.79 and it doesn't work there either.

Attached is a blend file which includes a slightly modified version of the operator_modal_view3d_raycast.py template. It now prints the indices into the console. I included two different test scenarios just in case.

Note, the only changes in the code are the debug print on line 63 and the change to execute the operator only on mouse press on line 84.

Exact steps for others to reproduce the error

Open the attached blend file
Open System Console
Run the "operator_modal_view3d_raycast.py" script

Case 1:
4. Select "Bevel + Decimate" object
5. See that the Decimate modifier's face count is 24
6. Search for "RayCast View Operator" and press Enter
7. Click on the object's faces and check the printed indices inside the Console
8. Confirm that they go above 23

Case 2:
4. Enable Developer Extras in Preferences > Interface
5. Select "Boolean + Bevel Applied" object
6. Switch into edit mode
7. Inside Viewport Overlays popover, enable indices
8. Select the top face and check the index
9. Switch into object mode
10. Search for "RayCast View Operator" and press Enter
11. Click on "Boolean + Bevel" object's top face and check the printed index inside the Console
12. Confirm that it doesn't match the index of the object which has the modifiers applied

ray_cast_debug.blend

**System Information** Operating system: Windows-7-6.1.7601-SP1 64 Bits Graphics card: AMD Radeon HD 6900 Series ATI Technologies Inc. 4.5.13399 Core Profile Context 15.200.1062.1003 **Blender Version** Broken: version: 2.82 (sub 3), branch: master, commit date: 2019-11-30 17:33, hash: `f1ac64921b` Worked: (optional) **Short description of error** With some modifier combinations, Object.ray_cast returns face indices that don't match the indices the object would have if the modifiers were applied or when using Object.evaluated_get. I tested this also in 2.79 and it doesn't work there either. Attached is a blend file which includes a slightly modified version of the operator_modal_view3d_raycast.py template. It now prints the indices into the console. I included two different test scenarios just in case. Note, the only changes in the code are the debug print on line 63 and the change to execute the operator only on mouse press on line 84. **Exact steps for others to reproduce the error** 1. Open the attached blend file 2. Open System Console 3. Run the "operator_modal_view3d_raycast.py" script Case 1: 4. Select "Bevel + Decimate" object 5. See that the Decimate modifier's face count is 24 6. Search for "RayCast View Operator" and press Enter 7. Click on the object's faces and check the printed indices inside the Console 8. Confirm that they go above 23 Case 2: 4. Enable Developer Extras in Preferences > Interface 5. Select "Boolean + Bevel Applied" object 6. Switch into edit mode 7. Inside Viewport Overlays popover, enable indices 8. Select the top face and check the index 9. Switch into object mode 10. Search for "RayCast View Operator" and press Enter 11. Click on "Boolean + Bevel" object's top face and check the printed index inside the Console 12. Confirm that it doesn't match the index of the object which has the modifiers applied [ray_cast_debug.blend](https://archive.blender.org/developer/F8300112/ray_cast_debug.blend)

Added subscriber: @Symstract

Added subscriber: @mano-wii

If I'm not mistaken, ray_cast now returns the loop_triangle index.
If I'm correct, you can get the poligon index like this:

 me.calc_loop_triangles()
 polygon_index = me.loop_triangles[0].polygon_index

I don't think this is a bug, can you confirm if this solves your problem?

If I'm not mistaken, `ray_cast` now returns the `loop_triangle` index. If I'm correct, you can get the poligon index like this: ``` me.calc_loop_triangles() polygon_index = me.loop_triangles[0].polygon_index ``` I don't think this is a bug, can you confirm if this solves your problem?

No, unfortunately that doesn't solve it. :(

This is from the API docs:

Return (result, location, normal, index):

result, Whether the ray successfully hit the geometry, boolean

location, The hit location of this ray cast, float array of 3 items in [-inf, inf]

normal, The face normal at the ray cast hit location, float array of 3 items in [-inf, inf]

index, The face index, -1 when original data isn’t available, int in [-inf, inf]

And this is from the template:

success, location, normal, face_index = obj.ray_cast(ray_origin_obj, ray_direction_obj)

No mention of loop triangles.

And it doesn't always behave wrong. A demonstration with my addon which has face highlighting:

2019-12-03 18-59-01.mp4

As we can see, only when I move the Wireframe modifier before the Solidify modifier, ray_cast starts to return wrong indices and the highlighting stops working.

No, unfortunately that doesn't solve it. :( This is from the API docs: > Return (result, location, normal, index): > > result, Whether the ray successfully hit the geometry, boolean > > location, The hit location of this ray cast, float array of 3 items in [-inf, inf] > > normal, The face normal at the ray cast hit location, float array of 3 items in [-inf, inf] > > **index, The face index**, -1 when original data isn’t available, int in [-inf, inf] And this is from the template: ``` success, location, normal, face_index = obj.ray_cast(ray_origin_obj, ray_direction_obj) ``` No mention of loop triangles. And it doesn't always behave wrong. A demonstration with my addon which has face highlighting: [2019-12-03 18-59-01.mp4](https://archive.blender.org/developer/F8187746/2019-12-03_18-59-01.mp4) As we can see, only when I move the Wireframe modifier before the Solidify modifier, ray_cast starts to return wrong indices and the highlighting stops working.

Added subscribers: @howardt, @ideasman42

Upon further investigation, I realized that this method always tries to return the original mesh index. (The index of the mesh without modifiers).
See: https://developer.blender.org/diffusion/B/browse/master/source/blender/makesrna/intern/rna_object_api.c$480

But apparently the CustomData_get_layer(&me_eval->pdata, CD_ORIGINDEX) of the Bevel modifier does not work that way.
@howardt, do you know why this happens?

We could return the actual mesh poly index or the looptri index.
This hasn't been done yet to keep the backward compatibility.

CC @ideasman42

Upon further investigation, I realized that this method always tries to return the original mesh index. (The index of the mesh without modifiers). See: https://developer.blender.org/diffusion/B/browse/master/source/blender/makesrna/intern/rna_object_api.c$480 But apparently the `CustomData_get_layer(&me_eval->pdata, CD_ORIGINDEX)` of the `Bevel modifier` does not work that way. @howardt, do you know why this happens? We could return the actual mesh poly index or the looptri index. This hasn't been done yet to keep the backward compatibility. CC @ideasman42

Added subscriber: @dr.sybren

Changed status from 'Confirmed' to: 'Needs User Info'

@Symstract Please provide a blend file that shows this issue without depending on a reimplementation of Blender code. As it is now it requires a developer to carefully read through your Python code, compare it to the current code in Blender, see if there are any differences, etc.

@dr.sybren I edited the description to include a mention of the minimal changes done to the template (debug print + execute only on mouse press). I also added comments for them in the code. I don't think redoing the code is necessary as the template is perfect for showcasing the issue?

In #72113#823243, @mano-wii wrote:
Upon further investigation, I realized that this method always tries to return the original mesh index.

I think the biggest issue here is that the function can return the index of either the original or the evaluated mesh, without telling the caller which one it is. It certainly makes the function's result hard to use.

I'm marking this as a Known Issue, as it's not a bug in Blender but working as designed. Design could (and IMO should) be better, but that doesn't make this a bug.

> In #72113#823243, @mano-wii wrote: > Upon further investigation, I realized that this method always tries to return the original mesh index. I think the biggest issue here is that the function can return the index of either the original or the evaluated mesh, without telling the caller which one it is. It certainly makes the function's result hard to use. I'm marking this as a Known Issue, as it's not a bug in Blender but working as designed. Design could (and IMO should) be better, but that doesn't make this a bug.

Changed status from 'Needs User Info' to: 'Confirmed'

Added subscriber: @elie

In #72113#862700, @dr.sybren wrote:
it's not a bug in Blender but working as designed. Design could (and IMO should) be better, but that doesn't make this a bug.

Could you ellaborate on the very design it is responding to? Because I feel totally unable to use this function since:

It may return an face index that is not a valid polygon index, even on an original mesh.

_, _, _, poly_index = obj.original.ray_cast(ray_origin, ray_direction)
print(poly_index) # 8
print(len(obj.original.data.polygons))  # 6

It returns a different face index than scene.ray_cast.

_, _, _, poly_index1, obj, _ = scene.ray_cast(depsgraph, ray_origin, ray_direction)
_, _, _, poly_index2 = obj.original.ray_cast(ray_origin, ray_direction)
_, _, _, poly_index3 = obj.evaluated_get(depsgraph).ray_cast(ray_origin, ray_direction)
print(poly_index1) # 24
print(poly_index2) # 8
print(poly_index3) # 8

Worst: the behavior changes during runtime. Unfortunately I don't know how to reproduce the bug reliably but on more complicated scenes I've noticed that it often returns correct indices at start-up, and after a few viewport operations that I could not identify for sure it suddenly changes the returned indices.

Doing further tests show that scene.ray_cast returns an index which is the correct polygon index of obj.evaluated_get(depsgraph), but the index of obj.ray_cast is valid neither as a polygon index nor as a loop_triangle index, nor as an index in the original object. So what is it?

Here is the sample scene I've used: blend. It is simply the default cube with a triangulate modifier and a subsurf modifier.

(Tested in 2.90 and 2.91)

> In #72113#862700, @dr.sybren wrote: > it's not a bug in Blender but working as designed. Design could (and IMO should) be better, but that doesn't make this a bug. Could you ellaborate on the very design it is responding to? Because I feel totally unable to use this function since: 1. It may return an face index that is not a valid polygon index, even on an original mesh. ``` _, _, _, poly_index = obj.original.ray_cast(ray_origin, ray_direction) print(poly_index) # 8 print(len(obj.original.data.polygons)) # 6 ``` 2. It returns a different face index than `scene.ray_cast`. ``` _, _, _, poly_index1, obj, _ = scene.ray_cast(depsgraph, ray_origin, ray_direction) _, _, _, poly_index2 = obj.original.ray_cast(ray_origin, ray_direction) _, _, _, poly_index3 = obj.evaluated_get(depsgraph).ray_cast(ray_origin, ray_direction) print(poly_index1) # 24 print(poly_index2) # 8 print(poly_index3) # 8 ``` 3. Worst: the behavior changes during runtime. Unfortunately I don't know how to reproduce the bug reliably but on more complicated scenes I've noticed that it often returns correct indices at start-up, and after a few viewport operations that I could not identify for sure it suddenly changes the returned indices. Doing further tests show that `scene.ray_cast` returns an index which is the correct polygon index of `obj.evaluated_get(depsgraph)`, but the index of `obj.ray_cast` is valid neither as a polygon index nor as a loop_triangle index, nor as an index in the original object. So what is it? Here is the sample scene I've used: [blend](https://pasteall.org/media/3/0/30deced87aedbfd695cc1fb3a971423c.blend). It is simply the default cube with a triangulate modifier and a subsurf modifier. (Tested in 2.90 and 2.91)

Added subscriber: @ckohl_art

Added subscriber: @Sombrero

Added subscriber: @mattli911

I wasn't able to even get this working at all, in 2.93. Although maybe I'm not fully understanding how it works. But wonder if it's more broken now after some API changes?

Added subscriber: @APEC

Added subscriber: @testure

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

PyAPI: Object.ray_cast returns wrong face indices with some modifier combinations #72113