X3D exports wrong normals #104437

New Issue

Alvaro-Segura · 2023-02-26T00:35:38+01:00

Alvaro-Segura commented

2023-02-26 00:35:38 +01:00

System Information
Operating system: Windows 10

Blender Version
Broken: 3.4.1
Worked: ?

Short description of error

Normals exported in X3D are not the same as those in Blender: They are fully smoothed at all vertices, ignoring any auto smooth that preserves sharp edges. This happens with either the Triangulate option enabled or not. Also, the normalIndex field is missing.
More details

The IndexedFaceSet node used (unless Triangulate option set) has separate indices arrays for vertices, texture coords and normals. The first two seem correct. But the indices for vertex normals (the normalIndex field) are missing.

In the IndexedTriangleSet node used when Triangulate is enabled, there is a single indices array (same for all three properties). Here vertices must be duplicated if they have different normal or UV in different faces. This appears correctly done for UV discontinuities, but not for normals.

Additionally, when Triangulate is set, the Normals checkbox is ignored and normals are always exported.
For reference, the OBJ exporter exports the correct normals. And OBJ is semantically very similar to an X3D IndexedFaceSet.

Exact steps for others to reproduce the error

This can be seen with just the default cube.

Export it as X3D (with or without Triangulate enabled).

If you look at the resulting file, you will not see any axis-aligned normals (like (1,0,0), (0,0,1), (0,-1,0)...) as they should. They are all oblique, as per smoothing between faces. If you export as OBJ you will see such expected normals.

**System Information** Operating system: Windows 10 **Blender Version** Broken: 3.4.1 Worked: ? **Short description of error** 1) Normals exported in X3D are not the same as those in Blender: They are fully smoothed at all vertices, ignoring any auto smooth that preserves sharp edges. This happens with either the *Triangulate* option enabled or not. Also, the *normalIndex* field is missing. 2) **More details** The `IndexedFaceSet` node used (unless *Triangulate* option set) has separate indices arrays for vertices, texture coords and normals. The first two seem correct. But the indices for vertex normals (the `normalIndex` field) are missing. In the `IndexedTriangleSet` node used when *Triangulate* is enabled, there is a single indices array (same for all three properties). Here vertices must be duplicated if they have different normal or UV in different faces. This appears correctly done for UV discontinuities, but not for normals. 3) Additionally, when *Triangulate* is set, the *Normals* checkbox is ignored and normals are *always* exported. 4) For reference, the OBJ exporter exports the correct normals. And OBJ is semantically very similar to an X3D IndexedFaceSet. **Exact steps for others to reproduce the error** This can be seen with just the default cube. - Export it as X3D (with or without Triangulate enabled). If you look at the resulting file, you will not see any axis-aligned normals (like (1,0,0), (0,0,1), (0,-1,0)...) as they should. They are all oblique, as per smoothing between faces. If you export as OBJ you will see such expected normals.

Alvaro-Segura added the

labels 2023-02-26 00:35:39 +01:00

Germano Cavalcante commented

2023-02-27 20:02:38 +01:00

Although the X3D specs show per-polygon support for Normal vectors, apparently BLender only implemented per-vertex support. So it is not surprising that in this case it does not show any value with axis alignment.

https://doc.x3dom.org/author/Rendering/Normal.html

Although the X3D specs show per-polygon support for Normal vectors, apparently BLender only implemented per-vertex support. So it is not surprising that in this case it does not show any value with axis alignment. https://doc.x3dom.org/author/Rendering/Normal.html

Alvaro-Segura commented

2023-02-27 22:18:31 +01:00

No, I don't think that is the problem. Per-vertex normals are the way to go, but they have to be correct (shared between faces in smooth edges and separate at sharp edges). The cube might not be a good example. I chose it because it is easy to spot, but the problem is with any shape. Use a cylinder, for example, which should have some sharp edges (circles) and some smooth edges (cylindrical sides). But the X3D export has all smoothed.

Blender is smart enough to auto smooth and leave sharp edges and smooth edges where needed, depending on the orientation of neighboring faces. And OBJ exports just fine, while X3D does not.

I've seen that these exporters are Python scripts and don't seem too complicated. Maybe I can do something...

No, I don't think that is the problem. Per-vertex normals are the way to go, but they have to be correct (shared between faces in smooth edges and separate at sharp edges). The cube might not be a good example. I chose it because it is easy to spot, but the problem is with any shape. Use a cylinder, for example, which should have some sharp edges (circles) and some smooth edges (cylindrical sides). But the X3D export has *all* smoothed. Blender is smart enough to auto smooth and leave sharp edges and smooth edges where needed, depending on the orientation of neighboring faces. And OBJ exports just fine, while X3D does not. I've seen that these exporters are Python scripts and don't seem too complicated. Maybe I can do something...

Germano Cavalcante commented

2023-02-27 22:21:46 +01:00

OBJ exports face normals
X3D exports vertex normals

In the Cube example, each of the 8 vertices has three connected polygons, which axis-aligned normal corresponds to the vertex?

OBJ exports face normals X3D exports vertex normals In the Cube example, each of the 8 vertices has three connected polygons, which axis-aligned normal corresponds to the vertex?

Scurest commented

2023-02-27 23:06:53 +01:00

@Alvaro-Segura

What mano-wii means when he says "per-vertex normals" is one normal is stored for every Blender vertex. These normals are (AFAIK) always the average of the normals of the surrounding faces. What he means by "per-polygon normals" is one normal is stored for each corner of a polygon. This is what you're talking about, where the three face corners that meet at one vertex of a cube each have their own normal. This is what is used for shading.

(X3D actually is able store one normal for each polygon, which is probably why this is confusing.)

Looking at X3D, it looks like it doesn't use the vertex/corner distinction like Blender/OBJ, but uses "one vertex = one tuple of attributes" like in OpenGL/glTF.

The X3D exporter creates one X3D vert for each Blender corner, deduping identical X3D verts with a hash table. So to support corner-domain normals, it looks like they need to be added to the same vertex_key function that handles the other corner-domain data, vertex colors and UVs.

		io_scene_x3d/export_x3d.py
		Lines 697 to 723 in deb7574efb
	
				                        if is_uv and is_col:

				                            slot_uv = 0

				                            slot_col = 1

				                            def vertex_key(lidx):

				                                return (

				                                    _tuple_from_rounded_iter(mesh_loops_uv[lidx].uv),

				                                    _tuple_from_rounded_iter(mesh_loops_col[lidx].color),

				                                )

				                        elif is_uv:

				                            slot_uv = 0

				                            def vertex_key(lidx):

				                                return (

				                                    _tuple_from_rounded_iter(mesh_loops_uv[lidx].uv),

				                                )

				                        elif is_col:

				                            slot_col = 0

				                            def vertex_key(lidx):

				                                return (

				                                    _tuple_from_rounded_iter(mesh_loops_col[lidx].color),

				                                )

				                        else:

				                            # ack, not especially efficient in this case

				                            def vertex_key(lidx):

				                                return None

@Alvaro-Segura What mano-wii means when he says "per-vertex normals" is one normal is stored for every Blender vertex. These normals are (AFAIK) always the average of the normals of the surrounding faces. What he means by "per-polygon normals" is one normal is stored for each corner of a polygon. This is what you're talking about, where the three face corners that meet at one vertex of a cube each have their own normal. This is what is used for shading. (X3D actually is able store one normal for each polygon, which is probably why this is confusing.) Looking at X3D, it looks like it doesn't use the vertex/corner distinction like Blender/OBJ, but uses "one vertex = one tuple of attributes" like in OpenGL/glTF. The X3D exporter creates one X3D vert for each Blender corner, deduping identical X3D verts with a hash table. So to support corner-domain normals, it looks like they need to be added to the same `vertex_key` function that handles the other corner-domain data, vertex colors and UVs. https://projects.blender.org/blender/blender-addons/src/commit/deb7574efbb8b8982d2b52c1411103416a6855e3/io_scene_x3d/export_x3d.py#L697-L723

Alvaro-Segura commented

2023-02-28 00:34:56 +01:00

OBJ exports face normals
X3D exports vertex normals

In the Cube example, each of the 8 vertices has three connected polygons, which axis-aligned normal corresponds to the vertex?

OBJ exports vertex normals (and vertex texture coordinates). But vertex normals
per-face. Not one normal (or one UV) for one corner, it's a separate normal for each vertex in each face. First there are lists of vectors and then indices on these lists for each face corner. If you look at the exported OBJ cube, you can see:

8 vertices ("v") // the 8 corners
6 vertex normals ("vn") // the 6 directions
14 texture coordinates ("vt") // this one is trickier

AND then come indices of these 3 attributes for each face (triangle). For example, the first triangle looks like this:

f 5/5/1 3/3/1 1/1/1 # that is => 3 x <point-index>/<texcoord-index>/<normal-index>

The last index for each vertex refers to a normal from the list of normals. Here all thee vertices have the same normal, with index 1. That is because the cube's faces are flat, so all three vertex normals are equal (the cube was a bad example). But this is a particular case. In, e.g. a cylinder the faces on the curved sides have different normal indices. That is OK.

X3D's IndexedFaceSet is very similar to the above, only written differently. It has the three lists of vectors (for coordinates, normals and texcoords), and then the three separate lists of indices pointing to each list, per face vertex. So, if OBJ is correct, there should be a way to make X3D correct, too.

OTOH, if we enable "Triangulate", then the exporter uses X3D's IndexedTriangleSet which can be more compact. This node does not have separate indices for the three attributes, it has one single index list (same index for coordinate, normal and texcoord). And I admit this export mode can be trickier.

Then, how can you have a sharp edge in a common vertex? The trick is to duplicate vertices in the list when they have different normal or different texcoord in different faces. E.g. the coordinate (1,1,1) will appear three times in the list. Well, this is already done for texcoords in triangulated mode: the triangulated exported X3D cube has 14 points in its coordinates list. Some points are duplicated because there are discontinuities in texture coordinates

It's a bit confusing. And again, I should have used a different example.

I'll take a look at the code, but will take some time.

> OBJ exports face normals > X3D exports vertex normals > > In the Cube example, each of the 8 vertices has three connected polygons, which axis-aligned normal corresponds to the vertex? OBJ exports vertex normals (and vertex texture coordinates). But *vertex normals per-face*. Not one normal (or one UV) for one corner, it's a separate normal for each vertex *in each face*. First there are lists of vectors and then indices on these lists for each face corner. If you look at the exported OBJ cube, you can see: 8 vertices ("v") // the 8 corners 6 vertex normals ("vn") // the 6 directions 14 texture coordinates ("vt") // this one is trickier *AND then* come indices of these 3 attributes for each face (triangle). For example, the first triangle looks like this: `f 5/5/1 3/3/1 1/1/1 # that is => 3 x <point-index>/<texcoord-index>/<normal-index>` The last index for each vertex refers to a normal from the list of normals. Here all thee vertices have the same normal, with index **1**. That is because the cube's faces are flat, so all three vertex normals are equal (the cube was a bad example). But this is a particular case. In, e.g. a cylinder the faces on the curved sides have different normal indices. That is OK. X3D's `IndexedFaceSet` is very similar to the above, only written differently. It has the three lists of vectors (for coordinates, normals and texcoords), and then the three separate lists of indices pointing to each list, per face vertex. So, if OBJ is correct, there should be a way to make X3D correct, too. OTOH, if we enable "Triangulate", then the exporter uses X3D's `IndexedTriangleSet` which can be more compact. This node does not have separate indices for the three attributes, it has one single `index` list (same index for coordinate, normal and texcoord). And I admit this export mode can be trickier. Then, how can you have a sharp edge in a common vertex? The trick is to duplicate vertices in the list when they have different normal or different texcoord in different faces. E.g. the coordinate (1,1,1) will appear three times in the list. Well, this is already done for texcoords in triangulated mode: the triangulated exported X3D cube has **14 points** in its coordinates list. Some points are duplicated because there are discontinuities in texture coordinates It's a bit confusing. And again, I should have used a different example. I'll take a look at the code, but will take some time.

Alvaro-Segura commented

2023-02-28 00:49:46 +01:00

@scurest

What mano-wii means when he says "per-vertex normals" is one normal is stored for every Blender vertex. These normals are (AFAIK) always the average of the normals of the surrounding faces. What he means by "per-polygon normals" is one normal is stored for each corner of a polygon. This is what you're talking about, where the three face corners that meet at one vertex of a cube each have their own normal. This is what is used for shading.

Sorry for the misunderstanding, I understood per-polygon normals as one single vector per face (yielding only flat faces). BTW, averaging is done only among touching faces with their normals forming an angle less than the crease angle limit (if auto-smooth is enabled).

(X3D actually is able store one normal for each polygon, which is probably why this is confusing.)

Yes, but I think I've rarely seen that.

Looking at X3D, it looks like it doesn't use the vertex/corner distinction like Blender/OBJ, but uses "one vertex = one tuple of attributes" like in OpenGL/glTF.

That is the case in the IndexedTriangleSet node (used when triangulate is enabled). Yes, that's a better way of saying it: "one vertex = one tuple of attributes". OTOH, the IndexedFaceSet which supports polygons of any size, does have separate indices per attribute, as in OBJ.

The X3D exporter creates one X3D vert for each Blender corner, deduping identical X3D verts with a hash table. So to support corner-domain normals, it looks like they need to be added to the same vertex_key function that handles the other corner-domain data, vertex colors and UVs.

io_scene_x3d/export_x3d.py Lines 697 to 723 in deb7574efb

if is_uv and is_col:

slot_uv = 0

slot_col = 1

def vertex_key(lidx):

return (

_tuple_from_rounded_iter(mesh_loops_uv[lidx].uv),

_tuple_from_rounded_iter(mesh_loops_col[lidx].color),

)

elif is_uv:

slot_uv = 0

def vertex_key(lidx):

return (

_tuple_from_rounded_iter(mesh_loops_uv[lidx].uv),

)

elif is_col:

slot_col = 0

def vertex_key(lidx):

return (

_tuple_from_rounded_iter(mesh_loops_col[lidx].color),

)

else:

# ack, not especially efficient in this case

def vertex_key(lidx):

return None

I think this might be the way to go: to add the normal to the vertex_key for this hash-table (which I assume now only has position and UV). This for the triangulated case.

In the non-triangulated case, the normals indices are just missing, and the normal vectors are all fully smoothed. I guess readers then assume these indices are the same as position indices (giving one normal per "corner"). But those indices can be different (the missing "normalIndex" array).

Thanks, I'll have a look.

@scurest > What mano-wii means when he says "per-vertex normals" is one normal is stored for every Blender vertex. These normals are (AFAIK) always the average of the normals of the surrounding faces. What he means by "per-polygon normals" is one normal is stored for each corner of a polygon. This is what you're talking about, where the three face corners that meet at one vertex of a cube each have their own normal. This is what is used for shading. Sorry for the misunderstanding, I understood per-polygon normals as one single vector per face (yielding only flat faces). BTW, averaging is done only among touching faces with their normals forming an angle less than the crease angle limit (if auto-smooth is enabled). > (X3D actually is able store one normal for each polygon, which is probably why this is confusing.) Yes, but I think I've rarely seen that. > Looking at X3D, it looks like it doesn't use the vertex/corner distinction like Blender/OBJ, but uses "one vertex = one tuple of attributes" like in OpenGL/glTF. That is the case in the `IndexedTriangleSet` node (used when triangulate is enabled). Yes, that's a better way of saying it: "one vertex = one tuple of attributes". OTOH, the `IndexedFaceSet` which supports polygons of any size, does have separate indices per attribute, as in OBJ. > The X3D exporter creates one X3D vert for each Blender corner, deduping identical X3D verts with a hash table. So to support corner-domain normals, it looks like they need to be added to the same `vertex_key` function that handles the other corner-domain data, vertex colors and UVs. > > https://projects.blender.org/blender/blender-addons/src/commit/deb7574efbb8b8982d2b52c1411103416a6855e3/io_scene_x3d/export_x3d.py#L697-L723 I think this might be the way to go: to add the normal to the `vertex_key` for this hash-table (which I assume now only has position and UV). This for the triangulated case. In the non-triangulated case, the normals indices are just missing, and the normal vectors are all fully smoothed. I guess readers then assume these indices are the same as position indices (giving one normal per "corner"). But those indices can be different (the missing "normalIndex" array). Thanks, I'll have a look.

Scurest commented

2023-02-28 01:02:54 +01:00

But those indices can be different (the missing "normalIndex" array).

Oh, you're right, I see now. The page I'm looking at is https://www.web3d.org/documents/specifications/19775-1/V4.0/Part01/components/geometry3D.html#IndexedFaceSet. So it does have the vertex/corner distinction. Same for UVs and colors. It doesn't seem like it has to split up vertices at all then. Hm.

> But those indices can be different (the missing "normalIndex" array). Oh, you're right, I see now. The page I'm looking at is https://www.web3d.org/documents/specifications/19775-1/V4.0/Part01/components/geometry3D.html#IndexedFaceSet. So it does have the vertex/corner distinction. Same for UVs and colors. It doesn't seem like it has to split up vertices at all then. Hm.

Cedric Steiert referenced this issue from extensions/io_scene_x3d

2024-08-06 23:47:37 +02:00

[TODO] Open bugs and tasks from blender/blender-addons #8

Sign in to join this conversation.