FBX export of Face smoothing info (without custom normals) #104434

New Issue

Yury · 2023-02-25T10:36:33+01:00

Yury commented

2023-02-25 10:36:33 +01:00

Issue

(Summary from original report and investigations by @Mysteryem, see comments below).

Current export of 'smooth faces' boolean flag, without exporting custom normals, is producing invalid imports in 3DSMax (which seems to interpret them as its 'smooth groups' - non-exclusive bitflags of face groups, each face can belong to more than one smooth groups).

FBX Review does show blender-exported files properly in that case though.

Unity never produces the expected result, regardless of how smooth face info is encoded (as done currently, with the complex bitflag way, etc.).

Original report:

When exporting model as FBX with smoothing option in exporter set to Face, it uses bool attribute of polygon "use_smooth" to define smoothing groups which is pretty useless (or at least I can't imagine a case where it would be useful):

code from export_fbx_bin.py

t_ps = array.array(data_types.ARRAY_INT32, (0,)) * len(me.polygons)
me.polygons.foreach_get("use_smooth", t_ps)

There are ways to calculate smoothing groups from vertex normals when importing to other 3D package, however they came with its flaws (at least 3dsMaxs one)

For example if we have a mesh made in Blender with hard edges set like this:

Export it as FBX with smoothing set to "Normals Only" (because "Face" just gives us one smoothing group without any of normal splits, also i had no luck with "Edge")
After importing in 3dsMax with Smoothing Groups checkbox enabled (in this case 3dsMax can't find smoothing groups inside FBX file and will generate smoothing groups from vertex normals). We get this result.

Notice that 3dsMax created only 2 smoothing groups, which isn't enough to preserve the look of original model (central vertex has no normal split). On more complex models this issue can be more devastating, requiring lots of manual work to fix.

Expected result should be like this:

I modified export_fbx_bin.py so it now can calculate smoothing groups. It works but far from being optimized due to my limited coding skills. I am sure it can be done much better by actual blender-addons devs. Thanks.

export_fbx_bin.py line 973

            # BEGINNING OF SMOOTHING GROUP PATCH

            smoothing_groups = (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32)

            #Calculate face groups (Can't really use_bitflags=True because it can put two face groups into one smoothing group even if they both have common "sharp" vert)
            face_groups = me.calc_smooth_groups(use_bitflags=False)[0]

            # Create dict {vertex_index: {facegroupIDs}}
            v_fg = {}
            for poly in me.polygons:
                for v_index in poly.vertices:        
                    polygroups = v_fg.get(v_index)
                    if not polygroups:
                        polygroups = set()
                    polygroups.add(face_groups[poly.index])
                    v_fg.update({v_index: polygroups})

            #Create dict {facegroupID: [polys]}
            fg_p = {}
            for poly in me.polygons:
                polys = fg_p.get(face_groups[poly.index])
                if not polys:
                    polys = []
                polys.append(poly)
                fg_p.update({face_groups[poly.index]: polys })

            #Create dict {facegroupID: {neighborfacegroupIDs}}
            fg_n = {}
            for fg in fg_p:
                neighbors = set()
                for poly in fg_p[fg]:
                    for v_index in poly.vertices:
                        neighbors.update(v_fg[v_index])
                neighbors.remove(fg)
                fg_n.update({fg: neighbors})
                        
            #Create dict {facegroupID: SmoothingGroupID}
            sg_of_fg = {}
            for fg in fg_n:
                if fg == 1:
                    sg = smoothing_groups[1]
                else:
                    neighbors = fg_n[fg]
                    neighbors_sgs = set()
                    for neighbor in neighbors:           
                        neighbor_sg = sg_of_fg.get(neighbor)
                        neighbors_sgs.add(neighbor_sg)
                    sg = min([sg for sg in smoothing_groups if sg not in neighbors_sgs and sg !=0])             
                sg_of_fg.update({fg: sg})
                
            #Create list [poly_index: SmoothingGroupID]
            smoothing_groups = list(map(sg_of_fg.get, face_groups))
            t_ps = [2**(x-1) for x in smoothing_groups]

            # END OF SMOOTHING GROUP PATCH
            
            # t_ps = array.array(data_types.ARRAY_INT32, (0,)) * len(me.polygons)
            # me.polygons.foreach_get("use_smooth", t_ps)

## Issue *(Summary from original report and investigations by @Mysteryem, see comments below).* Current export of 'smooth faces' boolean flag, without exporting custom normals, is producing invalid imports in 3DSMax (which seems to interpret them as its 'smooth groups' - non-exclusive bitflags of face groups, each face can belong to more than one smooth groups). FBX Review does show blender-exported files properly in that case though. Unity never produces the expected result, regardless of how smooth face info is encoded (as done currently, with the complex bitflag way, etc.). ## Original report: When exporting model as FBX with smoothing option in exporter set to Face, it uses bool attribute of polygon "use_smooth" to define smoothing groups which is pretty useless (or at least I can't imagine a case where it would be useful): code from export_fbx_bin.py ``` t_ps = array.array(data_types.ARRAY_INT32, (0,)) * len(me.polygons) me.polygons.foreach_get("use_smooth", t_ps) ``` There are ways to calculate smoothing groups from vertex normals when importing to other 3D package, however they came with its flaws (at least 3dsMaxs one) For example if we have a mesh made in Blender with hard edges set like this: ![1.jpg](/attachments/8d0a042e-ccf3-4a38-8dfe-4baea83f7845) ![2.jpg](/attachments/18a22c6c-b991-4d53-a6f6-19895c55bbe3) Export it as FBX with smoothing set to "Normals Only" (because "Face" just gives us one smoothing group without any of normal splits, also i had no luck with "Edge") After importing in 3dsMax with Smoothing Groups checkbox enabled (in this case 3dsMax can't find smoothing groups inside FBX file and will generate smoothing groups from vertex normals). We get this result. ![3.jpg](/attachments/2564c314-8363-4bd4-9b96-4ea3efc434b9) ![4.jpg](/attachments/aac3922a-864c-44a2-86a8-0e6371d68370) Notice that 3dsMax created only 2 smoothing groups, which isn't enough to preserve the look of original model (central vertex has no normal split). On more complex models this issue can be more devastating, requiring lots of manual work to fix. Expected result should be like this: ![5.jpg](/attachments/f1dba410-4100-4336-b182-a03d48a7be48) I modified export_fbx_bin.py so it now can calculate smoothing groups. It works but far from being optimized due to my limited coding skills. I am sure it can be done much better by actual blender-addons devs. Thanks. export_fbx_bin.py line 973 ``` # BEGINNING OF SMOOTHING GROUP PATCH smoothing_groups = (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32) #Calculate face groups (Can't really use_bitflags=True because it can put two face groups into one smoothing group even if they both have common "sharp" vert) face_groups = me.calc_smooth_groups(use_bitflags=False)[0] # Create dict {vertex_index: {facegroupIDs}} v_fg = {} for poly in me.polygons: for v_index in poly.vertices: polygroups = v_fg.get(v_index) if not polygroups: polygroups = set() polygroups.add(face_groups[poly.index]) v_fg.update({v_index: polygroups}) #Create dict {facegroupID: [polys]} fg_p = {} for poly in me.polygons: polys = fg_p.get(face_groups[poly.index]) if not polys: polys = [] polys.append(poly) fg_p.update({face_groups[poly.index]: polys }) #Create dict {facegroupID: {neighborfacegroupIDs}} fg_n = {} for fg in fg_p: neighbors = set() for poly in fg_p[fg]: for v_index in poly.vertices: neighbors.update(v_fg[v_index]) neighbors.remove(fg) fg_n.update({fg: neighbors}) #Create dict {facegroupID: SmoothingGroupID} sg_of_fg = {} for fg in fg_n: if fg == 1: sg = smoothing_groups[1] else: neighbors = fg_n[fg] neighbors_sgs = set() for neighbor in neighbors: neighbor_sg = sg_of_fg.get(neighbor) neighbors_sgs.add(neighbor_sg) sg = min([sg for sg in smoothing_groups if sg not in neighbors_sgs and sg !=0]) sg_of_fg.update({fg: sg}) #Create list [poly_index: SmoothingGroupID] smoothing_groups = list(map(sg_of_fg.get, face_groups)) t_ps = [2**(x-1) for x in smoothing_groups] # END OF SMOOTHING GROUP PATCH # t_ps = array.array(data_types.ARRAY_INT32, (0,)) * len(me.polygons) # me.polygons.foreach_get("use_smooth", t_ps) ```

4.jpg

46 KiB

1.jpg

62 KiB

5.jpg

37 KiB

2.jpg

38 KiB

3.jpg

26 KiB

Andrej commented

2023-02-25 10:38:37 +01:00

@mont29 can you please take a look?

Bastien Montagne commented

2023-03-06 16:19:19 +01:00

@Andrej-4 not sure whether the smoothing groups of FBX are supposed/expected to follow the smoothing groups handling of 3DSMax? Would be good to get evidence of this first.

Besides that, code here was very recently (speed-)improved by using numpy, so maybe @Mysteryem would be interested in this?

@Andrej-4 not sure whether the smoothing groups of FBX are supposed/expected to follow the smoothing groups handling of 3DSMax? Would be good to get evidence of this first. Besides that, code here was very recently (speed-)improved by using numpy, so maybe @Mysteryem would be interested in this?

Bastien Montagne added the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

FBX export of Face smoothing info (without custom normals) #104434

Issue

Original report: