OldNewMap handling of pointers in blenloader does not work correctly with FD_FLAGS_SWITCH_ENDIAN #81226

Closed
opened 2 years ago by zeldin · 10 comments
zeldin commented 2 years ago

System Information
Operating system: Linux/ppc64
Graphics card: llvmpipe

Blender Version
Broken: 2.83.4, master
Worked: ?

Short description of error
The functions switch_endian_bh4 and switch_endian_bh8 in readfile.c do not byteswap the "old" pointer field. In theory this is fine, as it is just used as an opaque handle, which is mapped to an actual pointer via the OldNewMap. However, in practice this does not work because it seems that the reference pointers to the object are byteswapped, and so if the pointer used as key to create the entry in the OldNewMap is not byteswapped as well, the values will not match and the lookup will fail.

This can easily fixed by just swapping the "old" pointer in the bh swapping functions. That also removes the need for the workaround in bh4_from_bh8. Please see the following patch:
blender.patch

Exact steps for others to reproduce the error

  1. Start blender on a big endian 64-bit system
  2. Startup crashes as it fails to load the builtin WM blendfile, getting a NULL back from oldnewmap_lookup_and_inc
**System Information** Operating system: Linux/ppc64 Graphics card: llvmpipe **Blender Version** Broken: 2.83.4, master Worked: ? **Short description of error** The functions switch_endian_bh4 and switch_endian_bh8 in readfile.c do not byteswap the "old" pointer field. In theory this is fine, as it is just used as an opaque handle, which is mapped to an actual pointer via the OldNewMap. However, in practice this does not work because it seems that the reference pointers to the object *are* byteswapped, and so if the pointer used as key to create the entry in the OldNewMap is not byteswapped as well, the values will not match and the lookup will fail. This can easily fixed by just swapping the "old" pointer in the bh swapping functions. That also removes the need for the workaround in bh4_from_bh8. Please see the following patch: [blender.patch](https://archive.blender.org/developer/F8928134/blender.patch) **Exact steps for others to reproduce the error** 1) Start blender on a big endian 64-bit system 2) Startup crashes as it fails to load the builtin WM blendfile, getting a NULL back from oldnewmap_lookup_and_inc
zeldin commented 2 years ago
Poster

Added subscriber: @zeldin

Added subscriber: @zeldin
Collaborator

Added subscribers: @JacquesLucke, @mano-wii

Added subscribers: @JacquesLucke, @mano-wii
Collaborator

Thanks for the report, unfortunately I don't have a big endian 64-bit system to test.

@JacquesLucke, if I'm not mistaken, you worked in this area recently.
Do you know what's going on here?

Thanks for the report, unfortunately I don't have a big endian 64-bit system to test. @JacquesLucke, if I'm not mistaken, you worked in this area recently. Do you know what's going on here?
Collaborator

Just last week I found this issue as well. My fix (which is essentially the same) is part of D9089.

@zeldin, can you provide a .blend file that has been saved on a big endian 64 bit system? Can you also check if D9089 fixes it for you?

Just last week I found this issue as well. My fix (which is essentially the same) is part of [D9089](https://archive.blender.org/developer/D9089). @zeldin, can you provide a .blend file that has been saved on a big endian 64 bit system? Can you also check if [D9089](https://archive.blender.org/developer/D9089) fixes it for you?
zeldin commented 2 years ago
Poster

@JacquesLucke Sure. Here you go:
untitled.blend

I'll try your patch later, need to make a new git checkout since the patch does not apply cleanly to 2.83.4. :-)

@JacquesLucke Sure. Here you go: [untitled.blend](https://archive.blender.org/developer/F8958873/untitled.blend) I'll try your patch later, need to make a new git checkout since the patch does not apply cleanly to 2.83.4. :-)
Collaborator

Thanks!

Did any older version of Blender work on your hardware?

Thanks! Did any older version of Blender work on your hardware?
zeldin commented 2 years ago
Poster

Yes. The "?" in "Worked" just means that I can't remember what the version number was. :-)

Well, I can't say for sure that it was possible to load little endian .blend files from disk, but running it in BE worked fine.

Yes. The "?" in "Worked" just means that I can't remember what the version number was. :-) Well, I can't say for sure that it was possible to load little endian .blend files from disk, but running it in BE worked fine.
Owner

This issue was referenced by ffb220590e

This issue was referenced by ffb220590ea76bc279eeb3498776a360467125c1
Owner

This issue was referenced by 724a4f4676

This issue was referenced by 724a4f4676586d65b718deab7f04f207e0ad6488
Owner

Changed status from 'Needs Triage' to: 'Resolved'

Changed status from 'Needs Triage' to: 'Resolved'
ideasman42 closed this issue 2 years ago
ideasman42 self-assigned this 2 years ago
Sign in to join this conversation.
No Label
good first issue
legacy module/Animation & Rigging
legacy module/Core
legacy module/Development Management
legacy module/Eevee & Viewport
legacy module/Grease Pencil
legacy module/Modeling
legacy module/Nodes & Physics
legacy module/Pipeline, Assets & IO
legacy module/Platforms, Builds, Tests & Devices
legacy module/Python API
legacy module/Rendering & Cycles
legacy module/Sculpt, Paint & Texture
legacy module/Triaging
legacy module/User Interface
legacy module/VFX & Video
legacy project/1.0.0-beta.2
legacy project/2.81
legacy project/2.82
legacy project/2.83
legacy project/2.90
legacy project/2.91
legacy project/2.92
legacy project/3.0
legacy project/3.1
legacy project/3.2
legacy project/3.3
legacy project/Alembic
legacy project/Animation & Rigging
legacy project/Asset Browser
legacy project/Asset Browser (Archived)
legacy project/Asset Browser Project Overview
legacy project/Audio
legacy project/Automated Testing
legacy project/BF Blender: 2.8
legacy project/BF Blender: After Release
legacy project/BF Blender: Next
legacy project/BF Blender: Regressions
legacy project/BF Blender: Unconfirmed
legacy project/Blender 2.70
legacy project/Blender Asset Bundle
legacy project/Code Quest
legacy project/Collada
legacy project/Compositing
legacy project/Core
legacy project/Cycles
legacy project/Datablocks and Libraries
legacy project/Dependency Graph
legacy project/Development Management
legacy project/Eevee
legacy project/EEVEE & Viewport
legacy project/Freestyle
legacy project/Game Animation
legacy project/Game Audio
legacy project/Game Data Conversion
legacy project/Game Engine
legacy project/Game Logic
legacy project/Game Physics
legacy project/Game Python
legacy project/Game Rendering
legacy project/Game UI
legacy project/Geometry Nodes
legacy project/Good First Issue
legacy project/GPU / Viewport
legacy project/Grease Pencil
legacy project/GSoC
legacy project/Images & Movies
legacy project/Import/Export
legacy project/Infrastructure: Websites
legacy project/LibOverrides - Usability and UX
legacy project/Line Art
legacy project/Masking
legacy project/Milestone 1: Basic, Local Asset Browser
legacy project/Modeling
legacy project/Modifiers
legacy project/Motion Tracking
legacy project/Nodes
legacy project/Nodes & Physics
legacy project/OpenGL Error
legacy project/Overrides
legacy project/Papercut
legacy project/Performance
legacy project/Physics
legacy project/Pipeline, Assets & I/O
legacy project/Platform: FreeBSD
legacy project/Platform: Linux
legacy project/Platform: macOS
legacy project/Platforms, Builds, Tests & Devices
legacy project/Platform: Windows
legacy project/Pose Library Basics
legacy project/Python API
legacy project/Render & Cycles
legacy project/Render Pipeline
legacy project/Retrospective
legacy project/Sculpt, Paint & Texture
legacy project/Text Editor
legacy project/Tracker Curfew
legacy project/Translations
legacy project/Triaging
legacy project/Undo
legacy project/USD
legacy project/User Interface
legacy project/UV Editing
legacy project/VFX & Video
legacy project/Video Sequencer
legacy project/Virtual Reality
legacy project/Wintab High Frequency
migration/requires-manual-verification
Module › Animation & Rigging
Module › Core
Module › Development Management
Module › Eevee & Viewport
Module › EEVEE & Viewport
Module › Grease Pencil
Module › Modeling
Module › Nodes & Physics
Module › Pipeline, Assets & IO
Module › Platforms, Builds Tests & Devices
Module › Platforms, Builds, Tests & Devices
Module › Python API
Module › Rendering & Cycles
Module › Sculpt, Paint & Texture
Module › Triaging
Module › User Interface
Module › VFX & Video
papercut
performance
Priority › High
Priority › Low
Priority › Normal
Priority › Unbreak Now!
Status › Archived
Status › Confirmed
Status › Duplicate
Status › Needs Information from Developers
Status › Needs Information from User
Status › Needs Triage
Status › Resolved
Type › Bug
Type › Design
Type › Known Issue
Type › Patch
Type › Report
Type › To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#81226
Loading…
There is no content yet.