[TODO] Create some sample wrl and x3d testing files #11

New Issue

Cedric Steiert · 2024-08-08T22:49:10+02:00

Cedric Steiert commented

2024-08-08 22:49:10 +02:00

It would be great to have a set of small example .wrl and .x3d files to test the importers (and exporters if time allows it) against various feature sets to detect breaking changes in code early.
Currently we'd have to test with the provided files from the issue reports or create our own for testing, which takes time, is not really consistent and uploading them into the repo might not be suitable as we can't guarantee GPL license compatibility with user reported files.
Thus I propose to create a small sample library of demo files covering the most important features of both file types as regression is always possible when implementing new features, fixing existing ones or maintaining the code to the newest API.

Have a better idea? Comment below. Always open for suggestions on how to improve the persistant quality of the project.

Steps to take:

Compare the features of the file types with the current implementation
Create simple demo files covering the most important features each (they should not be too big nor complex as the dependency tree might be harder to follow in case of brakings). Note that they have to be GPL compatible, thus only your own resources or works should be used. However user uploaded files can be taken as inspiration
Upload them into the repo, in a new folder, e.g. called sample_files
Propose a testing concept. Is it required to import each file on their own and compare the results manually or can some test cases be written for that or at least be partially automated?
Test: test the new files against the current implementation and create issues/fixes where necessary
Documentation: Create a new page in the wiki considering two main points: a) expected output of demo files b) proposed guidelines/testing concept

It would be great to have a set of small example `.wrl` and `.x3d` files to test the importers (and exporters if time allows it) against various feature sets to detect breaking changes in code early. Currently we'd have to test with the provided files from the issue reports or create our own for testing, which takes time, is not really consistent and uploading them into the repo might not be suitable as we can't guarantee GPL license compatibility with user reported files. Thus I propose to create a small sample library of demo files covering the most important features of both file types as regression is always possible when implementing new features, fixing existing ones or maintaining the code to the newest API. Have a better idea? Comment below. Always open for suggestions on how to improve the persistant quality of the project. Steps to take: - [ ] Compare the features of the file types with the current implementation - [ ] Create simple demo files covering the most important features each (they should not be too big nor complex as the dependency tree might be harder to follow in case of brakings). Note that they have to be GPL compatible, thus only your own resources or works should be used. However user uploaded files can be taken as inspiration - [ ] Upload them into the repo, in a new folder, e.g. called `sample_files` - [ ] Propose a testing concept. Is it required to import each file on their own and compare the results manually or can some test cases be written for that or at least be partially automated? - [ ] Test: test the new files against the current implementation and create issues/fixes where necessary - [ ] Documentation: Create a new page in the wiki considering two main points: a) expected output of demo files b) proposed guidelines/testing concept

Cedric Steiert added the

enhancement

help wanted

task

labels 2024-08-08 22:49:11 +02:00

Cedric Steiert commented

2024-08-08 22:51:52 +02:00

@Hombre57 maybe you're interested in creating and providing some sample wrl files? Is it even a good idea or just unnecassary work load?

Hombre57 commented

2024-08-09 15:32:11 +02:00

@Bujus_Krachus I can provide some VRML files, not sure for X3D ones. I can have access to NX, TopSolid and Solidworks. It'll only cover a small subset of VRML/X3D possibilities. I can also compile some broken files from the tickets closed recently.

I'll have a look in 1 or 2 weeks, if you don't mind.

@Bujus_Krachus I can provide some VRML files, not sure for X3D ones. I can have access to NX, TopSolid and Solidworks. It'll only cover a small subset of VRML/X3D possibilities. I can also compile some broken files from the tickets closed recently. I'll have a look in 1 or 2 weeks, if you don't mind.

Cedric Steiert commented

2024-08-09 19:10:31 +02:00

@Hombre57 thanks, would be great. Also don't worry about the time, it's absolutely by no means time sensitive.

The files would be great to catch regression introduced by new features/bugfixes or API changes early for the most widely used parts which create the basic feature set.
Take #2 for example, where i unknowingly destroyed huge parts of the importer by doing an initial tiny fix (which worked for that specific file but messed with any other) or #6 where updating to the new API resulted in changed material appearance. Unfortunately users most likely won't report a change in appearance, rather only if something goes horibly wrong (e.g. when exceptions are thrown); still that doesn't mean that everything is working accordingly.

It would be great to have at least some files covering the most basic and widely used features, which we can trust (as in the files are valid and according to technicall standard). Also to have some comparison images of a) the import at current state and b) of an importer we can trust (e.g. freeWRL). Of course we can't cover all (edge) cases (for that we rely on user reports), but i personally would prefer to keep noticeable regressions on code changes as little as possible. Would also make reviewing PRs as well as maintaining a basic feature set in the long run a lot easier.
The basic testing files shouldn't be too many tbh, as that would be quite timely to test them all each time, instead maybe differenciate between "basic" (most widely used features or stuff that breaks easily) and "advanced" (more specific or rarer use cases which are still considered important, but won't affect all users).

If creation of individual test files is too complex/time intensive then maybe just create a guide on how to create valid files with free and especially trusted software.

Usually i'd say to write unit tests, but that's currently not really recommended due to two reasons: a) unmaintable/untestable code base due to 14 years of ongoing development and no clear layered architecture b) importers rely on the visual aspect inside of blender.

Btw i gave you full commit rights to the repo as well, meaning you can directly push to the repo itself, merge branches, close/edit issues etc. If you want to work on your fork or directly in the repo it's your decision. Probably would be best to work on the repo in feature branches so that others could have access to them as well. For bigger changes/fixes it would still be great to do at least some kind of peer review (code style, regression checking etc.) via PRs.

@Hombre57 thanks, would be great. Also don't worry about the time, it's absolutely by no means time sensitive. The files would be great to catch regression introduced by new features/bugfixes or API changes early for the most widely used parts which create the basic feature set. Take https://projects.blender.org/extensions/io_scene_x3d/pulls/2 for example, where i unknowingly destroyed huge parts of the importer by doing an initial tiny fix (which worked for that specific file but messed with any other) or https://projects.blender.org/extensions/io_scene_x3d/pulls/6 where updating to the new API resulted in changed material appearance. Unfortunately users most likely won't report a change in appearance, rather only if something goes horibly wrong (e.g. when exceptions are thrown); still that doesn't mean that everything is working accordingly. It would be great to have at least some files covering the most basic and widely used features, which we can trust (as in the files are valid and according to technicall standard). Also to have some comparison images of a) the import at current state and b) of an importer we can trust (e.g. freeWRL). Of course we can't cover all (edge) cases (for that we rely on user reports), but i personally would prefer to keep noticeable regressions on code changes as little as possible. Would also make reviewing PRs as well as maintaining a basic feature set in the long run a lot easier. The basic testing files shouldn't be too many tbh, as that would be quite timely to test them all each time, instead maybe differenciate between "basic" (most widely used features or stuff that breaks easily) and "advanced" (more specific or rarer use cases which are still considered important, but won't affect all users). If creation of individual test files is too complex/time intensive then maybe just create a guide on how to create valid files with free and especially trusted software. Usually i'd say to write unit tests, but that's currently not really recommended due to two reasons: a) unmaintable/untestable code base due to 14 years of ongoing development and no clear layered architecture b) importers rely on the visual aspect inside of blender. Btw i gave you full commit rights to the repo as well, meaning you can directly push to the repo itself, merge branches, close/edit issues etc. If you want to work on your fork or directly in the repo it's your decision. Probably would be best to work on the repo in feature branches so that others could have access to them as well. For bigger changes/fixes it would still be great to do at least some kind of peer review (code style, regression checking etc.) via PRs.

Hombre57 commented

2024-08-09 23:31:00 +02:00

@Bujus_Krachus

Thanks for the commit rights Cedric.

About "the most basic and widely used features", which one are they ? I only use one : IndexedFaceShape. Anyway, I'll try to produce some files using most features of the VRML standard, excepted animation.

@Bujus_Krachus Thanks for the commit rights Cedric. About "_the most basic and widely used features_", which one are they ? I only use one : IndexedFaceShape. Anyway, I'll try to produce some files using most features of the VRML standard, excepted animation.

Cedric Steiert commented

2024-08-10 01:16:19 +02:00

About "the most basic and widely used features", which one are they

tldr: probable main use case CAD/3D-Printing support: all geo nodes including basic color material (face/vertex), normals, grouping and some transforms; more focus on features of IndexFaceSet. No focus/continued support on any other use cases, which vrml may have been used/intended for back in the days.

I trust you on this one as i haven't really used neither format (wrl/x3d); i usually stick to stl and fbx.

Campbells (OC) last words rgarding this extension were "as long as this remains working for basic IO".

I'd say this includes basic IO of all Geo nodes as well as simple materials/shading options (face & vertex) including normals and maybe basic grouping. As i think IndexedFaceShape is indeed the most commonly used node, there the test coverage should ideally be higher, meaning covering nodes like ImageTexture, PixelTexture, TextureTransform etc.

Maybe build a little snowman or sth other fun to combine different things for simplicity reasons ;)

The main goal is to lower probability of noticeable regression when doing simple fixes or changes for afore mentioned basic IO, like e.g. on the PRs listed above.

All the other stuff i'd personally not consider basic or essential. Meaning if the other stuff breaks for users repeatedly it either gets deactivated (like H3D Extensions for now), fixed sometime to work on a basic level again or removed entirely (like probable future of H3D if pending effort analysis says no).
From my research wrl is most commonly used nowadays for 3d (resin) (color) printing, meaning nodes like audio, light, sensors, world, bindable nodes etc. could get removed without worries if they get reported as broken at any time in the future. Btw that's also why i created a meta task for reintroducing a basic exporter (at least shape, color, normals, some transforms). For other use cases we have stl (regular printing), fbx/obj/gltf (gaming, web, app interchange) and many other supported formats inside blender or via extensions.

Hope this makes sense...please do correct me if i'm just tripping here.

Btw as you are also a contributor you can as well decide which direction this extension should take. If you're a big fan of animated spot lights whilst playing jingle bells, create some test files for that to ensure it's still working in future versions (so it can get fixed early when it starts to break (which it will, if it hasen't already lol)).

> About "the most basic and widely used features", which one are they _tldr: probable main use case CAD/3D-Printing support: all geo nodes including basic color material (face/vertex), normals, grouping and some transforms; more focus on features of IndexFaceSet. No focus/continued support on any other use cases, which vrml may have been used/intended for back in the days._ I trust you on this one as i haven't really used neither format (wrl/x3d); i usually stick to stl and fbx. Campbells (OC) last words rgarding this extension were "as long as this remains working for basic IO". I'd say this includes basic IO of all Geo nodes as well as simple materials/shading options (face & vertex) including normals and maybe basic grouping. As i think IndexedFaceShape is indeed the most commonly used node, there the test coverage should ideally be higher, meaning covering nodes like ImageTexture, PixelTexture, TextureTransform etc. Maybe build a little snowman or sth other fun to combine different things for simplicity reasons ;) The main goal is to lower probability of noticeable regression when doing simple fixes or changes for afore mentioned basic IO, like e.g. on the PRs listed above. All the other stuff i'd personally not consider basic or essential. Meaning if the other stuff breaks for users repeatedly it either gets deactivated (like H3D Extensions for now), fixed sometime to work on a basic level again or removed entirely (like probable future of H3D if pending effort analysis says no). From my research wrl is most commonly used nowadays for 3d (resin) (color) printing, meaning nodes like audio, light, sensors, world, bindable nodes etc. could get removed without worries if they get reported as broken at any time in the future. Btw that's also why i created a meta task for reintroducing a basic exporter (at least shape, color, normals, some transforms). For other use cases we have stl (regular printing), fbx/obj/gltf (gaming, web, app interchange) and many other supported formats inside blender or via extensions. Hope this makes sense...please do correct me if i'm just tripping here. Btw as you are also a contributor you can as well decide which direction this extension should take. If you're a big fan of animated spot lights whilst playing jingle bells, create some test files for that to ensure it's still working in future versions (so it can get fixed early when it starts to break (which it will, if it hasen't already lol)).

Sign in to join this conversation.