Add support for 'changelog' generation in API documentation automated process on the buildbot. #97663

Closed
opened 2022-04-27 18:24:45 +02:00 by James Tomkinson · 20 comments

The 'changelog' page of the Blender API documetation used to be generated manually as a separate, independent step, prior to the actual API doc generation.

Now that the API doc is generated automatically on the buildbots as part of the build process, the change log generation needs to be integrated there as well.

After talking with @Arnd, here is the proposal:

  • Store the API dump of a release together with its API doc on the server.
    • Switch from a dict-based py file storage to a JSON format, to avoid having potentially executable files on the server.
    • Have all available API dumps listed in another index JSON file at the root of the API doc directory.
  • As part of the API doc generation tool (sphinx_doc_gen.py):
    • Download the index file
    • Find the 'most recent' previous version available
    • Download found dump JSON file
    • Generate JSON dump file for current blender version, and update the index accordingly

    Generate the changelog page based on the diff between the two API dump files


ORIGINAL REPORT:

blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release

System Information
Operating system: Windows-10-10.0.19043-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65

Blender Version
Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: blender/blender@cc66d1020c
Worked: (newest version of Blender that worked as expected)

Short description of error
It appears that the python change-log information is not being updated.

Exact steps for others to reproduce the error
Go to Help >> Python API Reference
a web page is displayed, click on Change Log link
taken to https://docs.blender.org/api/3.1/change_log.html
In the title it says:

Blender API Change Log
2.83 to 2.90

The 'changelog' page of the Blender API documetation used to be generated manually as a separate, independent step, prior to the actual API doc generation. Now that the API doc is generated automatically on the buildbots as part of the build process, the change log generation needs to be integrated there as well. After talking with @Arnd, here is the proposal: - [x] Store the API dump of a release together with its API doc on the server. - [x] Switch from a dict-based `py` file storage to a JSON format, to avoid having potentially executable files on the server. - [x] Have all available API dumps listed in another index JSON file at the root of the API doc directory. - [x] As part of the API doc generation tool (`sphinx_doc_gen.py`): - Download the index file - Find the 'most recent' previous version available - Download found dump JSON file - Generate JSON dump file for current blender version, and update the index accordingly # Generate the changelog page based on the diff between the two API dump files ------------ **ORIGINAL REPORT:** # blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release **System Information** Operating system: Windows-10-10.0.19043-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65 **Blender Version** Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: `blender/blender@cc66d1020c` Worked: (newest version of Blender that worked as expected) **Short description of error** It appears that the python change-log information is not being updated. **Exact steps for others to reproduce the error** Go to Help >> Python API Reference a web page is displayed, click on Change Log link taken to https://docs.blender.org/api/3.1/change_log.html In the title it says: > Blender API Change Log > 2.83 to 2.90

Added subscriber: @jamestomk

Added subscriber: @jamestomk

blender/documentation#98757 was marked as duplicate of this issue

blender/documentation#98757 was marked as duplicate of this issue

Added subscriber: @lichtwerk

Added subscriber: @lichtwerk

Can confirm.

Can confirm.

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'

Added subscribers: @mont29, @ideasman42

Added subscribers: @mont29, @ideasman42

@ideasman42 , @mont29 : do you know if this was done by hand?

@ideasman42 , @mont29 : do you know if this was done by hand?

The changelog generator needs to be run manually. The main reason this wasn't automated is you need two different Blender versions of generate output, although if we kept the previous releases output in a fixed location we could compare against that which will work as long as the file format doesn't change.

The changelog generator needs to be run manually. The main reason this wasn't automated is you need two different Blender versions of generate output, although if we kept the previous releases output in a fixed location we could compare against that which will work as long as the file format doesn't change.
Philipp Oeser changed title from blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1.0 release to blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release 2022-06-10 08:23:46 +02:00

Added subscriber: @EAW

Added subscriber: @EAW

Will raise prio here since this is not a nice thing to have.

Will raise prio here since this is not a nice thing to have.

Added subscribers: @Arnd, @dmcgrath

Added subscribers: @Arnd, @dmcgrath

I believe this was intentional, at least temporarily.

@Arnd and I were speaking in chat on matters relating to a recent buildbot problem, when he mentioned:

short-term: i've changed the 'stable doc' to point to 3.2
but the /api/current is still pointing to 3.1.2 for now; so there's something there

I expect that at some point he will move it back to the proper versions, after he has got to the bottom of the build issue he was looking into.

I believe this was intentional, at least temporarily. @Arnd and I were speaking in chat on matters relating to a recent buildbot problem, when he mentioned: > short-term: i've changed the 'stable doc' to point to 3.2 > but the /api/current is still pointing to 3.1.2 for now; so there's something there I expect that at some point he will move it back to the proper versions, after he has got to the bottom of the build issue he was looking into.

Guess what: the python manual suffers from the same deficiency.

Guess what: the python manual suffers from the same deficiency.

This , at first glance, does not seem to be a buildbot related thing. The files that are produced in the build-steps are being delivered properly.
The remark above was related to the failure of the 3.2-upload process to adapt the 'current' and 'latest' links in docs. That's been fixed.

The Python changelog not being updated is something that, to me, looks to be part of the python-api-build-process itself. I'll look to see if I can see where exactly that is happening.
My intuition tells me it's not a buildbot issue.

This , at first glance, does not seem to be a buildbot related thing. The files that are produced in the build-steps are being delivered properly. The remark above was related to the failure of the 3.2-upload process to adapt the 'current' and 'latest' links in docs. That's been fixed. The Python changelog not being updated is something that, to me, looks to be part of the python-api-build-process itself. I'll look to see if I can see where exactly that is happening. My intuition tells me it's not a buildbot issue.

Think this was indeed never integrated in the automated API doc generation process.... Will look into how to do that locally first, then we can work with @Arnd into integrating it to the buildbot system.

Think this was indeed never integrated in the automated API doc generation process.... Will look into how to do that locally first, then we can work with @Arnd into integrating it to the buildbot system.
Bastien Montagne changed title from blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release to Add support for 'changelog' generation in API documentation automated process on the buildbot. 2022-06-13 17:45:28 +02:00

Added subscriber: @Blendify

Added subscriber: @Blendify
Aaron Carlisle self-assigned this 2022-06-14 00:42:46 +02:00

I will be looking into this and discuss a solution with Cambell and Arnd.

I will be looking into this and discuss a solution with Cambell and Arnd.
Aaron Carlisle was unassigned by Bastien Montagne 2022-06-14 09:50:33 +02:00
Bastien Montagne self-assigned this 2022-06-14 09:50:33 +02:00

@Blendify sorry it wasn't clear enough, am already working on this...

@Blendify sorry it wasn't clear enough, am already working on this...

Changed status from 'Confirmed' to: 'Resolved'

Changed status from 'Confirmed' to: 'Resolved'

This should now be resolved for current releases (2.93 LTS, 3.2 and future 3.3).

This should now be resolved for current releases (2.93 LTS, 3.2 and future 3.3).
This repo is archived. You cannot comment on issues.
No Milestone
No project
No Assignees
8 Participants
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: archive/blender-buildbot#97663
No description provided.