archive/blender-buildbot
Archived
1
0
Fork 0
Code Issues 3 Pull Requests Packages Projects Releases Wiki Activity

Add support for 'changelog' generation in API documentation automated process on the buildbot. #97663

New Issue
Closed
opened 2022-04-27 18:24:45 +02:00 by James Tomkinson · 20 comments
James Tomkinson commented 2022-04-27 18:24:45 +02:00
Copy Link

The 'changelog' page of the Blender API documetation used to be generated manually as a separate, independent step, prior to the actual API doc generation.

Now that the API doc is generated automatically on the buildbots as part of the build process, the change log generation needs to be integrated there as well.

After talking with @Arnd, here is the proposal:

  • Store the API dump of a release together with its API doc on the server.
    • Switch from a dict-based py file storage to a JSON format, to avoid having potentially executable files on the server.
    • Have all available API dumps listed in another index JSON file at the root of the API doc directory.
  • As part of the API doc generation tool (sphinx_doc_gen.py):
    • Download the index file
    • Find the 'most recent' previous version available
    • Download found dump JSON file
    • Generate JSON dump file for current blender version, and update the index accordingly

    Generate the changelog page based on the diff between the two API dump files

ORIGINAL REPORT:

blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release

System Information
Operating system: Windows-10-10.0.19043-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65

Blender Version
Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: blender/blender@cc66d1020c
Worked: (newest version of Blender that worked as expected)

Short description of error
It appears that the python change-log information is not being updated.

Exact steps for others to reproduce the error
Go to Help >> Python API Reference
a web page is displayed, click on Change Log link
taken to https://docs.blender.org/api/3.1/change_log.html
In the title it says:

Blender API Change Log
2.83 to 2.90

The 'changelog' page of the Blender API documetation used to be generated manually as a separate, independent step, prior to the actual API doc generation. Now that the API doc is generated automatically on the buildbots as part of the build process, the change log generation needs to be integrated there as well. After talking with @Arnd, here is the proposal: - [x] Store the API dump of a release together with its API doc on the server. - [x] Switch from a dict-based `py` file storage to a JSON format, to avoid having potentially executable files on the server. - [x] Have all available API dumps listed in another index JSON file at the root of the API doc directory. - [x] As part of the API doc generation tool (`sphinx_doc_gen.py`): - Download the index file - Find the 'most recent' previous version available - Download found dump JSON file - Generate JSON dump file for current blender version, and update the index accordingly # Generate the changelog page based on the diff between the two API dump files ------------ **ORIGINAL REPORT:** # blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release **System Information** Operating system: Windows-10-10.0.19043-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65 **Blender Version** Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: `blender/blender@cc66d1020c` Worked: (newest version of Blender that worked as expected) **Short description of error** It appears that the python change-log information is not being updated. **Exact steps for others to reproduce the error** Go to Help >> Python API Reference a web page is displayed, click on Change Log link taken to https://docs.blender.org/api/3.1/change_log.html In the title it says: > Blender API Change Log > 2.83 to 2.90
James Tomkinson commented 2022-04-27 18:24:45 +02:00
Author
Copy Link

Added subscriber: @jamestomk

Added subscriber: @jamestomk
blender-admin commented 2022-04-27 18:24:45 +02:00
Owner
Copy Link

blender/documentation#98757 was marked as duplicate of this issue

blender/documentation#98757 was marked as duplicate of this issue
Philipp Oeser commented 2022-04-28 09:34:00 +02:00
Copy Link

Added subscriber: @lichtwerk

Added subscriber: @lichtwerk
Philipp Oeser commented 2022-04-28 09:34:00 +02:00
Copy Link

Can confirm.

Can confirm.
Philipp Oeser commented 2022-04-28 09:34:10 +02:00
Copy Link

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'
Philipp Oeser commented 2022-05-05 16:18:31 +02:00
Copy Link

Added subscribers: @mont29, @ideasman42

Added subscribers: @mont29, @ideasman42
Philipp Oeser commented 2022-05-05 16:18:31 +02:00
Copy Link

@ideasman42 , @mont29 : do you know if this was done by hand?

@ideasman42 , @mont29 : do you know if this was done by hand?
Campbell Barton commented 2022-05-06 01:58:31 +02:00
Copy Link

The changelog generator needs to be run manually. The main reason this wasn't automated is you need two different Blender versions of generate output, although if we kept the previous releases output in a fixed location we could compare against that which will work as long as the file format doesn't change.

The changelog generator needs to be run manually. The main reason this wasn't automated is you need two different Blender versions of generate output, although if we kept the previous releases output in a fixed location we could compare against that which will work as long as the file format doesn't change.
Philipp Oeser changed title from blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1.0 release to blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release 2022-06-10 08:23:46 +02:00
Philipp Oeser commented 2022-06-10 08:24:41 +02:00
Copy Link

Added subscriber: @EAW

Added subscriber: @EAW
Philipp Oeser commented 2022-06-10 08:25:50 +02:00
Copy Link

Will raise prio here since this is not a nice thing to have.

Will raise prio here since this is not a nice thing to have.
Danny McGrath commented 2022-06-10 19:04:37 +02:00
Copy Link

Added subscribers: @Arnd, @dmcgrath

Added subscribers: @Arnd, @dmcgrath
Danny McGrath commented 2022-06-10 19:04:37 +02:00
Copy Link

I believe this was intentional, at least temporarily.

@Arnd and I were speaking in chat on matters relating to a recent buildbot problem, when he mentioned:

short-term: i've changed the 'stable doc' to point to 3.2
but the /api/current is still pointing to 3.1.2 for now; so there's something there

I expect that at some point he will move it back to the proper versions, after he has got to the bottom of the build issue he was looking into.

I believe this was intentional, at least temporarily. @Arnd and I were speaking in chat on matters relating to a recent buildbot problem, when he mentioned: > short-term: i've changed the 'stable doc' to point to 3.2 > but the /api/current is still pointing to 3.1.2 for now; so there's something there I expect that at some point he will move it back to the proper versions, after he has got to the bottom of the build issue he was looking into.
James Tomkinson commented 2022-06-12 18:55:10 +02:00
Author
Copy Link

Guess what: the python manual suffers from the same deficiency.

Guess what: the python manual suffers from the same deficiency.
Arnd Marijnissen commented 2022-06-13 14:47:48 +02:00
Copy Link

This , at first glance, does not seem to be a buildbot related thing. The files that are produced in the build-steps are being delivered properly.
The remark above was related to the failure of the 3.2-upload process to adapt the 'current' and 'latest' links in docs. That's been fixed.

The Python changelog not being updated is something that, to me, looks to be part of the python-api-build-process itself. I'll look to see if I can see where exactly that is happening.
My intuition tells me it's not a buildbot issue.

This , at first glance, does not seem to be a buildbot related thing. The files that are produced in the build-steps are being delivered properly. The remark above was related to the failure of the 3.2-upload process to adapt the 'current' and 'latest' links in docs. That's been fixed. The Python changelog not being updated is something that, to me, looks to be part of the python-api-build-process itself. I'll look to see if I can see where exactly that is happening. My intuition tells me it's not a buildbot issue.
Bastien Montagne commented 2022-06-13 16:09:48 +02:00
Copy Link

Think this was indeed never integrated in the automated API doc generation process.... Will look into how to do that locally first, then we can work with @Arnd into integrating it to the buildbot system.

Think this was indeed never integrated in the automated API doc generation process.... Will look into how to do that locally first, then we can work with @Arnd into integrating it to the buildbot system.
Bastien Montagne changed title from blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release to Add support for 'changelog' generation in API documentation automated process on the buildbot. 2022-06-13 17:45:28 +02:00
Aaron Carlisle commented 2022-06-14 00:42:46 +02:00
Copy Link

Added subscriber: @Blendify

Added subscriber: @Blendify
Aaron Carlisle self-assigned this 2022-06-14 00:42:46 +02:00
Aaron Carlisle commented 2022-06-14 00:42:46 +02:00
Copy Link

I will be looking into this and discuss a solution with Cambell and Arnd.

I will be looking into this and discuss a solution with Cambell and Arnd.
Aaron Carlisle was unassigned by Bastien Montagne 2022-06-14 09:50:33 +02:00
Bastien Montagne self-assigned this 2022-06-14 09:50:33 +02:00
Bastien Montagne commented 2022-06-14 09:50:33 +02:00
Copy Link

@Blendify sorry it wasn't clear enough, am already working on this...

@Blendify sorry it wasn't clear enough, am already working on this...
Bastien Montagne commented 2022-07-01 12:26:45 +02:00
Copy Link

Changed status from 'Confirmed' to: 'Resolved'

Changed status from 'Confirmed' to: 'Resolved'
Bastien Montagne commented 2022-07-01 12:26:45 +02:00
Copy Link

This should now be resolved for current releases (2.93 LTS, 3.2 and future 3.3).

This should now be resolved for current releases (2.93 LTS, 3.2 and future 3.3).
This repo is archived. You cannot comment on issues.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Priority::Low
Priority::Normal
Priority::Unbreak Now!
Status::Archived
Status::Confirmed
Status::Duplicate
Status::Needs Information from Developers
Status::Needs Triage
Status::Resolved
Type::Bug
Type::Report
Type::To Do
legacy module
Development Management
legacy module
Platforms, Builds, Tests & Devices
legacy module
Python API
legacy module
Rendering & Cycles
legacy project
3.3
legacy project
Development Management
legacy project
Documentation
legacy project
Infrastructure: Blender Buildbot
legacy project
Infrastructure: Websites
legacy project
Platform: Windows
legacy project
Platform: macOS
legacy project
Platforms, Builds, Tests & Devices
legacy project
Python API
legacy project
Render & Cycles
No Label Priority::Normal Status::Resolved Type::To Do
legacy module
Platforms, Builds, Tests & Devices
legacy module
Python API
legacy project
Documentation
legacy project
Infrastructure: Blender Buildbot
legacy project
Platforms, Builds, Tests & Devices
legacy project
Python API
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
blender-admin
No Assignees Bastien Montagne
8 Participants
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: archive/blender-buildbot#97663
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?