This information is accurate as of the 9th of August 2024

This README will cover the general workings and limitations of this tool.

---

The goal of this script is to be able to get a list of fix commits that occured in the current release that fixed issues in previous releases. For example, if a issue was in 4.1 and was fixed in 4.2, then we want to know about it so we can add it to the release notes. But if a issues was introduced in 4.2 and fixed in 4.2, then we don't care about it.

Philipp requested this as some groups tend to focus on the fix list rather than the new features list of a new release. And Blender has been lacking this for many years now.

---

How to use the script

• Open up the Python script in a code editor,
• Update the variables in the file parameters.py
• Open a command prompt or terminal in your Blender source code folder (The script assumes it's being run from there) and make sure your branches are up to date.
• Run the python script main.py.
• Wait for the script to finish (For a full release, this usually takes ~25 minutes).
• Go through the commits in the Commits that need manual review: section, go to the report they fixed, and update the "Broken" and "Working" fields with relevant information.

---

How the script works

Step 1: Gather a list of fix commits that occured between the previous release (4.1) and the current release (4.2).

Fix commits are commits that contain Fix #NUMBER in them (Other variations like "Fixes #NUMBER" and "Fixes RANDOM WORDS #NUMBER" are also accepted).

This is done by calling this command in the Blender source folder:

git --no-pager log v4.1.1..v4.2.0 --oneline -P -i --grep "Fix.*#+\d+"

Step 2: Sort the commits into different catagories. The catagories are:

• Commits that fixed old issues (What we're interested in)
• Commits that did not fix an old issue (What we want to discard)
• Commits that need manual checking (The script couldn't automatically sort them into the other catagories)
• Revert commits (Revert commits need manual sorting)

The process for sorting commits follows this flow:

• Is there "Revert" in the commit title? If yes, add to the revert list and continue onto the next commit.
• If it wasn't a revert commit, we then check if the bug report that was fixed has enough information in it to allow us to properly sort it into the other catagories. This is done by:
  • Extract the fixed report number from the commit message (E.g. Extract 123 from Fix #123).
  • Gather information about the report. The information currently used is "Report creation date" and "Report body".
  • "Was the report created before development of the current release started?" (E.g. The report was made in 2023 when development of the current release started in 2024). If yes, add to the Commits that fixed old issues list, otherwise do more checks.
  • If the commit wasn't sorted by date, then we need to sort it based on the information in the report. This is done by looking at the version numbers in the "Broken" and "Worked" fields.
    • If "Broken" is older than the current release, then add the commit to Commits that fixed old issues list.
      • This issue was in a older release so it should go in the Commits that fixed old issues.
    • If "Broken" and "Worked" are both the current release, then add the commit to Commits that did not fix an old issue list.
      • It was working in a earlier version of the current release, then broke at some point. This means the issue was introduced in this release.
    • If "Broken" is the current release and "Worked" is the previous release, then add the commit to Commits that did not fix an old issue list.
      • It worked in the previous release and is broken now, so it must of broken in this release.
    • For all other cases, add to the Manual checking list.
    • There are systems in place to accurately handle multiple version numbers in the "Broken" and "Worked" fields.

Step 3: Display the sorted commits to the user of the script.

---

Limitations and edge cases

The script has a few limitations and edge cases. Here are a list of the currently known ones:

• Revert commits could be handled automatically, but they aren't.
  • A revert commit tells you what commit was reverted. We should use that information to remove reverted commits from the "Fix list" as those fixes no longer exist.
• Some fix commits are missed.
  • At the moment we look for commits that fix a bug report (By checking for Fix #NUMBER). There are many other fixes that we are missing because of this (E.g. Fix: Bug in X feature)
    • Philipp suggested we don't bother looking for fix commits that don't have an associated report.
• Commits that have the word "Fix" then the fixed issue number on a new line are not detected due to the regular expression argument I used to find the Fix commits not running across new lines (And if it did run across new lines, then it will pick up any fix commit with a Pull Request: #NUMBER as a fix commit. Work could be done to try and avoid this).
• The system that checks the Broken and Working versions in the bug report has various issues. Here are a list of the ones I can remember:
  • Version number is extracted from the Broken and Working fields using regular expression looking for NUMBER.NUMBER. If the version number is not a valid Blender version, it is ignored. This can lead to the issues below.
  • Non Blender version numbers can be picked up, for example the date (2024.08.09). Most of these are filtered out, but some may accidentally match a valid Blender version number which can lead to a false positive/negative in the sorting process.
  • The version number extracter doesn't look at context, leading to the extraction of some wrong version numbers. E.g. Worked: I don't know, I tried 4.1 to 3.6 and they were all broken. In this example, the script tags 4.1 and 3.6 as working versions, which isn't true.
  • Sometimes the information about working and broken versions aren't in the "Broken" and "Working" fields, but in other places like the Short Description and comments. These are ignored as they are too dificult to filter.

---

What can the triaging team do to help the script?

Make sure the information in the "Broken" and "Worked" fields is accurate and remove any forgien version numbers (Anything that's NUMBER.NUMBER and not a Blender version. Examples include dates, or version numbers for other pieces of software, or the version number added to the wrong field (E.g. A broken version in the working field)).

Examples of fixes triaging team can make.

This all assumes that triaging is occuring during Blender 4.2 development and the script is being used to find the fixes we're interested in from 4.2.

Current release regression

There was a bug introduced in 4.2 and someone makes a bug report

Broken: 4.2.0 Alpha, ...
Worked: Newest version that worked.

If you tested the previous release (4.1) and find the issue isn't there, then update the report with Worked: 4.1 Or if you bisected it, add Worked: 4.2.0 Alpha...

Issue from previous version

There was a bug in a older version of Blender (E.g. 3.6), but it was reported with the current indevelopment version.

Broken: 4.2.0 Beta, ...
Worked: 3.5, ...

Add a older broken version to the broken list so the script can tell is was broken in a older version:

Broken: 4.2.0 Beta, ...
Borken: 4.1
Worked: 3.5, ...

Wrong number in the version field

Someone made a bug report and put the wrong version number in the relevant field.

Broken: 4.2.0 Alpha...
Worked: I don't know, I tried 4.1 to 3.6 and they were all broken.

Shift all broken versions to the broken field

Broken: 4.2.0 Alpha, ...
Broken: 4.1
Broken: 3.6
Worked: Unsure

Bug for a new feature

A new feature was added in 4.2, and the bug has been present ever since the feature was added.

Broken: 4.2.0 Alpha, ...
Worked: 4.1 as this feature wasn't in that version of Blender.

Or add a older version of 4.2 prior to the new feature to the working field.

Broken: 4.2.0 Alpha, ...
Worked: 4.2.0 Alpha, ... as the feature did not exist then.

And probably various other situations

---

Noticed quirks/TODOs

While using the script I noticed these quirks:

• Sometimes the script extracts a pull request as a "Fixed bug". Not sure if this is a issue with the script, or the commit message of the offending commits. Will need to investigate.
• I've primarily tested with the 4.1.1 -> 4.2.0 range of commits. Sometimes commits made in 4.2.0, then backported to 4.1.1 will appear as "A commit only made in 4.2.0". I'm not sure why this happens, but I might need to investigate.
• Philipp mentioned that we may want to add to the list something like this:
  • Commit hash - Commit title - Fixed in 4.2 and backported to 4.1.1.
  • The script currently does not have a system to handle these cases and I believe it's better to be done manually (Open the 4.1.1 backport task and just grab all the fix commits from there.)