Tools: Weekly Report Generator #110652

Germano Cavalcante · 2023-07-31T15:23:40+02:00

Germano Cavalcante commented

2023-07-31 15:23:40 +02:00

This commit implements 2 triage tools:

/tools/triage/weekly_report.py
/tools/triage/issues_needing_info.py

These tools automatically detect the username to list activities related to the user.

Solutions:

weekly_report.py: this is a report generator that mimics the old one seen in https://projects.blender.org/blender/blender-dev-tools/src/branch/blender-v3.4-release/utils/weekly_report.py. This tool uses the Gitea API that lists a user's activities. Once the Issues and Pull involved for the week are obtained, each of their links are accessed to see the events that do not appear in the activity feed (such as applying labels). With these labels it is possible to know why an issue was closed for example. The downside of this solution is that issues that the triager has tagged as confirmed or needing info but not commented on are not tracked.
issues_needing_info.py: this tool prints the links of issues that the user marked as needing information at least 1 week ago. It uses the Gitea API to first list all issues marked as needing information and updated more than a week ago, and then accesses the events for each one to filter them based on user.

Each of these files access the gitea_utils.py modules.

This commit implements 2 triage tools: - /tools/triage/weekly_report.py - /tools/triage/issues_needing_info.py These tools automatically detect the username to list activities related to the user. --- **Solutions:** - `weekly_report.py`: this is a report generator that mimics the old one seen in https://projects.blender.org/blender/blender-dev-tools/src/branch/blender-v3.4-release/utils/weekly_report.py. This tool uses the Gitea API that lists a user's activities. Once the Issues and Pull involved for the week are obtained, each of their links are accessed to see the events that do not appear in the activity feed (such as applying labels). With these labels it is possible to know why an issue was closed for example. The downside of this solution is that issues that the triager has tagged as confirmed or needing info but not commented on are not tracked. - `issues_needing_info.py`: this tool prints the links of issues that the user marked as needing information at least 1 week ago. It uses the Gitea API to first list all issues marked as needing information and updated more than a week ago, and then accesses the events for each one to filter them based on user. Each of these files access the `gitea_utils.py` modules.

Germano Cavalcante added the

Module

Triaging

label 2023-07-31 15:23:40 +02:00

Germano Cavalcante requested review from Brecht Van Lommel 2023-07-31 15:24:40 +02:00

Germano Cavalcante requested review from Philipp Oeser 2023-07-31 15:24:41 +02:00

Germano Cavalcante requested review from Arnd Marijnissen 2023-07-31 15:24:41 +02:00

Miguel Pozo added a new dependency 2023-07-31 15:41:59 +02:00

#110653 EEVEE Next: Volumes: Lighting integration improvements

Miguel Pozo removed a dependency 2023-07-31 15:42:06 +02:00

#110653 EEVEE Next: Volumes: Lighting integration improvements

Brecht Van Lommel commented

2023-08-01 15:03:16 +02:00

Is the idea that there will be just one script, and the question is which one is preferred? If not at least there should be a more informative name than v2.

I don't use these scripts, so maybe ask for feedback from @lichtwerk and others that are using these.

The purpose of the shell scripts is not clear to me, why not run the python file directly like other scripts in tools?

Is the idea that there will be just one script, and the question is which one is preferred? If not at least there should be a more informative name than v2. I don't use these scripts, so maybe ask for feedback from @lichtwerk and others that are using these. The purpose of the shell scripts is not clear to me, why not run the python file directly like other scripts in tools?

Germano Cavalcante commented

2023-08-01 17:24:55 +02:00

In fact the shell scripts are not needed here. (I tend to use it a lot in other cases and ended up forgetting that it's not necessary here).

Removed weekly_report_v2.py and shell scripts
Commits are now retrieved from the activity feed itself, this has resulted in a huge simplification of the script as git directories are no longer needed (except for getting the username)
Removed git_utils.py (Created a git_username_detect function in weekly_report.py itself)

In fact the shell scripts are not needed here. (I tend to use it a lot in other cases and ended up forgetting that it's not necessary here). - Removed `weekly_report_v2.py` and shell scripts - Commits are now retrieved from the activity feed itself, this has resulted in a huge simplification of the script as git directories are no longer needed (except for getting the username) - Removed `git_utils.py` (Created a `git_username_detect` function in `weekly_report.py` itself)

Brecht Van Lommel requested changes 2023-08-01 18:58:11 +02:00

Brecht Van Lommel left a comment

Nice simplification.

I only have minor comments on the implementation.

Nice simplification. I only have minor comments on the implementation.

tools/triage/gitea_utils.py Outdated

						
				@ -0,0 +7,4 @@

				def url_json_get(url):

				    import json

Brecht Van Lommel commented

2023-08-01 18:46:02 +02:00

These kinds of lazy imports are not needed in scripts like this. We do it in Blender for faster startup, but here it's not a factor.

mano-wii marked this conversation as resolved

tools/triage/gitea_utils.py Outdated

						
				@ -0,0 +201,4 @@

				            result.append(event)

				        if len(issue_events_json) < 50:

Brecht Van Lommel commented

2023-08-01 18:52:00 +02:00

This assumes the limit will always be 50, which I don't think is a guarantee.

What you could do is use &page={page}&limit={limit} above to specifically request this limit.

This assumes the limit will always be 50, which I don't think is a guarantee. What you could do is use `&page={page}&limit={limit}` above to specifically request this limit.

mano-wii marked this conversation as resolved

tools/triage/issues_needing_info.py Outdated

						
				@ -0,0 +4,4 @@

				"""

				# This script prints the URLs of all opened issues labeled

				# "Status/Needs Information from User" by a specified user

				# before a specified date.

Brecht Van Lommel commented

2023-08-01 18:56:06 +02:00

Might as well say 7 days since there is no argument to specify the date.

mano-wii marked this conversation as resolved

tools/triage/issues_needing_info.py Outdated

						
				@ -0,0 +56,4 @@

				    args = parser.parse_args()

				    username = args.username

				    if not username:

				        from weekly_report import git_username_detect

Brecht Van Lommel commented

2023-08-01 18:54:31 +02:00

Perhaps would make more sense to add this in gitea_utils since a dependency on the weekly reports script is not so obvious.

Perhaps would make more sense to add this in `gitea_utils` since a dependency on the weekly reports script is not so obvious.

mano-wii marked this conversation as resolved

tools/triage/weekly_report.py Outdated

						
				@ -0,0 +266,4 @@

				        start_date = end_date - datetime.timedelta(days=time_delta, hours=end_date.hour)

				    # Ensure friday :)

				    firday = start_date + datetime.timedelta(days=4)

Brecht Van Lommel commented

2023-08-01 18:32:57 +02:00

Typo: firday -> friday

mano-wii marked this conversation as resolved

Germano Cavalcante commented

2023-08-01 21:25:26 +02:00

Thanks for the feedback @brecht

In addition to what was suggested, I also created the url_json_get_all_pages function that uses the limit param.
Unfortunately in some cases the page and limit parameters fail when the page is 1. So they could not be used.

I also left the imports os and subprocess inside the git_username_detect function, as this function, besides not being a Gitea utility, is a bit of a workaround and therefore is treated differently.

Thanks for the feedback @brecht In addition to what was suggested, I also created the `url_json_get_all_pages` function that uses the `limit` param. Unfortunately in some cases the `page` and `limit` parameters fail when the page is 1. So they could not be used. I also left the imports `os` and `subprocess` inside the `git_username_detect` function, as this function, besides not being a Gitea utility, is a bit of a workaround and therefore is treated differently.

Brecht Van Lommel approved these changes 2023-08-01 22:41:04 +02:00

tools/triage/gitea_utils.py Outdated

						
				@ -0,0 +190,4 @@

				# WORKAROUND: This function doesn't involve Gitea, and the obtained username may not match the username used in Gitea.

				# However, it provides an option to fetch the configured username from the local Git,

				# in case the user does not explicitly supply the username.

				# Please note that this approach may not be suitable for all scenarios and might not be secure in some cases.

Brecht Van Lommel commented

2023-08-01 22:41:01 +02:00

I'm not sure what is insecure about this? Would leave that out unless there is a good reason.

mano-wii marked this conversation as resolved

Germano Cavalcante force-pushed weekly_report_generator from 2bec691eec to 6e98c3e64b

2023-08-04 19:26:53 +02:00

Compare

Germano Cavalcante merged commit b6ed70cd92 into main

2023-08-04 19:32:31 +02:00

Germano Cavalcante deleted branch weekly_report_generator

2023-08-04 19:32:33 +02:00

Germano Cavalcante referenced this issue from a commit

2023-08-04 19:32:33 +02:00

Triage: Add 'weekly_report' and 'issues_needing_info' tools

Philipp Oeser commented

2023-08-07 09:42:48 +02:00

Thx working on this! (wasnt able to review because I was on holidays)

I will add --calendarweek as an arg (as we discussed) and put up a PR for that

Thx working on this! (wasnt able to review because I was on holidays) I will add `--calendarweek` as an arg (as we discussed) and put up a PR for that

Sign in to join this conversation.

No reviewers

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Tools: Weekly Report Generator #110652