Move Blender UI translation repository from SVN to git/gitea #65

New Issue

Bastien Montagne · 2023-07-17T17:19:27+02:00

Bastien Montagne commented

2023-07-17 17:19:27 +02:00

Once we have a running instance of weblate in production, the current Blender SVN repository for the interface translation needs to be migrated to a git one.

Steps

Lock (read-only) SVN repository.
Import the SVN repository into a git one.
Update the translation tools.
Update related wiki pages.
Create project in weblate.
Communicate (chat + devtalk) about changes and new procedure to translate UI.

Import the SVN repository into a git one

Only the branches part needs to be imported.
The trunk is only automatically cleaned-up and compiled snapshots the the branches, neither it's content nor its history need to be kept.
Release tags status is unclear? Very long time ago Blender builds directly used the SVN repo for their translations... For now they will be ignored too.
Only a few admins, and the weblate website, will have commit access to this repository. See the Weblate section below for details.
Here are the commands used for the conversion, using git-svn and git-filter-repo:
- New repository name: blender-ui-translations.
- git svn clone -T https://svn.blender.org/svnroot/bf-translations/branches --ignore-paths=".*.mo$" https://svn.blender.org/svnroot/bf-translations
- git branch -m master main
- git filter-repo --force --mailmap=<path_to_remapping_file>
- Add README.md file.
- Rename uz to uz@latin (weblate considers uz as the Cyrillic version of Uzbek). !This also requires update to Blender repo, once rolled into production!
- Rename zh_CN to zh_HANS (weblate does not recognize this deprecated code). !This also requires update to Blender repo, once rolled into production!
- Rename zh_TW to zh_HANT (weblate does not recognize this deprecated code). !This also requires update to Blender repo, once rolled into production!
- Remove es_ES (has been deprecated for years, only active and maintained Spanish translation is es, and they conflict in weblate).
- Add license statement to all files: This file is distributed under the same license as the Blender package.

Here is a first test here: https://projects.blender.org/mont29/wip_ui_translations

WIP README.md file:

<!--
Keep this document short & concise,
linking to external resources instead of including content in-line.
-->

Blender UI Translation
======================

This repository contains the raw Blender UI translation files, using the GNUGettext format.

The translations used by Blender builds are a compacted and cleaned-up version of these files, stored directly in the [main Blender repository](https://projects.blender.org/blender/blender/src/branch/main/locale).

Project Pages
-------------

- [Blender Website](http://www.blender.org)
- [How To Translate Blender](https://wiki.blender.org/wiki/Process/Translate_Blender)

Development
-----------

- [Handling i18N in Blender code](https://wiki.blender.org/wiki/Source/Interface/Internationalization)

License
-------

The files in this repository are distributed under the same license as the Blender package.

2. Update the translation tools

The translation add-on needs to be updated for the new process and file layout.
- Update add-on part updating the translation repository, and the main blender repository.
- Updating or removing the 'translate from Blender UI' feature? Should not rely on a translation repo anymore, but either on single PO files, and/or using weblate API?

WIP branch: https://projects.blender.org/mont29/blender-addons/src/branch/tmp-ui-translate

3. Weblate

The conversion to Git will be done together with publishing a weblate website. The goal is then that all translations happens through that website, and translators do not have to deal directly with the underlying versioning system (the git repository).

Translators can create an account on the weblate site through their BlenderID, just like on gitea.
Proposed roles/teams:
- Guests (non-logged-in users) can see translations, and suggest edits.
- By default new users will have the same access and edit rights as guests.
- Users can be added to the Translators team. Translators can freely edit translations and manage the glossary, for both UI and Manual.
Using the weblate UI for translation is not mandatory, if they prefer translators can download a whole language's PO file, and upload it back later.
Weblate will have commit access to the ui translations git repository. Note that weblate tries to pack multiple consecutive edits from a single author into a single commit, so it won't be spamming git history with thousands of commits.
Update of the ui translations git from Blender source will remain manual for the time being.
Update of the main Blender repository from the ui translations git will also remain manual for the time being.

Tweak UI of weblate.
Investigate the TimeOut issue (for now timeout setting on the server has been extended to 10minutes).
Check warnings about configuration with IT.
Create Blender UI project.
- Add weblate as committer to the UI translation git repository.
- Use 'raw' GIT type of projects.
Create roles/teams:
- Guests (non-logged-in, can see translations, and suggest edits).
- Users (logged-in, same access and edit rights as Guests).
- Translators (logged-in, can edit translations and manage the glossary, for both UI and Manual).
Set-up website link, add short instruction text (mainly links to wiki pages).
Set-up 'source browser' link: https://projects.blender.org/blender/blender/src/branch/main/{{filename}}#L{{line}}
Define license for the project (GPL v3, see below).

4. License

Currently translation files (usually) have proper copyright/attribution information, but no license information at all.

All translation files in the repository will get an extra line in their header:
This file is distributed under the same license as the Blender package.
This will effectively make the whole translation licensed as GPL v3.

See also this discussion.

Once we have a running instance of weblate in production, the current Blender [SVN repository](https://svn.blender.org/svnroot/bf-translations/) for the interface translation needs to be migrated to a git one. ## Steps - [x] Lock (read-only) SVN repository. - [x] Import the SVN repository into a git one. - [x] Update the translation tools. - [x] Update related wiki pages. - [x] Create project in weblate. - [x] Communicate (chat + devtalk) about changes and new procedure to translate UI. ### Import the SVN repository into a git one * Only the `branches` part needs to be imported. * The `trunk` is only automatically cleaned-up and compiled snapshots the the branches, neither it's content nor its history need to be kept. * Release tags status is unclear? Very long time ago Blender builds directly used the SVN repo for their translations... For now they will be ignored too. * **Only a few admins, and the weblate website, will have commit access to this repository.** See the [Weblate](#3-weblate) section below for details. * Here are the commands used for the conversion, using `git-svn` and `git-filter-repo`: - [x] New repository name: `blender-ui-translations`. - [x] `git svn clone -T https://svn.blender.org/svnroot/bf-translations/branches --ignore-paths=".*.mo$" https://svn.blender.org/svnroot/bf-translations` - [x] `git branch -m master main` - [x] `git filter-repo --force --mailmap=<path_to_remapping_file>` - [x] Add `README.md` file. - [x] Rename `uz` to `uz@latin` *(weblate considers `uz` as the Cyrillic version of Uzbek)*. **!This also requires update to Blender repo, once rolled into production!** - [x] Rename `zh_CN` to `zh_HANS` *(weblate does not recognize this deprecated code)*. **!This also requires update to Blender repo, once rolled into production!** - [x] Rename `zh_TW` to `zh_HANT` *(weblate does not recognize this deprecated code)*. **!This also requires update to Blender repo, once rolled into production!** - [x] Remove `es_ES` *(has been deprecated for years, only active and maintained Spanish translation is `es`, and they conflict in weblate)*. - [x] Add license statement to all files: `This file is distributed under the same license as the Blender package.` Here is a first test here: https://projects.blender.org/mont29/wip_ui_translations WIP README.md file: ```  Blender UI Translation ====================== This repository contains the raw Blender UI translation files, using the GNUGettext format. The translations used by Blender builds are a compacted and cleaned-up version of these files, stored directly in the [main Blender repository](https://projects.blender.org/blender/blender/src/branch/main/locale). Project Pages ------------- - [Blender Website](http://www.blender.org) - [How To Translate Blender](https://wiki.blender.org/wiki/Process/Translate_Blender) Development ----------- - [Handling i18N in Blender code](https://wiki.blender.org/wiki/Source/Interface/Internationalization) License ------- The files in this repository are distributed under the same license as the Blender package. ``` ### 2. Update the translation tools * [The translation add-on](https://projects.blender.org/blender/blender-addons/src/branch/main/ui_translate) needs to be updated for the new process and file layout. - [x] Update add-on part updating the translation repository, and the main blender repository. - Updating or removing the 'translate from Blender UI' feature? Should not rely on a translation repo anymore, but either on single PO files, and/or using weblate API? WIP branch: https://projects.blender.org/mont29/blender-addons/src/branch/tmp-ui-translate ### 3. Weblate The conversion to Git will be done together with publishing a weblate website. The goal is then that all translations happens through that website, and translators do not have to deal directly with the underlying versioning system (the git repository). * Translators can create an account on the weblate site through their BlenderID, just like on gitea. * Proposed roles/teams: * Guests (non-logged-in users) can see translations, and suggest edits. * By default new users will have the same access and edit rights as guests. * Users can be added to the Translators team. Translators can freely edit translations and manage the glossary, for both UI and Manual. * Using the weblate UI for translation is not mandatory, if they prefer translators can download a whole language's PO file, and upload it back later. * Weblate will have commit access to the ui translations git repository. *Note that weblate tries to pack multiple consecutive edits from a single author into a single commit, so it won't be spamming git history with thousands of commits.* * Update of the ui translations git from Blender source will remain manual for the time being. * Update of the main Blender repository from the ui translations git will also remain manual for the time being. - [x] Tweak UI of weblate. - [x] Investigate the TimeOut issue _(for now timeout setting on the server has been extended to 10minutes)_. - [x] Check warnings about configuration with IT. - [x] Create `Blender UI` project. - [x] Add weblate as committer to the UI translation git repository. - [x] Use 'raw' GIT type of projects. - [x] Create roles/teams: - [x] Guests (non-logged-in, can see translations, and suggest edits). - [x] Users (logged-in, same access and edit rights as Guests). - [x] Translators (logged-in, can edit translations and manage the glossary, for both UI and Manual). - [x] Set-up website link, add short instruction text (mainly links to wiki pages). - [x] Set-up 'source browser' link: `https://projects.blender.org/blender/blender/src/branch/main/{{filename}}#L{{line}}` - [x] Define license for the project (GPL v3, see below). ### 4. License Currently translation files (usually) have proper copyright/attribution information, but no license information at all. All translation files in the repository will get an extra line in their header: `This file is distributed under the same license as the Blender package.` This will effectively make the whole translation licensed as GPL v3. See also [this discussion](https://devtalk.blender.org/t/proposed-license-for-blenders-po-ui-translation-files/30440).

👍 1 🎉 1

Arnd Marijnissen was assigned by Bastien Montagne

2023-07-17 17:19:27 +02:00

Francesco Siddi was assigned by Bastien Montagne

2023-07-17 17:19:27 +02:00

Bastien Montagne referenced this issue from a commit

2023-07-18 18:29:23 +02:00

UITranslate: Initial refactor/changes for upcoming repositories changes.

Damien Picard commented

2023-07-20 23:13:15 +02:00

You might be interested in this comment, as I had a look before at converting the SVN repo to git. Particularly the mailmap part.

About the licensing, I don’t know if it’s generally possible to attach a new license to an existing work, except with authorization from all authors.
I find CC-BY-ND a poor choice of license, as derivatives are an integral part of free software and it would not allow actual contribution by individual translators. AFAIK the usual license choice for free software projects’ localization is the same as the localized software, so that would be the GPL v2 or later for Blender. It may also allow us to attach this license without getting approval from each translator, as it can be argued that using the same license as the rest of Blender is kind of implied.

You might be interested in [this comment](https://projects.blender.org/blender/blender-manual/issues/68588#issuecomment-6250), as I had a look before at converting the SVN repo to git. Particularly the mailmap part. About the licensing, I don’t know if it’s generally possible to attach a new license to an existing work, except with authorization from all authors. I find CC-BY-ND a poor choice of license, as derivatives are an integral part of free software and it would not allow actual contribution by individual translators. AFAIK the usual license choice for free software projects’ localization is the same as the localized software, so that would be the GPL v2 or later for Blender. It may also allow us to attach this license without getting approval from each translator, as it can be argued that using the same license as the rest of Blender is kind of implied.

Bastien Montagne commented

2023-07-21 11:03:37 +02:00

Eeeek, I meant CC-BY-NC, not ND, indeed no derivative does not really makes any sense here. Will fix the original post.

I find it a bit weird to use GPL or other code license on PO files, since these are really not code at all, just data... But I do not have any expertise in the area for sure. I just think it would be time to add an official license to this work. ;)

I did not follow recent updates to #68588, as I thought this was (too) old and not active any more. My own experiment is similar to yours, although I would not change the current layout of /branches unless there is a very good reason to (among other reasons, because I think it's nicer for each language to have their own 'working directory' where the team can freely put whatever extra material they may need).

Am also working on a mapping file, but if you can post here yours that could save me some time indeed!

Eeeek, I meant CC-BY-NC, not ND, indeed no derivative does not really makes any sense here. Will fix the original post. I find it a bit weird to use GPL or other code license on PO files, since these are really not code at all, just data... But I do not have any expertise in the area for sure. I just think it would be time to add an official license to this work. ;) I did not follow recent updates to #68588, as I thought this was (too) old and not active any more. My own experiment is similar to yours, although I would not change the current layout of /branches unless there is a very good reason to (among other reasons, because I think it's nicer for each language to have their own 'working directory' where the team can freely put whatever extra material they may need). Am also working on a mapping file, but if you can post here yours that could save me some time indeed!

Damien Picard commented

2023-07-21 17:32:46 +02:00

I meant CC-BY-NC, not ND

I’m pretty sure CC-BY-NC is not compatible with the GPL either because the Non-Commercial clause makes it a non-free license (freedom 0: The freedom to run the program as you wish, for any purpose). See for instance this page from the GNU project.

CC-BY on the other hand might be a candidate, as version 4 is explicitly compatible with GPLv3 but this use is discourage by Creative Commons themselves.

I find it a bit weird to use GPL or other code license on PO files, since these are really not code at all, just data...

My understanding is that the translations need to be compiled to machine code (.mo files), and are thus actually pretty much source code. They are linked into the program and need to be compatible with the licence of the composite work. This is even in the default template on execution of xgettext: This file is distributed under the same license as the %s package.

I just think it would be time to add an official license to this work. ;)

Agreed!

Am also working on a mapping file, but if you can post here yours that could save me some time indeed!

(The mailmap was sent through another channel.)

> I meant CC-BY-NC, not ND I’m pretty sure CC-BY-NC is not compatible with the GPL either because the Non-Commercial clause makes it a non-free license (freedom 0: The freedom to run the program as you wish, for any purpose). See for instance [this page ](https://www.gnu.org/licenses/license-list.html#CC-BY-NC) from the GNU project. CC-BY on the other hand _might_ be a candidate, as version 4 is [explicitly compatible with GPLv3](https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses) but this use is [discourage by Creative Commons](https://creativecommons.org/faq/#can-i-apply-a-creative-commons-license-to-software) themselves. > I find it a bit weird to use GPL or other code license on PO files, since these are really not code at all, just data... My understanding is that the translations need to be compiled to machine code (.mo files), and are thus actually pretty much source code. They are linked into the program and need to be compatible with the licence of the composite work. This is even in the default template on execution of `xgettext`: `This file is distributed under the same license as the %s package.` > I just think it would be time to add an official license to this work. ;) Agreed! > Am also working on a mapping file, but if you can post here yours that could save me some time indeed! (The mailmap was sent through another channel.)

Bastien Montagne commented

2023-07-25 11:32:18 +02:00

I meant CC-BY-NC, not ND

I’m pretty sure CC-BY-NC is not compatible with the GPL either because the Non-Commercial clause makes it a non-free license (freedom 0: The freedom to run the program as you wish, for any purpose). See for instance this page from the GNU project.

CC-BY on the other hand might be a candidate, as version 4 is explicitly compatible with GPLv3 but this use is discourage by Creative Commons themselves.

After discussion with Ton and some admins here at Blender HQ, we decided to propose to use the CC0 license, see https://devtalk.blender.org/t/proposed-license-for-blenders-po-ui-translation-files/30440.

I find it a bit weird to use GPL or other code license on PO files, since these are really not code at all, just data...

My understanding is that the translations need to be compiled to machine code (.mo files), and are thus actually pretty much source code. They are linked into the program and need to be compatible with the licence of the composite work. This is even in the default template on execution of xgettext: This file is distributed under the same license as the %s package.

No, PO (and MO) are pure data, they do not contain any form of logic that could make them code or executables. MOs are essentially like a mapping (python dictionary), or a very basic form of data-base. In that sense, compiling' a MO into a PO is exactly as converting an ASCII text FBX file into a binary one.

Further more, they are not linked into Blender, the MO files are distributed with Blender, and opened like any other data file during its execution to extract their translations.

Am also working on a mapping file, but if you can post here yours that could save me some time indeed!

(The mailmap was sent through another channel.)

Thanks!

> > I meant CC-BY-NC, not ND > > I’m pretty sure CC-BY-NC is not compatible with the GPL either because the Non-Commercial clause makes it a non-free license (freedom 0: The freedom to run the program as you wish, for any purpose). See for instance [this page ](https://www.gnu.org/licenses/license-list.html#CC-BY-NC) from the GNU project. > > CC-BY on the other hand _might_ be a candidate, as version 4 is [explicitly compatible with GPLv3](https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses) but this use is [discourage by Creative Commons](https://creativecommons.org/faq/#can-i-apply-a-creative-commons-license-to-software) themselves. After discussion with Ton and some admins here at Blender HQ, we decided to propose to use the CC0 license, see https://devtalk.blender.org/t/proposed-license-for-blenders-po-ui-translation-files/30440. > > I find it a bit weird to use GPL or other code license on PO files, since these are really not code at all, just data... > > My understanding is that the translations need to be compiled to machine code (.mo files), and are thus actually pretty much source code. They are linked into the program and need to be compatible with the licence of the composite work. This is even in the default template on execution of `xgettext`: `This file is distributed under the same license as the %s package.` No, PO (and MO) are pure data, they do not contain any form of logic that could make them code or executables. MOs are essentially like a mapping (python dictionary), or a very basic form of data-base. In that sense, compiling' a MO into a PO is exactly as converting an ASCII text FBX file into a binary one. Further more, they are not linked into Blender, the MO files are distributed with Blender, and opened like any other data file during its execution to extract their translations. > > Am also working on a mapping file, but if you can post here yours that could save me some time indeed! > > (The mailmap was sent through another channel.) Thanks!

Damien Picard commented

2023-07-25 11:57:37 +02:00

After discussion with Ton and some admins here at Blender HQ, we decided to propose to use the CC0 license

For what it’s worth I think it’s a good decision :)

No, PO (and MO) are pure data, they do not contain any form of logic that could make them code or executables.

I didn’t mean to start a discussion about code vs data, just stating what I think is the rationale behind this practice in free software translation, but I may be wrong.

> After discussion with Ton and some admins here at Blender HQ, we decided to propose to use the CC0 license For what it’s worth I think it’s a good decision :) > No, PO (and MO) are pure data, they do not contain any form of logic that could make them code or executables. I didn’t mean to start a discussion about code vs data, just stating what I think is the rationale behind this practice in free software translation, but I may be wrong.

👍 1

Bastien Montagne commented

2023-08-15 10:35:05 +02:00

Updated the licensing part, whole translation repo will be GPL v3, see also https://devtalk.blender.org/t/proposed-license-for-blenders-po-ui-translation-files/30440.