Gitea: User manual Git migration and online editing #51
Labels
No Label
Service
Buildbot
Service
Chat
Service
Gitea
Service
Translate
Type
Bug
Type
Config
Type
Deployment
Type
Feature
Type
Setup
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: infrastructure/blender-projects-platform#51
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Contributing to the user manual is currently quite technical, and requires using subversion. The first step will be moving the manual to a Git repository with LFS. This already will allow some level of online editing.
Git migration:
Online editing:
blender-merged
CC @blendify
I think keeping everything on Gitea would be sufficient. Adding more would just complicate the process in my opinion.
Adding the rst preview would be a nice to have though.
I also didn't find a good candidate for an alternative online editor, so perhaps it's the best we can do for now.
For rst preview, Gitea needs a command to convert rst to html. The basic pandoc command from the docs generates would have wrong URLs for images,
:doc:
links are not generated, and probably various other sphinx features are not supported. Pandoc has various customization options so perhaps there is a way to improve this.But perhaps better, a script could be written that uses sphinx to generate html from a single rst file? Not sure if there is a convenient way to do it. But basic working links and images would be nice even if they can't be verified to be correct.
The other approach of course would be to convert the manual to markdown or support both rst and md pages. That would likely help attract contributors and developers, but it's also a complicated change that involves a lot work and trade-offs.
This is possible with the command:
sphinx-build -b html -j auto manual build/html <path_to_rst_file>
I would avoid this, like you said there are many trade offs and rst is much more powerful than markdown and leaves a lot to be desired.
The issue is that it gets the contents of a single rst file and nothing else from the repository. That command assumes it has the whole repository available.
Making a checkout of the full repository would be too slow for to do for every preview. Unless perhaps some clever caching mechanism is implemented, but that seems complicated.
For security we'd also need to ensure we're not executing custom Python code written by users in the conf.py in their fork of the repository.
Links and images can still work even without the full repository as long as we set them to the right URL. However sphinx wants to validate that the files exist, which is not helpful in this case and maybe we need some way to generate the links without validation.
I managed to hack together something to render a single page:
https://projects.blender.org/brecht/gitea-custom/src/branch/sphinx/sphinx
https://github.com/go-gitea/gitea/pull/23333
But this will need changes to ensure security before this can be deployed. Probably running as an isolated user or a web service.
And links are missing due to other pages not being available.
I made
:doc:
links work.:ref:
and:term:
can't work based on just a single RST files so those links don't lead anywhere for now.Security should also be much better now though there are some specifics to be figured out for deployment.
I did a few more fixes to improve the preview, based on testing with the actual manual.
For reference this is how I generated the Git repo.
I had to also install
git-filter-repo
.User manual online editingto User manual Git migration and online editing@Blendify @Arnd. I've created a test repo here, which also includes branches, tags and commit authors that match user accounts. See the attached scripts and mailmap for how it was done.
https://projects.blender.org/brecht/blender-manual
What I could not get working is nice merge commits, not sure it is even supported by
svn2git
. It just showsMerge branch 'blender-3.5-release'
without any relation to the commits in the branch. I guess this is ok.The branches have been renamed from e.g.
blender-3.5-release
toblender-v3.5-release
for consistency with the Blender repo.I've put a translation repo here.
https://projects.blender.org/brecht/blender-manual-translations
This repository is also quite big, but we can't use LFS for it since the translation files are text and treating them as binary files would not give good delta compression.
The author mapping is not as good for this one, since I don't have easy access to the developer.blender.org username to full name mapping. I've attached the mailmap in case anyone wants to try improving it.
There's also just one branch. We could add per release branches and get them used on the buildbot if desired. I'm not sure what is better, translations of unchanged text may be added later, but changed text may also delete translations.
The uatest buildbot is now building from Git. This includes nightly builds, incremental builds (without delivery), and build status reporting of commits. Pull request builds and
@blender-bot build
are not there yet, that will be for after the initial deployment.This one was built from Git:
https://docs.blender.org/UATEST/manual/en/dev/
@Blendify what I need from you before we can deploy this:
@Arnd, I need some help from you to deploy the RST preview, particularly regarding user isolation. This is not a blocker but would be great to have available alongside this.
https://projects.blender.org/infrastructure/gitea-custom/src/branch/main/sphinx/
Git repositories are now online and buildbot was updated.
https://code.blender.org/2023/05/sunsetting-subversion/
I'll keep this issue open for the remaining tasks.
Bastien Montagne referenced this issue2023-07-18 18:35:08 +02:00
User manual Git migration and online editingto Gitea: User manual Git migration and online editingFor reference:
There is a bug report in Gitea about this: go-gitea/gitea#9017