[Draft] New Developer Documentation Infrastructure Proposal (Replacing Wiki?) #105348
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
4 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#105348
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This task looks at the infrastructure for technical documentation. This seems crucial for the goal of improving the technical documentation culture for Blender, so that developers write, maintain and use technical documentation as part of their every day life.
To achieve that, the proposed platform uses developer centric tooling. Technical documentation is moved close to the code and uses the same workflow ("docs-as-code"). Other big organizations claim this had a major positive impact on their documentation culture.
More specifically the proposal is a Git repository with Markdown files, which can be generated into static HTML using MkDocs or Material for MkDocs.
Motivation
Generally the technical documentation for Blender is lacking, it does not receive enough attention. This is the main motivation behind attempting to improve the technical documentation culture.
There are a number of issues with the current infrastructure, and various opportunities.
A developer handbook could include all general information, with all the important stuff upfront (build instructions, guidelines, important tools, testing), less crucial stuff in the back (unity builds, bisecting with event simulation, etc) [TODO create a task on developer handbook]).
Docs-As-Code (or docs-like-code)
Various big organizations have switched to a Docs Like Code infrastructure (e.g. Google, Microsoft, Spotify, Twitter, ...), using a similar technology stack (Markdown + Git, typically using MkDocs). They claim that this "solved" the big issues they had with technical documentation, and brought major improvements for their documentation and overall engineering culture. [TODO references]
Some of the benefits of docs like code are:
(Material for) MkDocs
MkDocs and Material for MkDocs are well known, established projects enabling a Docs Like Code workflow using Markdown and Git. The resulting documentation is enjoyable to work with, and so is the writing workflow.
Some practical benefits:
Does this replace the Wiki?
With the developer documentation moved out of the Wiki, one of the main purposes for the Wiki is gone. Rather than introducing yet another platform, it's better to see if the Wiki can be "removed" entirely.
MkDocs or Sphinx?
The Blender manual uses Sphinx, and it's a fine framework. However, feedback from developers is that they don't enjoy working with it much, mostly because ReStructuredText is a rather unfamiliar syntax for them. Markdown is very common even outside of Blender, and used across most Blender development platforms. This familiarity seems important to minimize the barrier of entry for developers writing documentation. Gitea supports Markdown with live-preview, but not ReStructuredText (which some have considered crucial). Otherwise there doesn't seem to be much of a difference in functionality, both are greatly extensible through extensions and plugins (could be investigated and elaborated on further).
There is an extention to use Markdown within Sphinx, but mixing markup languages seems weak, and we'll end up converting files between languages. Rather make a choice and stick to it. This could still be investigated further.
New Developer Documentation Infrastructure (replacing Wiki?)to Proposal Draft: New Developer Documentation Infrastructure (replacing Wiki?)Proposal Draft: New Developer Documentation Infrastructure (replacing Wiki?)to [Draft] New Developer Documentation Infrastructure Proposal (Replacing Wiki?)These are mostly written by developers, and if they prefer markdown it seems logical to use it here as well.
As you suggested to me earlier, this could move to a repository on projects.blender.org for every user, e.g.
projects.blender.org/brecht/weekly-reports
. Gitea can render markdown and for this purpose it seems good enough, no need for MkDocs.This definitely belongs in the developer docs I think.
Perhaps a solution here would be to make a branch for every release, and merge it all together for publishing? That way a checkout could still have a reasonable size if the big files are in LFS.
+1 for using Gitea's markdown formatting directly (without MkDocs) - noting against MkDocs, it's just another tool to have to setup locally / as part of our infrastructure. There will likely be subtle differences python-markdown & goldmark (used by MkDocs and Gitea respectively) and having to be aware of all this adds some additional burden.
Personally I'd be happy to use Sphinx/RST for developer docs, although I can't argue that markdown is now more popular for developers.
Gitea doesn't offer good navigation or any search for docs. To me it seems necessary to publish the developer docs to a website for better presentation. Especially for more user facing things like build instructions and release notes.
I agree the difference in markdown dialect and extensions between Gitea and MkDocs will cause friction though.
We could perhaps support previewing MkDocs markdown on Gitea similar to what we plan to do for RST for the manual (infrastructure/blender-projects-platform#51). Though the way that currently works, there are still differences to be aware of.
In this case I'd much prefer to use Sphinx/RST. Developers already need to use this for the user-manual. We know how to write extensions for this system and it it avoids introducing yet-another technology to have to become familier with (when we already have quite a few if you count Sphinx/MediaWiki/Doxygen/Gitea).
Developers prefer to write code than docs, so I'm not convinced the technology used will make a significant difference.
One other point is that the markdown used in Devtalk/Gitea is liteweight - some rich text and a few conventions for dot-points, literal text and so on.
The kind of markup we'll need for larger more structured document requires inter-linking sections - conventions for referencing commits/issues/PR's, this will be something people will have to learn and document in it's self.
While I'm noting some down-sides with the proposal, I'd expect we could move from the Wiki to Sphinx/MkDocs and both would work out well enough. Nevertheless, noting my preference here - it may well be that markdown is a more popular choice for others.
That's what I thought too. For me also mediawiki syntax is not an impediment. But other developers say they dislike it to the point of it discouraging writing dev docs at all.
Personally I find markdown more pleasant to write, but don't feel very strongly about it. So other developers are probably in a better position to make the case for that.
For me Markdown (together with a generator like MkDocs and the extentions it provides) is clearly the preferable choice. Some points for it:
Personally, I find the writing experience with Material for MkDocs very pleasant. It just makes it very easy to produce quality, feature rich & great looking documentation out of the box. My experiences with Sphinx have been quite far from that. Although I see why it makes a lot more sense in a Python context of course.
I'm also looking some more into vitepress as alternative to MkDocs (markdown too). The Blender Studio started using this for documentation, and it seems quite promising.
I'd disagree, I think the technology used can have a large impact on the technical writing culture. If we want devs to work on documentation regularly, we need to make it as pleasant as possible for them.
Case in point, Google, Twitter and Spotify have moved to a "docs as code" approach and claim it completely changed their technical documentation culture. Great summary: https://www.doctave.com/blog/2021/09/07/how-google-twitter-and-spotify-build-culture-of-documentation.html (Although I find the intro text makes a different point than the rest of the article.)
As a note, @Jeroen-Bakker, @JacquesLucke and @filedescriptor have started contributing documentation in my devdocs repository. This should help us get feedback. We're also looking into setting up buildbots and servers to build & host the docs entirely on blender.org, shouldn't take long.
Here's some content written by @Jeroen-Bakker, as a live demo: https://julianeisel.github.io/devdocs/eevee_and_viewport/gpu/
Feel free to browse around some more, but most pages are dummies.
I do think that we should consider different aspects of documentation and the audience (both writers and readers). If only considering writing documentation media-wiki is fine. But looking at the diversity of our reading audience, or involvement it might not be the case.
So for writing it might not matter that much, but for reading, the reason why we are writing documentation, it does. I believe that selecting the right tool, use different writing styles, would increase involvement not only in the documentation area.
For sure, but I disagree with the characterization as one system as great and everything else as unpleasant (although I personally have a strong preference for anything that can be edited locally and managed with version control).
Linux, Firefox, LLVM & CMake use Sphinx for their developer docs. Not to say this automatically means we should too - just pointing out other project have managed this and I don't see why we couldn't too, especially since it's already used for the User-Manual & Python API.
In general +1 for having docs closer to the code, I just don't see this as being isolated to a particular technology.