The attachments should now be rendered using `{attachment slug}` instead
of `@[slug]`. The `link` attribute can be specified in the shortcode
(for attachments that support it), rather than in the attachment itself.
The attachment subdocument is now reduced to `{oid: File ObjectID}`, and
nodes without attachments should NOT have an `attachment` property at
all (previously it would be an empty dict). This makes querying for
nodes with/out attachments easier.
The CLI command `upgrade_attachment_schema` can do dry-run and remove
empty attachments:
- Added --go to actually perform the database changes.
- Remove empty attachments, so that a node either has one or more
attachments or no attachments sub-document at all.
The CLI command `upgrade_attachment_usage` converts `@[slug]` to
`{attachment slug}`. It also takes into account 'link' and 'link_custom'
fields on the attachment. After conversion those fields are removed from
the attachment itself.
Simplified maintentance CLI commands that iterate over all projects:
I've moved the common approach (either run on one project or all of
them, skipping deleted ones, giving a message upon dry-run, and showing
duration of the command) to a new _db_projects() function. The new
function is now used by two recently-touched CLI commands; more of them
could be migrated to use this.
Previously all keys starting with an underscore were ignored (so changes
to _created wouldn't count as "different"), but this clashes with saving
Markdown output to _xxx_html keys.
@manager.option also registers the function as command, so the double use
is generally unnecessary.
Furthermore, @manager.command will register CLI options based on the
function parameters, which potentially conflict with the ones registered
with the following @manager.options decorators.
Note that positional arguments should be given in reverse order.
`manage.py operations merge_project src_url dst_url` moves all nodes and
files from the project with `src_url` to the project with `dst_url`.
This also moves soft-deleted files/nodes, as it ignores the _deleted
field. The actual files on the storage backend are copied rather than
moved.
Note that this may invalidate the nodes, as their node type definition
may differ between projects. Since we use direct MongoDB queries the
nodes are moved to the new project anyway. This allows for a
move-first-then-fix approach).
This command fixes the filename in the Content-Disposition header of file
variations on Google Cloud Storage. This is to fix the existing files after
fixing T51477.
This ensures the thumbnail URL is public so that it won't expire.
Since this now requires API calls to Google, I've increased the number of
parallel threads used for indexing, since they'll be waiting for network
I/O more.
It's marginally faster (on our production DB user reindexing goes down from
5+ minutes to 4 minutes), but will likely become significantly faster when
we run ElasticSearch on its own machine.
This uses the orphan-files.txt file output by find_orphan_files() to
mark those files as deleted. This allows for a two-stage approach, where
file IDs are found on one machine (against a read-only MongoDB slave, for
example) and soft-deleted on another machine (against a writable master).
