It looks like with the new Eve (or one of its dependencies) empty lists
aren't stored any more; rather than storing `{'org_roles': []}`, it skips
the `'org_roles'` key altogether. Not sure what caused this, as it was
mentioned in neither the Eve nor the PyMongo changelog.
Custom types became rather useless in Cerberus 1.0 since the type checker
is cripled (doesn't know field name, cannot return useful/detailed error
messages). Instead we use a validator now.
Now BLENDER_ID_ENDPOINT is used for the Blender ID OAuth config,
and it's directly accessed when building requests for Blender ID token
validation (without using utility functions).
The attachments should now be rendered using `{attachment slug}` instead
of `@[slug]`. The `link` attribute can be specified in the shortcode
(for attachments that support it), rather than in the attachment itself.
The attachment subdocument is now reduced to `{oid: File ObjectID}`, and
nodes without attachments should NOT have an `attachment` property at
all (previously it would be an empty dict). This makes querying for
nodes with/out attachments easier.
The CLI command `upgrade_attachment_schema` can do dry-run and remove
empty attachments:
- Added --go to actually perform the database changes.
- Remove empty attachments, so that a node either has one or more
attachments or no attachments sub-document at all.
The CLI command `upgrade_attachment_usage` converts `@[slug]` to
`{attachment slug}`. It also takes into account 'link' and 'link_custom'
fields on the attachment. After conversion those fields are removed from
the attachment itself.
Simplified maintentance CLI commands that iterate over all projects:
I've moved the common approach (either run on one project or all of
them, skipping deleted ones, giving a message upon dry-run, and showing
duration of the command) to a new _db_projects() function. The new
function is now used by two recently-touched CLI commands; more of them
could be migrated to use this.
Added shortcodes 2.5.0 as dependency; Earlier versions corrupted
non-ASCII characters, see
https://github.com/dmulholland/shortcodes/issues/6
The rendered elements have a `shortcode` CSS class.
The YouTube shortcode supports various ways to refer to a video:
- `{youtube VideoID}`
- `{youtube youtube.com or youtu.be URL}`
URLs containing an '=' should be quoted, or otherwise the shortcodes
library will parse it as "key=value" pair.
The IFrame shortcode supports the `cap` and `nocap` attributes. `cap`
indicates the required capability the user should have in order to
render the tag. If `nocap` is given, its contents are shown as a message
to users who do not have this tag; without it, the iframe is silently
hidden.
`{iframe src='https://source' cap='subscriber' nocap='Subscribe to view'}`
Merged test code + added HTML class for shortcode iframes
This is done via coercion rules. To cache the field 'content' in the
database, include this in your Eve schema:
{'content': {'type': 'string', 'coerce': 'markdown'},
'_content_html': {'type': 'string'}}
The `_content_html` field will be filled automatically when saving the
document via Eve.
To display the cached HTML, and fall back to display-time rendering if it
is not there, use `{{ document | markdowned('content') }}` in your template.
Still needs unit testing, a CLI command for regenerating the caches, and
a CLI command for migrating the node type definitions in existing projects.
Static files are now served with an 8-character hash before the last
extension. For example, `tutti.min.js` is now served as
`tutti.min.abcd1234.js`. When doing a request the hash is removed before
serving the static file.
The hash must be 8 characters long, and is taken from STATIC_FILE_HASH.
It is up to the deployment to change this configuration variable
whenever static files change. This forces browsers that download newly
deployed HTML to also refresh the dependencies (most importantly
JS/CSS).
For this to work, the URL must be built with `url_for('static_xxx',
filename='/path/to/file')`. The 'static' module still returns regular,
hashless URLs.
generate_local_thumbnails() now uses pathlib and f-string formatting too,
making the code a lot simpler. Furthermore, I removed unused bits of
resize_and_crop() and simplified the rest.
`manage.py operations merge_project src_url dst_url` moves all nodes and
files from the project with `src_url` to the project with `dst_url`.
This also moves soft-deleted files/nodes, as it ignores the _deleted
field. The actual files on the storage backend are copied rather than
moved.
Note that this may invalidate the nodes, as their node type definition
may differ between projects. Since we use direct MongoDB queries the
nodes are moved to the new project anyway. This allows for a
move-first-then-fix approach).
This is done via a custom PATCH due to the lack of transactions of MongoDB;
we cannot undelete both project-referenced files and file-referenced
projects in one atomic operation.
Note that undeleting files cannot be done via Eve, as it doesn't support
PATCHing collections. Instead, direct MongoDB modification is used to set
_deleted=False and provide new _etag and _updated values.
This means it's no longer GCS-specific, and can be tested using the local
storage implementation.
Required implementation of a rename operation. To mirror Google's API, I've
implemented the renaming of a Blob as a function on the Bucket class.
To me this makes sense, as it requires creating a new Blob instance, which
shouldn't be done by another Blob.
Rather than making them sortable, I made them automatically sorted upon
saving the node. The colour map comes first, then the other maps in
alphabetical order.
This is a two-stage approach that happens when a new token is verified
with Blender ID and stored in our local MongoDB:
- Given the remote IP address of the HTTP request, compute and store the
org roles in the token document.
- Recompute the user's roles based on their own roles, regular org roles,
and the roles stored in non-expired token documents.
This happens once per hour, since that's how long we store tokens in our
database.
