Commit Graph

397 Commits

Author SHA1 Message Date
2ad5b20880 Quick hack to get /p/{url}/jstree working again
Apparently Eve is now stricter in checking against MONGO_QUERY_BLACKLIST,
and blocks our use of $regex when getting child nodes. See
`jstree.py::jstree_get_children()`
2018-08-30 13:59:23 +02:00
1c430044b9 More urljoin() instead of string concatenation 2018-08-29 14:28:24 +02:00
73bc084417 Cerberus or Eve apparently changed validator._id to document_id 2018-08-29 14:18:24 +02:00
2c40665271 Use urljoin() to compose OAuth URLs instead of string concatenation
String concatenation is bound to mess up; in this case it was producing
double slashes instead of single ones when `BLENDER_ID_ENDPOINT` ends in
a slash. Since URLs generally end in a slash, this should be supported.
2018-08-29 14:17:17 +02:00
d655d2b749 Users schema: don't supply schema when allow_known=True
Apparently the new Cerberus doesn't like this, and will check against the
schema only and ignore `allow_unknown` when it's there.
2018-08-29 11:23:19 +02:00
a58e616769 Markdown validator: gracefully handle partial document validation
Validation of partial documents can happen when validating an update.
Missing data is fine then.
2018-08-29 11:22:39 +02:00
1649591d75 Create a copy in the validator's self.document
This ensures that further modifications (like setting '_etag' etc.) aren't
done in-place.
2018-08-28 17:45:44 +02:00
6737aa1123 Markdown validator now also updates the doc with post_internal
The post_internal function does `document = validator.document`, replacing
the to-be-posted document by the copy that Cerberus made (and which we
cannot add keys to because it iterates over the keys and the dict size thus
isn't allowed to change).

I hope this doesn't break other validators who expect to be able to write
to `self.document`.
2018-08-28 17:29:29 +02:00
469f24d113 Fix for {validate: markdown} when used in Eve
Eve's Validator has not only a validate() function, but also
validate_update() and validate_replace(). Those set
self.persisted_document, so if that attribute exists we just use it.
2018-07-13 17:14:06 +02:00
8a0f582a80 Removed dependency on flask_pymongo 2018-07-13 17:08:06 +02:00
559e212c55 Removed debug prints + added TODO(fsiddi) 2018-07-13 17:04:23 +02:00
61278730c6 De-indent the code a bit 2018-07-13 17:02:47 +02:00
0fdcbc3947 Restored MarkDown conversion using 'validator': 'markdown' 2018-07-13 17:02:38 +02:00
8dc3296bd5 Schema change for IP range, use validator instead of type
Custom types became rather useless in Cerberus 1.0 since the type checker
is cripled (doesn't know field name, cannot return useful/detailed error
messages). Instead we use a validator now.
2018-07-13 15:03:35 +02:00
7da741f354 Re-enabled PATCH handler for organisations 2018-07-13 13:36:59 +02:00
41369d134c Fix bloody Eve raising exceptions instead of returning status code 2018-07-13 12:45:58 +02:00
61ed083218 Don't change the global schema! 2018-07-13 12:33:22 +02:00
46777f7f8c Removed unnecessary ['shema'] 2018-07-13 12:06:48 +02:00
ef94c68177 Re-enabled the 'valid_properties': True in nodes_schema 2018-07-13 12:06:38 +02:00
c607eaf23d Added magic custom validation rule schemas in docstrings 2018-07-13 12:02:18 +02:00
549cf0a3e8 WIP on libraries upgrade 2018-07-12 15:23:57 +02:00
9ee7b742ab Make more consistent use of BLENDER_ID_ENDPOINT
Now BLENDER_ID_ENDPOINT is used for the Blender ID OAuth config,
and it's directly accessed when building requests for Blender ID token
validation (without using utility functions).
2018-06-22 19:38:27 +02:00
0e14bdd09f Introduce rating functions
These hotness and confidence calculation algorithms come from Reddit
and have been tweaked based on our experience on the Dillo project.
2018-06-03 02:09:20 +02:00
ce6df542cc Add ratings_embedded_schema to node_types
Ratings, like attachments, are a common feature in node_types.
By adding this schema definition, we reduce code duplication.
No functional changes are introduced introduced in this commit.
2018-05-11 01:32:39 +02:00
67e8e7c082 Disallow spaces in attachment slugs
Slugs shouldn't have spaces. It also interferes with using slugs in
shortcodes.
2018-04-03 13:59:31 +02:00
a86920fc73 Disallow spaces in attachment slugs 2018-04-03 12:24:42 +02:00
3b452d14ce Render attachments with shortcodes rather than slugs
The attachments should now be rendered using `{attachment slug}` instead
of `@[slug]`. The `link` attribute can be specified in the shortcode
(for attachments that support it), rather than in the attachment itself.

The attachment subdocument is now reduced to `{oid: File ObjectID}`, and
nodes without attachments should NOT have an `attachment` property at
all (previously it would be an empty dict). This makes querying for
nodes with/out attachments easier.

The CLI command `upgrade_attachment_schema` can do dry-run and remove
empty attachments:

- Added --go to actually perform the database changes.
- Remove empty attachments, so that a node either has one or more
  attachments or no attachments sub-document at all.

The CLI command `upgrade_attachment_usage` converts `@[slug]` to
`{attachment slug}`. It also takes into account 'link' and 'link_custom'
fields on the attachment. After conversion those fields are removed from
the attachment itself.

Simplified maintentance CLI commands that iterate over all projects:
I've moved the common approach (either run on one project or all of
them, skipping deleted ones, giving a message upon dry-run, and showing
duration of the command) to a new _db_projects() function. The new
function is now used by two recently-touched CLI commands; more of them
could be migrated to use this.
2018-04-03 10:59:20 +02:00
0841d52dd1 Removed unused imports 2018-04-03 10:48:40 +02:00
8288455468 Fixed a KeyError when editing a comment. 2018-03-28 12:36:03 +02:00
d3ff88e5cf Also replace node types when key with underscore changed
Previously all keys starting with an underscore were ignored (so changes
to _created wouldn't count as "different"), but this clashes with saving
Markdown output to _xxx_html keys.
2018-03-27 17:56:38 +02:00
f22dc4d92a Fixed PATCHing comments 2018-03-27 17:42:29 +02:00
540dd28861 Short-circuit check_permissions() when logged in as CLI user
The CLI user should just be able to do anything.
2018-03-27 17:42:12 +02:00
dfaac59e20 Cache Markdown'ed HTML in database
This is done via coercion rules. To cache the field 'content' in the
database, include this in your Eve schema:

    {'content': {'type': 'string', 'coerce': 'markdown'},
     '_content_html': {'type': 'string'}}

The `_content_html` field will be filled automatically when saving the
document via Eve.

To display the cached HTML, and fall back to display-time rendering if it
is not there, use `{{ document | markdowned('content') }}` in your template.

Still needs unit testing, a CLI command for regenerating the caches, and
a CLI command for migrating the node type definitions in existing projects.
2018-03-27 16:34:32 +02:00
08ce84fe31 Drop 'template' from blog node type 2018-03-27 15:56:06 +02:00
d24715a224 Smarter upgrades of node type definitions
- No changes are applied unless the new --go CLI arg is used.
- Differences to node types are actually shown.
- Dynamic form definitions are kept.
2018-03-27 12:03:18 +02:00
dee0b18429 utils.doc_diff() now also supports list values 2018-03-27 11:50:23 +02:00
318ccb2d95 Reduce log level
WARNING and higher are sent to Sentry, which isn't necessary here.
2018-03-27 11:50:23 +02:00
0cf45c0d78 Use capability check instead of role check in strip_link_and_variations() 2018-03-23 14:23:47 +01:00
e4f229cc70 Fix T51678: 16bit greyscale PNG images thumbnailing fails
generate_local_thumbnails() now uses pathlib and f-string formatting too,
making the code a lot simpler. Furthermore, I removed unused bits of
resize_and_crop() and simplified the rest.
2018-03-22 17:53:14 +01:00
2bf0bf1064 Formatting 2018-03-21 02:15:07 +01:00
df6297d40f Fixed project search
The project ID wasn't used at all when searching in a project's context.
2018-03-13 12:24:29 +01:00
257793dcd5 Simplified some code 2018-03-13 12:21:41 +01:00
450dde56b7 Pass our OAuth2 client ID to Blender ID when validating tokens
This is a security measure, as it ensures that valid Blender ID OAuth2
tokens that were not generated for Blender Cloud are rejected.
2018-02-21 10:49:33 +01:00
854bc7cfaf Sentry: include extra user information
We perform authentication of the user while handling the request,
but Sentry calls get_user_info() in a before-request handler. This means
that Sentry would miss user info in many cases. This fixes that.
2018-02-14 13:52:52 +01:00
1f2fb774b4 Converted another datetime.utcnow() to utils.utcnow() 2018-02-13 16:50:11 +01:00
de801e41e3 CLI command for moving all nodes+files to another project
`manage.py operations merge_project src_url dst_url` moves all nodes and
files from the project with `src_url` to the project with `dst_url`.
This also moves soft-deleted files/nodes, as it ignores the _deleted
field. The actual files on the storage backend are copied rather than
moved.

Note that this may invalidate the nodes, as their node type definition
may differ between projects. Since we use direct MongoDB queries the
nodes are moved to the new project anyway. This allows for a
move-first-then-fix approach).
2018-02-13 15:52:21 +01:00
cd42ce6cba Moving blobs between nodes now uses storage API
Instead of only being GCS-specific, it now works for all storage
backends.
2018-02-13 15:36:11 +01:00
eb18e5b933 Formatting 2018-02-13 14:36:23 +01:00
350cf85cd2 Removed unused imports 2018-02-13 14:36:16 +01:00
f2888069db Added pillar.api.utils.utcnow() which returns a datetime for 'now'
This replaces pillar.web.utils.datetime_now() and can be used in a wider
setting (since we don't import web stuff in the api, but we do vice versa).
2018-02-13 14:36:05 +01:00