Commit Graph

397 Commits

Author SHA1 Message Date
0c238284b0 Typo 2018-02-01 15:18:43 +01:00
06b2adf923 Added all the in-use texture map types to the texture node type
Note that we now name 'occlusion' (as it's used in production) as
'ambient occlusion'. The database needs to be migrated for this.
2018-02-01 14:13:50 +01:00
d86c215c34 More texture map types
These are actually in use in production.
2018-02-01 11:10:16 +01:00
1b57b333df Removed the URLer service
We don't have a need for it any more, so it can go.
2018-01-31 14:33:41 +01:00
08a814525b Implement project undelete as PATCH
This is done via a custom PATCH due to the lack of transactions of MongoDB;
we cannot undelete both project-referenced files and file-referenced
projects in one atomic operation.
2018-01-31 14:15:23 +01:00
f8ff30fb4d (un)delete on project also (un)delete file documents.
Note that undeleting files cannot be done via Eve, as it doesn't support
PATCHing collections. Instead, direct MongoDB modification is used to set
_deleted=False and provide new _etag and _updated values.
2018-01-31 10:08:17 +01:00
7d1b08bf58 refresh_links_for_backend: log to backend-specific child logger. 2018-01-31 10:08:17 +01:00
60abf6d4a9 on_pre_get_files: remove refresh of files
It is possible to perform a GET request that has an empty `lookup`
param, like {'_id': {'$in': ['objectID1', 'objectID2', ...]}}. Such a
request would cause a refresh of *all* file documents, which is far too
heavy to do in one client HTTP request.
2018-01-31 10:08:17 +01:00
7c384d1f45 Link refresh: allow refreshing links for soft-deleted projects.
Since files aren't deleted (yet) when projects are deleted, it can happen
that a file is refreshed but then cannot reference a deleted project.
By removing the project ID from the PATCH, Eve doesn't have enough info to
check this, and it'll work fine.
2018-01-31 10:08:17 +01:00
f18d5580c1 Fixed unicode/bytes issues in CDN file path hashing. 2018-01-31 10:08:17 +01:00
9177f77e69 Link refresh: gracefully handle case where 'file_path' is not set.
In this case an error is logged and the entire link regeneration is
aborted. In other words, variations aren't refreshed either.
2018-01-31 10:08:17 +01:00
4b5a961e14 Speed up authentication by trusting g.current_user if set. 2018-01-31 10:08:17 +01:00
de1eab4596 GCS: the slash is a separator, and not part of the directory name itself. 2018-01-26 14:58:42 +01:00
de1c227ccd GCS storage: use self.subdir instead of hard-coded '_' 2018-01-26 14:58:25 +01:00
230b2c669c Implement rename-after-Zencoder-is-done using our Storage API.
This means it's no longer GCS-specific, and can be tested using the local
storage implementation.

Required implementation of a rename operation. To mirror Google's API, I've
implemented the renaming of a Blob as a function on the Bucket class.
To me this makes sense, as it requires creating a new Blob instance, which
shouldn't be done by another Blob.
2018-01-26 14:54:34 +01:00
2e2314c16b Replace storage backend 'pillar' with 'local'
The backend 'pillar' is obsolete; 'local' is the modernity and uses our
nice storage API.
2018-01-26 14:13:03 +01:00
f8c3408f18 Properly handle errors when saving updated file doc after Zencoder notif.
We never checked the return values from the put_internal() call, so errors
would have passed silently into the night.
2018-01-26 12:49:15 +01:00
89ca0516a9 Better handling of Zencoder error notifications. 2018-01-26 12:29:22 +01:00
987d6d03a6 Fix T49280: Make texture files (in texture node) sortable
Rather than making them sortable, I made them automatically sorted upon
saving the node. The colour map comes first, then the other maps in
alphabetical order.
2018-01-26 11:42:42 +01:00
741cdf6e12 Elastic: regenerate picture URL before inserting into ElasticSearch
This ensures the thumbnail URL is public so that it won't expire.
Since this now requires API calls to Google, I've increased the number of
parallel threads used for indexing, since they'll be waiting for network
I/O more.
2018-01-26 10:29:28 +01:00
fd3e795824 Store IP-based org-given roles in the user document.
This is a two-stage approach that happens when a new token is verified
with Blender ID and stored in our local MongoDB:

  - Given the remote IP address of the HTTP request, compute and store the
    org roles in the token document.
  - Recompute the user's roles based on their own roles, regular org roles,
    and the roles stored in non-expired token documents.

This happens once per hour, since that's how long we store tokens in our
database.
2018-01-25 14:01:28 +01:00
d3f97358d9 Work around Eve not supporting returning binary data 2018-01-25 14:01:28 +01:00
c44f0489bc Backend support for organization IP ranges.
We can now store IP ranges with Organizations. The aim is to have any user
logging in with a remote IP address within such a race will get the
organization roles assigned to the user object stored in the Flask session.

This commit just contains the MongoDB storage and querying, and not yet the
updates to the user.
2018-01-25 14:01:28 +01:00
7245dac1ca Elastic: reverted previous two commits
The ngrams cause too much noise in the search results.
2018-01-12 16:27:09 +01:00
6748fd0006 Elastic: use ngrams for user search but not assets 2018-01-12 15:36:15 +01:00
b2bd01117e Elastic: tweaked user indexing
This makes it a bit more "fuzzy", so users are also matched on N-grams
and not just N-grams-from-the-start-of-the-word.
2018-01-12 15:28:33 +01:00
31ca4f3d23 Gracefully handle Elastic errors in user search 2018-01-12 15:27:56 +01:00
8326d8e7fe Elastic query: remove matching on document type.
ElasticSearch 6 removes support for different document types in one index,
so we don't have to match for it any more. Furthermore, elasticsearch_dsl
seems to save with the generic 'doc' type, so this check is
counter-productive.
2018-01-12 11:17:52 +01:00
3e5ccaf8fd Elastic indexing: explicitly save into the configured index. 2018-01-12 11:16:42 +01:00
68b6e43649 Upgraded ElasticSearch to 6.1.x. 2018-01-11 10:29:15 +01:00
61673ef273 Search: implemented pagination
- Got rid of the nasty off-by-one logic in the JavaScript.
- Implemented pagination at the API.
2018-01-10 17:07:21 +01:00
36da289746 user search: boost exact matches on username 2018-01-10 15:12:39 +01:00
7f892601f4 Reformatting 2018-01-10 15:12:22 +01:00
740e088cc5 Elastic: Doc & return type polish 2018-01-09 16:21:08 +01:00
4fdcd2a343 Elastic: include exact searches on email address 2018-01-09 16:16:38 +01:00
6e40b9a44a Elastic: delegated common user search functionality to a single function 2018-01-09 16:16:26 +01:00
c20aa41b5c Formatting 2018-01-09 16:09:49 +01:00
284d822a8a Asset search: order by newest-first on empty query 2018-01-05 17:29:59 +01:00
9d39995d0f Asset search: debug-log instead of print query + result 2018-01-05 17:29:47 +01:00
83a38ff50e Admin user search: include aggregations 2018-01-05 17:03:09 +01:00
33c051bf28 Search: include explicit indices to search
This makes it possible to search for 'everything' without explicitly
stating the document type.
2018-01-05 16:42:08 +01:00
b6f7958dfe Admin user search: Include aggregations 2018-01-05 16:41:36 +01:00
acd7a40fe6 Renamed do_search → do_node_search 2018-01-05 16:40:54 +01:00
af69b4fa58 Admin user search: boost user ID matching 2018-01-05 16:02:49 +01:00
01d8ad5ca2 Admin user search: also match on user ID
And removed user ID matching for regular user search.
2018-01-05 15:52:25 +01:00
57ce554feb Log search queries instead of printing them 2018-01-05 15:52:01 +01:00
75a6de18b2 Regular user search now also finds by email address 2018-01-05 15:51:30 +01:00
dcd67b6114 Simplified ElasticSearch connection 2018-01-05 15:24:57 +01:00
f6cf8d29f0 Elastic: search indexing logging tweaks 2018-01-05 15:24:47 +01:00
2433a1b981 minor documentation / annotation fixes 2018-01-05 11:58:33 +01:00