Commit Graph

1703 Commits

Author SHA1 Message Date
b4e6716cfe Bumped version to 2.2-dev8 v2.2-dev8 2018-11-29 11:22:59 +01:00
de7e3d0016 Gracefully handle timeout error on May-I-Run task when shutting down 2018-11-28 17:14:37 +01:00
9632b1b2c1 Declare PreTaskCheckParams as Iterable
This allows both tuple and list.
2018-11-28 10:16:56 +01:00
2a0790617c Pass FFmpeg binary as ffmpeg_cmd and split the command
This is for consistency with the `blender_cmd` setting of Blender render
commands. The splitting is done so that extra CLI commands can be given
in the Manager config.
2018-11-23 16:12:38 +01:00
9d793b41c5 Bumped version to 2.2-dev7 2018-11-22 16:11:26 +01:00
fba404521c Gracefully handle aborting a command without self.proc attribute 2018-11-22 16:09:57 +01:00
d2e3791cbe Added command 'create_video' to run FFmpeg
This requires FFmpeg to be installed (not just for the command, but also
for running the accompanying unit test). Because of this external
dependency, the worker has to declare task_type='video-encoding' in its
configuration file before it gets such tasks. This is not enabled by
default.
2018-11-22 16:09:57 +01:00
fc8f12de5b Document that Python 3.7 is required now 2018-11-22 16:09:57 +01:00
4cea32d876 Stricter type declarations in commands.py 2018-11-22 16:09:57 +01:00
45cd62b558 Fix warning in PyCharm by explicitly declaring abc.ABC as superclass
This fixes a warning that `AbstractSubprocessCommand` doesn't implement all
abstract methods.
2018-11-22 16:09:57 +01:00
feace5409f Added MyPy runner to unit tests 2018-11-22 16:09:57 +01:00
d004d961fa Fixed last MyPy error 2018-11-21 11:30:19 +01:00
421ec4b658 Only stop task if task-to-stop is the same as currently-executing-task
The 'may-i-run' endpoint may say 'no' to us running task X, but when the
response comes in and is handled the worker may already be working on task
Y. This case is now recognised, and the 'no' will be ignored.
2018-11-16 15:45:39 +01:00
4161328115 Less verbose debug log when POSTing to Manager
The POSTed logs can get huge, so we limit to the first 80 characters now.
2018-11-16 10:35:59 +01:00
dda04b98aa Some more debug log reduction 2018-11-15 15:28:20 +01:00
9911e9e6c2 Use pid=%d instead of PID %r for consistency with other PID logging 2018-11-15 14:44:15 +01:00
e7d12ccff8 Master → Manager 2018-11-15 14:22:54 +01:00
29c46feaf2 Log fetch_task stuff in a separate logger 2018-11-15 14:14:32 +01:00
b4c38f0e26 Log pre-task sanity checking in sub-logger
This allows us to hide those specific logs from the debug log.
2018-11-15 14:09:57 +01:00
9bde5d0e93 Splice '.line' into logger name for debug log
Now the log is sent to
`flamenco_worker.commands.blender_render.line.(task_id=xxx, command_idx=0)`
instead of
`flamenco_worker.commands.blender_render.(task_id=xxx, command_idx=0).line`

This makes configuration of log levels easier.
2018-11-15 13:53:11 +01:00
9978f280bf Include proper PIDs in unit tests 2018-11-15 13:52:15 +01:00
a350ed2e5a Absolute imports for tests to make PyCharm happy 2018-11-15 13:52:05 +01:00
55d0ce87ea Only send render status updates every 30 seconds
Previously it was done for every line of Blender output that contained
render time info, which was way too often.
2018-11-15 12:48:14 +01:00
6acbc2cc25 Fix logging failed command 2018-11-15 12:43:22 +01:00
49b53b71f3 Send subprocess read-lines to separate logger
This allows the logging config to mask those out if required.
2018-11-15 12:38:42 +01:00
e5205687e3 Don't log 'unable to find remaining time', it's too spammy 2018-11-15 12:35:40 +01:00
d3aab505b3 Better logging when a command fails 2018-11-15 12:23:29 +01:00
252af665fa Bumped version to 2.2-dev6 2018-11-15 12:14:25 +01:00
3a218da3cb Fixed race condition in subprocess PID file handling
This doesn't fix the root cause for multiple subprocesses running, but it
does kill better when it finds another subprocess in a race condition.
2018-11-15 12:13:51 +01:00
cb6fa7e893 Upgraded from Python 3.5 → 3.7 2018-11-15 12:13:51 +01:00
8c6039f65f Added TODO 2018-11-15 11:43:37 +01:00
ac7a5aefd0 More type annotations 2018-11-15 11:43:37 +01:00
826f53b345 Changed from 'task' to 'future'
This makes a clearer distinction between Flamenco tasks ('task') and
AsyncIO tasks ('future'). The AsyncIO Task class is a subclass of its
Future class.
2018-11-15 11:43:37 +01:00
73439de819 Ignore more flamenco-worker logs 2018-11-15 10:44:24 +01:00
518cf77b8c Include mypy in development packages + solved lots of warnings
Haven't written the mypy-running unit test yet, that'll come when all
warnings are solved. This commit doesn't solve all of them, just some
simple ones.
2018-11-15 10:43:59 +01:00
fe901ffd7a Log more process IDs
Also changed 'PID=...' to 'pid=...' in the logs sent to Flamenco Manager,
so that all our logs use lower-case 'pid'.
2018-11-15 10:42:04 +01:00
04c611b122 Fixed bug in deploy script 2018-11-13 17:26:04 +01:00
72aa77e637 Added automated deployment script for Blender Animation Studio 2018-11-13 10:26:55 +01:00
6e1b1b585d Updated changelog 2018-11-13 10:13:24 +01:00
9e55ccd6af Moved from requirements.txt to Pipfile
This also means that we need a `tests/__init__.py` file; this file makes
py.test undertand what is our sources directory, and thus allows them to
`import flamenco_worker`. As a result, the test imports from tests/*.py
need to change to relative imports.
2018-11-13 10:00:58 +01:00
4492405aaf Don't late-import psutil and other modules
Early detection of missing dependencies trumps a few ms of startup time.
2018-11-13 09:46:45 +01:00
17709f1c08 Bumped version to 2.2-dev5 2018-11-12 15:11:05 +01:00
0b456b95be Include the process PID in the log lines
This will help debugging the crazy double-Blender-running we're seeing
in the studio.
2018-11-12 15:09:04 +01:00
c1bc3fc167 Write PID file for subprocesses
This prevents multiple subprocess commands running at once. They shouldn't
be doing that in the first place, but we have a strange bug where multiple
Blenders seem to be running on the same machine.
2018-11-12 15:09:04 +01:00
cb0aa251f9 Added missing unit tests 2018-11-12 13:34:01 +01:00
af32227a2d Typo 2018-11-12 13:26:43 +01:00
2ce7358cf9 Fixed sanity check for Python 3.5 2018-11-12 13:24:03 +01:00
bc16faccff Bumped version to 2.2-dev4 2018-11-12 12:45:49 +01:00
b9a25c2f3b Added pre-task sanity check
This check is performed every time before trying to fetch a task. It checks
for readability or writability of certain filesystem paths, to prevent
tasks from failing when the shared storage hasn't been mounted. Instead,
the worker will go to `error` status and sleep for 10 minutes before trying
again.
2018-11-12 12:43:20 +01:00
3386072774 Bumped attrs (16.3.0 → 18.2.0) and requests (2.12.4 → 2.20.1) libraries 2018-11-12 12:12:02 +01:00