Flamenco 3 run-worker scripts

This repository contains run-worker scripts for Flamenco 3.

The run-flamenco-worker scripts allow running more than one worker on the same machine. It provides each worker with $FLAMENCO_HOME and $FLAMENCO_WORKER_NAME environment variables.

The run-flamenco-worker-gpu scripts target a multi-GPU environment where each worker uses its own GPU. This is useful for Eevee (OpenGL) rendering, which uses only 1 GPU at a time.

Windows workflow (multiple workers)

Use the windows\run-flamenco-worker.cmd script to run multiple Flamenco Workers on the same PC:

• Run run-flamenco-worker.cmd WORKERNAME to run a worker with a given name.

• This wil register as {PCNAME}-{WORKERNAME} with Flamenco Manager and use %APPDATA%\Blender Foundation\Flamenco\worker-{WORKERNAME} as worker home directory.

For example, running run-flamenco-worker.cmd worker1 on Windows computer MYDESKTOP:

• The worker would register as MYDESKTOP-worker1 with Flamenco Manager;
• The worker would use %APPDATA%\Blender Foundation\Flamenco\worker-worker1 as worker home directory.

Windows workflow (assigning a specific GPU to each worker)

• Prerequisites:

  • For Windows, you'll need a compiled, modified version of Flamenco (Pull Request is pending). This picks up the FLAMENCO_BLENDER_PATH environment variable (see below);
  • GPU-specific software (e.g. NVIDIA Control Panel) to assign a GPU to an executable.

• Create some copies of blender.exe in order to assign different GPUs to different Blender instances, e.g.:

  • Copy blender.exe to blender-gpu0.exe
  • Copy blender.exe to blender-gpu1.exe
  • Do this for each GPU.

• Use your GPU driver to assign specific GPUs to these executables.

  • For blender-gpu0.exe, configure your first GPU as OpenGL Rendering GPU (for Eevee) and CUDA - GPU (for Cycles).
  • For blender-gpu1.exe, configure your second GPU.
  • Do this for each GPU
  • Example: For NVidia Control Panel, go to Manage 3D Settings, Program Settings:

  

• Make sure the manager sends 'blender' to the worker instead of a full path. This way the worker will check the FLAMENCO_BLENDER_PATH in order to find the Blender executable they should use.

  # flamenco-manager.yaml
  
  variables:
    blender:
      values:
      - platform: windows
        value: blender
  

• Finally, use the windows\run-flamenco-worker-gpu.cmd script to run Flamenco Workers:

  • First check the BLENDER_PATH in the script;
  • run run-flamenco-worker-gpu.cmd -g=0 to run a worker on the first GPU (using `blender-gpu0.exe)
  • run run-flamenco-worker-gpu.cmd -g=1 to run a worker on the second GPU (using `blender-gpu1.exe)

Each worker will be provided with the following environment variables:

• FLAMENCO_BLENDER_PATH: The Blender executable the worker will use
• FLAMENCO_WORKER_NAME: Unique worker name, containing the name of the computer and (if available) the name of the Graphics Card, e.g. RENDERPC-0-GeForce-GTX-1080-Ti (or RENDERPC-0-gpu0)
• FLAMENCO_HOME: Unique worker directory in AppData\Blender Foundation\Flamenco\worker-{NAME}

Linux workflow (multiple workers)

Use the linux\run-flamenco-worker script to run multiple Flamenco Workers on the same server:

• Run run-flamenco-worker WORKERNAME to run a worker with a given name.

• This wil register as {SERVERNAME}-{WORKERNAME} with Flamenco Manager and use ~/.local/share/flamenco/worker-{WORKERNAME} as worker home directory.

For example, running run-flamenco-worker worker1 on Windows computer RENDERSERVER:

• The worker would register as RENDERSERVER-worker1 with Flamenco Manager;
• The worker would use ~/.local/share/flamenco/worker-worker1 as worker home directory.

Linux Workflow (assigning a specific GPU to each worker, NVIDIA only)

• Prerequisites:

  • The Linux implementation uses the NVIDIA Prime Render Offload to specify which GPU each worker should use. It's in the NVIDIA driver by default.
  • Install glxinfo (sudo apt-get install mesa-utils) to enable the script to determine the GPU Model and provide a better worker name.

• Use the linux\run-flamenco-worker-gpu script to run Flamenco 3 workers:

  • run run-flamenco-worker-gpu -g=0 to run a worker on the first GPU
  • run run-flamenco-worker-gpu -g=1 to run a worker on the second GPU

Each worker will be provided with the following environment variables:

• __NV_PRIME_RENDER_OFFLOAD and __GLX_VENDOR_LIBRARY_NAME to specify the GPU to use;
• FLAMENCO_WORKER_NAME: Unique worker name, containing the name of the host and (if available) the name of the Graphics Card, e.g. RENDER-0-GeForce-GTX-1080-Ti (or RENDER-0-gpu0);
• FLAMENCO_HOME: Unique worker directory in ~/.local/share/flamenco/worker-{NAME}

License

These scripts are licensed under the GPLv3+ license.