Flamenco Manager configuration should be in fixed directory #102229
Labels
No Label
Good First Issue
Priority
High
Priority
Low
Priority
Normal
Status
Archived
Status
Confirmed
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Job Type
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
7 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: studio/flamenco#102229
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Flamenco Manager's files should be loaded from a fixed directory, just like Flamenco Worker is doing (docs). This will make it easier to upgrade, and places files where people tend to expect them.
$HOME/.local/share/flamenco/manager
(and similar on other platforms).-move-config
that moves all the Manager-local files to that location, and updatesflamenco-manager.yaml
to match.Original report:
Flamenco manager database files are placed in the user's root folder on macOS
The files in question are
and are located in by default in the root folder of the user:
This is rather unexpected and appears to be an overlook for the macOS platform.
The expected location for these files is
since the database files for the worker are placed there correctly.
Added subscriber: @iclemens
Added subscriber: @Memento
Added subscriber: @dr.sybren
This is as designed, it's saving to the current working directory. I want to revisit that design, though. It'll also make upgrading the Manager easier if it uses a fixed location.
Flamenco manager database files are placed in the user's root folder on macOSto Flamenco Manager configuration should be in fixed directoryHello!
I'm in the process of upstreaming a NixOS module for flamenco. It sets up the package (manager, worker or both) as a systemd service. I was trying to define the service and I got really confused when I was trying to set up the paths.
I have two ideas about file management, focusing specifically on the manager, for consideration:
local_manager_storage_path and
getSuitableStorageRoot
Currently,
local_manager_storage_path
uses a directory path relative to the executable, discovered by using thegetSuitableStorageRoot
function.From what I understand, part of Flamenco's design philosophy is that it could be used as an portable application. Maybe it could be possible to make
local_manager_storage_path
bypass thegetSuitableStorageRoot
logic only if the path is an absolute path. This way, default behavior is preserved, while expanding functionality for other uses.Go provides a function provided with
path/filepath
,isAbs()
, which can tell if a path is an absolute path on multiple platforms, Unix and Windows both included.Configuration file location
A lot of application I've seen has an config file path argument
-config
.Again, following the portability philosophy, default behavior can be that the configuration file is generated and read in the current working directory, but then people can use
-config
to set an path to an existing configuration file, if they need it.Currently, the NixOS module deals with these caveats by setting up an temporary working directory with the configuration file inside it, and by adding "../../../../" in front of the
local_manager_storage_path
, which will effectively setlocal_manager_storage_path
to an absolute path from NixOS' software install directory (in NixOS, the program is stored in the nix store, an read only folder that houses all system files).These changes would greatly reduce the complexity of hosting flamenco manager on a server, while retaining its ability to run as a portable application.
Cool, thanks!
That's not explicitly part of the design, but I certainly am not opposed to the idea. A clearer separation between these two approaches would be good:
~/Library/Application Support/Flamenco
on macOS, as mentioned in this issue's description).Currently the way the config files & databases are handled is a bit of a mess, and mix up these two approaches. That's what I want to get rid of.
That seems reasonable to me.
Both the Manager and the Worker need multiple files from their "config location".
flamenco-manager.yaml
flamenco-manager.sqlite
flamenco-worker.yaml
(which may or may not be shared among all workers in a farm)flamenco-worker-credentials.yaml
(which is unique to each worker)flamenco-worker.sqlite
(also unique to that worker)So one
-config
option isn't going to cut it. For the Manager it could point to a single directory, but for the Worker there's the potentially-shared config as well as the worker-unique files.I'm thinking of switching the entire configuration management of both Worker and Manager to Viper. That should make config management a bit easier.
@Hubble-the-Wolverine as you are packaging flamenco, perhaps you could use the compile time options to set the paths you want to use?
You can already bypass this at compile time by specifying the
configFilename
variable.See how these options are used to setup flamenco as a system service on our studio computers:
There the manager is setup look for its config in
/etc/flamenco_manager/config.yaml
(like most other system services)My preference would be for manager files to reside at
~/Library/Application Support/Flamenco
which worker ones already use.That notwithstanding, do note that many command line unix programs nowadays abide by the XDG Base Directory Specification and place config files in a subfolder of
$XDG_CONFIG_HOME
, which, when unset, defaults to$HOME/.config
.I agree with Jorge, it will be more convenient to have manager files at ~/Library/Application Support/Flamenco (or in the Flamenco folder) but not at Volume/Users/xxx/
It's really weird... :/