Unexpected behavior of --config-file global command line option

[per1234]

Bug Report

Current behavior

The --config-file global command line option currently only defines the directory containing the configuration file. It does not allow configuring the specific configuration file within that directory.

arduino-cli expects the path specified via --config-file to contain a file named with the prefix arduino-cli followed by a file extension associated with one of the supported formats.

$ ./arduino-cli config dump --verbose --config-file ./config
INFO[0000] Using config file: /home/per/ArduinoIDE/arduino-cli_nightly/config/arduino-cli.json

Even if you specify a filename, it will be ignored:

$ ./arduino-cli config dump --verbose --config-file ./config/foo.json
INFO[0000] Using config file: /home/per/ArduinoIDE/arduino-cli_nightly/config/arduino-cli.json

Unless the file doesn't exist:

$ ./arduino-cli config dump --verbose --config-file ./config/nonexistent.json
INFO[0000] Config file not found, using default values  

If the configuration directory contains multiple configuration files, there is a specific order of precedence that determines which one is used, regardless of the filename provided to the --config-file option:

$ ./arduino-cli config dump --verbose --config-file ./config/arduino-cli.toml
INFO[0000] Using config file: /home/per/ArduinoIDE/arduino-cli_nightly/config/arduino-cli.json 

(note that I specified the .toml file, but the .json file was used instead)
The order of precedence:

• JSON
• TOML
• YAML
• Java properties file
• HCL
• envfile
• INI

Expected behavior

The --config-file option's documentation seems somewhat misleading to me:

--config-file string        The custom config file (if not specified the default will be used).

From reading that, I would expect to be able to specify any arbitrary filename (with the appropriate file extension for the file's format).

I don't have a problem with the current behavior of arduino-cli regarding custom configuration file locations, it's just not the behavior I expect from the option name and command line help. I don't think requiring a specific filename (arduino-cli) is onerous, and I don't think it likely the user will ever have configuration files of multiple formats in the configuration directory (I only discovered this because I was verifying support for all the formats). So a simple solution for this would be keep the current behavior, but change the option name to something like --config-dir (hopefully keeping --config-file as an undocumented alias for backwards compatibility), and update the command line help accordingly.

Environment

• CLI version (output of arduino-cli version): 0.11.0-rc1 Commit: 5373961
• OS and platform: Ubuntu 20.04, Windows 10

[ianfixes]

This explains so much. Setting aside what an awful time it was trying to troubleshoot / google this, I'd like to draw attention to the error message:

INFO[0000] Config file not found, using default values

It doesn't tell you which config file it looked for, which is pretty important considering that the one it looks for isn't the one you specified with --config-file!

So in the short term, the messaging should be updated to be more verbose. Otherwise you get headscratchers like this:

$ cd /home/me
$ ls arduino-cli.yaml
arduino-cli.yaml
$ arduino-cli --config-file arduino-cli.yaml config dump --verbose | head -n 1
INFO[0000] Using config file: /home/me/arduino-cli.yaml
$ arduino-cli --config-file CUSTOM-arduino-cli.yaml config dump --verbose | head -n 1
INFO[0000] Using config file: /home/me/arduino-cli.yaml
$ # ^ WTF moment #1
$ mv arduino-cli.yaml CUSTOM-arduino-cli.yaml
$ ls CUSTOM-arduino-cli.yaml
CUSTOM-arduino-cli.yaml
$ arduino-cli --config-file CUSTOM-arduino-cli.yaml config dump --verbose | head -n 1
INFO[0000] Config file not found, using default values  
$ # ^ WTF moment #2

What the INFO is really saying here is "I tried a bunch of places to look for the config file and none of them were actually what you typed in..."

[ubidefeo]

@ianfixes
have you tried running the latest Nightly Build?
We have reworked this behaviour quite a lot over the last few weeks

[ianfixes]

I am trying to use it as a backend for arduino_ci, so for now I am only going to worry about a specific released version. But that's good to know! It will make some of my testing work a lot easier when it rolls out.

[ubidefeo]

@ianfixes take into account that we're about to roll out a new version soon and there is one main breaking change related to not automatically generating a sketch folder build directory unless a setting in the config file or an ENV variables are set to true.
I invite you to try the nightly and play with the commands because there's a lot more in there :)

[ianfixes]

For the purposes of my own project's releases, I will have to pin to specific versions of ardunio-cli as a backend.

That said, I can think of a few very useful features in your project that would support mine (e.g. asking what compiler flags to set for a particular board). What's the proper medium for an open-ended discussion on that?

[umbynos]

I would say this is solved (0.34.1):

$ cd /home/umberto/.arduino15/
$ ls arduino-cli.yaml 
arduino-cli.yaml
$ arduino-cli --config-file arduino-cli.yaml config dump --verbose | head -n 1
INFO[0000] Using config file: /home/umberto/.arduino15/arduino-cli.yaml 
$ arduino-cli --config-file CUSTOM-arduino-cli.yaml config dump --verbose | head -n 1
INFO[0000] Config file not found, using default values
$ cp arduino-cli.yaml CUSTOM-arduino-cli.yaml
$ ls CUSTOM-arduino-cli.yaml 
CUSTOM-arduino-cli.yaml
$ arduino-cli --config-file CUSTOM-arduino-cli.yaml config dump --verbose | head -n 1
INFO[0000] Using config file: /home/umberto/.arduino15/CUSTOM-arduino-cli.yaml