lib install --git-url with a local path does not work as expected

[alranel]

Bug Report

Current behavior

$ arduino-cli lib install --git-url /path/to/a/local/MyLibrary
--git-url and --zip-path flags allow installing untrusted files, use it at your own risk.
Enumerating objects: 305, done.
Counting objects: 100% (305/305), done.
Compressing objects: 100% (159/159), done.
Total 305 (delta 132), reused 280 (delta 126)
Installed Library from Git URL

It looks like a local path is accepted as argument for --git-url. However the library is installed incorrectly, because it looks like it is copied in the root of the libraries directory instead of under its own directory:

ls ~/Documents/Arduino/libraries
README.adoc			docs			examples		keywords.txt		library.properties	src

Expected behavior

Either:

ls ~/Documents/Arduino/libraries
MyLibrary

or:

Error: invalid URL supplied.

Note that it works correctly if I supply a proper git URL (https://github.com/foo/bar.git).

Environment

• CLI version (output of arduino-cli version): arduino-cli alpha Version: 0.14.0 Commit: a86b21d99e2af9e0857da0ce4ab80baf1d3afb55 Date:
• OS and platform: macOS

Additional context

I was looking for a way to install a library located in my filesystem. A --path argument could be handy in addition to --git-url and --zip-file. Incidentally, my local library is a clone of a git repo so I tried to use --git-url so I discovered this bug.

[alranel]

A workaround could be to supply the --libraries argument to compile, however it is my understanding that it will not override any existing library with the same name installed in the libraries folder.

[ubidefeo]

@alranel the --git-url is supposed to be a .git URL, hence it needs to be hosted somewhere as a .git repo.
when it comes to --libraries you can point such flag to a folder containing one or more libraries folders (not directly to the content of the library folder) and it will pick those up, however the scoring system to choose the preferred library needs to be reworked, and I've already made tasks for this to happen after the holidays.
I have thought about adding a flag --library which points to the content of a library (i.e. where the src folder is) and it can be used multiple times so

arduino-cli compile -b arduino:samd:nano_33_iot --library /Users/xxx/customLibrary --library /Users/xxx/customLibrary2
``

will make sure that if `customLibrary` and `customLibrary2` are already in one of the standard paths the ones pointed at in the command will be preferred.

Some extra work is required, of course

[alranel]

The help message for --libraries should be more clear on that point, because the current wording suggests that it can be used to point directly to additional libraries. :)

Anyway, the main issue reported here is that --git-url is happily accepting a local path instead of a URL, and it's even using it but it's doing wrong things. So if we don't want to support local git repositories, we should validate the argument and reject it if it's not a URL :)

[ubidefeo]

this is bread for @silvanocerza when he's back :)

[vinay-lanka]

Hey, this might've been an oversight when I suggested this on #648.
I used go-git to clone the repository from the given URL.
I'll try to fix this at the earliest.
Also, I'm sorry for the inconvenience as this contribution seems to be making more issues than its solving :/

[silvanocerza]

I can't reproduce this, I cloned the WiFi101 library locally and tried installing it using the absolute path and it's installed correctly in ~/Arduino/libraries/WiFi101.

Also local paths to Git repos are valid URLs as specified by Git documentation.

[alranel]

@silvanocerza I just tried again and I can reproduce the issue consistently, even if I install another library or I try to supply a relative path instead of an absolute path.
Maybe is it a macOS-specific issue?

This is my config:

% arduino-cli config dump
board_manager:
  additional_urls: []
daemon:
  port: "50051"
directories:
  data: /Users/alranel/Library/Arduino15
  downloads: /Users/alranel/Library/Arduino15/staging
  user: /Users/alranel/Documents/Arduino
library:
  enable_unsafe_install: true
logging:
  file: ""
  format: text
  level: info
sketch:
  always_export_binaries: false
telemetry:
  addr: :9090
  enabled: true

[silvanocerza]

Maybe is it a macOS-specific issue?

That would be strange but wouldn't surprise me one bit. @ubidefeo could you give it a try?

[alranel]

Now after doing something I cannot replicate it anymore, but I'm noticing something strange:

% arduino-cli lib install --git-url . Arduino_MCHPTouch
--git-url and --zip-path flags allow installing untrusted files, use it at your own risk.
Error installing Git Library: repository already exists

However, there's no Arduino_MCHPTouch directory in my <directories.user>/libraries. It looks like that error comes from some other (temporary?) location involved in the process. I wonder if this can be related to the main issue I was having.

[silvanocerza]

Have you actually called it like that or it the . a typo? Because if you actually did it's trying to install the current directory as a library, that one is causing the error.

The Arduino_MCHPTouch instead is completely ignored when calling lib install with --git-url. When using --git-url only the first arg is used and the rest discarded, we could probably enhance this so that you could pass multiple paths.

[alranel]

I added . on purpose because I am in the cloned repository and I want to install it. I later discovered that when using --git-url it is not necessary to supply the library name :) This is very nice but should be explained in the reference.

Anyway, I found the bug:

arduino-cli/arduino/libraries/librariesmanager/install.go

Lines 115 to 121 in 8f5d38b

	libsDir := lm.getUserLibrariesDir()
	if libsDir == nil {
	return fmt.Errorf("User directory not set")
	}
	i := strings.LastIndex(url, "/")
	folder := strings.TrimRight(url[i+1:], ".git")
	path := libsDir.Join(folder)

This is the logic that composes the destination path where the repo is cloned to. It assumes that the last token in the path is the name of the directory, optionally followed by .git, however this is not true whenever a path ending with .., . or / is supplied. In those cases, the folder variable is empty and the resulting destination path is just <directories.user>/libraries which causes the two issues reported above.

[silvanocerza]

Well yes, that's a bug then. Could you open another issue about it so we can keep track of it separately from the opening one?