IDE library manager leaves temporary directory name when libs are in a dropbox monitored path

[anno73]

When File->Preferences->Sketchbook location is set to a directory which is monitored by dropbox client, installation/upgrade of libraries interfere with dropbox client and do not get renamed to the proper final name of the lib but remain as temporary folder arduino-\d+.
Upon upgrade, this leads to multiple libraries in different directories and the builder not being able to consistently select the most recent version (see also issue https://github.com/arduino/arduino-builder/issues/366).
This issue is discussed on the forum thread https://forum.arduino.cc/index.php?topic=724169.

I tested with ArduinoIDE 1.8.13 on Windows 10 and a recent hourly build 2021-01-24 ... both showing the same behavior.

Attached a trace with Sysinternals Process Manager: Logfile.zip
Filter settings: ProcessMonitorFilterSettings
I suppose the interesting two lines are the ones with Result equal to SHARING VIOLATION and INVALID DEVICE REQUEST.

I also tested the IDE with directories monitored with the latest versions of nextcloud client and megasync client. Both worked well during fresh installs and up-/downgrades of libraries.
On the other hand, arduino-cli worked well with all three sync clients.

[matthijskooijman]

Haven't looked at the logs or the issue in detail, but I wonder if the problem is that dropbox locks files while it syncs them, preventing the IDE from renaming them? If so, I'm not sure if the IDE can actually do anything to fix this (except for retrying until it works, which doesn't seem right).

Does the IDE give any error messages when this happens?

[anno73]

Hmm, arduino-cli manages to update. So I assume that the ArduinoIDE could be able too.

No, there is no error message in the IDE. At least not in the "default settings". Maybe there is some debug option that could be tried?

[matthijskooijman]

Hmm, arduino-cli manages to update.

Maybe that uses a different method (i.e. without first downloading into new directory or something like that), but that -cli manages it is a sign that maybe the IDE could be changed to work as well indeed.

No, there is no error message in the IDE. At least not in the "default settings". Maybe there is some debug option that could be tried?

You could try running it on the commandline (you need arduino-debug.exe then) and see if that outputs anything else (though in practice, it should only show more startup messages, I think, but it would be worth trying). I wouldn't be surprised if the error is suppressed in the IDE code somewhere, though...

[anno73]

Hmm, arduino-cli manages to update.

Maybe that uses a different method (i.e. without first downloading into new directory or something like that), but that -cli manages it is a sign that maybe the IDE could be changed to work as well indeed.

I fear, I 'lied' to you ... I just found empty package-\d+ directories. And in addition I got error message from arduino-cli:

d:\CloudStorage\Dropbox\Work\Arduino\libraries>arduino-cli lib install "Arduino Cloud Provider Examples"@1.0.0
Arduino Cloud Provider Examples@1.0.0 depends on Arduino Cloud Provider Examples@1.0.0
Downloading Arduino Cloud Provider Examples@1.0.0...
Arduino Cloud Provider Examples@1.0.0 already downloaded
Installing Arduino Cloud Provider Examples@1.0.0...
Installed Arduino Cloud Provider Examples@1.0.0

d:\CloudStorage\Dropbox\Work\Arduino\libraries>arduino-cli lib install "Arduino Cloud Provider Examples"@1.1.0
Arduino Cloud Provider Examples@1.1.0 depends on Arduino Cloud Provider Examples@1.1.0
Downloading Arduino Cloud Provider Examples@1.1.0...
Arduino Cloud Provider Examples@1.1.0 already downloaded
Installing Arduino Cloud Provider Examples@1.1.0...
Replacing Arduino_Cloud_Provider_Examples@1.0.0 with Arduino Cloud Provider Examples@1.1.0...
Installed Arduino Cloud Provider Examples@1.1.0

d:\CloudStorage\Dropbox\Work\Arduino\libraries>arduino-cli lib install "Arduino Cloud Provider Examples"@1.2.0
Arduino Cloud Provider Examples@1.2.0 depends on Arduino Cloud Provider Examples@1.2.0
Downloading Arduino Cloud Provider Examples@1.2.0...
Arduino Cloud Provider Examples@1.2.0 already downloaded
Installing Arduino Cloud Provider Examples@1.2.0...
Replacing Arduino_Cloud_Provider_Examples@1.1.0 with Arduino Cloud Provider Examples@1.2.0...
Error installing name:"Arduino Cloud Provider Examples" versionRequired:"1.2.0": moving extracted archive to destination dir: rename d:\CloudStorage\Dropbox\Work\Arduino\libraries\package-120976539\Arduino_Cloud_Provider_Examples-1.2.0 d:\CloudStorage\Dropbox\Work\Arduino\libraries\Arduino_Cloud_Provider_Examples: Access is denied.

This didn't happen on the previous tests I did.

No, there is no error message in the IDE. At least not in the "default settings". Maybe there is some debug option that could be tried?

You could try running it on the commandline (you need arduino-debug.exe then) and see if that outputs anything else (though in practice, it should only show more startup messages, I think, but it would be worth trying). I wouldn't be surprised if the error is suppressed in the IDE code somewhere, though...

When I start arduino_debug.exe I get following output when installing a library:

2021-01-26T00:04:47.140Z INFO c.a.u.n.FileDownloaderCache:131 [LibraryManager Installer Thread] The cache is not enable.
2021-01-26T00:04:47.141Z INFO c.a.u.n.HttpConnectionManager:153 [LibraryManager Installer Thread] Connect to https://downloads.arduino.cc/libraries/github.com/arduino/Arduino_Cloud_Provider_Examples-1.0.0.zip, method=GET, request id=CBD03A7E16B648C2
2021-01-26T00:04:47.796Z INFO c.a.u.n.HttpConnectionManager:157 [LibraryManager Installer Thread] Request complete URL="https://downloads.arduino.cc/libraries/github.com/arduino/Arduino_Cloud_Provider_Examples-1.0.0.zip", method=GET, response code=206, request id=CBD03A7E16B648C2, headers={null=[HTTP/1.1 206 Partial Content], X-Cache=[Miss from cloudfront], Server=[cloudflare], CF-Ray=[6175fa7f9dd8d72d-FRA], Content-Range=[bytes 0-10031/10032], X-Amz-Cf-Pop=[DUB2-C1], Connection=[keep-alive], Last-Modified=[Tue, 30 Apr 2019 16:53:17 GMT], x-amz-version-id=[Vq994o_5Zoga8IcDKbjSJtJdnpCNq3q.], Date=[Tue, 26 Jan 2021 00:04:47 GMT], Via=[1.1 2ad54f65c83b790e81bb57464047598b.cloudfront.net (CloudFront)], x-amz-replication-status=[COMPLETED], CF-Cache-Status=[MISS], ETag=["5793092fc64995c12f2db19268c84573"], X-Amz-Cf-Id=[64lP-F1NxrfgOfAZfXvQ7yyPa43jGKW8eBTRADMIdI3u1GQpecgEZw==], Vary=[Accept-Encoding], Set-Cookie=[__cfduid=dbbc93fd6becdc047f9ebb957ef090b631611619486; expires=Thu, 25-Feb-21 00:04:46 GMT; path=/; domain=.arduino.cc; HttpOnly; SameSite=Lax], x-amz-request-id=[D210F315921EFE31], alt-svc=[h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400], x-amz-id-2=[TdR9nMC1KvxhaRoQE6DOBogjYZbbNKzgv5+gR7H3WO0fy4ETGDBhCZwJU58SxploB3J9R2xlEiA=], Content-Length=[10032], Expect-CT=[max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"], cf-request-id=[07dd9ae3bf0000d72dff334000000001], Content-Type=[application/zip]}

and basically more of the same upon upgrade.