Skip to content

CLI cannot gracefully handle invalid 3rd party URLs when initializing the index #2254

Closed
@kittaakos

Description

@kittaakos

Describe the problem

I am working on arduino/arduino-ide#2021 after #2119. I experience error messages when I initialize the directories.data folder via core update-index, and an invalid 3rd party URL is defined. The same problem is present via the gRPC API.

To reproduce

I have a bleeding edge CLI from 1a85d16. Probably, the issue can be reproduced from 62bb4c4.

./arduino-cli version                         
arduino-cli  Version: git-snapshot Commit: 1a85d168 Date: 2023-08-02T10:28:17Z

This is my CLI directories and additional_urls config:

cat ~/.arduinoIDE/arduino-cli.yaml | yq .directories
{
  "builtin": {
    "libraries": "/Users/a.kitta/Library/Arduino15/libraries"
  },
  "data": "/Users/a.kitta/Library/Arduino15",
  "downloads": "/Users/a.kitta/Library/Arduino15/staging",
  "user": "/Users/a.kitta/Documents/Arduino"
}
cat ~/.arduinoIDE/arduino-cli.yaml | yq .board_manager.additional_urls
[
  "https://invalidUrl"
]

My directories.data location is "empty":

ls ~/Library/Arduino15
inventory.yaml

❌ Update the indexes using the CLI config. It fails:

./arduino-cli core update-index --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}

❌ Try to run a platform search. It also fails:

./arduino-cli core search "arduino:avr"  --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}

Expected behavior

The CLI can gracefully handle when a 3rd party package URL is invalid during the index update.
The CLI reports if the URL is invalid. This is working as expected in IDE 2.1.1 with CLI 0.33.1.

Screenshot 2023-08-02 at 14 00 35

If I use the default text output, I see the expected no such host error. But it's missing from the JSON and gRPC levels.

./arduino-cli core search "arduino:avr"  --config-file ~/.arduinoIDE/arduino-cli.yaml             
Downloading index: package_index.tar.bz2 downloaded                                             
Downloading index: . Get "https://invalidUrl": dial tcp: lookup invalidUrl: no such host
Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory
Downloading index: package_index.tar.bz2 downloaded                                             
Downloading index: . Get "https://invalidUrl": dial tcp: lookup invalidUrl: no such host
Some indexes could not be updated.

Arduino CLI version

62bb4c4

Operating system

macOS

Operating system version

13.4.1

Additional context

I have noticed that although there is an error message, the primary package and the libraries index files are there, and the tools exist.

rm -rf ~/Library/Arduino15
./arduino-cli core update-index --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}
tree ~/Library/Arduino15
/Users/a.kitta/Library/Arduino15
├── inventory.yaml
├── library_index.json
├── library_index.json.sig
├── package_index.json
├── package_index.json.sig
├── packages
│   └── builtin
│       └── tools
│           ├── ctags
│           │   └── 5.8-arduino11
│           │       └── ctags
│           ├── dfu-discovery
│           │   └── 0.1.2
│           │       ├── LICENSE.txt
│           │       └── dfu-discovery
│           ├── mdns-discovery
│           │   └── 1.0.9
│           │       ├── LICENSE.txt
│           │       └── mdns-discovery
│           ├── serial-discovery
│           │   └── 1.4.0
│           │       ├── LICENSE.txt
│           │       └── serial-discovery
│           └── serial-monitor
│               └── 0.13.0
│                   ├── LICENSE.txt
│                   └── serial-monitor
├── staging
│   └── packages
│       ├── ctags-5.8-arduino11-pm-x86_64-apple-darwin.zip
│       ├── dfu-discovery_v0.1.2_macOS_64bit.tar.gz
│       ├── mdns-discovery_v1.0.9_macOS_64bit.tar.gz
│       ├── serial-discovery_v1.4.0_macOS_64bit.tar.gz
│       └── serial-monitor_v0.13.0_macOS_64bit.tar.gz
└── tmp

16 directories, 19 files

I have also noticed that executing any subsequent core search commands produce the same error message, while core list does not.

./arduino-cli core search "" --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}
./arduino-cli core list --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
[]

Issue checklist

  • I searched for previous reports in the issue tracker
  • I verified the problem still occurs when using the nightly build
  • My report contains all necessary details

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic: codeRelated to content of the project itselftype: imperfectionPerceived defect in any part of project

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions