Pluggable Serial or Visualization Monitor

[PaulStoffregen]

Now that Pluggable Discovery is wrapped at at version 1, let's talk of a Pluggable Serial Monitor, and perhaps a Pluggable Visualization Monitor.

With Pluggable Serial Monitor, the IDE would run program which communicates with the target hardware using its own protocol, and then in turn communicates a stream of bytes to the IDE as if the IDE was using serial or network protocol.

With Pluggable Visualization Monitor, the IDE would run a program which communicates with the target hardware using its own protocol, where that program opens a window or other GUI and manages all of its update to the GUI.

[PaulStoffregen]

Teensy already does something like Pluggable Serial Monitor with IDE 1.8.5 to 1.8.15 using a large patch the Java code. A very simple scheme using stdin & stdout for the bidirectional data flow and stderr for status update is used.

One unexpected (at least to me) issue that came up with Teensy's implementation is the poor handling of buffering and flow control on Windows. While stdin / stdout offers a very simple interface, if we're already seeing problems with Teensy 4.0 able to use about 50% of the USB 2.0 bandwidth (480 Mbit/sec), in the not too distant future of microcontrollers with USB 3.0 bandwidth (5 Gbit/sec) we will need an interprocess communication method which gives reliable flow control.

[cmaglie]

I've made some benchmarks: https://github.com/cmaglie/data-transfers-benchmarks#readme to compare the "pure" performance of stdio vs TCP/IP, and it seems that stdio is not so bad after all (especially on windows it looks like it will outperform TCP/IP! the opposite happens on Linux, did not test on mac).

BTW is this huge data transfer from the monitor to the IDE is really needed? Can't we throttle (drop) data directly from the monitor tool if the IDE cannot keep up?

My rationale is:

• we have a data flow like: port -> monitor tool -> IDE
• the monitor tool proxies data from port to the IDE as fast as it can
• IDE GUIs are slow and eventually cannot keep up displaying the stream at full-speed (maybe with a GPU rendering :-))
• to avoid crashing, the IDE must begin to gracefully drop the incoming data

So, since the IDE will not be able to display it anyway, isn't better to throttle at the monitor level?
On the other hand I realize that implementing a flow control system is tricky to say the least...

A very simple scheme using stdin & stdout for the bidirectional data flow and stderr for status update is used.

Another question that comes to my mind is: how do you change port settings? AFAIU stderr is monodirectional from the "monitor tool" to the IDE so there no way on the IDE to communicate changes on the monitor.

In my mind I'd like to use TCP/IP to stream data in and out from the port and leave stdin/stdout only for signaling and control.

[PaulStoffregen]

TCP/IP for bidirectional data and stdin/stdout for status sounds good to me.

[PaulStoffregen]

Flow control is essential. When flow control works, the speed automatically throttles as needed by the GUI. I'm optimistic TCP/IP will give flow control on all platforms. According to official documentation, stdin/stdout should too, but my experience shows Windows doesn't.

Some protocols lack flow control. Usually those are fairly slow, but for those specific protocols data does indeed need to be discarded when the incoming buffer is full. For Pluggable Serial Monitor, my hope is the IDE can use TCP/IP and the pluggable utility program is responsible for handling the scenario where the TCP socket can't accept more data without blocking.

Flow control alone doesn't solve all IDE issues. The IDE must constrain its memory allocation. But that's a design issue entirely within the IDE, separate from Pluggable Serial Monitor or which protocol is used. I have a huge patch which solves this in the Java IDE, but it's far from simple. If you ever want to put that work into merging it, I'm happy to help (and of course the code is open sources & always available on my public github account). But if the new IDE is your main focus, I completely understand.

I'd like to move forward with a TCP/IP test program. Can you give me your thoughts on the stdout / stdin status messages you want to see? JSON both directions, or something simpler? Messages for the IP address and port numbers to use? Does the IDE listen for an incoming TCP connect from the pluggable utility, or the other way around?

[cmaglie]

@PaulStoffregen, sorry it took a bit of time to make up my mind...

I'm writing down the RFC but the general idea is:

• on stdio we will talk a protocol very similar to the Pluggable Discovery, so:
  • command sent the monitor are single line ASCII commands
  • responses from the monitor are in json format
• one of the commands will be CONNECT <address>. It will connect with the underlying port and start to listen to a localhost TCP port. The listening port it's sent back in the json response and the IDE should connect to that TCP port to start receiving data back and forth.

Hope this is enough for you to start experimenting.

[PaulStoffregen]

I'm a little confused by 2 uses of the word "it", and also the word "connect".

It will connect with the underlying port and start to listen to a localhost TCP port. The listening port it's sent back in the json response and the IDE should connect to that TCP port to start receiving data back and forth.

If we use the BSD sockets API terminology (socket, listen, bind, connect) and avoid pronouns, I believe this might mean:

The pluggable serial utility will bind() to a localhost TCP port and listen() for incoming connections from the IDE. The IDE transmits CONNECT <address> to the pluggable serial utility's stdin stream, which responds on stdout with a JSON message to inform the IDE of the port number where it is listening for incoming connections. Then the IDE will connect(). The two start can communicate data back and forth as long as the connection is active.

But I don't understand why the IDE would transmit <address> during this dialogue?

[cmaglie]

The pluggable serial utility will bind() to a localhost TCP port and listen() for incoming connections from the IDE. The IDE transmits CONNECT

to the pluggable serial utility's stdin stream, which responds on stdout with a JSON message to inform the IDE of the port number where it is listening for incoming connections. Then the IDE will connect(). The two start can communicate data back and forth as long as the connection is active.

Correct

But I don't understand why the IDE would transmit

during this dialogue?

I mean the port address, like CONNECT /dev/ttyACM0, otherwise, the pluggable serial tool does not know which port to open.

A complete example may be:

1. IDE launches the pluggable serial tool
2. IDE sends the command CONNECT /dev/ttyACM0 via stdout
3. The pluggable serial tool opens ttyACM0 and start to listen to a random TCP port (let suppose it listens to 127.0.0.1:40123)
4. The pluggable serial tool sends the response to the IDE via stdin: { "event":"connected", "data_stream_at":"127.0.0.1:40123" }
5. The IDE connects via TCP/IP to 127.0.0.1:40123 to actually get the stream data from the serial port

Now that I'm writing it I don't know if the pluggable serial tool should open ttyACM0 before TCP-listening. Maybe a better strategy would be to first establish the TCP connection and open ttyACM0 just after, like:

1. IDE launches the pluggable serial tool
2. IDE sends the command CONNECT /dev/ttyACM0 via stdout
3. The pluggable serial tool start to listen to a random TCP port (let suppose it listens to 127.0.0.1:40123)
4. The pluggable serial tool sends the response to the IDE via stdin: { "event":"connected", "data_stream_at":"127.0.0.1:40123", "port_address": "/dev/ttyACM0" }
5. The IDE connects via TCP/IP to 127.0.0.1:40123 to actually get the stream data from the serial port
6. The pluggable serial tool opens ttyACM0 and starts streaming the serial data.

(the JSON responses are just examples, they will be surely some small changes in the final specification)

[PaulStoffregen]

Ah yes, that's much clearer.

Before I dive into this, I'd like to ask if you're sure you want the pluggable utility to act as a server (calls listen) and the IDE acting as a client (calls connect)?

If the roles were reversed, we could avoid the scenario where a TCP connection is established but no hardware access is working, because the pluggable utility would be responsible for starting the connection only when the hardware access is confirmed working. And in the less likely scenario where hardware works but the TCP connection isn't possible, the hardware could be promptly closed before any data is read from it.

[cmaglie]

so what you propose is:

1. IDE launches the pluggable serial tool
2. IDE start listening to a random TCP/IP port (let suppose it listens to 127.0.0.1:40123)
3. IDE sends the command CONNECT 127.0.0.1:40123 /dev/ttyACM0 via stdout (that means "please open ttyACM0 and send me data over the TCP port 40123")
4. The pluggable serial tool opens ttyACM0
5. The pluggable serial tool connects to 127.0.0.1:40123 and start streaming serial data
6. The pluggable serial tool sends the response to the IDE via stdin: { "event":"connected", "message":"OK" }

if the serial tool could not open the port:

5. The pluggable serial tool sends an error to the IDE via stdin: { "event":"connected", "error":true, "message":"port not found" }

Sounds good to me, let's do it this way.

[cmaglie]

Here the first draft: #6

[PaulStoffregen]

@cmaglie - Any chance this was meant to be "eventType" rather than "event"? Also, any idea what I should send if my monitor tool has no configurable settings?

[PaulStoffregen]

Well, I'm stuck. Pretty sure it's supposed to be "eventType" looking at arduino/monitor/monitor.go, and because I get an out of sync error if I use event rather than eventType.

But I must be missing something else. I'm sending this in response to the DESCRIBE command.

{
  "eventType": "describe",
  "message": "ok",
  "port_description": {
    "protocol": "Teensy",
    "configuration_parameters": {
      "baudrate": {
        "label": "Baudrate",
        "type": "enum",
        "value": [
          "1200", "2400", "4800", "9600",
          "19200", "38400", "57600", "115200"
        ],
        "selected": "9600"
      }
    }
  }
}

But I always seem to get "Error getting port settings details".

INFO[0000] Starting monitor process                      monitor=teensy-monitor
INFO[0000] Monitor process started successfully!         monitor=teensy-monitor
INFO[0000] sending command                               command="HELLO 1 \"arduino-cli nightly-20220120\"" monitor=teensy-monitor
INFO[0000] received message                              error=false event_type=hello message=OK monitor=teensy-monitor
INFO[0000] sending command                               command=DESCRIBE monitor=teensy-monitor
INFO[0000] received message                              error=false event_type=describe message=ok monitor=teensy-monitor
INFO[0000] sending command                               command=QUIT monitor=teensy-monitor
ERRO[0000] stopped decode loop                           monitor=teensy-monitor
Error getting port settings details: Port monitor error: command 'describe' failed: ok
ERRO[0000] Error getting port settings details: Port monitor error: command 'describe' failed: ok 

Tried reading the Go code. I see a called to EnumerateMonitorPortSettings() in cli/monitor/monitor.go at line 84 seems to be failing. Go is an unfamiliar language, but my best guess is the call to m.Describe in commands/monitor/settings.go at line 44 might be failing? Maybe? Perhaps this is calling Describe in arduino/monitor/monitor.go at line 313? But it seems like the only 2 errors are not being able to send or not getting a reply.

I see a bunch of json fields in data structures near the top of arduino/monitor/monitor.go. Most of it makes some sense, even though I don't know Go language, but I don't quite understand this line.

        ConfigurationParameters map[string]*PortParameterDescriptor `json:"configuration_parameters,omitempty"`

Sadly, with so little info in the verbose messages about why my JSON reply wasn't accepted, and my lack of familiarity with Go and trying to unravel how this code works, I'm afraid I'm completely stuck. Something is wrong with my reply to DESCRIBE, causing arduino-cli to immediately send QUIT and print that error, but I just can't figure out what it's really expecting.