Error in BearSSL when attempting to connect to secure MQTT broker

[schnorea]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: ESP-12

• Core Version: SDK:2.2.2-dev(38a443e)/Core:2.7.4=20704000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-30-g92add50/BearSSL:5c771be and with straight from master.

• Development Env: Arduino IDE

• Operating System: Ubuntu & MacOS

Settings in IDE

• Module: Nodemcu
• Flash Mode: qio
• Flash Size: 4MB
• lwip Variant: v2 Lower Memory
• Reset Method: nodemcu
• Flash Frequency: ?
• CPU Frequency: 160MHz
• Upload Using: SERIAL
• Upload Speed: 115200

Problem Description

Attempting to connect to AWS IoT. MQTT client sometime is rarely successful most of the times it errors out in BearSSL. I know the AWS IoT setup works as the credentials used on other platforms and as I said it sometimes works. Same setup used on ESP8266 with sometime success. When success works as expected. When it fails the wifi connects, the SNTP gets the time, the MQTT client gets an IP addess from AWS, then attempts to connect and fails which causes and exception. The exception is either a 28 or a 9 depending on the configuration but is always associated with the same line of code ( br_ssl_hs_client_run at src/ssl/ssl_hs_client.c line 1871).

This version uses the MQTT client but the PubSubClient has the same behavior so i don't think it has to do with these libraries.

I have built this on both my MAC Powerbook and Ubuntu 18.04 to the same effect.

I have attempted to use GDB with some success but tracing this problem usually results in a segfault on target and then GDB exits abruptly. This eliminates the ability to do a backtrace.

I have looked at the max stack usage and this doesn't look like the problem. One interesting thing is that when it fails the max has like three different levels. When successful it is always the same value.

When it fails it looks like this:

Max stack 4580, 4532, 4564, 4548 or something else.

When successful it is always:

Max stack 4596

When successful the code will print out messages sent via the AWS IOT Core test client as would be expected.

What i show is with 2.7.4 but i have pulled master and it has very similar behavior but doesn't print out the exception stack so i didn't include that.

MCVE Sketch

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <MQTT.h>
#include <ArduinoJson.h>
#include <time.h>
#include <StackThunk.h>

#define emptyString String()

#include "secrets.h"

const int MQTT_PORT = 8883;
const char MQTT_SUB_TOPIC[] = "robo/cmds";
const char MQTT_PUB_TOPIC[] = "$aws/things/" THINGNAME "/shadow/update";

#ifdef USE_SUMMER_TIME_DST
uint8_t DST = 1;
#else
uint8_t DST = 0;
#endif

WiFiClientSecure net;

BearSSL::X509List cert(cacert);
BearSSL::X509List client_crt(client_cert);
BearSSL::PrivateKey key(privkey);

MQTTClient client;

unsigned long lastMillis = 0;
time_t now;
time_t nowish = 1510592825;

void NTPConnect(void)
{
  Serial.print("Setting time using SNTP");
  configTime(TIME_ZONE * 3600, DST * 3600, "pool.ntp.org", "time.nist.gov");
  now = time(nullptr);
  while (now < nowish)
  {
    delay(500);
    Serial.print(".");
    now = time(nullptr);
  }
  Serial.println("done!");
  struct tm timeinfo;
  gmtime_r(&now, &timeinfo);
  Serial.print("Current time: ");
  Serial.print(asctime(&timeinfo));
}

void messageReceived(String &topic, String &payload)
{
  Serial.println("Received [" + topic + "]: " + payload);
}

void lwMQTTErr(lwmqtt_err_t reason)
{
  if (reason == lwmqtt_err_t::LWMQTT_SUCCESS)
    Serial.print("Success");
  else if (reason == lwmqtt_err_t::LWMQTT_BUFFER_TOO_SHORT)
    Serial.print("Buffer too short");
  else if (reason == lwmqtt_err_t::LWMQTT_VARNUM_OVERFLOW)
    Serial.print("Varnum overflow");
  else if (reason == lwmqtt_err_t::LWMQTT_NETWORK_FAILED_CONNECT)
    Serial.print("Network failed connect");
  else if (reason == lwmqtt_err_t::LWMQTT_NETWORK_TIMEOUT)
    Serial.print("Network timeout");
  else if (reason == lwmqtt_err_t::LWMQTT_NETWORK_FAILED_READ)
    Serial.print("Network failed read");
  else if (reason == lwmqtt_err_t::LWMQTT_NETWORK_FAILED_WRITE)
    Serial.print("Network failed write");
  else if (reason == lwmqtt_err_t::LWMQTT_REMAINING_LENGTH_OVERFLOW)
    Serial.print("Remaining length overflow");
  else if (reason == lwmqtt_err_t::LWMQTT_REMAINING_LENGTH_MISMATCH)
    Serial.print("Remaining length mismatch");
  else if (reason == lwmqtt_err_t::LWMQTT_MISSING_OR_WRONG_PACKET)
    Serial.print("Missing or wrong packet");
  else if (reason == lwmqtt_err_t::LWMQTT_CONNECTION_DENIED)
    Serial.print("Connection denied");
  else if (reason == lwmqtt_err_t::LWMQTT_FAILED_SUBSCRIPTION)
    Serial.print("Failed subscription");
  else if (reason == lwmqtt_err_t::LWMQTT_SUBACK_ARRAY_OVERFLOW)
    Serial.print("Suback array overflow");
  else if (reason == lwmqtt_err_t::LWMQTT_PONG_TIMEOUT)
    Serial.print("Pong timeout");
}

void lwMQTTErrConnection(lwmqtt_return_code_t reason)
{
  if (reason == lwmqtt_return_code_t::LWMQTT_CONNECTION_ACCEPTED)
    Serial.print("Connection Accepted");
  else if (reason == lwmqtt_return_code_t::LWMQTT_UNACCEPTABLE_PROTOCOL)
    Serial.print("Unacceptable Protocol");
  else if (reason == lwmqtt_return_code_t::LWMQTT_IDENTIFIER_REJECTED)
    Serial.print("Identifier Rejected");
  else if (reason == lwmqtt_return_code_t::LWMQTT_SERVER_UNAVAILABLE)
    Serial.print("Server Unavailable");
  else if (reason == lwmqtt_return_code_t::LWMQTT_BAD_USERNAME_OR_PASSWORD)
    Serial.print("Bad UserName/Password");
  else if (reason == lwmqtt_return_code_t::LWMQTT_NOT_AUTHORIZED)
    Serial.print("Not Authorized");
  else if (reason == lwmqtt_return_code_t::LWMQTT_UNKNOWN_RETURN_CODE)
    Serial.print("Unknown Return Code");
}

void connectToMqtt(bool nonBlocking = false)
{
  Serial.print("MQTT connecting ");
  while (!client.connected())
  {
    if (client.connect(THINGNAME))
    {
      Serial.println("connected!");
      if (!client.subscribe(MQTT_SUB_TOPIC))
        lwMQTTErr(client.lastError());
    }
    else
    {
      Serial.print("SSL Error Code: ");
      Serial.println(net.getLastSSLError());
      Serial.print("failed, reason -> ");
      lwMQTTErrConnection(client.returnCode());
      if (!nonBlocking)
      {
        Serial.println(" < try again in 5 seconds");
        delay(5000);
      }
      else
      {
        Serial.println(" <");
      }
    }
    Serial.printf("Max stack %d.", stack_thunk_get_max_usage());
    if (nonBlocking)
      break;
  }
}

void connectToWiFi(String init_str)
{
  if (init_str != emptyString)
    Serial.print(init_str);
  while (WiFi.status() != WL_CONNECTED)
  {
    Serial.print(".");
    delay(1000);
  }
  if (init_str != emptyString)
    Serial.println("ok!");
}

void checkWiFiThenMQTT(void)
{
  connectToWiFi("Checking WiFi");
  connectToMqtt();
}

unsigned long previousMillis = 0;
const long interval = 5000;

void checkWiFiThenMQTTNonBlocking(void)
{
  connectToWiFi(emptyString);
  if (millis() - previousMillis >= interval && !client.connected()) {
    previousMillis = millis();
    connectToMqtt(true);
  }
}

void checkWiFiThenReboot(void)
{
  connectToWiFi("Checking WiFi");
  Serial.print("Rebooting");
  ESP.restart();
}

void sendData(void)
{
  DynamicJsonDocument jsonBuffer(JSON_OBJECT_SIZE(3) + 100);
  JsonObject root = jsonBuffer.to<JsonObject>();
  JsonObject state = root.createNestedObject("state");
  JsonObject state_reported = state.createNestedObject("reported");
  state_reported["value"] = random(100);
  Serial.printf("Sending  [%s]: ", MQTT_PUB_TOPIC);
  serializeJson(root, Serial);
  Serial.println();
  char shadow[measureJson(root) + 1];
  serializeJson(root, shadow, sizeof(shadow));
  if (!client.publish(MQTT_PUB_TOPIC, shadow, false, 0))
    lwMQTTErr(client.lastError());
}

void setup()
{
  Serial.begin(115200);
  //gdbstub_init();
  delay(5000);
  Serial.println();
  Serial.println();
  WiFi.hostname(THINGNAME);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, pass);
  connectToWiFi(String("Attempting to connect to SSID: ") + String(ssid));

  NTPConnect();

  net.setTrustAnchors(&cert);
  net.setClientRSACert(&client_crt, &key);
  client.begin(MQTT_HOST, MQTT_PORT, net);
  client.onMessage(messageReceived);

  connectToMqtt();
}

void loop()
{
  //now = time(nullptr);
  if (!client.connected())
  {
    Serial.print("Not Connected");
    checkWiFiThenMQTT();
    //checkWiFiThenMQTTNonBlocking();
    //checkWiFiThenReboot();
  }
  else
  {
    client.loop();
    if (millis() - lastMillis > 5000)
    {
      lastMillis = millis();
      sendData();
    }
  }
}

Debug Messages

SDK:2.2.2-dev(38a443e)/Core:2.7.4=20704000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-30-g92add50/BearSSL:5c771be
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 0 (4)
reconnect


scandone
Attempting to connect to SSID: ARSnet...scandone
state: 0 -> 2 (b0)
.state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 15
cnt 

connected with ARSnet, channel 6
dhcp client start...
ip:192.168.2.122,mask:255.255.255.0,gw:192.168.2.1
ok!
Setting time using SNTP..................pm open,type:2 0
...........done!
Current time: Thu Dec 31 00:24:09 2020
MQTT connecting BSSL:_connectSSL: start connection
BSSL:_run_until: Timeout
BSSL:_wait_for_handshake: failed
BSSL:Couldn't connect. Error = 'Unknown error code.'
SSL Error Code: 0
failed, reason -> Connection Accepted < try again in 5 seconds
Max stack 4580.Fatal exception 28(LoadProhibitedCause):
epc1=0x4022451e, epc2=0x00000000, epc3=0x00000000, excvaddr=0x0000000c, depc=0x00000000

Decoded Exception

Exception 28: LoadProhibited: A load referenced a page mapped with an attribute that does not permit loads
PC: 0x4022309a: br_ssl_hs_client_run at src/ssl/ssl_hs_client.c line 1871
EXCVADDR: 0x0000000c

Decoding stack results
0x402229e5: br_ssl_hs_client_run at src/ssl/ssl_hs_client.c line 1528
0x40221120: jump_handshake at src/ssl/ssl_engine.c line 1081
0x40221605: br_ssl_engine_recvrec_ack at src/ssl/ssl_engine.c line 1206
0x402060de: BearSSL::WiFiClientSecure::_run_until(unsigned int, bool) at /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp line 540
0x40206f08: HardwareSerial::write(unsigned char const*, unsigned int) at /home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266/HardwareSerial.h line 165
0x402072a1: Print::write(char const*) at /home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266/Print.h line 62
0x4020628d: BearSSL::WiFiClientSecure::available() at /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp line 386
0x402053aa: BearSSL::WiFiClientSecure::connected() at /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp line 260
0x40208b2d: __delay(unsigned long) at /home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266/core_esp8266_wiring.cpp line 57
0x4020a65d: MQTTClient::connected() at /home/ccdocker/Arduino/libraries/MQTT/src/MQTTClient.cpp line 398
0x40202330: connectToMqtt(bool) at /home/ccdocker/projects/2020/robot_iot_3/robot_iot_3.ino line 139
0x4020d7de: strdup at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/string/strdup.c line 11
0x402024d1: setup() at /home/ccdocker/projects/2020/robot_iot_3/robot_iot_3.ino line 259
0x402085dc: loop_wrapper() at /home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266/core_esp8266_main.cpp line 194

[earlephilhower]

Can you please enable full debugging in the IDE and get the logs? That often gives a better idea about handshake difficulties and such. There were also 3.0.0 changes to fix reporting of some errors that you may profit from, so consider trying the master, too.

Your crash is pretty obviously a nullptr deref inside MQTTClient (the execVADDR = 0x0000 + some offset). You would need to check the library code for that, it's not part of our stuff. Could be an OOM error on a new or a malloc not being handled properly (you need 26KB+ free, mostly contiguous for SSL, and the JSON libraries I've seen are all monster memory hogs).

[schnorea]

On the logs I will give it a go and post them.
I tried 3.0.0 and the only difference was that I didn't get the exception print out so I need to find out where that get's enabled.
I also tried multiple MQTTClients had the same issue pointing to BearSSL.
I don't need JSON lib so i will remove that and try again.

Thanks for the pointers.

[schnorea]

Removing JSON didn't change anything except it loads a lot quicker.

[schnorea]

IDE Build Log

/home/ccdocker/bin/arduino-1.8.13/arduino-builder -dump-prefs -logger=machine -hardware /home/ccdocker/bin/arduino-1.8.13/hardware -hardware /home/ccdocker/.arduino15/packages -hardware /home/ccdocker/Arduino/hardware -tools /home/ccdocker/bin/arduino-1.8.13/tools-builder -tools /home/ccdocker/bin/arduino-1.8.13/hardware/tools/avr -tools /home/ccdocker/.arduino15/packages -built-in-libraries /home/ccdocker/bin/arduino-1.8.13/libraries -libraries /home/ccdocker/Arduino/libraries -fqbn=esp8266com:esp8266:nodemcuv2:xtal=160,vt=flash,exception=disabled,ssl=basic,eesz=4M,led=2,ip=lm2f,dbg=Serial,lvl=SSLTLS_MEMHTTP_CLIENTHTTP_SERVERCOREWIFIHTTP_UPDATEUPDATEROTAOOMMDNS,wipe=all,baud=115200 -vid-pid=10C4_EA60 -ide-version=10813 -build-path /tmp/arduino_build_649514 -warnings=all -build-cache /tmp/arduino_cache_141023 -prefs=build.warn_data_percentage=75 -verbose /home/ccdocker/projects/2020/robot_iot_3a/robot_iot_3a.ino
/home/ccdocker/bin/arduino-1.8.13/arduino-builder -compile -logger=machine -hardware /home/ccdocker/bin/arduino-1.8.13/hardware -hardware /home/ccdocker/.arduino15/packages -hardware /home/ccdocker/Arduino/hardware -tools /home/ccdocker/bin/arduino-1.8.13/tools-builder -tools /home/ccdocker/bin/arduino-1.8.13/hardware/tools/avr -tools /home/ccdocker/.arduino15/packages -built-in-libraries /home/ccdocker/bin/arduino-1.8.13/libraries -libraries /home/ccdocker/Arduino/libraries -fqbn=esp8266com:esp8266:nodemcuv2:xtal=160,vt=flash,exception=disabled,ssl=basic,eesz=4M,led=2,ip=lm2f,dbg=Serial,lvl=SSLTLS_MEMHTTP_CLIENTHTTP_SERVERCOREWIFIHTTP_UPDATEUPDATEROTAOOMMDNS,wipe=all,baud=115200 -vid-pid=10C4_EA60 -ide-version=10813 -build-path /tmp/arduino_build_649514 -warnings=all -build-cache /tmp/arduino_cache_141023 -prefs=build.warn_data_percentage=75 -verbose /home/ccdocker/projects/2020/robot_iot_3a/robot_iot_3a.ino
Using board 'nodemcuv2' from platform in folder: /home/ccdocker/Arduino/hardware/esp8266com/esp8266
Using core 'esp8266' from platform in folder: /home/ccdocker/Arduino/hardware/esp8266com/esp8266
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/python3/python3 /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/makecorever.py --build_path /tmp/arduino_build_649514 --platform_path /home/ccdocker/Arduino/hardware/esp8266com/esp8266 --version unix-2.7.4
Detecting libraries used...
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++ -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip2/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include -I/tmp/arduino_build_649514/core -c -w -Os -g -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++11 -ffunction-sections -fdata-sections -fno-exceptions -DNEW_OOM_ABORT -DBEARSSL_SSL_BASIC -w -x c++ -E -CC -DNONOSDK22x_190703=1 -DF_CPU=160000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_OOM -DDEBUG_ESP_MDNS -DARDUINO=10813 -DARDUINO_ESP8266_NODEMCU -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_NODEMCU\"" -DLED_BUILTIN=2 -DFLASHMODE_DIO -DESP8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/variants/nodemcu /tmp/arduino_build_649514/sketch/robot_iot_3a.ino.cpp -o /dev/null -DARDUINO_LIB_DISCOVERY_PHASE
Alternatives for ESP8266WiFi.h: [ESP8266WiFi@1.0]
ResolveLibrary(ESP8266WiFi.h)
  -> candidates: [ESP8266WiFi@1.0]
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++ -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip2/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include -I/tmp/arduino_build_649514/core -c -w -Os -g -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++11 -ffunction-sections -fdata-sections -fno-exceptions -DNEW_OOM_ABORT -DBEARSSL_SSL_BASIC -w -x c++ -E -CC -DNONOSDK22x_190703=1 -DF_CPU=160000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_OOM -DDEBUG_ESP_MDNS -DARDUINO=10813 -DARDUINO_ESP8266_NODEMCU -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_NODEMCU\"" -DLED_BUILTIN=2 -DFLASHMODE_DIO -DESP8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/variants/nodemcu -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src /tmp/arduino_build_649514/sketch/robot_iot_3a.ino.cpp -o /dev/null -DARDUINO_LIB_DISCOVERY_PHASE
Alternatives for MQTT.h: [MQTT@2.4.8]
ResolveLibrary(MQTT.h)
  -> candidates: [MQTT@2.4.8]
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++ -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip2/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include -I/tmp/arduino_build_649514/core -c -w -Os -g -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++11 -ffunction-sections -fdata-sections -fno-exceptions -DNEW_OOM_ABORT -DBEARSSL_SSL_BASIC -w -x c++ -E -CC -DNONOSDK22x_190703=1 -DF_CPU=160000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_OOM -DDEBUG_ESP_MDNS -DARDUINO=10813 -DARDUINO_ESP8266_NODEMCU -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_NODEMCU\"" -DLED_BUILTIN=2 -DFLASHMODE_DIO -DESP8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/variants/nodemcu -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src -I/home/ccdocker/Arduino/libraries/MQTT/src /tmp/arduino_build_649514/sketch/robot_iot_3a.ino.cpp -o /dev/null -DARDUINO_LIB_DISCOVERY_PHASE
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/BearSSLHelpers.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/CertStoreBearSSL.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFi.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFiAP.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFiGeneric.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFiGratuitous.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFiMulti.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFiSTA-WPS.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFiSTA.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/ESP8266WiFiScan.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiClient.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiClientSecureAxTLS.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiServer.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiServerSecureAxTLS.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiServerSecureBearSSL.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiUdp.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/libraries/MQTT/src/MQTTClient.cpp
Using cached library dependencies for file: /home/ccdocker/Arduino/libraries/MQTT/src/lwmqtt/client.c
Using cached library dependencies for file: /home/ccdocker/Arduino/libraries/MQTT/src/lwmqtt/helpers.c
Using cached library dependencies for file: /home/ccdocker/Arduino/libraries/MQTT/src/lwmqtt/packet.c
Using cached library dependencies for file: /home/ccdocker/Arduino/libraries/MQTT/src/lwmqtt/string.c
Generating function prototypes...
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++ -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip2/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include -I/tmp/arduino_build_649514/core -c -w -Os -g -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++11 -ffunction-sections -fdata-sections -fno-exceptions -DNEW_OOM_ABORT -DBEARSSL_SSL_BASIC -w -x c++ -E -CC -DNONOSDK22x_190703=1 -DF_CPU=160000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_OOM -DDEBUG_ESP_MDNS -DARDUINO=10813 -DARDUINO_ESP8266_NODEMCU -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_NODEMCU\"" -DLED_BUILTIN=2 -DFLASHMODE_DIO -DESP8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/variants/nodemcu -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src -I/home/ccdocker/Arduino/libraries/MQTT/src /tmp/arduino_build_649514/sketch/robot_iot_3a.ino.cpp -o /tmp/arduino_build_649514/preproc/ctags_target_for_gcc_minus_e.cpp -DARDUINO_LIB_DISCOVERY_PHASE
/home/ccdocker/bin/arduino-1.8.13/tools-builder/ctags/5.8-arduino11/ctags -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives /tmp/arduino_build_649514/preproc/ctags_target_for_gcc_minus_e.cpp
Compiling sketch...
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/python3/python3 /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/signing.py --mode header --publickey /home/ccdocker/projects/2020/robot_iot_3a/public.key --out /tmp/arduino_build_649514/core/Updater_Signing.h
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++ -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip2/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include -I/tmp/arduino_build_649514/core -c -Wall -Wextra -Os -g -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++11 -MMD -ffunction-sections -fdata-sections -fno-exceptions -DNEW_OOM_ABORT -DBEARSSL_SSL_BASIC -DNONOSDK22x_190703=1 -DF_CPU=160000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_OOM -DDEBUG_ESP_MDNS -DARDUINO=10813 -DARDUINO_ESP8266_NODEMCU -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_NODEMCU\"" -DLED_BUILTIN=2 -DFLASHMODE_DIO -DESP8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/variants/nodemcu -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src -I/home/ccdocker/Arduino/libraries/MQTT/src /tmp/arduino_build_649514/sketch/robot_iot_3a.ino.cpp -o /tmp/arduino_build_649514/sketch/robot_iot_3a.ino.cpp.o
Compiling libraries...
Compiling library "ESP8266WiFi"
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiAP.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++ -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip2/include -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include -I/tmp/arduino_build_649514/core -c -Wall -Wextra -Os -g -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++11 -MMD -ffunction-sections -fdata-sections -fno-exceptions -DNEW_OOM_ABORT -DBEARSSL_SSL_BASIC -DNONOSDK22x_190703=1 -DF_CPU=160000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_ESP_OOM -DDEBUG_ESP_MDNS -DARDUINO=10813 -DARDUINO_ESP8266_NODEMCU -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_NODEMCU\"" -DLED_BUILTIN=2 -DFLASHMODE_DIO -DESP8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/cores/esp8266 -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/variants/nodemcu -I/home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src -I/home/ccdocker/Arduino/libraries/MQTT/src /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/BearSSLHelpers.cpp -o /tmp/arduino_build_649514/libraries/ESP8266WiFi/BearSSLHelpers.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/CertStoreBearSSL.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiGeneric.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiGratuitous.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiMulti.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiSTA-WPS.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiScan.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiClient.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiSTA.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiClientSecureBearSSL.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiServer.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiServerSecureBearSSL.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiServerSecureAxTLS.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiClientSecureAxTLS.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiUdp.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/BearSSLHelpers.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/CertStoreBearSSL.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiAP.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiGeneric.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiGratuitous.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiMulti.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiSTA-WPS.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiSTA.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFiScan.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiClient.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiClientSecureAxTLS.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiClientSecureBearSSL.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiServer.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiServerSecureAxTLS.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiServerSecureBearSSL.cpp.o
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-ar cru /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/ESP8266WiFi/WiFiUdp.cpp.o
Compiling library "MQTT"
Using previously compiled file: /tmp/arduino_build_649514/libraries/MQTT/MQTTClient.cpp.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/client.c.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/helpers.c.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/packet.c.o
Using previously compiled file: /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/string.c.o
Compiling core...
Using precompiled core: /tmp/arduino_cache_141023/core/core_26e1ec1ef89b3cf4115e9604b2a00718.a
Linking everything together...
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc -CC -E -P -DVTABLES_IN_FLASH /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/ld/eagle.app.v6.common.ld.h -o /tmp/arduino_build_649514/local.eagle.app.v6.common.ld
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc -fno-exceptions -DNEW_OOM_ABORT -Wl,-Map -Wl,/tmp/arduino_build_649514/robot_iot_3a.ino.map -g -Wall -Wextra -Os -nostdlib -Wl,--no-check-sections -u app_entry -u _printf_float -u _scanf_float -Wl,-static -L/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lib -L/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/lib/NONOSDK22x_190703 -L/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/ld -L/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/lib -Teagle.flash.4m.ld -Wl,--gc-sections -Wl,-wrap,system_restart_local -Wl,-wrap,spi_flash_read -o /tmp/arduino_build_649514/robot_iot_3a.ino.elf -Wl,--start-group /tmp/arduino_build_649514/sketch/robot_iot_3a.ino.cpp.o /tmp/arduino_build_649514/libraries/ESP8266WiFi/ESP8266WiFi.a /tmp/arduino_build_649514/libraries/MQTT/MQTTClient.cpp.o /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/client.c.o /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/helpers.c.o /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/packet.c.o /tmp/arduino_build_649514/libraries/MQTT/lwmqtt/string.c.o /tmp/arduino_cache_141023/core/core_26e1ec1ef89b3cf4115e9604b2a00718.a -lhal -lphy -lpp -lnet80211 -llwip2-536-feat -lwpa -lcrypto -lmain -lwps -lbearssl -laxtls -lespnow -lsmartconfig -lairkiss -lwpa2 -lstdc++ -lm -lc -lgcc -Wl,--end-group -L/tmp/arduino_build_649514
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/python3/python3 /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/elf2bin.py --eboot /home/ccdocker/Arduino/hardware/esp8266com/esp8266/bootloaders/eboot/eboot.elf --app /tmp/arduino_build_649514/robot_iot_3a.ino.elf --flash_mode dio --flash_freq 40 --flash_size 4M --path /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin --out /tmp/arduino_build_649514/robot_iot_3a.ino.bin
Creating BIN file "/tmp/arduino_build_649514/robot_iot_3a.ino.bin" using "/home/ccdocker/Arduino/hardware/esp8266com/esp8266/bootloaders/eboot/eboot.elf" and "/tmp/arduino_build_649514/robot_iot_3a.ino.elf"
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/python3/python3 /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/signing.py --mode sign --privatekey /home/ccdocker/projects/2020/robot_iot_3a/private.key --bin /tmp/arduino_build_649514/robot_iot_3a.ino.bin --out /tmp/arduino_build_649514/robot_iot_3a.ino.bin.signed --legacy /tmp/arduino_build_649514/robot_iot_3a.ino.bin.legacy_sig
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/python3/python3 /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/sizes.py --elf /tmp/arduino_build_649514/robot_iot_3a.ino.elf --path /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin
Executable segment sizes:
IROM   : 366252          - code in flash         (default or ICACHE_FLASH_ATTR) 
IRAM   : 28600   / 32768 - code in IRAM          (ICACHE_RAM_ATTR, ISRs...) 
DATA   : 1340  )         - initialized variables (global, static) in RAM/HEAP 
RODATA : 2248  ) / 81920 - constants             (global, static) in RAM/HEAP 
BSS    : 25584 )         - zeroed variables      (global, static) in RAM/HEAP 
Using library ESP8266WiFi at version 1.0 in folder: /home/ccdocker/Arduino/hardware/esp8266com/esp8266/libraries/ESP8266WiFi 
Using library MQTT at version 2.4.8 in folder: /home/ccdocker/Arduino/libraries/MQTT 
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-size -A /tmp/arduino_build_649514/robot_iot_3a.ino.elf
Sketch uses 398440 bytes (38%) of program storage space. Maximum is 1044464 bytes.
Global variables use 29172 bytes (35%) of dynamic memory, leaving 52748 bytes for local variables. Maximum is 81920 bytes.
/home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/python3/python3 /home/ccdocker/Arduino/hardware/esp8266com/esp8266/tools/upload.py --chip esp8266 --port /dev/ttyUSB0 --baud 115200 erase_flash --before default_reset --after hard_reset write_flash 0x0 /tmp/arduino_build_649514/robot_iot_3a.ino.bin 
esptool.py v2.8
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 60:01:94:2b:ce:38
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Erasing flash (this may take a while)...
Chip erase completed successfully in 6.1s
Compressed 402592 bytes to 293099...
Wrote 402592 bytes (293099 compressed) at 0x00000000 in 25.9 seconds (effective 124.2 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

[earlephilhower]

Thanks, but we need the Serial log to tell what's going on w/the connection.

[schnorea]

Failing output log with all debug enabled.

This log has an example of failure (the exception is decoded above).

SDK:2.2.2-dev(38a443e)/Core:2.7.4=20704000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-30-g92add50/BearSSL:5c771be
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 15
cnt 

connected with ARSnet, channel 6
dhcp client start...
wifi evt: 0
ip:192.168.2.122,mask:255.255.255.0,gw:192.168.2.1
wifi evt: 3


Attempting to connect to SSID: ARSnet...scandone
ok!
Setting time using SNTP...pm open,type:2 0
............done!
Current time: Thu Dec 31 04:13:13 2020
MQTT connecting [hostByName] request IP for: a3kf0d3b3a0ocb-ats.iot.us-east-1.amazonaws.com
[hostByName] Host: a3kf0d3b3a0ocb-ats.iot.us-east-1.amazonaws.com IP: 3.222.90.31
:ref 1
BSSL:_connectSSL: start connection
:wr 139 0
:wrc 139 139 0
:ack 139
:rn 90
:rd 5, 90, 0
:rdi 90, 5
:rd 85, 90, 5
:rdi 85, 85
:c0 85, 90
:rn 536
:rd 5, 536, 0
:rdi 536, 5
:rd 531, 536, 5
:rdi 531, 531
:c0 531, 536
:rn 536
:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
:rn 536
:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
:rn 536
:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
BSSL:_run_until: Timeout
BSSL:_wait_for_handshake: failed
BSSL:Couldn't connect. Error = 'Unknown error code.'
SSL Error Code: 0
failed, reason -> Connection Accepted < try again in 5 seconds
:rn 536
Max stack 4580.:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
Fatal exception 28(LoadProhibitedCause):
epc1=0x402238a6, epc2=0x00000000, epc3=0x00000000, excvaddr=0x0000000c, depc=0x00000000

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Exception (28):
epc1=0x402238a6 epc2=0x00000000 epc3=0x00000000 excvaddr=0x0000000c depc=0x00000000

>>>stack>>>

ctx: bearssl
sp: 3fff0cd0 end: 3fff1140 offset: 0190
3fff0e60:  00000055 3fff3560 0000099b 402231f1  
3fff0e70:  deadbeef deadbeef deadbeef deadbeef  
3fff0e80:  deadbe4d deadbeef deadbeef deadbeef  
3fff0e90:  deadbeef deadbeef deadbeef deadbeef  
3fff0ea0:  deadbeef deadbeef deadbeef deadbeef  
3fff0eb0:  deadbeef deadbeef deadbeef deadbeef  
3fff0ec0:  deadbeef deadbeef deadbeef deadbeef  
3fff0ed0:  deadbeef deadbeef deadbeef deadbeef  
3fff0ee0:  deadbeef deadbeef deadbeef deadbeef  
3fff0ef0:  deadbeef deadbeef deadbeef deadbeef  
3fff0f00:  deadbeef deadbeef deadbeef deadbeef  
3fff0f10:  deadbeef deadbeef deadbeef deadbeef  
3fff0f20:  deadbeef deadbeef deadbeef deadbeef  
3fff0f30:  deadbeef deadbeef deadbeef deadbeef  
3fff0f40:  deadbeef deadbeef deadbeef deadbeef  
3fff0f50:  deadbeef deadbeef deadbeef deadbeef  
3fff0f60:  deadbeef deadbeef deadbeef deadbeef  
3fff0f70:  deadbeef deadbeef deadbeef deadbeef  
3fff0f80:  deadbeef deadbeef deadbeef deadbeef  
3fff0f90:  deadbeef deadbeef deadbeef deadbeef  
3fff0fa0:  deadbeef deadbeef deadbeef deadbeef  
3fff0fb0:  deadbeef deadbeef deadbeef deadbeef  
3fff0fc0:  deadbeef deadbeef deadbeef deadbeef  
3fff0fd0:  deadbeef deadbeef deadbeef deadbeef  
3fff0fe0:  deadbeef deadbeef deadbeef deadbeef  
3fff0ff0:  deadbeef deadbeef deadbeef deadbeef  
3fff1000:  deadbeef deadbeef deadbeef deadbeef  
3fff1010:  deadbeef deadbeef deadbeef deadbeef  
3fff1020:  deadbeef deadbeef deadbeef deadbeef  
3fff1030:  deadbeef deadbeef deadbeef deadbeef  
3fff1040:  deadbeef deadbeef deadbeef deadbeef  
3fff1050:  deadbeef deadbeef deadbeef deadbeef  
3fff1060:  deadbeef deadbeef deadbeef deadbeef  
3fff1070:  deadbeef deadbeef deadbeef deadbeef  
3fff1080:  deadbeef deadbeef deadbeef deadbeef  
3fff1090:  deadbeef 4025526f deadbeef deadbeef  
3fff10a0:  3fff3280 3fff3304 deadbeef 3fff3260  
3fff10b0:  deadbeef 3fff2a7c 3fff3280 deadbeef  
3fff10c0:  deadbeef 00000055 3fff2a60 deadbeef  
3fff10d0:  deadbeef deadbeef deadbeef 3fff347c  
3fff10e0:  3fff357c 00000000 3fff2a7c 4022192c  
3fff10f0:  3fff3260 00000218 deadbeef deadbeef  
3fff1100:  00000250 00000005 deadbeef 3ffeeb38  
3fff1110:  00003a98 0000089f 3fff2a7c 40221e11  
3fff1120:  0000008b deadbeef deadbeef deadbeef  
3fff1130:  deadbeef 00000000 3ffee9f8 40202660  

ctx: cont
sp: 3ffffec0 end: 3fffffc0 offset: 0000
3ffffec0:  00003a98 00000000 00000010 40206506  
3ffffed0:  00000ab7 00000000 73646e6f 40209b59  
3ffffee0:  00000000 00008bf4 00000008 00000000  
3ffffef0:  40105761 00aaa38c 000011e4 00000000  
3fffff00:  00000000 00000000 3ffee9f8 402066c5  
3fffff10:  3ffe88e4 3ffee940 3ffee9f8 40205782  
3fffff20:  40209161 00001388 3ffee940 4020acbd  
3fffff30:  00000000 3ffee940 3ffeeb00 402022f8  
3fffff40:  00000000 000022b3 3ffee940 4020df96  
3fffff50:  00000000 00000000 3ffee940 3ffeec40  
3fffff60:  3fffdad0 3ffee940 3ffee9f8 402024b5  
3fffff70:  3fff2400 0025002f 80efeffe 3fff2400  
3fffff80:  0025002f 80efeffe 6e535200 fe007465  
3fffff90:  80efeffe 3fff2200 001f001f 80efeffe  
3fffffa0:  feefeffe 00000000 3ffeec00 40208bfc  
3fffffb0:  feefeffe feefeffe 3ffe853c 401012a1  
<<<stack<<<

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 3584, room 16 
tail 0
chksum 0xb0
csum 0xb0
v2843a5ac
~ld

[schnorea]

Successful output log

This is an example of when it works with full debug info.

embedded in the below log is this line:

Received [robo/cmds]: Robo cmd 0x81

which is the result of a MQTT message sent via AWS IoT Core.

SDK:2.2.2-dev(38a443e)/Core:2.7.4=20704000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-30-g92add50/BearSSL:5c771be
wifi evt: 2
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 0 (4)
reconnect
wifi evt: 1
STA disconnect: 4


scandone
Attempting to connect to SSID: ARSnet...scandone
state: 0 -> 2 (b0)
.state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 15
cnt 

connected with ARSnet, channel 6
dhcp client start...
wifi evt: 0
ip:192.168.2.122,mask:255.255.255.0,gw:192.168.2.1
wifi evt: 3
ok!
Setting time using SNTP..................pm open,type:2 0
...........done!
Current time: Thu Dec 31 04:13:58 2020
MQTT connecting [hostByName] request IP for: a3kf0d3b3a0ocb-ats.iot.us-east-1.amazonaws.com
[hostByName] Host: a3kf0d3b3a0ocb-ats.iot.us-east-1.amazonaws.com IP: 52.20.209.4
:ref 1
BSSL:_connectSSL: start connection
:wr 139 0
:wrc 139 139 0
:ack 139
:rn 90
:rd 5, 90, 0
:rdi 90, 5
:rd 85, 90, 5
:rdi 85, 85
:c0 85, 90
:rn 536
:rd 5, 536, 0
:rdi 536, 5
:rd 531, 536, 5
:rdi 531, 531
:c0 531, 536
:rn 1072
:rd 1072, 1072, 0
:rdi 536, 536
:c 536, 536, 1072
:rdi 536, 536
:c0 536, 536
:rn 536
:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
:rn 536
:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
:rn 1072
:rd 1072, 1072, 0
:rdi 536, 536
:c 536, 536, 1072
:rdi 536, 536
:c0 536, 536
:rn 536
:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
:rn 536
:rd 536, 536, 0
:rdi 536, 536
:c0 536, 536
:rn 109
:rd 63, 109, 0
:rdi 109, 63
:rd 5, 109, 63
:rdi 46, 5
:rd 32, 109, 68
:rdi 41, 32
:rd 5, 109, 100
:rdi 9, 5
:rd 4, 109, 105
:rdi 4, 4
:c0 4, 109
:wr 517 0
:wrc 517 517 0
:wr 517 0
:wrc 517 517 0
:ack 517
:ack 517
:wr 378 0
:wrc 378 378 0
:wr 6 0
:wrc 6 6 0
:wr 85 0
:wrc 85 85 0
:ack 378
:ack 91
:rn 91
:rd 5, 91, 0
:rdi 6, 5
:rd 1, 91, 5
:rdi 1, 1
:c 1, 6, 91
:rd 5, 85, 0
:rdi 85, 5
:rd 80, 85, 5
:rdi 80, 80
:c0 80, 85
BSSL:Connected!
:wr 85 0
:wrc 85 85 0
:ack 85
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
connected!
:wr 85 0
:wrc 85 85 0
:ack 85
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
Max stack 4596.:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:rn 85
:rd 5, 85, 0
:rdi 85, 5
:rd 80, 85, 5
:rdi 80, 80
:c0 80, 85
Received [robo/cmds]: Robo cmd 0x81
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69
:wr 69 0
:wrc 69 69 0
:ack 69
:rn 69
:rd 5, 69, 0
:rdi 69, 5
:rd 64, 69, 5
:rdi 64, 64
:c0 64, 69

[earlephilhower]

Sorry, but that doesn't look like it has BearSSL debugging enabled. The certificates sent and received (they're public certs and no private cert/key dumped so should be safe to post) as well as more handshake info is generally dumped. Make sure you're using the 2nd to last setting (longest one) as that also catches malloc fails and full logs elsewhere.

For example:

Trying: api.github.com:443...[hostByName] request IP for: api.github.com
[hostByName] Host: api.github.com IP: 192.30.255.117
:ur 1
:dsrcv 1367
:del
:ref 1
BSSL:_connectSSL: start connection

_iobuf_in:       0x3fff0bcc
_iobuf_out:      0x3fffb85c
_iobuf_in_size:  16709
_iobuf_out_size: 597
:wr 137 0
:wrc 137 137 0
:ack 137
:rn 536
:rch 536, 536
:rch 1072, 536
:rch 1608, 536
:rd 5, 2144, 0
:rdi 536, 5
:rd 90, 2144, 5
:rdi 531, 90
:rd 5, 2144, 95
:rdi 441, 5
:rd 512, 2144, 100
:rdi 436, 436
:c 436, 536, 2144
:rdi 536, 76
BSSL:CERT: 30 82 06 d4 30 82 05 bc a0 03 02 01 02 02 10 0c 07 3b 67 6f 67 45 78 f9 99 81 48 52 84 46 51 30 0d 06 09 2a 86 48 86 f7 0d 01 01 0b 05 00 30 70 31 0b 30 09 06 03 55 04 06 13 02 55 53 31 15 30 13 06 03 55 04 0a 13 0c 44 69 67 69 43 65 72 74 20 49 6e 63 31 19 30 17 06 03 55 04 0b 13 10 77 77 77 2e 64 69 67 69 63 65 72 74 2e 63 6f 6d 31 2f 30 2d 06 03 55 04 03 13 26 44 69 67 69 43 65 72 74 20 53 48 41 32 20 48 69 67 68 20 41 73 73 75 72 61 6e 63 65 20 53 65 72 76 65 72 20 43 41 30 1e 17 0d 32 30 30 36 32 32 30 30 30 30 30 30 5a 17 0d 32 32 30 38 31 37 31 32 30 30 30 30 5a 30 68 31 0b 30 09 06 03 55 04 06 13 02 55 53 31 13 30 11 06 03 55 04 08 13 0a 43 61 6c 69 66 6f 72 6e 69 61 31 16 30 14 06 03 55 04 07 13 0d 53 61 6e 20 46 72 61 6e 63 69 73 63 6f 31 15 30 13 
:rd 5, 1608, 76
:rdi 460, 5
:rd 512, 1608, 81
:rdi 455, 455
:c 455, 536, 1608
:rdi 536, 57
BSSL:CERT: 06 03 55 04 0a 13 0c 47 69 74 48 75 62 2c 20 49 6e 63 2e 31 15 30 13 06 03 55 04 03 0c 0c 2a 2e 67 69 74 68 75 62 2e 63 6f 6d 30 82 01 22 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 82 01 0f 00 30 82 01 0a 02 82 01 01 00 a0 e6 d3 87 ac 6f 4e 3b:rch 1072, 536
 29 75 60 4c 4a 1e fa dd af 81 0a 37 c4 89 ad b5 8e 9d 1d 0c 55 da a4 b1 cb ab d3 14 bc d2 6d b8 d1 7c 36 1f 45 a1 06 25 32 63 7f 94 4c f4 d6 97 06 3f 24 f2 85 f5 83 8c 27 8a 7f 6a c8 46 e8 04 6f 4c 5f 4c 48 9a a6 80 c1 08 db 9c 6e 81 8b 54 59 f0 c6 6d 58 2a 3d 42 ea da 5d aa 6b 90 7b af 12 34 30 1b 22 5c af cd ee f2 3c 08 90 99 91 be 41 16 c6 e0 95 59 a9 d6 52 39 de e9 a3 02 e2 68 e3 f9 b5 56 ce ae 62 27 5e ff a3 94 1f 89 82 0f 5d ea 82 4d af de 0f 3b aa 04 4a 6f a4 85 43 80 11 35 f1 3b d6 66 80 68 97 6e 0a e9 79 
BSSL:CERT: 57 63 44 91 c1 e2 45 db dd 2c b9 2d 3d 16 76 af 0b a4 04 80 c0 10 35 26 f2 9f 38 43 a8 1d a5 19 79 f0 b0 60 98 40 f8 4c 54 53 5f 32 0e de 86 65 e7 4f 5c 21 7a c7 35 7f de a3 e0 8e b2 d3 02 42 64 40 14 21 20 96 14 09 54 1f 53 02 03 01 00 01 a3 82 03 70 30 82 03 6c 30 1f 06 03 55 1d 23 04 18 30 16 80 14 51 68 ff 90 af 02 07 75 3c cc d9 65 64 62 a2 12 b8 59 72 3b 30 1d 06 03 55 1d 0e 04 16 04 14 79 53 13 17 5:rch 1608, 424
3 da 1e a9 73 a9 9e 88 8d c2 53 d9 36 e2 e5 2a 30 23 06 03 55 1d 11 04 1c 30 1a 82 0c 2a 2e 67 69 74 68 75 62 2e 63 6f 6d 82 0a 67 69 74 68 75 62 2e 63 6f 6d 30 0e 06 03 55 1d 0f 01 01 ff 04 04 03 02 05 a0 30 1d 06 03 55 1d 25 04 16 30 14 06 08 2b 06 01 05 05 07 03 01 06 08 2b 06 01 05 05 07 03 02 30 75 06 03 55 1d 1f 04 6e 30 6c 30 34 a0 32 a0 30 86 2e 68 
:rd 5, 2032, 57
:rdi 479, 5
:rd 512, 2032, 62
:rdi 474, 474
:c 474, 536, 2032
:rdi 536, 38
BSSL:CERT: 74 74 70 3a 2f 2f 63 72 6c 33 2e 64 69 67 69 63 65 72 74 2e 63 6f 6d 2f 73 68 61 32 2d 68 61 2d 73 65 72 76 65 72 2d 67 36 2e 63 72 6c 30 34 a0 32 a0 30 86 2e 68 74 74 70 3a 2f 2f 63 72 6c 34 2e 64 69 67 69 63 65 72 74 2e 63 6f 6d 2f 73 68 61 32 2d 68 61 2d 73 65 72 76 65 72 2d 67 36 2e 63 72 6c 30 4c 06 03 55 1d 20 04 45 30 43 30 37 06 09 60 86 48 01 86 fd 6c 01 01 30 2a 30 28 06 08 2b 06 01 05 05 07 02 01 16 1c 68 74 74 70 73 3a 2f 2f 77 77 77 2e 64 69 67 69 63 65 72 74 2e 63 6f 6d 2f 43 50 53 30 08 06 06 67 81 0c 01 02 02 30 81 83 06 08 2b 06 01 05 05 07 01 01 04 77 30 75 30 24 06 08 2b 06 01 05 05 07 30 01 86 18 68 74 74 70 3a 2f 2f 6f 63 73 70 2e 64 69 67 69 63 65 72 74 2e 63 6f 6d 30 4d 06 08 2b 06 01 05 05 07 30 02 86 41 68 74 74 70 3a 2f 2f 63 61 63 
BSSL:CERT: 65 72 74 73 2e 64 69 67 69 63 65 72 74 2e 63 6f 6d 2f 44 69 67 69 43 65 72 74 53 48 41 32 48 69 67 68 41 73 73 75 72 61 6e 63 65 53 65 72 76 65 72 43 41 2e 63 72 74 30 0c 06 03 55 1d 13 01 01 ff 04 02 30 00 30 82 01 7b 06 0a 2b 06 01 04 01 d6 79 02 04 02 04 82 01 6b 04 82 01 67 01 65 00 75 00 29 79 be f0 9e 39 39 21 f0 56 73 9f 63 a5 77 e5 be 57 7d 9c 60 0a f8 f9 4d 5d 26 5c 25 5d c7 84 00 00 01 72 dd b6 2a de 00 00 04 03 00 46 30 44 02 20 66 d5 ea ba af 15 0e 8d 17 0d 2c 63 6a e1 f9 b7 22 7e d9 dc 01 10 b6 ea e6 b4 4d 2c 87 cf fb fb 02 20 55 2b 93 7a 7a 61 d4 8f 0f 1a 57 4e 39 03 56 b3 ad bf 41 3b 55 cf 9b 3f 6b 99 65 25 8b 5f d7 60 00 75 00 41 c8 ca b1 df 22 46 4a 10 c6 a1 3a 09 42 87 5e 4e 31 8b 1b 03 eb eb 4b c7 68 f0 90 62 96 06 f6 00 00 01 72 dd b6 2a 
:rd 5, 1496, 38

[earlephilhower]

Quickly ooking at the code, it's related to the BearSSL x509 context info where the x509 cert is null. W/o code, can't really say how that could happen. If you can carve out a MCVE w/o the 3rd party libs it would be helpful. W/o it, you're going to need to be self sufficient as it's using a private client cert, to a private AWS instance, using 3rd party code which means nobody but you can repro your runs. :(

[schnorea]

I'm also using the Basic SSL package (Low Memory)

[schnorea]

Yeah, a hear you:(

When you say "third party libs" which ones you talking about so I am clear about what I need to do?

I'll give it a try. Thanks for the encouragement.

[earlephilhower]

The JSON lib and MQTT libs aren't part of the core. Something simple and minimal that does a SSL connection attempt and gets to your failed state is what we'd need.

That said, I still think you're just out of memory and not catching it properly somewhere or in a library. It's not stack usage related, the present amount is fine for every case we know of, your reported usage is small, and when an overflow occurs it is reported as a SSL stack overflow via canaries or via massive memory corruption (not nulls).

[schnorea]

I woke up with a similar thought. What if that transaction is processing things to slowly? The evidence is this:

BSSL:_run_until: Timeout

And, if i had turned on time stamps you would see that this time out is always 15 seconds. What if sometimes that transaction takes less than 15 seconds and succeeds but most of the time it doesn't because inadequate hardware performance.

No, I like your explination better. Low memory issue draws program execution out in the sticks (exception) and it can't get back in time.

Okay the logs from above were without the JSON Lib. Next I'll try pulling the MQTT to get at the raw SSL connection.

Thanks again.