Strange error from new httpclient

[mouridis]

Hardware:

Board: WEMOS LOLIN32
Core Installation/update date: Commit c3ec91f
IDE name: Arduino IDE 1.8.6
Flash Frequency: 80Mhz
PSRAM enabled: NoUpload Speed: 921600
Computer OS: Windows 10 x64

Description:

Updating to the latest commit (c3ec91f) yesterday broke my app in a very strange way. After hours trying to figure out what's wrong, it seems that when a String object is passed in a function that uses httpclient it results in a Guru Meditation Error.

You should have no problem reproducing the issue using the following sketch:

#include <WiFi.h>
#include <HTTPClient.h>

const String APSSID = "xxxxxxx";
const String APPassphrase =  "yyyyyyy";
const String unavailableString = "unavailable";
const String serverURL = "http://example.com/";
const unsigned int timeoutInSeconds = 8;

void setup() {
  Serial.begin(115200);
  delay(1000);
  WiFi.mode(WIFI_STA);
  WiFi.begin(APSSID.c_str(), APPassphrase.c_str());
  Serial.print("Connecting.");
  int i = 0;
  while ((WiFi.status() != WL_CONNECTED) && (WiFi.status() != WL_NO_SSID_AVAIL) && (WiFi.status() != WL_CONNECT_FAILED) && (i < (timeoutInSeconds))) {
          delay(1000);
          Serial.print(".");
          i++;
  }
  Serial.println();
  if ((WiFi.status() == WL_CONNECTED)) {
    Serial.println("Connected to the WiFi network");
    Serial.println(getPayloadFromServer("nikos"));
      } else {
    Serial.println("Failed to connect to the WiFi network");
  }
  WiFi.disconnect();
}

void loop() {
}

String getPayloadFromServer(String message) {
  HTTPClient http;
  Serial.println("Attempting server connection.");
  http.begin(serverURL);
  Serial.println("Sending GET request.");
  int httpCode = http.GET();
  if (httpCode > 0) {
    Serial.println("Server reached and responded to GET request.");
    if (httpCode == HTTP_CODE_OK) {
      Serial.print("Server responded with requested payload: ");
      String payload = http.getString();
      Serial.println(payload);
      http.end();
      return payload;
    } else {
      Serial.println("Server error: Server did not provide the requested payload.");
    }
  } else {
    Serial.println("Server could not be reached or server did not reply to GET request.");
  }
  http.end();
  return unavailableString;
}

Just edit APSSID and APPassphrase to your situation and this will cause a Guru Meditation Error with this trace:

Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC      : 0x4015e078  PS      : 0x00060e30  A0      : 0x800d3130  A1      : 0x3ffb1e50  
A2      : 0x3ffb1ecc  A3      : 0x00000000  A4      : 0x00000013  A5      : 0x0000ff00  
A6      : 0x00ff0000  A7      : 0xff000000  A8      : 0x800d4d42  A9      : 0x3ffb1e50  
A10     : 0x3ffb1f00  A11     : 0x3ffb7d84  A12     : 0x000000ff  A13     : 0x0000ff00  
A14     : 0x00ff0000  A15     : 0xff000000  SAR     : 0x00000018  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000010  LBEG    : 0x400013f9  LEND    : 0x4000140d  LCOUNT  : 0xfffffffb  

Backtrace: 0x4015e078:0x3ffb1e50 0x400d312d:0x3ffb1e70 0x400d40c9:0x3ffb1e90 0x400d1bf6:0x3ffb1ec0 0x400d1d5e:0x3ffb1f60 0x4013f7cb:0x3ffb1fb0 0x4008ba0d:0x3ffb1fd0

Using ESP Exception Decoder with the above trace, I get this:

PC: 0x4015e078: HTTPClient::connected() at C:\Users\Nikos\Documents\Arduino\hardware\espressif\esp32\libraries\HTTPClient\src\HTTPClient.cpp line 381
EXCVADDR: 0x00000010

Decoding stack results
0x4015e078: HTTPClient::connected() at C:\Users\Nikos\Documents\Arduino\hardware\espressif\esp32\libraries\HTTPClient\src\HTTPClient.cpp line 381
0x400d312d: HTTPClient::disconnect() at C:\Users\Nikos\Documents\Arduino\hardware\espressif\esp32\libraries\HTTPClient\src\HTTPClient.cpp line 347
0x400d40c9: HTTPClient::begin(String) at C:\Users\Nikos\Documents\Arduino\hardware\espressif\esp32\libraries\HTTPClient\src\HTTPClient.cpp line 336
0x400d1bf6: getPayloadFromServer(String) (C:\Users\Nikos\OneDrive\Workspace\Electronics\Projects\TimeSquare2\Code\DeviceBeforeVersioning\LOLIN32 at ESP32)\HelpingScripts\BasicHttpClientMod/BasicHttpClientMod.ino line 40
0x400d1d5e: setup() (C:\Users\Nikos\OneDrive\Workspace\Electronics\Projects\TimeSquare2\Code\DeviceBeforeVersioning\LOLIN32 at ESP32)\HelpingScripts\BasicHttpClientMod/BasicHttpClientMod.ino line 25
0x4013f7cb: loopTask(void*) at C:\Users\Nikos\Documents\Arduino\hardware\espressif\esp32\cores\esp32\main.cpp line 15
0x4008ba0d: vPortTaskWrapper at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/freertos/port.c line 141

Now... somehow this problem doesn't happen if the String type parameter is removed from the getPayloadFromServer function. Allow me to offer the same sketch with only two changes in two lines to reflect the parameter removal from the getPayloadFromServer function:

#include <WiFi.h>
#include <HTTPClient.h>

const String APSSID = "xxxxxxx";
const String APPassphrase =  "yyyyyyy";
const String unavailableString = "unavailable";
const String serverURL = "http://example.com/";
const unsigned int timeoutInSeconds = 8;

void setup() {
  Serial.begin(115200);
  delay(1000);
  WiFi.mode(WIFI_STA);
  WiFi.begin(APSSID.c_str(), APPassphrase.c_str());
  Serial.print("Connecting.");
  int i = 0;
  while ((WiFi.status() != WL_CONNECTED) && (WiFi.status() != WL_NO_SSID_AVAIL) && (WiFi.status() != WL_CONNECT_FAILED) && (i < (timeoutInSeconds))) {
          delay(1000);
          Serial.print(".");
          i++;
  }
  Serial.println();
  if ((WiFi.status() == WL_CONNECTED)) {
    Serial.println("Connected to the WiFi network");
    Serial.println(getPayloadFromServer());
  } else {
    Serial.println("Failed to connect to the WiFi network");
  }
  WiFi.disconnect();
}

void loop() {
}

String getPayloadFromServer() {
  HTTPClient http;
  Serial.println("Attempting server connection.");
  http.begin(serverURL);
  Serial.println("Sending GET request.");
  int httpCode = http.GET();
  if (httpCode > 0) {
    Serial.println("Server reached and responded to GET request.");
    if (httpCode == HTTP_CODE_OK) {
      Serial.print("Server responded with requested payload: ");
      String payload = http.getString();
      Serial.println(payload);
      http.end();
      return payload;
    } else {
      Serial.println("Server error: Server did not provide the requested payload.");
    }
  } else {
    Serial.println("Server could not be reached or server did not reply to GET request.");
  }
  http.end();
  return unavailableString;
}

This only changes lines 25 and 36 from the previous sketch and it executes fine!

Another way to avoid the issue is to fallback to commit b70737d, right before httpclient lib was updated to 1.2.

Can anybody offer any possible explanation why this happens?

I hope it's clear that I'm not looking for workarounds. In the above example sketch passing the String object as a parameter is useless (the message variable is not even used anywhere in the function). But in my full app I really need to pass a String object to the function. I am well aware that there are other ways to access a piece of text from the function (setting a global var, using char arrays etc).

I really can't understand how passing a String object that is not even used affects httpclient...

Thanks

[chegewara]

Did you try to initialize String var = "nicos"; before you pass it to function? Maybe nicos passed as argument is treated as const char* instead of string.

[mouridis]

@chegewara Yes, I did. And just for triple-checking's sake, I tried again after your suggestion. Replacing

Serial.println(getPayloadFromServer("nikos"));

with this

String var = "nikos";
Serial.println(getPayloadFromServer(var));

makes no difference. I still get the Guru Meditation Error.

I also tried to declare var as a const String. Same result.

From the Exception Decoder output and the fact that the sketch works with the older commit just before httpclient update, it seems the reason for the error has to do with the new httpclient. But I cannot figure out why the new httpclient has a problem with a String that is not even passed to it.

Maybe @Jeroen88 who has clear understanding of the library inner workings can shed some light...

[Jeroen88]

This is a strange error indeed. I had a quick glance and can not determine it's cause yet. Could you enable debugging and add the output?
One thing I am thinking of, if the response from your server is big, it is copied around several times. I think the stack is too small for this. Might this be the problem? The output from the server is copied into a stream string internally and this is copied into payload and then printed, all using the stack.
In the meantime try the example StreamHttpClient, this might help.

[mouridis]

@Jeroen88 The response from my server is a 60 byte JSON. It should not be a problem. Besides, when I do not pass an irrelevant String to the function, this sketch works fine for payloads bigger than 1Kb. So, this is not the issue.

I suppose you suggest the StreamHttpClient example to handle a possibly big payload, but this is not the case as my payload is very small. Besides, the StreamHttpClient example is identical to my sketch. Its changes start after calling the GET() method, while the crash happens earlier in begin(). If I run the example as-is, I'm sure it will have no problem running. But as soon as I convert the example to a function that accepts a String parameter it will crash.

Regarding enabling debugging, if you mean adding:

Serial.setDebugOutput(true);

after:

Serial.begin(115200);

I tried it but the Serial output is exactly the same.
(I 've never used this before so please bear with me if I'm doing something wrong)

[mouridis]

Did any of you by any chance tried to flash my sketch in your dev boards and confirm the issue?

Please feedback if you did because at this point I start questioning my sanity. This error makes no sense... maybe I'm doing something profoundly wrong in my setup.

If I could get a "you're not the only one seeing this" it would help me decide the direction where I should search more for the solution. Thanks!

[mouridis]

OK - important new piece of info:
If I move the construction of the http object outside of the function (meaning changing its scope to global), the function can accept the String parameter with no Guru Meditation Error.

I just move the following line:

HTTPClient http;

from inside the getPayloadFromServer() function to the top of the sketch just above setup().

This is kind of a reasonable workaround but it makes the http object global, occupying memory all the time.

To sum up, for this example sketch:

• httpclient 1.1 works fine all the time no matter if the object is constructed inside the function, or outside of it and no matter if there are String parameters to the function or not. Of course, it has other issues that httpclient 1.2 solves.
• httpclient 1.2 works fine if the object is constructed outside the function, no matter if the function has a String parameter or not.
• httpclient 1.2 works fine if the object is constructed inside the function and the function has no String parameter.
• httpclient 1.2 causes a Guru Meditation Error if the object is constructed inside a function which has a String parameter

I think it's best to keep this issue open for two reasons:

1. It's still an issue if you don't want your httpclient objects to be global.
2. In my main app which is kind of huge, httpclient 1.2 causes the same error when the httpclient object is constructed inside a function, even without passing a String to that function. It seems the String parameter is only one way to easily reproduce the issue but it's not the only one. I temporarily solved the issue in my app by using the workaround of this comment.

I hope all this helps to find the reason.

[Jeroen88]

@mouridis I could reproduce your error. An uninitialized pointer in the HTTPClient seemed to be the problem. I created PR #2097 to solve it. The first sketch you provided runs fine with this fix.

[Jeroen88]

@mouridis

Regarding enabling debugging, if you mean adding: Serial.setDebugOutput(true);

No I meant

#include <esp_log.h>
and in setup()
esp_log_level_set("*", ESP_LOG_DEBUG);
This enables debugging of the libraries and can give you lots of clues why things don't work.

Thanks for the abundant information that helped to find the bug.

[mouridis]

@Jeroen88 You are the man!

I confirm this not only fixes the sketch above but also my main app issue. Thanks!

I guess your PR will be merged soon so I'm closing this.

[Jeroen88]

@mouridis that is good news :). Maybe it is better to reopen the issue and keep it open until the PR is merged?

[mouridis]

I hear you...

[mouridis]

Addressed in commit 0640964