Backward-compatible refactoring of the Json and MessagePack integration .

[mathieucarbou]

This PR:

• refactors the macros to activate or not Json and MessagePack support
• regroups the code for both since it is similar in many ways
• add deprecations where applicable
• add the ability to easily add a JsonVariant callback with server.on("/testJsonPost", HTTP_POST, [](AsyncWebServerRequest *request, JsonVariant &json)
• Improve examples

This PR is tested.

[mathieucarbou]

Fix: #300

[willmmiles]

Integration of AsyncMessagePack and AsyncJson is a good idea. The usage of feature-detect macros is also a good step forward.

I'm not 100% sure about directly including ArduinoJson in ESPAsyncWebServer.h, though. I'm concerned that might have unintentional side effects with PlatformIO's library dependency finder -- in particular, it may cause it to pull ArduinoJson in to projects that didn't have it previously, potentially increasing firmware size. I don't think it's clever enough to interpret __has_include() at that point in the build process, and I don't know what semantics it ought to have even if it did -- PIO still doesn't have a working story for optional dependencies. Can you check if the firmware size is impacted with a test external project? I don't think building the examples in-tree will reflect the LDF behaviour from a separate project.

[mathieucarbou]

Integration of AsyncMessagePack and AsyncJson is a good idea. The usage of feature-detect macros is also a good step forward.

I'm not 100% sure about directly including ArduinoJson in ESPAsyncWebServer.h, though. I'm concerned that might have unintentional side effects with PlatformIO's library dependency finder -- in particular, it may cause it to pull ArduinoJson in to projects that didn't have it previously, potentially increasing firmware size. I don't think it's clever enough to interpret __has_include() at that point in the build process, and I don't know what semantics it ought to have even if it did -- PIO still doesn't have a working story for optional dependencies. Can you check if the firmware size is impacted with a test external project? I don't think building the examples in-tree will reflect the LDF behaviour from a separate project.

PIO is impacted by the lib_compat_mode and lib_ldf_mode flags.

We advise in the doc to use lib_compat_mode = strict and lib_ldf_mode = chain which are also those widely advised. Jason from Tasmota who is maintaining pioaduino does not advise to use any other combination, especially deep, because it does not correctly work.

So I will test different use cases but only with the flags we do support:

lib_compat_mode = strict
lib_ldf_mode = chain

[mathieucarbou]

Also, considering the late refactorings, it is highly possible that any firmware size change is negligeable in comparison... So I will test with different base lines.

[mathieucarbou]

@willmmiles : we have a firmware size reduction already from v3.8.1 to main, and we have a further firmware size reudction when we compare some use cases between v3.8.1 and this branch:

3.8.1 / No Json

[platformio]
src_dir = examples/FirmwareSize
[env]
framework = arduino
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31/platform-espressif32.zip
board = esp32dev
build_type = release
build_flags = -Wall -Wextra
lib_compat_mode = strict
lib_ldf_mode = chain
lib_deps =
  ESP32Async/AsyncTCP @ 3.4.9
  ESP32Async/ESPAsyncWebServer @ 3.8.1
board_build.partitions = partitions-4MB.csv
board_build.filesystem = littlefs
[env:default]

#include <Arduino.h>
#include <WiFi.h>
#include <ESPAsyncWebServer.h>

static AsyncWebServer server(80);

void setup() {
  Serial.begin(115200);

  WiFi.mode(WIFI_AP);
  WiFi.softAP("esp-captive");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(404, "text/plain", "Not found");
  });

  server.begin();
}

void loop() { delay(100); }

RAM:   [=         ]  13.4% (used 43824 bytes from 327680 bytes)
Flash: [=====     ]  53.7% (used 1020651 bytes from 1900544 bytes)

PR / No Json

[platformio]
lib_dir = .
src_dir = examples/FirmwareSize
[env]
framework = arduino
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31/platform-espressif32.zip
board = esp32dev
build_type = release
build_flags = -Wall -Wextra
lib_compat_mode = strict
lib_ldf_mode = chain
lib_deps =
  ESP32Async/AsyncTCP @ 3.4.9
board_build.partitions = partitions-4MB.csv
board_build.filesystem = littlefs
[env:default]

#include <Arduino.h>
#include <WiFi.h>
#include <ESPAsyncWebServer.h>

static AsyncWebServer server(80);

void setup() {
  Serial.begin(115200);

  WiFi.mode(WIFI_AP);
  WiFi.softAP("esp-captive");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(404, "text/plain", "Not found");
  });

  server.begin();
}

void loop() { delay(100); }

RAM:   [=         ]  13.4% (used 43824 bytes from 327680 bytes)
Flash: [=====     ]  53.7% (used 1020567 bytes from 1900544 bytes)

=> 84 bytes smaller

3.8.1 / With Json

[platformio]
src_dir = examples/FirmwareSize
[env]
framework = arduino
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31/platform-espressif32.zip
board = esp32dev
build_type = release
build_flags = -Wall -Wextra
lib_compat_mode = strict
lib_ldf_mode = chain
lib_deps =
  ESP32Async/AsyncTCP @ 3.4.9
  ESP32Async/ESPAsyncWebServer @ 3.8.1
  bblanchon/ArduinoJson @ 7.4.2
board_build.partitions = partitions-4MB.csv
board_build.filesystem = littlefs
[env:default]

#include <Arduino.h>
#include <WiFi.h>
#include <ESPAsyncWebServer.h>
#include <AsyncJson.h>

static AsyncWebServer server(80);

void setup() {
  Serial.begin(115200);

  WiFi.mode(WIFI_AP);
  WiFi.softAP("esp-captive");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(404, "text/plain", "Not found");
  });

  server.on("/json1", HTTP_GET, [](AsyncWebServerRequest *request) {
    AsyncJsonResponse *response = new AsyncJsonResponse();
    JsonObject root = response->getRoot().to<JsonObject>();
    root["hello"] = "world";
    response->setLength();
    request->send(response);
  });

  server.begin();
}

void loop() { delay(100); }

RAM:   [=         ]  13.4% (used 43824 bytes from 327680 bytes)
Flash: [=====     ]  54.1% (used 1028799 bytes from 1900544 bytes)

PR / With Json

[platformio]
lib_dir = .
src_dir = examples/FirmwareSize
[env]
framework = arduino
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31/platform-espressif32.zip
board = esp32dev
build_type = release
build_flags = -Wall -Wextra
lib_compat_mode = strict
lib_ldf_mode = chain
lib_deps =
  ESP32Async/AsyncTCP @ 3.4.9
  bblanchon/ArduinoJson @ 7.4.2
board_build.partitions = partitions-4MB.csv
board_build.filesystem = littlefs
[env:default]

#include <Arduino.h>
#include <WiFi.h>
#include <ESPAsyncWebServer.h>

static AsyncWebServer server(80);

void setup() {
  Serial.begin(115200);

  WiFi.mode(WIFI_AP);
  WiFi.softAP("esp-captive");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(404, "text/plain", "Not found");
  });

  server.on("/json1", HTTP_GET, [](AsyncWebServerRequest *request) {
    AsyncJsonResponse *response = new AsyncJsonResponse();
    JsonObject root = response->getRoot().to<JsonObject>();
    root["hello"] = "world";
    response->setLength();
    request->send(response);
  });

  server.begin();
}

void loop() { delay(100); }

RAM:   [=         ]  13.4% (used 43824 bytes from 327680 bytes)
Flash: [=====     ]  54.1% (used 1028387 bytes from 1900544 bytes)

=> 412 bytes smaller

3.8.1 / ArduinoJson in lib_deps / Not using Json

[platformio]
src_dir = examples/FirmwareSize
[env]
framework = arduino
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31/platform-espressif32.zip
board = esp32dev
build_type = release
build_flags = -Wall -Wextra
lib_compat_mode = strict
lib_ldf_mode = chain
lib_deps =
  ESP32Async/AsyncTCP @ 3.4.9
  ESP32Async/ESPAsyncWebServer @ 3.8.1
  bblanchon/ArduinoJson @ 7.4.2
board_build.partitions = partitions-4MB.csv
board_build.filesystem = littlefs
[env:default]

#include <Arduino.h>
#include <WiFi.h>
#include <ESPAsyncWebServer.h>

static AsyncWebServer server(80);

void setup() {
  Serial.begin(115200);

  WiFi.mode(WIFI_AP);
  WiFi.softAP("esp-captive");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(404, "text/plain", "Not found");
  });

  server.begin();
}

void loop() { delay(100); }

RAM:   [=         ]  13.4% (used 43824 bytes from 327680 bytes)
Flash: [=====     ]  53.8% (used 1022747 bytes from 1900544 bytes)

=> 2096 bytes more than 3.8.1 / No Json

PR / ArduinoJson in lib_deps / Not using Json

[platformio]
lib_dir = .
src_dir = examples/FirmwareSize
[env]
framework = arduino
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31/platform-espressif32.zip
board = esp32dev
build_type = release
build_flags = -Wall -Wextra
lib_compat_mode = strict
lib_ldf_mode = chain
lib_deps =
  ESP32Async/AsyncTCP @ 3.4.9
  ; ESP32Async/ESPAsyncWebServer @ 3.8.1
  bblanchon/ArduinoJson @ 7.4.2
board_build.partitions = partitions-4MB.csv
board_build.filesystem = littlefs
[env:default]

#include <Arduino.h>
#include <WiFi.h>
#include <ESPAsyncWebServer.h>

static AsyncWebServer server(80);

void setup() {
  Serial.begin(115200);

  WiFi.mode(WIFI_AP);
  WiFi.softAP("esp-captive");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(404, "text/plain", "Not found");
  });

  server.begin();
}

void loop() { delay(100); }

RAM:   [=         ]  13.4% (used 43824 bytes from 327680 bytes)
Flash: [=====     ]  53.8% (used 1022295 bytes from 1900544 bytes)

=> −452 bytes smaller than 3.8.1 / ArduinoJson in lib_deps / Not using Json

=> 1728 bytes more than PR / No Json

[mathieucarbou]

In summary: if the user has ArduinoJson in its lib_dep but is not using it, then he better removes it.
And on that, there is no change between this PR and v3.8.1.

Otherwise the PR (plus new code in main) will have a lower firmware size in all cases.

[willmmiles]

In summary: if the user has ArduinoJson in its lib_dep but is not using it, then he better removes it. And on that, there is no change between this PR and v3.8.1.

Otherwise the PR (plus new code in main) will have a lower firmware size in all cases.

Perfect. Thanks for checking, maybe I'm just overly cautious any time the LDF gets involved...