From 12a664dcebc71ee54d0cfe02da7845604769dbdd Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Wed, 25 Mar 2020 18:23:51 +0100 Subject: [PATCH 01/16] Alternative HTTP Client directly using the Stream, alternative to ESP8266HTTPClient / HTTPClient --- libraries/ClientHTTP/README.md | 112 +++ .../examples/Authorization/Authorization.ino | 156 ++++ .../BasicClientHTTP/BasicClientHTTP.ino | 84 ++ .../BasicClientHTTPS/BasicClientHTTPS.ino | 153 ++++ .../DataClientHTTPS/DataClientHTTPS.ino | 189 +++++ .../POSTClientHTTPS/POSTClientHTTPS.ino | 152 ++++ .../POSTJsonClientHTTPS.ino | 206 +++++ .../POSTSizerClientHTTPS.ino | 160 ++++ .../ReuseConnectionClientHTTPS.ino | 174 ++++ libraries/ClientHTTP/keywords.txt | 130 +++ libraries/ClientHTTP/library.properties | 9 + libraries/ClientHTTP/src/ClientHTTP.cpp | 788 ++++++++++++++++++ libraries/ClientHTTP/src/ClientHTTP.h | 269 ++++++ 13 files changed, 2582 insertions(+) create mode 100644 libraries/ClientHTTP/README.md create mode 100644 libraries/ClientHTTP/examples/Authorization/Authorization.ino create mode 100644 libraries/ClientHTTP/examples/BasicClientHTTP/BasicClientHTTP.ino create mode 100644 libraries/ClientHTTP/examples/BasicClientHTTPS/BasicClientHTTPS.ino create mode 100644 libraries/ClientHTTP/examples/DataClientHTTPS/DataClientHTTPS.ino create mode 100644 libraries/ClientHTTP/examples/POSTClientHTTPS/POSTClientHTTPS.ino create mode 100644 libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino create mode 100644 libraries/ClientHTTP/examples/POSTSizerClientHTTPS/POSTSizerClientHTTPS.ino create mode 100644 libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino create mode 100644 libraries/ClientHTTP/keywords.txt create mode 100644 libraries/ClientHTTP/library.properties create mode 100644 libraries/ClientHTTP/src/ClientHTTP.cpp create mode 100644 libraries/ClientHTTP/src/ClientHTTP.h diff --git a/libraries/ClientHTTP/README.md b/libraries/ClientHTTP/README.md new file mode 100644 index 00000000000..a1d68703952 --- /dev/null +++ b/libraries/ClientHTTP/README.md @@ -0,0 +1,112 @@ +# ClientHTTP +The ClientHTTP class implements support for REST API client calls to a HTTP server. +It inherits from Client and thus implements a superset of that class' interface. + +The ClientHTTP class decorates the underlying stream with HTTP request headers and +reads the HTTP response headers from the stream before the stream is made available to +the program. Also chunked responses are handled 'under the hood' offering a clean stream +to which a request payload (if any) can be written using the standard Print::write() +functions and from which response payloads (if any) can be read using the standard +Stream::read() functions. + +The ClientHTTP class has a smaller memory footprint than the alternative HTTPClient class. +Moreover: because of the Stream oriented nature of ClientHTTP no memory buffers are +needed, neither for sending payload, nor for receiving payload. It still remains possible +to use such memory buffers though. Chunked response from the stream is correctly implemented. +No Strings are used to prevent heap fragmentation. Only small stack buffers are used. Two +exceptions are: the host name is copied into a heap char array, and the requestHeaders and +responseHeaders are dynamic standard containers (map). + +ClientHTTP is able to handle redirections, but support for this may be omitted by clearing +the #define SUPPORT_REDIRECTS in the header file, thus saving some programm memory bytes. + +Request headers can easily be set by ClientHTTP. Response headers can be easily collected by +ClientHTTP. + +Any client may be used, e.g. WiFiClient, WiFiClientSecure and EthernetClient. + +A HTTP GET, HEAD, POST, PUT, PATCH or DELETE always follows the same skeleton. + +Integration with ArduinoJson, version 5 and 6, both for sending and receiving Json payloads +is very intuitive. + +The same library is defined for both the ESP8266 and the ESP32. + +The following examples are available. All but one are HTTPS / TLS 1.2 examples: +- Authorization: how to use basic authorization +- BasicClientHTTP: how to GET a payload from a HTTP server and print it to Serial +- BasicClientHTTPS: how to GET a paylaod from a HTTPS server and print it to Serial +- DataClientHTTPS: how to GET a payload from a HTTPS server and handle the data using + StreamString or a buffer +- POSTClientHTTPS: how to POST a payload to a HTTPS server and print the response to + Serial +- POSTJsonClientHTTPS: how to POST an ArduinoJson to a HTTPS server and deserialize the + response into another ArduinJson without buffers. All data is buffered into the + ArduinoJson only +- POSTSizerClientHTTPS: how to determine the size of a payload by printing to a utility + class Sizer first. Next the resulting length is used to set the Content-Length request + header by calling POST +- ReuseConnectionHTTPS: how to reuse a connection, including correct handling of the + Connection: close response header. This example DOES NOT WORK on the ESP8266 YET! + +Still to be done: +- Support for redirection to a different host +- Fixing the response header handling. I think an Allocator needs to be defined for this + but I could use some help on this one :) + +## Request and response headers +Request headers can be set by just defining them before calling the REST method (`GET`, +`POST`, etc): +```cpp + http.requestHeaders["Content-Type"] = "application/json"; + http.requestHeaders["Connection"] = "close"; +``` +Response headers can be collected by just defining them before `::status()` is called: +```cpp + http.responseHeaders["Content-Type"]; +``` +After the `::status()` command `responseHeaders['Content-Type']` is either `NULL` if the server +did not send the response header, or it is populated with the value sent. + +## Skeleton program +A HTTP REST API call always follows the following skeleton +- Prepare a transport client, like `WiFiClient client` or `WiFiClientSecure client` +- Prepare a `ClientHTTP http(client)`, passing the transport client into the constructor +- Set request headers +- Define response headers to be collected +- `htpp.connect(host, port)` +- Call the REST API method, e.g. `http.POST(payload, payloadLength)` +- Send the payload, if any, to the server using standard `Print::write()` commands +- Call `http.status()` which reads the response headers from the stream and returns + the HTTP response code (greater than 0) or an error code (less than 0) +- Check any response headers, if needed +- Read the payload sent by the server, if any, using standard `Stream::read()` commands +- Close the connection calling `http.stop()`, or reuse the connection if the server + did not sent a `Connection: close` response header, nor closed it's side of the + connection + +## ArduinoJson integration +ArduinoJson documents, both version 5 and version 6, can directly be POSTed to a REST server: + +```cpp + DynamicJsonDocument requestDocument(requestCapacity); + ... + http.requestHeaders["Content-Type"] = "application/json"; + http.POST("/post", measureJson(requestDocument)); + serializeJson(requestDocument, http); + ... +``` +Also ArduinoJson documents, both version 5 and 6, can directly be deserialized from the +response: +```cpp + ... + http.responseHeaders["Content-Type"]; + http.status(); + DynamicJsonDocument responseDocument(responseCapacity); + if(http.responseHeaders["Content-Type"] != NULL && strcasecmp(http.responseHeaders["Content-Type"], "application/json") == 0) { + deserializeJson(responseDocument, http); + } +``` + + + diff --git a/libraries/ClientHTTP/examples/Authorization/Authorization.ino b/libraries/ClientHTTP/examples/Authorization/Authorization.ino new file mode 100644 index 00000000000..741934e19dd --- /dev/null +++ b/libraries/ClientHTTP/examples/Authorization/Authorization.ino @@ -0,0 +1,156 @@ +/* + * ClientHTTP example sketch + * + * GET using Basic Authorization on a WiFiClientSecure with a CA certificate + * + * This example sketch demonstrates how to use the ClientHTTP library + * to fetch from a web server using GET and Basic Authorization. HTTPS (TLS 1.2) + * secure communication is used with a CA certificate. + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include + +#include + +#include + +#ifndef STASSID +#define STASSID "your-ssid" +#define STAPSK "your-password" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + +// Use web browser to view and copy +// This certificate expires on 1/19/2038, 12:59:59 AM (Central European Standard Time) +const char * rootCACert = R"EOF( +-----BEGIN CERTIFICATE----- +MIIF3jCCA8agAwIBAgIQAf1tMPyjylGoG7xkDjUDLTANBgkqhkiG9w0BAQwFADCB +iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl +cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV +BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTAw +MjAxMDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV +BAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0plcnNleSBDaXR5MR4wHAYDVQQKExVU +aGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNVBAMTJVVTRVJUcnVzdCBSU0EgQ2Vy +dGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK +AoICAQCAEmUXNg7D2wiz0KxXDXbtzSfTTK1Qg2HiqiBNCS1kCdzOiZ/MPans9s/B +3PHTsdZ7NygRK0faOca8Ohm0X6a9fZ2jY0K2dvKpOyuR+OJv0OwWIJAJPuLodMkY +tJHUYmTbf6MG8YgYapAiPLz+E/CHFHv25B+O1ORRxhFnRghRy4YUVD+8M/5+bJz/ +Fp0YvVGONaanZshyZ9shZrHUm3gDwFA66Mzw3LyeTP6vBZY1H1dat//O+T23LLb2 +VN3I5xI6Ta5MirdcmrS3ID3KfyI0rn47aGYBROcBTkZTmzNg95S+UzeQc0PzMsNT +79uq/nROacdrjGCT3sTHDN/hMq7MkztReJVni+49Vv4M0GkPGw/zJSZrM233bkf6 +c0Plfg6lZrEpfDKEY1WJxA3Bk1QwGROs0303p+tdOmw1XNtB1xLaqUkL39iAigmT +Yo61Zs8liM2EuLE/pDkP2QKe6xJMlXzzawWpXhaDzLhn4ugTncxbgtNMs+1b/97l +c6wjOy0AvzVVdAlJ2ElYGn+SNuZRkg7zJn0cTRe8yexDJtC/QV9AqURE9JnnV4ee +UB9XVKg+/XRjL7FQZQnmWEIuQxpMtPAlR1n6BB6T1CZGSlCBst6+eLf8ZxXhyVeE +Hg9j1uliutZfVS7qXMYoCAQlObgOK6nyTJccBz8NUvXt7y+CDwIDAQABo0IwQDAd +BgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rIDZsswDgYDVR0PAQH/BAQDAgEGMA8G +A1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEMBQADggIBAFzUfA3P9wF9QZllDHPF +Up/L+M+ZBn8b2kMVn54CVVeWFPFSPCeHlCjtHzoBN6J2/FNQwISbxmtOuowhT6KO +VWKR82kV2LyI48SqC/3vqOlLVSoGIG1VeCkZ7l8wXEskEVX/JJpuXior7gtNn3/3 +ATiUFJVDBwn7YKnuHKsSjKCaXqeYalltiz8I+8jRRa8YFWSQEg9zKC7F4iRO/Fjs +8PRF/iKz6y+O0tlFYQXBl2+odnKPi4w2r78NBc5xjeambx9spnFixdjQg3IM8WcR +iQycE0xyNN+81XHfqnHd4blsjDwSXWXavVcStkNr/+XeTWYRUc+ZruwXtuhxkYze +Sf7dNXGiFSeUHM9h4ya7b6NnJSFd5t0dCy5oGzuCr+yDZ4XUmFF0sbmZgIn/f3gZ +XHlKYC6SQK5MNyosycdiyA5d9zZbyuAlJQG03RoHnHcAP9Dc1ew91Pq7P8yF1m9/ +qS3fuQL39ZeatTXaw2ewh0qpKJ4jjv9cJ2vhsE/zB+4ALtRZh8tSQZXq9EfX7mRB +VXyNWQKV3WKdwrnuWih0hKWbt5DHDAff9Yk2dDLWKMGwsAvgnEzDHNb842m1R0aB +L6KCq9NjRHDEjf8tM7qtj3u1cIiuPhnPQCjY/MiQu12ZIvVS5ljFH4gxQ+6IHdfG +jjxDah2nGN59PRbxYvnKkKj9 +-----END CERTIFICATE----- +)EOF"; + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + +#if defined(ESP8266) + configTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif +#if defined(ESP32) + configTzTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif + + Serial.print("Waiting for time"); + time_t epoch; + for(;;) { + Serial.write('.'); + time(&epoch); + if(epoch >= 3600) break; + delay(500); + } + Serial.print("Current time: "); + Serial.println(asctime(localtime(&epoch))); + + // Use WiFiClientSecure class to create TLS connection + WiFiClientSecure client; +#if defined(ESP8266) + BearSSL::X509List cert(rootCACert); + client.setTrustAnchors(&cert); +#endif +#if defined(ESP32) + client.setCACert(rootCACert); +#endif + + ClientHTTP http(client); + http.setTimeout(10000); + + String credentials = base64::encode((const uint8_t *)"guest:guest", sizeof "guest:guest" - 1); + String auth = String("Basic ") + credentials; + http.requestHeaders["Authorization"] = (char *)auth.c_str(); + http.requestHeaders["Connection"] = "close"; + + if (!http.connect("jigsaw.w3.org", 443)) { + Serial.println("Connection failed"); + return; + } + + if(!http.GET("/HTTP/Basic/")) { + Serial.println("Failed to GET"); + return; + } + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK) { + Serial.println("Succesfully authenticated!"); + } + + http.printTo(Serial); + Serial.println(); + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + Serial.println("Closing connection"); + http.stop(); +} + +void loop() { +} diff --git a/libraries/ClientHTTP/examples/BasicClientHTTP/BasicClientHTTP.ino b/libraries/ClientHTTP/examples/BasicClientHTTP/BasicClientHTTP.ino new file mode 100644 index 00000000000..d24f82fbb55 --- /dev/null +++ b/libraries/ClientHTTP/examples/BasicClientHTTP/BasicClientHTTP.ino @@ -0,0 +1,84 @@ +/* + * ClientHTTP example sketch + * + * GET using Basic Authorization + * + * This example sketch demonstrates how to use the ClientHTTP library + * to fetch from a web server using GET and Basic Authorization. + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include + +#include + +#ifndef STASSID +#define STASSID "your-ssid" +#define STAPSK "your-password" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + + // Use WiFiClient class to create connection + WiFiClient client; + + ClientHTTP http(client); + http.setTimeout(10000); + + if (!http.connect("jigsaw.w3.org", 80)) { + Serial.println("Connection failed"); + return; + } + + if(!http.GET("/HTTP/connection.html")) { + Serial.println("Failed to GET"); + return; + } + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK) { + Serial.println("Page succesfully fetched!"); + } + + http.printTo(Serial); + Serial.println(); + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + Serial.println("Closing connection"); + http.stop(); +} + +void loop() { +} diff --git a/libraries/ClientHTTP/examples/BasicClientHTTPS/BasicClientHTTPS.ino b/libraries/ClientHTTP/examples/BasicClientHTTPS/BasicClientHTTPS.ino new file mode 100644 index 00000000000..66b5f355882 --- /dev/null +++ b/libraries/ClientHTTP/examples/BasicClientHTTPS/BasicClientHTTPS.ino @@ -0,0 +1,153 @@ +/* + * ClientHTTP example sketch + * + * GET on a WiFiClientSecure with a CA certificate + * + * This example sketch demonstrates how to use the ClientHTTP library + * to fetch from a web server using GET. HTTPS (TLS 1.2) secure + * communication is used with a CA certificate. + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include + +#include + +#ifndef STASSID +//#define STASSID "your-ssid" +//#define STAPSK "your-password" +#define STASSID "H369A394602" +#define STAPSK "445396F996E9" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + +// Use web browser to view and copy +// This certificate expires on 1/19/2038, 12:59:59 AM (Central European Standard Time) +const char * rootCACert = R"EOF( +-----BEGIN CERTIFICATE----- +MIIF3jCCA8agAwIBAgIQAf1tMPyjylGoG7xkDjUDLTANBgkqhkiG9w0BAQwFADCB +iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl +cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV +BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTAw +MjAxMDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV +BAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0plcnNleSBDaXR5MR4wHAYDVQQKExVU +aGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNVBAMTJVVTRVJUcnVzdCBSU0EgQ2Vy +dGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK +AoICAQCAEmUXNg7D2wiz0KxXDXbtzSfTTK1Qg2HiqiBNCS1kCdzOiZ/MPans9s/B +3PHTsdZ7NygRK0faOca8Ohm0X6a9fZ2jY0K2dvKpOyuR+OJv0OwWIJAJPuLodMkY +tJHUYmTbf6MG8YgYapAiPLz+E/CHFHv25B+O1ORRxhFnRghRy4YUVD+8M/5+bJz/ +Fp0YvVGONaanZshyZ9shZrHUm3gDwFA66Mzw3LyeTP6vBZY1H1dat//O+T23LLb2 +VN3I5xI6Ta5MirdcmrS3ID3KfyI0rn47aGYBROcBTkZTmzNg95S+UzeQc0PzMsNT +79uq/nROacdrjGCT3sTHDN/hMq7MkztReJVni+49Vv4M0GkPGw/zJSZrM233bkf6 +c0Plfg6lZrEpfDKEY1WJxA3Bk1QwGROs0303p+tdOmw1XNtB1xLaqUkL39iAigmT +Yo61Zs8liM2EuLE/pDkP2QKe6xJMlXzzawWpXhaDzLhn4ugTncxbgtNMs+1b/97l +c6wjOy0AvzVVdAlJ2ElYGn+SNuZRkg7zJn0cTRe8yexDJtC/QV9AqURE9JnnV4ee +UB9XVKg+/XRjL7FQZQnmWEIuQxpMtPAlR1n6BB6T1CZGSlCBst6+eLf8ZxXhyVeE +Hg9j1uliutZfVS7qXMYoCAQlObgOK6nyTJccBz8NUvXt7y+CDwIDAQABo0IwQDAd +BgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rIDZsswDgYDVR0PAQH/BAQDAgEGMA8G +A1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEMBQADggIBAFzUfA3P9wF9QZllDHPF +Up/L+M+ZBn8b2kMVn54CVVeWFPFSPCeHlCjtHzoBN6J2/FNQwISbxmtOuowhT6KO +VWKR82kV2LyI48SqC/3vqOlLVSoGIG1VeCkZ7l8wXEskEVX/JJpuXior7gtNn3/3 +ATiUFJVDBwn7YKnuHKsSjKCaXqeYalltiz8I+8jRRa8YFWSQEg9zKC7F4iRO/Fjs +8PRF/iKz6y+O0tlFYQXBl2+odnKPi4w2r78NBc5xjeambx9spnFixdjQg3IM8WcR +iQycE0xyNN+81XHfqnHd4blsjDwSXWXavVcStkNr/+XeTWYRUc+ZruwXtuhxkYze +Sf7dNXGiFSeUHM9h4ya7b6NnJSFd5t0dCy5oGzuCr+yDZ4XUmFF0sbmZgIn/f3gZ +XHlKYC6SQK5MNyosycdiyA5d9zZbyuAlJQG03RoHnHcAP9Dc1ew91Pq7P8yF1m9/ +qS3fuQL39ZeatTXaw2ewh0qpKJ4jjv9cJ2vhsE/zB+4ALtRZh8tSQZXq9EfX7mRB +VXyNWQKV3WKdwrnuWih0hKWbt5DHDAff9Yk2dDLWKMGwsAvgnEzDHNb842m1R0aB +L6KCq9NjRHDEjf8tM7qtj3u1cIiuPhnPQCjY/MiQu12ZIvVS5ljFH4gxQ+6IHdfG +jjxDah2nGN59PRbxYvnKkKj9 +-----END CERTIFICATE----- +)EOF"; + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + +#if defined(ESP8266) + configTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif +#if defined(ESP32) + configTzTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif + + Serial.print("Waiting for time"); + time_t epoch; + for(;;) { + Serial.write('.'); + time(&epoch); + if(epoch >= 3600) break; + delay(500); + } + Serial.print("Current time: "); + Serial.println(asctime(localtime(&epoch))); + + // Use WiFiClientSecure class to create TLS connection + WiFiClientSecure client; +#if defined(ESP8266) + BearSSL::X509List cert(rootCACert); + client.setTrustAnchors(&cert); +#endif +#if defined(ESP32) + client.setCACert(rootCACert); +#endif + + ClientHTTP http(client); + http.setTimeout(10000); + + http.requestHeaders["Connection"] = "close"; + + if (!http.connect("jigsaw.w3.org", 443)) { + Serial.println("Connection failed"); + return; + } + + if(!http.GET("/HTTP/connection.html")) { + Serial.println("Failed to GET"); + return; + } + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK) { + Serial.println("Page succesfully fetched!"); + } + + http.printTo(Serial); + Serial.println(); + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + Serial.println("Closing connection"); + http.stop(); +} + +void loop() { +} diff --git a/libraries/ClientHTTP/examples/DataClientHTTPS/DataClientHTTPS.ino b/libraries/ClientHTTP/examples/DataClientHTTPS/DataClientHTTPS.ino new file mode 100644 index 00000000000..7721455a403 --- /dev/null +++ b/libraries/ClientHTTP/examples/DataClientHTTPS/DataClientHTTPS.ino @@ -0,0 +1,189 @@ +/* + * ClientHTTP example sketch + * + * GET on a WiFiClientSecure with a CA certificate + * + * This example sketch demonstrates how to use the ClientHTTP library + * to fetch from a web server using GET. HTTPS (TLS 1.2) secure + * communication is used with a CA certificate. How to handle response + * data sent by the server is demonstrated + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include +#include // Not necessary to include if a buffer is used in stead of a StreamString + +#include + +#ifndef STASSID +//#define STASSID "your-ssid" +//#define STAPSK "your-password" +#define STASSID "H369A394602" +#define STAPSK "445396F996E9" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + +// Use web browser to view and copy +// This certificate expires on 1/19/2038, 12:59:59 AM (Central European Standard Time) +const char * rootCACert = R"EOF( +-----BEGIN CERTIFICATE----- +MIIF3jCCA8agAwIBAgIQAf1tMPyjylGoG7xkDjUDLTANBgkqhkiG9w0BAQwFADCB +iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl +cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV +BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTAw +MjAxMDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV +BAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0plcnNleSBDaXR5MR4wHAYDVQQKExVU +aGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNVBAMTJVVTRVJUcnVzdCBSU0EgQ2Vy +dGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK +AoICAQCAEmUXNg7D2wiz0KxXDXbtzSfTTK1Qg2HiqiBNCS1kCdzOiZ/MPans9s/B +3PHTsdZ7NygRK0faOca8Ohm0X6a9fZ2jY0K2dvKpOyuR+OJv0OwWIJAJPuLodMkY +tJHUYmTbf6MG8YgYapAiPLz+E/CHFHv25B+O1ORRxhFnRghRy4YUVD+8M/5+bJz/ +Fp0YvVGONaanZshyZ9shZrHUm3gDwFA66Mzw3LyeTP6vBZY1H1dat//O+T23LLb2 +VN3I5xI6Ta5MirdcmrS3ID3KfyI0rn47aGYBROcBTkZTmzNg95S+UzeQc0PzMsNT +79uq/nROacdrjGCT3sTHDN/hMq7MkztReJVni+49Vv4M0GkPGw/zJSZrM233bkf6 +c0Plfg6lZrEpfDKEY1WJxA3Bk1QwGROs0303p+tdOmw1XNtB1xLaqUkL39iAigmT +Yo61Zs8liM2EuLE/pDkP2QKe6xJMlXzzawWpXhaDzLhn4ugTncxbgtNMs+1b/97l +c6wjOy0AvzVVdAlJ2ElYGn+SNuZRkg7zJn0cTRe8yexDJtC/QV9AqURE9JnnV4ee +UB9XVKg+/XRjL7FQZQnmWEIuQxpMtPAlR1n6BB6T1CZGSlCBst6+eLf8ZxXhyVeE +Hg9j1uliutZfVS7qXMYoCAQlObgOK6nyTJccBz8NUvXt7y+CDwIDAQABo0IwQDAd +BgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rIDZsswDgYDVR0PAQH/BAQDAgEGMA8G +A1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEMBQADggIBAFzUfA3P9wF9QZllDHPF +Up/L+M+ZBn8b2kMVn54CVVeWFPFSPCeHlCjtHzoBN6J2/FNQwISbxmtOuowhT6KO +VWKR82kV2LyI48SqC/3vqOlLVSoGIG1VeCkZ7l8wXEskEVX/JJpuXior7gtNn3/3 +ATiUFJVDBwn7YKnuHKsSjKCaXqeYalltiz8I+8jRRa8YFWSQEg9zKC7F4iRO/Fjs +8PRF/iKz6y+O0tlFYQXBl2+odnKPi4w2r78NBc5xjeambx9spnFixdjQg3IM8WcR +iQycE0xyNN+81XHfqnHd4blsjDwSXWXavVcStkNr/+XeTWYRUc+ZruwXtuhxkYze +Sf7dNXGiFSeUHM9h4ya7b6NnJSFd5t0dCy5oGzuCr+yDZ4XUmFF0sbmZgIn/f3gZ +XHlKYC6SQK5MNyosycdiyA5d9zZbyuAlJQG03RoHnHcAP9Dc1ew91Pq7P8yF1m9/ +qS3fuQL39ZeatTXaw2ewh0qpKJ4jjv9cJ2vhsE/zB+4ALtRZh8tSQZXq9EfX7mRB +VXyNWQKV3WKdwrnuWih0hKWbt5DHDAff9Yk2dDLWKMGwsAvgnEzDHNb842m1R0aB +L6KCq9NjRHDEjf8tM7qtj3u1cIiuPhnPQCjY/MiQu12ZIvVS5ljFH4gxQ+6IHdfG +jjxDah2nGN59PRbxYvnKkKj9 +-----END CERTIFICATE----- +)EOF"; + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + +#if defined(ESP8266) + configTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif +#if defined(ESP32) + configTzTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif + + Serial.print("Waiting for time"); + time_t epoch; + for(;;) { + Serial.write('.'); + time(&epoch); + if(epoch >= 3600) break; + delay(500); + } + Serial.print("Current time: "); + Serial.println(asctime(localtime(&epoch))); + + // Use WiFiClientSecure class to create TLS connection + WiFiClientSecure client; +#if defined(ESP8266) + BearSSL::X509List cert(rootCACert); + client.setTrustAnchors(&cert); +#endif +#if defined(ESP32) + client.setCACert(rootCACert); +#endif + + ClientHTTP http(client); + http.setTimeout(10000); + + http.requestHeaders["Connection"] = "close"; + + if (!http.connect("jigsaw.w3.org", 443)) { + Serial.println("Connection failed"); + return; + } + + if(!http.GET("/HTTP/connection.html")) { + Serial.println("Failed to GET"); + return; + } + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK) { + Serial.println("Data succesfully fetched!"); + } + + // Either read data using a buffer... + uint8_t buffer[1024]; + uint8_t * head = buffer; + // leftToRead() equals: + // - the size of the payload if a 'Content-Length' header is sent by the server + // - the size of the current chunk if no 'Content-Length' header is sent but a 'Transfer-Encoding: chunked' + // In the latter case leftToRead() will only become 0 (zero) after the last chunkhas been read + while(http.leftToRead()) { + size_t bytesToRead = http.available(); + if(bytesToRead == 0) { // No daat available yet + delay(100); + continue; + } + + if(bytesToRead > buffer + sizeof buffer - head) { + bytesToRead = buffer + sizeof buffer - head; + Serial.println("Warning: response truncated because buffer is too small"); + } + if(bytesToRead == 0) { + break; + } + size_t bytesRead = http.read(head, bytesToRead); + head += bytesRead; + } + size_t bytesReceived = head - buffer; + Serial.printf("Received a buffer of %u bytes\n", bytesReceived); + Serial.write(buffer, bytesReceived); // Because data is text it can be written to Serial + + // ... or read data using a StreamString: +/* + StreamString response; + http.printTo(response); + Serial.printf("Received a string of %u characters\n", response.length()); + Serial.println("Received:"); + Serial.println(response); +*/ + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + Serial.println("Closing connection"); + http.stop(); +} + +void loop() { +} diff --git a/libraries/ClientHTTP/examples/POSTClientHTTPS/POSTClientHTTPS.ino b/libraries/ClientHTTP/examples/POSTClientHTTPS/POSTClientHTTPS.ino new file mode 100644 index 00000000000..0adc4db960a --- /dev/null +++ b/libraries/ClientHTTP/examples/POSTClientHTTPS/POSTClientHTTPS.ino @@ -0,0 +1,152 @@ +/* + * ClientHTTP example sketch + * + * POST form data on a WiFiClientSecure with a CA certificate + * + * This example sketch demonstrates how to use the ClientHTTP library + * to send data to a web server using POST. HTTPS (TLS 1.2) secure + * communication is used with a CA certificate. The Postman echo server + * is used for this example. + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include + +#include + +#ifndef STASSID +#define STASSID "your-ssid" +#define STAPSK "your-password" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + +// Use the command: +// openssl s_client -showcerts -connect postman-echo.com:443 +// and copy the last certificate from the chain +// This certificate expires on +const char * rootCACert = R"EOF( +-----BEGIN CERTIFICATE----- +MIIEdTCCA12gAwIBAgIJAKcOSkw0grd/MA0GCSqGSIb3DQEBCwUAMGgxCzAJBgNV +BAYTAlVTMSUwIwYDVQQKExxTdGFyZmllbGQgVGVjaG5vbG9naWVzLCBJbmMuMTIw +MAYDVQQLEylTdGFyZmllbGQgQ2xhc3MgMiBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0 +eTAeFw0wOTA5MDIwMDAwMDBaFw0zNDA2MjgxNzM5MTZaMIGYMQswCQYDVQQGEwJV +UzEQMA4GA1UECBMHQXJpem9uYTETMBEGA1UEBxMKU2NvdHRzZGFsZTElMCMGA1UE +ChMcU3RhcmZpZWxkIFRlY2hub2xvZ2llcywgSW5jLjE7MDkGA1UEAxMyU3RhcmZp +ZWxkIFNlcnZpY2VzIFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IC0gRzIwggEi +MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDVDDrEKvlO4vW+GZdfjohTsR8/ +y8+fIBNtKTrID30892t2OGPZNmCom15cAICyL1l/9of5JUOG52kbUpqQ4XHj2C0N +Tm/2yEnZtvMaVq4rtnQU68/7JuMauh2WLmo7WJSJR1b/JaCTcFOD2oR0FMNnngRo +Ot+OQFodSk7PQ5E751bWAHDLUu57fa4657wx+UX2wmDPE1kCK4DMNEffud6QZW0C +zyyRpqbn3oUYSXxmTqM6bam17jQuug0DuDPfR+uxa40l2ZvOgdFFRjKWcIfeAg5J +Q4W2bHO7ZOphQazJ1FTfhy/HIrImzJ9ZVGif/L4qL8RVHHVAYBeFAlU5i38FAgMB +AAGjgfAwge0wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0O +BBYEFJxfAN+qAdcwKziIorhtSpzyEZGDMB8GA1UdIwQYMBaAFL9ft9HO3R+G9FtV +rNzXEMIOqYjnME8GCCsGAQUFBwEBBEMwQTAcBggrBgEFBQcwAYYQaHR0cDovL28u +c3MyLnVzLzAhBggrBgEFBQcwAoYVaHR0cDovL3guc3MyLnVzL3guY2VyMCYGA1Ud +HwQfMB0wG6AZoBeGFWh0dHA6Ly9zLnNzMi51cy9yLmNybDARBgNVHSAECjAIMAYG +BFUdIAAwDQYJKoZIhvcNAQELBQADggEBACMd44pXyn3pF3lM8R5V/cxTbj5HD9/G +VfKyBDbtgB9TxF00KGu+x1X8Z+rLP3+QsjPNG1gQggL4+C/1E2DUBc7xgQjB3ad1 +l08YuW3e95ORCLp+QCztweq7dp4zBncdDQh/U90bZKuCJ/Fp1U1ervShw3WnWEQt +8jxwmKy6abaVd38PMV4s/KCHOkdp8Hlf9BRUpJVeEXgSYCfOn8J3/yNTd126/+pZ +59vPr5KW7ySaNRB6nJHGDn2Z9j8Z3/VyVOEVqQdZe4O/Ui5GjLIAZHYcSNPYeehu +VsyuLAOQ1xk4meTKCRlb/weWsKh/NEnfVqn3sF/tM+2MR7cwA130A4w= +-----END CERTIFICATE----- +)EOF"; + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + +#if defined(ESP8266) + configTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif +#if defined(ESP32) + configTzTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif + + Serial.print("Waiting for time"); + time_t epoch; + for(;;) { + Serial.write('.'); + time(&epoch); + if(epoch >= 3600) break; + delay(500); + } + Serial.print("Current time: "); + Serial.println(asctime(localtime(&epoch))); + + // Use WiFiClientSecure class to create TLS connection + WiFiClientSecure client; +#if defined(ESP8266) + BearSSL::X509List cert(rootCACert); + client.setTrustAnchors(&cert); +#endif +#if defined(ESP32) + client.setCACert(rootCACert); +#endif + + ClientHTTP http(client); + http.setTimeout(10000); + + http.requestHeaders["Content-Type"] = "application/x-www-form-urlencoded"; + http.requestHeaders["Connection"] = "close"; + + if (!http.connect("postman-echo.com", 443)) { + Serial.println("Connection failed"); + return; + } + + // If the Content-Type: application/x-www-form-urlencoded is used, like in this example, data should be url encoded! + const char payload[] = "field1=value1&field2=value2"; + + if(!http.POST("/post", sizeof payload - 1)) { // The terminating '\0' should not be send + Serial.println("Failed to POST"); + return; + } + + http.write((const uint8_t *)payload, sizeof payload - 1); // The terminating '\0' should not be send + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK || status == ClientHTTP::NO_CONTENT) { + Serial.println("Payload succesfully send!"); + } + + http.printTo(Serial); + Serial.println(); + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + Serial.println("Closing connection"); + http.stop(); +} + +void loop() { +} diff --git a/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino b/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino new file mode 100644 index 00000000000..dd8aa0e1ff3 --- /dev/null +++ b/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino @@ -0,0 +1,206 @@ +/* + * ClientHTTP example sketch + * + * POST json data on a WiFiClientSecure with a CA certificate + * + * This example sketch demonstrates how to use the ClientHTTP library + * to send json data to a web server using POST. HTTPS (TLS 1.2) secure + * communication is used with a CA certificate. ArduinoJson is used to + * serialize and deserialize Json data. The Postman echo server is used + * for this example. + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include +#include + +#include + +#ifndef STASSID +#define STASSID "your-ssid" +#define STAPSK "your-password" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + +// Use the command: +// openssl s_client -showcerts -connect postman-echo.com:443 +// and copy the last certificate from the chain +// This certificate expires on +const char * rootCACert = R"EOF( +-----BEGIN CERTIFICATE----- +MIIEdTCCA12gAwIBAgIJAKcOSkw0grd/MA0GCSqGSIb3DQEBCwUAMGgxCzAJBgNV +BAYTAlVTMSUwIwYDVQQKExxTdGFyZmllbGQgVGVjaG5vbG9naWVzLCBJbmMuMTIw +MAYDVQQLEylTdGFyZmllbGQgQ2xhc3MgMiBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0 +eTAeFw0wOTA5MDIwMDAwMDBaFw0zNDA2MjgxNzM5MTZaMIGYMQswCQYDVQQGEwJV +UzEQMA4GA1UECBMHQXJpem9uYTETMBEGA1UEBxMKU2NvdHRzZGFsZTElMCMGA1UE +ChMcU3RhcmZpZWxkIFRlY2hub2xvZ2llcywgSW5jLjE7MDkGA1UEAxMyU3RhcmZp +ZWxkIFNlcnZpY2VzIFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IC0gRzIwggEi +MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDVDDrEKvlO4vW+GZdfjohTsR8/ +y8+fIBNtKTrID30892t2OGPZNmCom15cAICyL1l/9of5JUOG52kbUpqQ4XHj2C0N +Tm/2yEnZtvMaVq4rtnQU68/7JuMauh2WLmo7WJSJR1b/JaCTcFOD2oR0FMNnngRo +Ot+OQFodSk7PQ5E751bWAHDLUu57fa4657wx+UX2wmDPE1kCK4DMNEffud6QZW0C +zyyRpqbn3oUYSXxmTqM6bam17jQuug0DuDPfR+uxa40l2ZvOgdFFRjKWcIfeAg5J +Q4W2bHO7ZOphQazJ1FTfhy/HIrImzJ9ZVGif/L4qL8RVHHVAYBeFAlU5i38FAgMB +AAGjgfAwge0wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0O +BBYEFJxfAN+qAdcwKziIorhtSpzyEZGDMB8GA1UdIwQYMBaAFL9ft9HO3R+G9FtV +rNzXEMIOqYjnME8GCCsGAQUFBwEBBEMwQTAcBggrBgEFBQcwAYYQaHR0cDovL28u +c3MyLnVzLzAhBggrBgEFBQcwAoYVaHR0cDovL3guc3MyLnVzL3guY2VyMCYGA1Ud +HwQfMB0wG6AZoBeGFWh0dHA6Ly9zLnNzMi51cy9yLmNybDARBgNVHSAECjAIMAYG +BFUdIAAwDQYJKoZIhvcNAQELBQADggEBACMd44pXyn3pF3lM8R5V/cxTbj5HD9/G +VfKyBDbtgB9TxF00KGu+x1X8Z+rLP3+QsjPNG1gQggL4+C/1E2DUBc7xgQjB3ad1 +l08YuW3e95ORCLp+QCztweq7dp4zBncdDQh/U90bZKuCJ/Fp1U1ervShw3WnWEQt +8jxwmKy6abaVd38PMV4s/KCHOkdp8Hlf9BRUpJVeEXgSYCfOn8J3/yNTd126/+pZ +59vPr5KW7ySaNRB6nJHGDn2Z9j8Z3/VyVOEVqQdZe4O/Ui5GjLIAZHYcSNPYeehu +VsyuLAOQ1xk4meTKCRlb/weWsKh/NEnfVqn3sF/tM+2MR7cwA130A4w= +-----END CERTIFICATE----- +)EOF"; + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + +#if defined(ESP8266) + configTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif +#if defined(ESP32) + configTzTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif + + Serial.print("Waiting for time"); + time_t epoch; + for(;;) { + Serial.write('.'); + time(&epoch); + if(epoch >= 3600) break; + delay(500); + } + Serial.print("Current time: "); + Serial.println(asctime(localtime(&epoch))); + + // Use WiFiClientSecure class to create TLS connection + WiFiClientSecure client; +#if defined(ESP8266) + BearSSL::X509List cert(rootCACert); + client.setTrustAnchors(&cert); +#endif +#if defined(ESP32) + client.setCACert(rootCACert); +#endif + + ClientHTTP http(client); + http.setTimeout(10000); + + http.requestHeaders["Content-Type"] = "application/json"; + http.requestHeaders["Connection"] = "close"; + http.responseHeaders["Content-Type"]; + + if (!http.connect("postman-echo.com", 443)) { + Serial.println("Connection failed"); + return; + } + + // If the Content-Type: application/json is used, like in this example, data should be a serialized json! + const char json[] = "{\"sensor\":\"gps\",\"time\":1351824120,\"data\":[48.756080,2.302038]}"; + const size_t requestCapacity = JSON_OBJECT_SIZE(3) + JSON_ARRAY_SIZE(2) + 60; +#if ARDUINOJSON_VERSION_MAJOR == 6 + DynamicJsonDocument requestDocument(requestCapacity); + DeserializationError error = deserializeJson(requestDocument, json); + // Test if parsing succeeds. + if(error) { + Serial.print(F("deserializeJson() failed: ")); + Serial.println(error.c_str()); + return; + } + size_t payloadLength = measureJson(requestDocument); +#endif +#if ARDUINOJSON_VERSION_MAJOR == 5 + DynamicJsonBuffer jsonBuffer(requestCapacity); + JsonObject& root = jsonBuffer.parseObject(json); + // Test if parsing succeeds. + if(!root.success()) { + Serial.println("parseObject() failed"); + return; + } + size_t payloadLength = root.measureLength(); +#endif + + if(!http.POST("/post", payloadLength)) { + Serial.println("Failed to POST"); + return; + } + +#if ARDUINOJSON_VERSION_MAJOR == 6 + serializeJson(requestDocument, http); +#endif +#if ARDUINOJSON_VERSION_MAJOR == 5 + root.printTo(http); +#endif + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK || status == ClientHTTP::NO_CONTENT) { + Serial.println("Payload succesfully send!"); + } + + const size_t responseCapacity = 2*JSON_ARRAY_SIZE(2) + 3*JSON_OBJECT_SIZE(0) + 2*JSON_OBJECT_SIZE(3) + JSON_OBJECT_SIZE(6) + JSON_OBJECT_SIZE(7) + 618; +#if ARDUINOJSON_VERSION_MAJOR == 6 + DynamicJsonDocument responseDocument(responseCapacity); + DeserializationError error = deserializeJson(responseDocument, http); + // Test if parsing succeeds. + if(error) { + Serial.print(F("deserializeJson() failed: ")); + Serial.println(error.c_str()); + return; + } + Serial.println("Received json:"); + serializeJsonPretty(responseDocument, Serial); +#endif +#if ARDUINOJSON_VERSION_MAJOR == 5 + DynamicJsonBuffer jsonBuffer(responseCapacity); + JsonObject& rootReceived = jsonBuffer.parseObject(http); + // Test if parsing succeeds. + if(!rootReceived.success()) { + Serial.println("parseObject() failed"); + return; + } + Serial.println("Received json:"); + rootReceived.prettyPrintTo(Serial); +#endif + + Serial.println("\n"); + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + Serial.println("Closing connection"); + http.stop(); +} + +void loop() { +} diff --git a/libraries/ClientHTTP/examples/POSTSizerClientHTTPS/POSTSizerClientHTTPS.ino b/libraries/ClientHTTP/examples/POSTSizerClientHTTPS/POSTSizerClientHTTPS.ino new file mode 100644 index 00000000000..14292eb5faa --- /dev/null +++ b/libraries/ClientHTTP/examples/POSTSizerClientHTTPS/POSTSizerClientHTTPS.ino @@ -0,0 +1,160 @@ +/* + * ClientHTTP example sketch + * + * POST form data on a WiFiClientSecure with a CA certificate + * + * This example sketch demonstrates how to use the ClientHTTP library + * to send data to a web server using POST. HTTPS (TLS 1.2) secure + * communication is used with a CA certificate. The Postman echo server + * is used for this example. This example assumes that the length of the + * payload is not known beforehand and needs to be computed. The utility + * class Sizer is used for this. + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include + +#include + +#ifndef STASSID +#define STASSID "your-ssid" +#define STAPSK "your-password" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + +// Use the command: +// openssl s_client -showcerts -connect postman-echo.com:443 +// and copy the last certificate from the chain +// This certificate expires on +const char * rootCACert = R"EOF( +-----BEGIN CERTIFICATE----- +MIIEdTCCA12gAwIBAgIJAKcOSkw0grd/MA0GCSqGSIb3DQEBCwUAMGgxCzAJBgNV +BAYTAlVTMSUwIwYDVQQKExxTdGFyZmllbGQgVGVjaG5vbG9naWVzLCBJbmMuMTIw +MAYDVQQLEylTdGFyZmllbGQgQ2xhc3MgMiBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0 +eTAeFw0wOTA5MDIwMDAwMDBaFw0zNDA2MjgxNzM5MTZaMIGYMQswCQYDVQQGEwJV +UzEQMA4GA1UECBMHQXJpem9uYTETMBEGA1UEBxMKU2NvdHRzZGFsZTElMCMGA1UE +ChMcU3RhcmZpZWxkIFRlY2hub2xvZ2llcywgSW5jLjE7MDkGA1UEAxMyU3RhcmZp +ZWxkIFNlcnZpY2VzIFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IC0gRzIwggEi +MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDVDDrEKvlO4vW+GZdfjohTsR8/ +y8+fIBNtKTrID30892t2OGPZNmCom15cAICyL1l/9of5JUOG52kbUpqQ4XHj2C0N +Tm/2yEnZtvMaVq4rtnQU68/7JuMauh2WLmo7WJSJR1b/JaCTcFOD2oR0FMNnngRo +Ot+OQFodSk7PQ5E751bWAHDLUu57fa4657wx+UX2wmDPE1kCK4DMNEffud6QZW0C +zyyRpqbn3oUYSXxmTqM6bam17jQuug0DuDPfR+uxa40l2ZvOgdFFRjKWcIfeAg5J +Q4W2bHO7ZOphQazJ1FTfhy/HIrImzJ9ZVGif/L4qL8RVHHVAYBeFAlU5i38FAgMB +AAGjgfAwge0wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0O +BBYEFJxfAN+qAdcwKziIorhtSpzyEZGDMB8GA1UdIwQYMBaAFL9ft9HO3R+G9FtV +rNzXEMIOqYjnME8GCCsGAQUFBwEBBEMwQTAcBggrBgEFBQcwAYYQaHR0cDovL28u +c3MyLnVzLzAhBggrBgEFBQcwAoYVaHR0cDovL3guc3MyLnVzL3guY2VyMCYGA1Ud +HwQfMB0wG6AZoBeGFWh0dHA6Ly9zLnNzMi51cy9yLmNybDARBgNVHSAECjAIMAYG +BFUdIAAwDQYJKoZIhvcNAQELBQADggEBACMd44pXyn3pF3lM8R5V/cxTbj5HD9/G +VfKyBDbtgB9TxF00KGu+x1X8Z+rLP3+QsjPNG1gQggL4+C/1E2DUBc7xgQjB3ad1 +l08YuW3e95ORCLp+QCztweq7dp4zBncdDQh/U90bZKuCJ/Fp1U1ervShw3WnWEQt +8jxwmKy6abaVd38PMV4s/KCHOkdp8Hlf9BRUpJVeEXgSYCfOn8J3/yNTd126/+pZ +59vPr5KW7ySaNRB6nJHGDn2Z9j8Z3/VyVOEVqQdZe4O/Ui5GjLIAZHYcSNPYeehu +VsyuLAOQ1xk4meTKCRlb/weWsKh/NEnfVqn3sF/tM+2MR7cwA130A4w= +-----END CERTIFICATE----- +)EOF"; + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + +#if defined(ESP8266) + configTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif +#if defined(ESP32) + configTzTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif + + Serial.print("Waiting for time"); + time_t epoch; + for(;;) { + Serial.write('.'); + time(&epoch); + if(epoch >= 3600) break; + delay(500); + } + Serial.print("Current time: "); + Serial.println(asctime(localtime(&epoch))); + + // Use WiFiClientSecure class to create TLS connection + WiFiClientSecure client; +#if defined(ESP8266) + BearSSL::X509List cert(rootCACert); + client.setTrustAnchors(&cert); +#endif +#if defined(ESP32) + client.setCACert(rootCACert); +#endif + + ClientHTTP http(client); + http.setTimeout(10000); + + http.requestHeaders["Content-Type"] = "application/x-www-form-urlencoded"; + http.requestHeaders["Connection"] = "close"; + + if (!http.connect("postman-echo.com", 443)) { + Serial.println("Connection failed"); + return; + } + + // If the Content-Type: application/x-www-form-urlencoded is used, like in this example, data should be url encoded! + const char payload[] = "field1=value1&field2=value2"; + + // Determine the size of the payload using the utility class Sizer + Sizer sizer; + sizer.print(payload); + size_t payloadLength = sizer.getSize(); + Serial.printf("Payload length is %u\n", payloadLength); + + if(!http.POST("/post", payloadLength)) { + Serial.println("Failed to POST"); + return; + } + + http.print(payload); + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK || status == ClientHTTP::NO_CONTENT) { + Serial.println("Payload succesfully send!"); + } + + http.printTo(Serial); + Serial.println(); + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + Serial.println("Closing connection"); + http.stop(); +} + +void loop() { +} diff --git a/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino new file mode 100644 index 00000000000..a8a981c49d8 --- /dev/null +++ b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino @@ -0,0 +1,174 @@ +/* + * ClientHTTP example sketch + * + * GET on a WiFiClientSecure with a CA certificate + * + * This example sketch demonstrates how to use the ClientHTTP library + * to fetch from a web server using GET. HTTPS (TLS 1.2) secure + * communication is used with a CA certificate. The connection is + * reused to fetch from the same server every 10 seconds. + * + * Created by Jeroen Döll on 25 March 2020 + * This example is in the public domain + */ + +#if defined(ESP8266) +#include +#endif +#if defined(ESP32) +#include +#endif + +#include + +#include + +#ifndef STASSID +#define STASSID "your-ssid" +#define STAPSK "your-password" +#endif + +const char * ssid = STASSID; +const char * password = STAPSK; + +// Use web browser to view and copy +// This certificate expires on 1/19/2038, 12:59:59 AM (Central European Standard Time) +const char * rootCACert = R"EOF( +-----BEGIN CERTIFICATE----- +MIIF3jCCA8agAwIBAgIQAf1tMPyjylGoG7xkDjUDLTANBgkqhkiG9w0BAQwFADCB +iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl +cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV +BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTAw +MjAxMDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV +BAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0plcnNleSBDaXR5MR4wHAYDVQQKExVU +aGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNVBAMTJVVTRVJUcnVzdCBSU0EgQ2Vy +dGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK +AoICAQCAEmUXNg7D2wiz0KxXDXbtzSfTTK1Qg2HiqiBNCS1kCdzOiZ/MPans9s/B +3PHTsdZ7NygRK0faOca8Ohm0X6a9fZ2jY0K2dvKpOyuR+OJv0OwWIJAJPuLodMkY +tJHUYmTbf6MG8YgYapAiPLz+E/CHFHv25B+O1ORRxhFnRghRy4YUVD+8M/5+bJz/ +Fp0YvVGONaanZshyZ9shZrHUm3gDwFA66Mzw3LyeTP6vBZY1H1dat//O+T23LLb2 +VN3I5xI6Ta5MirdcmrS3ID3KfyI0rn47aGYBROcBTkZTmzNg95S+UzeQc0PzMsNT +79uq/nROacdrjGCT3sTHDN/hMq7MkztReJVni+49Vv4M0GkPGw/zJSZrM233bkf6 +c0Plfg6lZrEpfDKEY1WJxA3Bk1QwGROs0303p+tdOmw1XNtB1xLaqUkL39iAigmT +Yo61Zs8liM2EuLE/pDkP2QKe6xJMlXzzawWpXhaDzLhn4ugTncxbgtNMs+1b/97l +c6wjOy0AvzVVdAlJ2ElYGn+SNuZRkg7zJn0cTRe8yexDJtC/QV9AqURE9JnnV4ee +UB9XVKg+/XRjL7FQZQnmWEIuQxpMtPAlR1n6BB6T1CZGSlCBst6+eLf8ZxXhyVeE +Hg9j1uliutZfVS7qXMYoCAQlObgOK6nyTJccBz8NUvXt7y+CDwIDAQABo0IwQDAd +BgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rIDZsswDgYDVR0PAQH/BAQDAgEGMA8G +A1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEMBQADggIBAFzUfA3P9wF9QZllDHPF +Up/L+M+ZBn8b2kMVn54CVVeWFPFSPCeHlCjtHzoBN6J2/FNQwISbxmtOuowhT6KO +VWKR82kV2LyI48SqC/3vqOlLVSoGIG1VeCkZ7l8wXEskEVX/JJpuXior7gtNn3/3 +ATiUFJVDBwn7YKnuHKsSjKCaXqeYalltiz8I+8jRRa8YFWSQEg9zKC7F4iRO/Fjs +8PRF/iKz6y+O0tlFYQXBl2+odnKPi4w2r78NBc5xjeambx9spnFixdjQg3IM8WcR +iQycE0xyNN+81XHfqnHd4blsjDwSXWXavVcStkNr/+XeTWYRUc+ZruwXtuhxkYze +Sf7dNXGiFSeUHM9h4ya7b6NnJSFd5t0dCy5oGzuCr+yDZ4XUmFF0sbmZgIn/f3gZ +XHlKYC6SQK5MNyosycdiyA5d9zZbyuAlJQG03RoHnHcAP9Dc1ew91Pq7P8yF1m9/ +qS3fuQL39ZeatTXaw2ewh0qpKJ4jjv9cJ2vhsE/zB+4ALtRZh8tSQZXq9EfX7mRB +VXyNWQKV3WKdwrnuWih0hKWbt5DHDAff9Yk2dDLWKMGwsAvgnEzDHNb842m1R0aB +L6KCq9NjRHDEjf8tM7qtj3u1cIiuPhnPQCjY/MiQu12ZIvVS5ljFH4gxQ+6IHdfG +jjxDah2nGN59PRbxYvnKkKj9 +-----END CERTIFICATE----- +)EOF"; + + +// Use WiFiClientSecure class to create TLS connection +WiFiClientSecure client; +ClientHTTP http(client); + + +void setup() { + Serial.begin(115200); + Serial.println(); + + Serial.printf("Connecting to %s", ssid); + WiFi.mode(WIFI_STA); + WiFi.begin(ssid, password); + while (WiFi.status() != WL_CONNECTED) { + delay(500); + Serial.print("."); + } + Serial.println(); + Serial.println("WiFi connected"); + Serial.print("IP address: "); + Serial.println(WiFi.localIP()); + +#if defined(ESP8266) + configTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif +#if defined(ESP32) + configTzTime("CET-1CEST-2,M3.5.0/02:00:00,M10.5.0/03:00:00", "pool.ntp.org", "time.nist.gov"); +#endif + + Serial.print("Waiting for time"); + time_t epoch; + for(;;) { + Serial.write('.'); + time(&epoch); + if(epoch >= 3600) break; + delay(500); + } + Serial.print("Current time: "); + Serial.println(asctime(localtime(&epoch))); + +#if defined(ESP8266) + BearSSL::X509List cert(rootCACert); + client.setTrustAnchors(&cert); +#endif +#if defined(ESP32) + client.setCACert(rootCACert); +#endif + + http.setTimeout(10000); + + http.requestHeaders["Connection"] = "Keep-Alive"; + http.responseHeaders["Connection"]; +} + + +void wait() { + for(size_t index = 0; index < 10; index++) { + Serial.print("."); + delay(1000); + } + Serial.println(); +} + +void loop() { + if(!http.connected()) { + if (!http.connect("jigsaw.w3.org", 443)) { + Serial.print("Connection failed, waiting 10s before retrying"); + wait(); + return; + } + } + + if(!http.GET("/HTTP/connection.html")) { + Serial.println("Failed to GET, waiting 10s before retrying"); + wait(); + return; + } + + ClientHTTP::http_code_t status = http.status(); + + if(status > 0) { + Serial.printf("HTTP status code is %d\n", status); + if(status == ClientHTTP::OK) { + Serial.println("Page succesfully fetched!"); + } + + http.printTo(Serial); + Serial.println(); + } else { + Serial.printf("Request failed with error code %d\n", status); + } + + if(http.responseHeaders["Connection"] != NULL && strcasecmp(http.responseHeaders["Connection"], "close") == 0) { + Serial.println("Closing connection because the server requested to do so"); + http.stop(); + } else { + Serial.println("Reusing the connection"); + } + + Serial.print("Wait 10s for the next loop"); + wait(); +} diff --git a/libraries/ClientHTTP/keywords.txt b/libraries/ClientHTTP/keywords.txt new file mode 100644 index 00000000000..03e7defe9db --- /dev/null +++ b/libraries/ClientHTTP/keywords.txt @@ -0,0 +1,130 @@ +####################################### +# Syntax Coloring Map For ESP8266HTTPClient +####################################### + +####################################### +# Library (KEYWORD3) +####################################### + +ClientHTTP KEYWORD3 RESERVED_WORD + +####################################### +# Datatypes (KEYWORD1) +####################################### + +Sizer KEYWORD1 DATA_TYPE +ClientHTTP KEYWORD1 DATA_TYPE +t KEYWORD1 DATA_TYPE +follow_redirects_t KEYWORD1 DATA_TYPE +http_version_t KEYWORD1 DATA_TYPE +client_state_t KEYWORD1 DATA_TYPE + +####################################### +# Methods and Functions (KEYWORD2) +####################################### + +reset KEYWORD2 +getSize KEYWORD2 +connect KEYWORD2 +write KEYWORD2 +available KEYWORD2 +read KEYWORD2 +peek KEYWORD2 +flush KEYWORD2 +stop KEYWORD2 +connected KEYWORD2 +GET KEYWORD2 +HEAD KEYWORD2 +POST KEYWORD2 +PUT KEYWORD2 +PATCH KEYWORD2 +DELETE KEYWORD2 +status KEYWORD2 +leftToRead KEYWORD2 +leftToWrite KEYWORD2 +printTo KEYWORD2 +writeRequestHeaders KEYWORD2 +readResponseHeaders KEYWORD2 + +####################################### +# Constants (LITERAL1) +####################################### + +HTTP_ERROR LITERAL1 RESERVED_WORD_2 +REDIRECT_LIMIT_REACHED LITERAL1 RESERVED_WORD_2 +REDIRECT_CONNECTION_FAILED LITERAL1 RESERVED_WORD_2 +REDIRECT_UNKOWN_METHOD LITERAL1 RESERVED_WORD_2 +REDIRECT_FAILURE LITERAL1 RESERVED_WORD_2 +RESPONSE_HEADER_TIMEOUT LITERAL1 RESERVED_WORD_2 +OUT_OF_MEMORY LITERAL1 RESERVED_WORD_2 +CHUNK_HEADER_TIMEOUT LITERAL1 RESERVED_WORD_2 +CHUNK_HEADER_INVALID LITERAL1 RESERVED_WORD_2 +CONTINUE LITERAL1 RESERVED_WORD_2 +SWITCHING_PROTOCOLS LITERAL1 RESERVED_WORD_2 +PROCESSING LITERAL1 RESERVED_WORD_2 +OK LITERAL1 RESERVED_WORD_2 +CREATED LITERAL1 RESERVED_WORD_2 +ACCEPTED LITERAL1 RESERVED_WORD_2 +NON_AUTHORITATIVE_INFORMATION LITERAL1 RESERVED_WORD_2 +NO_CONTENT LITERAL1 RESERVED_WORD_2 +RESET_CONTENT LITERAL1 RESERVED_WORD_2 +PARTIAL_CONTENT LITERAL1 RESERVED_WORD_2 +MULTI_STATUS LITERAL1 RESERVED_WORD_2 +ALREADY_REPORTED LITERAL1 RESERVED_WORD_2 +IM_USED LITERAL1 RESERVED_WORD_2 +MULTIPLE_CHOICES LITERAL1 RESERVED_WORD_2 +MOVED_PERMANENTLY LITERAL1 RESERVED_WORD_2 +FOUND LITERAL1 RESERVED_WORD_2 +SEE_OTHER LITERAL1 RESERVED_WORD_2 +NOT_MODIFIED LITERAL1 RESERVED_WORD_2 +USE_PROXY LITERAL1 RESERVED_WORD_2 +TEMPORARY_REDIRECT LITERAL1 RESERVED_WORD_2 +PERMANENT_REDIRECT LITERAL1 RESERVED_WORD_2 +BAD_REQUEST LITERAL1 RESERVED_WORD_2 +UNAUTHORIZED LITERAL1 RESERVED_WORD_2 +PAYMENT_REQUIRED LITERAL1 RESERVED_WORD_2 +FORBIDDEN LITERAL1 RESERVED_WORD_2 +NOT_FOUND LITERAL1 RESERVED_WORD_2 +METHOD_NOT_ALLOWED LITERAL1 RESERVED_WORD_2 +NOT_ACCEPTABLE LITERAL1 RESERVED_WORD_2 +PROXY_AUTHENTICATION_REQUIRED LITERAL1 RESERVED_WORD_2 +REQUEST_TIMEOUT LITERAL1 RESERVED_WORD_2 +CONFLICT LITERAL1 RESERVED_WORD_2 +GONE LITERAL1 RESERVED_WORD_2 +LENGTH_REQUIRED LITERAL1 RESERVED_WORD_2 +PRECONDITION_FAILED LITERAL1 RESERVED_WORD_2 +PAYLOAD_TOO_LARGE LITERAL1 RESERVED_WORD_2 +URI_TOO_LONG LITERAL1 RESERVED_WORD_2 +UNSUPPORTED_MEDIA_TYPE LITERAL1 RESERVED_WORD_2 +RANGE_NOT_SATISFIABLE LITERAL1 RESERVED_WORD_2 +EXPECTATION_FAILED LITERAL1 RESERVED_WORD_2 +MISDIRECTED_REQUEST LITERAL1 RESERVED_WORD_2 +UNPROCESSABLE_ENTITY LITERAL1 RESERVED_WORD_2 +LOCKED LITERAL1 RESERVED_WORD_2 +FAILED_DEPENDENCY LITERAL1 RESERVED_WORD_2 +UPGRADE_REQUIRED LITERAL1 RESERVED_WORD_2 +PRECONDITION_REQUIRED LITERAL1 RESERVED_WORD_2 +TOO_MANY_REQUESTS LITERAL1 RESERVED_WORD_2 +REQUEST_HEADER_FIELDS_TOO_LARGE LITERAL1 RESERVED_WORD_2 +INTERNAL_SERVER_ERROR LITERAL1 RESERVED_WORD_2 +NOT_IMPLEMENTED LITERAL1 RESERVED_WORD_2 +BAD_GATEWAY LITERAL1 RESERVED_WORD_2 +SERVICE_UNAVAILABLE LITERAL1 RESERVED_WORD_2 +GATEWAY_TIMEOUT LITERAL1 RESERVED_WORD_2 +HTTP_VERSION_NOT_SUPPORTED LITERAL1 RESERVED_WORD_2 +VARIANT_ALSO_NEGOTIATES LITERAL1 RESERVED_WORD_2 +INSUFFICIENT_STORAGE LITERAL1 RESERVED_WORD_2 +LOOP_DETECTED LITERAL1 RESERVED_WORD_2 +NOT_EXTENDED LITERAL1 RESERVED_WORD_2 +NETWORK_AUTHENTICATION_REQUIRED LITERAL1 RESERVED_WORD_2 +DISABLE LITERAL1 RESERVED_WORD_2 +STRICT LITERAL1 RESERVED_WORD_2 +FORCE LITERAL1 RESERVED_WORD_2 +HTTP_10 LITERAL1 RESERVED_WORD_2 +HTTP_11 LITERAL1 RESERVED_WORD_2 +CLIENT_READY LITERAL1 RESERVED_WORD_2 +CLIENT_REQUEST_HEADER_SENT LITERAL1 RESERVED_WORD_2 +CLIENT_RESPONSE_HEADER LITERAL1 RESERVED_WORD_2 +CLIENT_RESPONSE_PAYLOAD LITERAL1 RESERVED_WORD_2 +RESPONSE_HEADER_LINE_SIZE LITERAL1 RESERVED_WORD_2 +PRINT_BUFFER_SIZE LITERAL1 RESERVED_WORD_2 diff --git a/libraries/ClientHTTP/library.properties b/libraries/ClientHTTP/library.properties new file mode 100644 index 00000000000..77d47e7d8f3 --- /dev/null +++ b/libraries/ClientHTTP/library.properties @@ -0,0 +1,9 @@ +name=ClientHTTP +version=0.1 +author=Jeroen Döll +maintainer=Jeroen Döll +sentence=HTTP Client for ESP8266 and ESP32 +paragraph= +category=Communication +url=https://github.com/espressif/arduino-esp32/tree/master/libraries/ClientHTTP +architectures=esp32,esp8266 diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp new file mode 100644 index 00000000000..c2e17848677 --- /dev/null +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -0,0 +1,788 @@ +/* + * ClientHTTP.h - A HTTP Client class using the standard Client and Stream interface, + * while handling the request and response headers and chunked + * Transfer-Encoding 'under the hood' + * + * Copyright (C) 2020 Jeroen Döll + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + + #include "ClientHTTP.h" + + + ClientHTTP::ClientHTTP(Client &client): _client(client) { + _host = NULL; + _state = CLIENT_READY; + _location = NULL; + _redirectCount = 0; + strcpy(_originalMethod, ""); +} + + + ClientHTTP::~ClientHTTP() { + free(_location); + +// TO DO RESPONSE HEADERS +// for(auto responseHeader: responseHeaders) { +// free(responseHeader.second); // !!!!!! NOT SURE IF THESE POINTERS MUST BE FREE-ED +// } + + free(_host); +} + +int ClientHTTP::connect(IPAddress ip, uint16_t port) { + if(_host) { + free(_host); + _host = NULL; + } + + _ip = ip; + _port = port; + +#if defined(ESP32) + log_v("Connect to %s at port %u", ip.toString().c_str(), port); +#endif + + return _client.connect(ip, port); +} + +int ClientHTTP::connect(const char * host, uint16_t port) { + _ip = INADDR_NONE; + + if(_host) { + if(strcasecmp(_host,host) != 0) { // Only make necessary heap mutations + free(_host); + _host = strdup(host); + } + } else { + _host = strdup(host); + } + if(_host == NULL) { +#if defined(ESP32) + log_e("Out of memory"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Out of memory"); +#endif + + return 0; + } + + _port = port; + +#if defined(ESP32) + log_v("Connect to %s at port %u", host, port); +#endif + + return _client.connect(host, port); +} + +size_t ClientHTTP::write(uint8_t c) { + if(_state != CLIENT_REQUEST_HEADER_SENT) { +#if defined(ESP32) + log_e("Invalid state: write"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Invalid state: write"); +#endif + + return 0; + } + + if(_leftToWrite == 0) { +#if defined(ESP32) + log_v("No more bytes to write"); +#endif + + return 0; + } + + size_t bytesWritten = _client.write(c); + _leftToWrite -= bytesWritten; + if(_leftToWrite == 0) { +// _client.flush(); // !!!!!!!! DOES THIS FLUSH THE WRITE BUFFER ONLY (SHOULD NOT FLUSH READ BUFFER) + _state = CLIENT_RESPONSE_HEADER; + } + + return bytesWritten; +} + +size_t ClientHTTP::write(const uint8_t *buf, size_t size) { + if(_state != CLIENT_REQUEST_HEADER_SENT) { +#if defined(ESP32) + log_e("Invalid state: write"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Invalid state: write"); +#endif + + return 0; + } + + if(_leftToWrite == 0) { +#if defined(ESP32) + log_v("No more bytes to write"); +#endif + + return 0; + } + + size_t bytesToWrite = size; + if(bytesToWrite > _leftToWrite) { +#if defined(ESP32) + log_d("Too many bytes to write: truncated %u bytes", bytesToWrite - _leftToWrite); +#endif + + bytesToWrite = _leftToWrite; + } + + size_t bytesWritten = _client.write(buf, bytesToWrite); + if(_leftToWrite == 0) { // Chunked write, is a chunked request part of the HTTP 1.1 standard? + if(bytesWritten != 0 || size == 0) { + if(!_firstChunkToWrite) { + _client.print("\r\n"); + } else { + _firstChunkToWrite = false; + } + _client.printf("%x\r\n", bytesWritten); + if(size == 0) { // empty write to mark end of chunked write + _state = CLIENT_RESPONSE_HEADER; + } + } + } else { + _leftToWrite -= bytesWritten; + if(_leftToWrite == 0) { + // _client.flush(); // !!!!!!!! DOES THIS FLUSH THE WRITE BUFFER ONLY (SHOULD NOT FLUSH READ BUFFER) + _state = CLIENT_RESPONSE_HEADER; + } + } + + return bytesWritten; +} + +int ClientHTTP::available() { + if(_state == CLIENT_RESPONSE_PAYLOAD) { + if(_leftToRead == 0) { + if(_chunked) { + char chunkHeader[7]; + size_t bytesRead = _client.readBytesUntil('\n', chunkHeader, sizeof chunkHeader); + if(bytesRead == 0) { +#if defined(ESP32) + log_e("Timeout reading chunk size"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Timeout reading chunk size"); +#endif + + return 0; // Time out reading chunk header + } + + if(bytesRead != 1 || chunkHeader[0] != '\r') { +#if defined(ESP32) + log_e("Chunked transport out of sync"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Chunked transport out of sync"); +#endif + + return 0; + } + + bytesRead = _client.readBytesUntil('\n', chunkHeader, sizeof chunkHeader); + if(bytesRead == 0) { +#if defined(ESP32) + log_e("Timeout reading chunk size"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Timeout reading chunk size"); +#endif + + return 0; // Time out reading chunk header + } + + if(chunkHeader[bytesRead - 1] != '\r') { +#if defined(ESP32) + log_e("Invalid chunk header"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Invalid chunk header"); +#endif + + return 0; // Invalid chunk header + } + + _leftToRead = strtol(chunkHeader, NULL, 16); +#if defined(ESP32) + chunkHeader[bytesRead] = '\0'; + log_v("Chunk header '%s', chunk size %04x (%u)", chunkHeader, _leftToRead, _leftToRead); +#endif + + if(_leftToRead == 0) { + _state = CLIENT_READY; + + return 0; + } else { + + return _client.available(); + } + } else { + _state = CLIENT_READY; + + return 0; + } + } else { + + return _client.available(); + } + } else if(_leftToRead == 0) { + _state = CLIENT_READY; + } else { +#if defined(ESP32) + log_e("Invalid state (%u): available", _state); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Invalid state (%u): available", _state); +#endif + } + + return 0; +} + + +int ClientHTTP::read() { + uint8_t c; + if(read(&c, 1) != 1) return -1; + + return c; +} + +int ClientHTTP::read(uint8_t *buf, size_t size) { + if(available()) { + size_t bytesRead = _client.read(buf, size); + _leftToRead -= bytesRead; + + // To prevent _leftToRead being 0 before the end of the stream, so _leftToRead == 0 can be used to check for the end of the stream + if(_chunked && _leftToRead == 0 && _state != CLIENT_READY) { + available(); + } + + return bytesRead; + } else { + + return 0; + } +} + + +int ClientHTTP::peek() { + if(available()) { + return _client.peek(); + } else { + return -1; + } +} + + +void ClientHTTP::flush() { + _client.flush(); +} + + +void ClientHTTP::stop() { +#if defined(ESP32) + log_v("Stop"); +#endif + + _client.stop(); +} + + +uint8_t ClientHTTP::connected() { + return _client.connected(); +} + + + ClientHTTP::operator bool() { + return _client; +} + + +bool ClientHTTP::writeRequestHeaders(const char * method, const char * uri, size_t contentLength) { + if(_host == NULL && _ip == INADDR_NONE) { +#if defined(ESP32) + log_e("Host not set"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Host not set"); +#endif + + return false; + } + if(!_client.connected()) { + Serial.println("(writeRequestHeaders) Not connected"); +#if defined(ESP32) + log_e("Not connected"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Not connected"); +#endif + + return false; + } + + if(_state == CLIENT_RESPONSE_PAYLOAD) { + if(_leftToRead) { +#if defined(ESP32) + log_v("Cleaning up %yy bytes", _leftToRead); +#endif + while(_leftToRead) { + if(available() == 0) { + delay(100); + continue; + } + read(); + } + } + _state = CLIENT_READY; + } + + if(_state != CLIENT_READY) { + Serial.printf("(writeRequestHeaders) Invalid state (%u)\n", _state); +#if defined(ESP32) + log_e("Invalid state (%u)", _state); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Invalid state (%u)", _state); +#endif + + return false; + } + +#if defined(ESP32) + log_v("%s %s, content length is %u", method, uri, contentLength); + if(_redirectCount) { + log_v("%u time(-s) redirected", _redirectCount); + } +#endif + + if(_redirectCount == 0) { + strcpy(_originalMethod, method); + } + +// for(auto responseHeader: responseHeaders) { +// free(responseHeader.second); // !!!!!! NOT SURE IF THESE POINTERS MUST BE FREE-ED +// responseHeader.second = NULL; +//Serial.printf("Freed response header '%s', value is '%s'\n", responseHeader.first, responseHeader.second != NULL ? responseHeader.second : "null"); +// } + + _leftToWrite = 0; + _firstChunkToWrite = true; + + _client.printf("%s %s HTTP/1.1\r\n", method, uri); + if(_host) { + _client.printf("Host: %s\r\n", _host); + } else { + _client.printf("Host: %s\r\n", _ip.toString().c_str()); + } + + for(auto requestHeader: requestHeaders) { // std::pair + _client.printf("%s: %s\r\n", requestHeader.first, requestHeader.second); +#if defined(ESP32) + log_v("Request header '%s: %s'", requestHeader.first, requestHeader.second); +#endif + } + + if(strcmp(method, "POST") == 0) { + _leftToWrite = contentLength; + _client.printf("Content-Length: %u\r\n", contentLength); +#if defined(ESP32) + log_v("Request header 'Content-Length: %u'", contentLength); +#endif + } + + if(requestHeaders.find("User-Agent") == requestHeaders.end()) { + _client.print("User-Agent: ESPClientHTTP\r\n"); +#if defined(ESP32) + log_v("Request header 'User-Agent: ESPClientHTTP'"); +#endif + } + + if(httpVersion == HTTP_10 && requestHeaders.find("Connection") == requestHeaders.end()) { + _client.print("Connection: keep-alive\r\n"); +#if defined(ESP32) + log_v("Request header 'Connection: keep-alive'"); +#endif + } + + _client.printf("\r\n"); + + _client.flush(); + + return true; +} + + +bool ClientHTTP::GET(const char * uri) { + bool success = writeRequestHeaders("GET", uri); + _state = CLIENT_RESPONSE_HEADER; + + return success; +} + + +bool ClientHTTP::HEAD(const char * uri) { + bool success = writeRequestHeaders("HEAD", uri); + _state = CLIENT_RESPONSE_HEADER; + + return success; +} + + +bool ClientHTTP::POST(const char * uri, size_t contentLength) { + bool success = writeRequestHeaders("POST", uri, contentLength); + _state = CLIENT_REQUEST_HEADER_SENT; + + return success; +} + + +bool ClientHTTP::PUT(const char * uri, size_t contentLength) { + bool success = writeRequestHeaders("PUT", uri, contentLength); + _state = CLIENT_REQUEST_HEADER_SENT; + + return success; +} + + +bool ClientHTTP::PATCH(const char * uri, size_t contentLength) { + bool success = writeRequestHeaders("PATCH", uri, contentLength); + _state = CLIENT_REQUEST_HEADER_SENT; + + return success; +} + + +bool ClientHTTP::DELETE(const char * uri, size_t contentLength) { + bool success = writeRequestHeaders("DELETE", uri, contentLength); + _state = CLIENT_REQUEST_HEADER_SENT; + + return success; +} + + +ClientHTTP::http_code_t ClientHTTP::status() { + if(_state != CLIENT_RESPONSE_HEADER) { + return HTTP_ERROR; + } + + ClientHTTP::http_code_t statusCode = readResponseHeaders(); + +#if defined(SUPPORT_REDIRECTS) + // Redirect if status code says so + while(statusCode == MOVED_PERMANENTLY || statusCode == TEMPORARY_REDIRECT || statusCode == FOUND || statusCode == SEE_OTHER) { + if(_redirectCount++ == redirectLimit) { +#if defined(ESP32) + log_e("Redirect limit reached"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Redirect limit reached"); +#endif + + return REDIRECT_LIMIT_REACHED; + } + + if(leftToRead()) { + while(leftToRead()) read(); // Replace this line with 'printTo(Serial)' to view the dropped payload +#if defined(ESP32) + log_v("Dropped sent payload"); +#endif + } + +#if defined(ESP32) + log_v("Redirect to '%s'", _location); +#endif + if(strchr(_location, ':')) { +#if defined(ESP32) + log_e("Redirection to (another) host not supported (yet)"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Redirection to (another) host not supported (yet)"); +#endif + + return HTTP_ERROR; + } + + if(!connected()) { +#if defined(ESP32) + log_v("(re-)connect"); +#endif + int connectionStatus; + if(_host != NULL) { + connectionStatus = connect(_host, _port); + } else { + connectionStatus = connect(_ip, _port); + } + if(!connectionStatus) { + Serial.println("(status) Failed to (re-)connect to host"); +#if defined(ESP32) + log_e("Failed to (re-)connect to host"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Failed to (re-)connect to host"); +#endif + _redirectCount = 0; + strcpy(_originalMethod, ""); + + return REDIRECT_CONNECTION_FAILED; + } + } + + _state = CLIENT_READY; + + bool methodResult; + if((statusCode == MOVED_PERMANENTLY || statusCode == TEMPORARY_REDIRECT) && (strcmp(_originalMethod, "GET") != 0 || strcmp(_originalMethod, "HEAD") != 0)) { + if(strcmp(_originalMethod, "POST") == 0) { + methodResult = POST(_location); + } else if(strcmp(_originalMethod, "PUT") == 0) { + methodResult = PUT(_location); + } else if(strcmp(_originalMethod, "PATCH") == 0) { + methodResult = PATCH(_location); + } else if(strcmp(_originalMethod, "DELETE") == 0) { + methodResult = DELETE(_location); + } else { + Serial.printf("(status) Redirection with method '%s' not supported", _originalMethod); +#if defined(ESP32) + log_e("Redirection with method '%s' not supported", _originalMethod); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Redirection with method '%s' not supported", _originalMethod); +#endif + _redirectCount = 0; + strcpy(_originalMethod, ""); + + return REDIRECT_UNKOWN_METHOD; + } + } else { + methodResult = GET(_location); + } + + if(!methodResult) { +#if defined(ESP32) + log_e("Failed to redirect to '%s'", _location); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Failed to redirect to '%s'", _location); +#endif + _redirectCount = 0; + strcpy(_originalMethod, ""); + + return REDIRECT_FAILURE; + } + + statusCode = readResponseHeaders(); + } + + _redirectCount = 0; + strcpy(_originalMethod, ""); +#endif + + return statusCode; +} + + +ClientHTTP::http_code_t ClientHTTP::readResponseHeaders() { + + char headerLine[RESPONSE_HEADER_LINE_SIZE]; + size_t bytesRead = _client.readBytesUntil('\n', headerLine, sizeof headerLine - 1); + headerLine[bytesRead] = '\0'; + char * statusPtr = strchr(headerLine, ' '); + if(!statusPtr) return HTTP_ERROR; + + int status = atoi(statusPtr); +#if defined(ESP32) + log_v("Response status line is '%s', status is %u", headerLine, status); +#endif + + _chunked = false; + _leftToRead = 0; + +#if defined(ESP32) + log_v("Client timeout is %u\n", _client.getTimeout()); +#endif + + for(;;) { + size_t bytesRead = _client.readBytesUntil('\n', headerLine, sizeof headerLine - 1); + if(bytesRead == 0) { +#if defined(ESP32) + log_e("Timeout reading response headers"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Timeout reading response headers"); +#endif + + return RESPONSE_HEADER_TIMEOUT; + } + + if(headerLine[0] == '\r') break; + + headerLine[bytesRead - 1] = '\0'; +// Serial.printf("HEADER '%s'\n", headerLine); + + char * valuePtr = strchr(headerLine, ':'); + if(valuePtr) { + *valuePtr++ = '\0'; + while(*valuePtr && isspace(*valuePtr)) valuePtr++; // Skip leading space characters + if(*valuePtr != '\0') { +#if defined(ESP32) + log_v("Read response header '%s: %s'", headerLine, valuePtr); +#endif + + // Set response header TO DO + auto search = responseHeaders.find(headerLine); + if(search != responseHeaders.end()) { +#if defined(ESP32) + log_v("Set response header '%s: %s'", search->first, valuePtr); +#endif +// free(responseHeader.second); // !!!!!! NOT SURE IF THESE POINTERS MUST BE FREE-ED +// responseHeader.second = strdup(valuePtr); + } + } + } + + if(!_chunked && strcasecmp(headerLine, "Transfer-Encoding") == 0) { + for(size_t index = sizeof "Transfer-Encoding"; headerLine[index] != '\0'; index++) { + headerLine[index] = tolower(headerLine[index]); // Transfer-Encoding values should be compared case insensitively + } + _chunked = (strstr(headerLine + sizeof "Transfer-Encoding", "chunked") != NULL); +#if defined(ESP32) + log_v("Found Transfer-Encoding response header, chunked is %s", _chunked ? "true": "false"); +#endif + } + + if(!_chunked && _leftToRead == 0 && strcasecmp(headerLine, "content-length") == 0) { + _leftToRead = strtol(valuePtr, NULL, 10); +#if defined(ESP32) + log_v("Found 'Content-Length: %u' response header", _leftToRead); +#endif + } + + if(strcasecmp(headerLine, "location") == 0) { +#if defined(ESP32) + log_v("Found 'Location: %s' response header", valuePtr); +#endif + free(_location); + _location = strdup(valuePtr); + if(_location == NULL) { + Serial.println("(readResponseHeaders) Out of memory"); +#if defined(ESP32) + log_e("Out of memory"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Out of memory"); +#endif + + return OUT_OF_MEMORY; + } + } + } + + if(_chunked) { + char chunkHeader[7]; + size_t bytesRead = _client.readBytesUntil('\n', chunkHeader, sizeof chunkHeader); + if(bytesRead == 0) { +#if defined(ESP32) + log_e("Timeout reading chunk header"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Timeout reading chunk header"); +#endif + + return CHUNK_HEADER_TIMEOUT; // Time out reading chunk header + } + + if(chunkHeader[bytesRead - 1] != '\r') { +#if defined(ESP32) + log_e("Invalid chunk header"); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("Invalid chunk header"); +#endif + + return CHUNK_HEADER_INVALID; // Invalid chunk header + } + + _leftToRead = strtol(chunkHeader, NULL, 16); +#if defined(ESP32) + log_v("Chunk size %04x (%u)", _leftToRead, _leftToRead); +#endif + } + + _state = CLIENT_RESPONSE_PAYLOAD; + + return (http_code_t)status; +} + + +/* +const char * ClientHTTP::errorToString(http_code_t error) { + switch(error) { + case ClientHTTPOUT_OF_MEMORY: + return "Out of memory"; + break; + default: + return "No error string available"; + } +} + + +ClientHTTPhttp_code_t ClientHTTP::returnError(http_code_t error) { + if(error < 0) { +#if defined(ESP32) + log_e("Error %d: %s", error, errorToString(error)); +#endif +#if defined(ESP8266) + DEBUG_HTTP_MSG("(ClientHTTPreturnError) Error %d: %s\n", error, errorToString(error)); +#endif + + } + + return error; +} +*/ + + +size_t ClientHTTP::leftToRead() { + return _leftToRead; +} + + +size_t ClientHTTP::leftToWrite() { + return _leftToWrite; +} + + +void ClientHTTP::printTo(Print &printer) { + while(leftToRead()) { + uint8_t printBuffer[PRINT_BUFFER_SIZE]; + + size_t bytesToRead = available(); + if(bytesToRead > PRINT_BUFFER_SIZE) bytesToRead = PRINT_BUFFER_SIZE; + + size_t bytesRead = read(printBuffer, bytesToRead); + printer.write(printBuffer, bytesRead); + } +} diff --git a/libraries/ClientHTTP/src/ClientHTTP.h b/libraries/ClientHTTP/src/ClientHTTP.h new file mode 100644 index 00000000000..8dd20306c7e --- /dev/null +++ b/libraries/ClientHTTP/src/ClientHTTP.h @@ -0,0 +1,269 @@ +/* + * ClientHTTP.h - A HTTP Client class using the standard Client and Stream interface, + * while handling the request and response headers and chunked + * Transfer-Encoding 'under the hood' + * + * Copyright (C) 2020 Jeroen Döll + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +// TO DO +// map char * allocation and free. responseHeaders are not collected yet +// redirect to a different host +// Possibly add gzip stream + +#pragma once + +#include +#include +#include + +// If SUPPORT_REDIRECTS is not defined, HTTP requests are not automatically redirected if the server sends +// a 'Location: ' response header. On an ESP32 this saves about 700 bytes of code. Redirects can still be handled +// by the sketch if redirects are not supported by the client. +#define SUPPORT_REDIRECTS + +#if defined(ESP8266) +#ifdef DEBUG_ESP_PORT +#define DEBUG_HTTP_MSG(...) DEBUG_ESP_PORT.printf( __VA_ARGS__ ) +#else +#define DEBUG_HTTP_MSG(...) +#endif +#endif // defined(ESP8266) + +/* + * class Sizer is a helper class to determine the payload size + * Usage: + * - print all payload to a Sizer instance first + * - read the payload size with getSize() + * - use this size in httpClient::POST() + * - print the exact payload used in Sizer to the httpClient + * See examples for an example + */ +class Sizer: public Print { +public: + Sizer(): _size(0) {} + + virtual size_t write(uint8_t) { + _size++; + } + + virtual size_t write(const uint8_t *buf, size_t size) { + _size += size; + } + + void reset() { + _size = 0; + } + + size_t getSize() { + return _size; + } +private: + size_t _size; +}; + +// ::readResponseHeaders() uses a small buffer on the stack. A full header line must fit into this buffer. HTTP does not specifiy a maximum size for headers +#define RESPONSE_HEADER_LINE_SIZE (128) + +// ::prinTo() uses a small buffer on the stack +#define PRINT_BUFFER_SIZE (64) + +class ClientHTTP: public Client { +public: + ClientHTTP(Client &client); + + ~ClientHTTP(); + + // HTTP codes see RFC7231 + typedef enum { + HTTP_ERROR = -1, + REDIRECT_LIMIT_REACHED = -2, + REDIRECT_CONNECTION_FAILED = -3, + REDIRECT_UNKOWN_METHOD = -4, + REDIRECT_FAILURE = -5, + RESPONSE_HEADER_TIMEOUT = -6, + OUT_OF_MEMORY = -7, + CHUNK_HEADER_TIMEOUT = -8, + CHUNK_HEADER_INVALID = -9, + CONTINUE = 100, + SWITCHING_PROTOCOLS = 101, + PROCESSING = 102, + OK = 200, + CREATED = 201, + ACCEPTED = 202, + NON_AUTHORITATIVE_INFORMATION = 203, + NO_CONTENT = 204, + RESET_CONTENT = 205, + PARTIAL_CONTENT = 206, + MULTI_STATUS = 207, + ALREADY_REPORTED = 208, + IM_USED = 226, + MULTIPLE_CHOICES = 300, + MOVED_PERMANENTLY = 301, + FOUND = 302, + SEE_OTHER = 303, + NOT_MODIFIED = 304, + USE_PROXY = 305, + TEMPORARY_REDIRECT = 307, + PERMANENT_REDIRECT = 308, + BAD_REQUEST = 400, + UNAUTHORIZED = 401, + PAYMENT_REQUIRED = 402, + FORBIDDEN = 403, + NOT_FOUND = 404, + METHOD_NOT_ALLOWED = 405, + NOT_ACCEPTABLE = 406, + PROXY_AUTHENTICATION_REQUIRED = 407, + REQUEST_TIMEOUT = 408, + CONFLICT = 409, + GONE = 410, + LENGTH_REQUIRED = 411, + PRECONDITION_FAILED = 412, + PAYLOAD_TOO_LARGE = 413, + URI_TOO_LONG = 414, + UNSUPPORTED_MEDIA_TYPE = 415, + RANGE_NOT_SATISFIABLE = 416, + EXPECTATION_FAILED = 417, + MISDIRECTED_REQUEST = 421, + UNPROCESSABLE_ENTITY = 422, + LOCKED = 423, + FAILED_DEPENDENCY = 424, + UPGRADE_REQUIRED = 426, + PRECONDITION_REQUIRED = 428, + TOO_MANY_REQUESTS = 429, + REQUEST_HEADER_FIELDS_TOO_LARGE = 431, + INTERNAL_SERVER_ERROR = 500, + NOT_IMPLEMENTED = 501, + BAD_GATEWAY = 502, + SERVICE_UNAVAILABLE = 503, + GATEWAY_TIMEOUT = 504, + HTTP_VERSION_NOT_SUPPORTED = 505, + VARIANT_ALSO_NEGOTIATES = 506, + INSUFFICIENT_STORAGE = 507, + LOOP_DETECTED = 508, + NOT_EXTENDED = 510, + NETWORK_AUTHENTICATION_REQUIRED = 511 + } http_code_t; + + virtual int connect(IPAddress ip, uint16_t port); + + virtual int connect(const char * host, uint16_t port); + + virtual size_t write(uint8_t); + + virtual size_t write(const uint8_t *buf, size_t size); + + virtual int available(); + + virtual int read(); + + virtual int read(uint8_t *buf, size_t size); + + virtual int peek(); + + virtual void flush(); + + virtual void stop(); + + virtual uint8_t connected(); + + virtual operator bool(); + + // Handle redirections as stated in RFC document: + // https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html + typedef enum { + DISABLE, + STRICT, + FORCE + } follow_redirects_t; + follow_redirects_t redirects = ClientHTTP::STRICT; + + uint16_t redirectLimit = 10; + + bool GET(const char * uri); + + bool HEAD(const char * uri); + + bool POST(const char * uri, size_t contentLength = 0); + + bool PUT(const char * uri, size_t contentLength = 0); + + bool PATCH(const char * uri, size_t contentLength = 0); + + bool DELETE(const char * uri, size_t contentLength = 0); + + http_code_t status(); + + size_t leftToRead(); + + size_t leftToWrite(); + + void printTo(Print &printer); + + std::map requestHeaders; + + // Helper struct for case insensitive comparison of keys in the response headers map + struct cmp_str { + bool operator()(char const *a, char const *b) const { + return strcasecmp(a, b) < 0; + } + }; + std::map responseHeaders; + + typedef enum http_version_t { + HTTP_10, + HTTP_11 + }; + http_version_t httpVersion = HTTP_11; + +protected: + bool writeRequestHeaders(const char * method, const char * uri, size_t contentLength = 0); + + http_code_t readResponseHeaders(); + +/* + const char * errorToString(http_code_t error); + + http_code_t returnError(http_code_t error); +*/ + + Client & _client; + + IPAddress _ip; + char * _host; + uint16_t _port; + + // for GET and HEAD status changes from CLIENT_READY -> CLIENT_RESPONSE_HEADER -> CLIENT_RESPONSE_PAYLOAD -> CLIENT_READY + // for POST, PUT and PATCH status changes from CLIENT_READY -> CLIENT_REQUEST_HEADER_SENT -> CLIENT_RESPONSE_HEADER -> CLIENT_RESPONSE_PAYLOAD -> CLIENT_READY + typedef enum client_state_t { + CLIENT_READY, // No pending HTTP communication + CLIENT_REQUEST_HEADER_SENT, // Request header sent, ready to write payload that is to be send to the server via stream + CLIENT_RESPONSE_HEADER, // Ready to receive response header from server + CLIENT_RESPONSE_PAYLOAD, // Ready to read payload sent by server from stream + }; + client_state_t _state; + + bool _chunked; + size_t _leftToRead; + + size_t _leftToWrite; + bool _firstChunkToWrite; + + char * _location; + uint16_t _redirectCount; + char _originalMethod[16]; +}; From 02258373931561a429befe360b661ea0c1a80d36 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Wed, 25 Mar 2020 19:02:52 +0100 Subject: [PATCH 02/16] Add library to MakeLists.txt --- CMakeLists.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CMakeLists.txt b/CMakeLists.txt index cf8981f9aad..ae3912afa5c 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -48,6 +48,7 @@ set(LIBRARY_SRCS libraries/FS/src/FS.cpp libraries/FS/src/vfs_api.cpp libraries/HTTPClient/src/HTTPClient.cpp + libraries/ClientHTTP/src/ClientHTTP.cpp libraries/HTTPUpdate/src/HTTPUpdate.cpp libraries/NetBIOS/src/NetBIOS.cpp libraries/Preferences/src/Preferences.cpp @@ -187,6 +188,7 @@ set(COMPONENT_ADD_INCLUDEDIRS libraries/FFat/src libraries/FS/src libraries/HTTPClient/src + libraries/ClientHTTP/src libraries/HTTPUpdate/src libraries/NetBIOS/src libraries/Preferences/src From 2288d105bbc76a7943671e88cc7fb6de0c3fcb6e Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Wed, 25 Mar 2020 19:22:29 +0100 Subject: [PATCH 03/16] Add return values to class Sizer --- libraries/ClientHTTP/src/ClientHTTP.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/libraries/ClientHTTP/src/ClientHTTP.h b/libraries/ClientHTTP/src/ClientHTTP.h index 8dd20306c7e..b2329c6a42c 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.h +++ b/libraries/ClientHTTP/src/ClientHTTP.h @@ -59,10 +59,14 @@ class Sizer: public Print { virtual size_t write(uint8_t) { _size++; + + return 1; } virtual size_t write(const uint8_t *buf, size_t size) { _size += size; + + return size; } void reset() { @@ -224,10 +228,10 @@ class ClientHTTP: public Client { }; std::map responseHeaders; - typedef enum http_version_t { + typedef enum { HTTP_10, HTTP_11 - }; + } http_version_t; http_version_t httpVersion = HTTP_11; protected: @@ -249,12 +253,12 @@ class ClientHTTP: public Client { // for GET and HEAD status changes from CLIENT_READY -> CLIENT_RESPONSE_HEADER -> CLIENT_RESPONSE_PAYLOAD -> CLIENT_READY // for POST, PUT and PATCH status changes from CLIENT_READY -> CLIENT_REQUEST_HEADER_SENT -> CLIENT_RESPONSE_HEADER -> CLIENT_RESPONSE_PAYLOAD -> CLIENT_READY - typedef enum client_state_t { + typedef enum { CLIENT_READY, // No pending HTTP communication CLIENT_REQUEST_HEADER_SENT, // Request header sent, ready to write payload that is to be send to the server via stream CLIENT_RESPONSE_HEADER, // Ready to receive response header from server CLIENT_RESPONSE_PAYLOAD, // Ready to read payload sent by server from stream - }; + } client_state_t; client_state_t _state; bool _chunked; From 65d3032b23719deabc2d7ed91e58a17b45d74f1b Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Wed, 25 Mar 2020 21:36:18 +0100 Subject: [PATCH 04/16] Change the map key/value pair to --- libraries/ClientHTTP/README.md | 76 ++++++------------- .../examples/Authorization/Authorization.ino | 2 +- .../POSTJsonClientHTTPS.ino | 2 + .../ReuseConnectionClientHTTPS.ino | 3 +- libraries/ClientHTTP/src/ClientHTTP.cpp | 11 ++- libraries/ClientHTTP/src/ClientHTTP.h | 14 ++-- 6 files changed, 45 insertions(+), 63 deletions(-) diff --git a/libraries/ClientHTTP/README.md b/libraries/ClientHTTP/README.md index a1d68703952..c45d9d1f28c 100644 --- a/libraries/ClientHTTP/README.md +++ b/libraries/ClientHTTP/README.md @@ -1,62 +1,39 @@ # ClientHTTP -The ClientHTTP class implements support for REST API client calls to a HTTP server. -It inherits from Client and thus implements a superset of that class' interface. - -The ClientHTTP class decorates the underlying stream with HTTP request headers and -reads the HTTP response headers from the stream before the stream is made available to -the program. Also chunked responses are handled 'under the hood' offering a clean stream -to which a request payload (if any) can be written using the standard Print::write() -functions and from which response payloads (if any) can be read using the standard -Stream::read() functions. - -The ClientHTTP class has a smaller memory footprint than the alternative HTTPClient class. -Moreover: because of the Stream oriented nature of ClientHTTP no memory buffers are -needed, neither for sending payload, nor for receiving payload. It still remains possible -to use such memory buffers though. Chunked response from the stream is correctly implemented. -No Strings are used to prevent heap fragmentation. Only small stack buffers are used. Two -exceptions are: the host name is copied into a heap char array, and the requestHeaders and -responseHeaders are dynamic standard containers (map). - -ClientHTTP is able to handle redirections, but support for this may be omitted by clearing -the #define SUPPORT_REDIRECTS in the header file, thus saving some programm memory bytes. +The ClientHTTP class implements support for REST API client calls to a HTTP server. It inherits from Client and thus implements a superset of that class' interface. + +The ClientHTTP class decorates the underlying stream with HTTP request headers and reads the HTTP response headers from the stream before the stream is made available to the program. Also chunked responses are handled 'under the hood' offering a clean stream to which a request payload (if any) can be written using the standard Print::write() functions and from which response payloads (if any) can be read using the standard Stream::read() functions. + +The ClientHTTP class has a smaller memory footprint than the alternative HTTPClient class. Moreover: because of the Stream oriented nature of ClientHTTP no memory buffers are needed, neither for sending payload, nor for receiving payload. It still remains possible to use such memory buffers though. Chunked response from the stream is correctly implemented. +No Strings are used to prevent heap fragmentation. Only small stack buffers are used. Two exceptions are: the host name is copied into a heap char array, and the requestHeaders and responseHeaders are dynamic standard containers (map) with `std::string` key and `std::string` value. + +ClientHTTP is able to handle redirections. Support for this may be omitted by clearing the `#define SUPPORT_REDIRECTS` in the header file, thus saving some program memory bytes. Request headers can easily be set by ClientHTTP. Response headers can be easily collected by ClientHTTP. -Any client may be used, e.g. WiFiClient, WiFiClientSecure and EthernetClient. +Any client may be used, e.g. `WiFiClient`, `WiFiClientSecure` and `EthernetClient`. -A HTTP GET, HEAD, POST, PUT, PATCH or DELETE always follows the same skeleton. +A HTTP `GET`, `HEAD`, `POST`, `PUT`, `PATCH` or `DELETE` always follows the same skeleton. -Integration with ArduinoJson, version 5 and 6, both for sending and receiving Json payloads -is very intuitive. +Integration with ArduinoJson, version 5 and 6, both for sending and receiving Json payloads is very intuitive. The same library is defined for both the ESP8266 and the ESP32. The following examples are available. All but one are HTTPS / TLS 1.2 examples: - Authorization: how to use basic authorization -- BasicClientHTTP: how to GET a payload from a HTTP server and print it to Serial -- BasicClientHTTPS: how to GET a paylaod from a HTTPS server and print it to Serial -- DataClientHTTPS: how to GET a payload from a HTTPS server and handle the data using - StreamString or a buffer -- POSTClientHTTPS: how to POST a payload to a HTTPS server and print the response to - Serial -- POSTJsonClientHTTPS: how to POST an ArduinoJson to a HTTPS server and deserialize the - response into another ArduinJson without buffers. All data is buffered into the - ArduinoJson only -- POSTSizerClientHTTPS: how to determine the size of a payload by printing to a utility - class Sizer first. Next the resulting length is used to set the Content-Length request - header by calling POST -- ReuseConnectionHTTPS: how to reuse a connection, including correct handling of the - Connection: close response header. This example DOES NOT WORK on the ESP8266 YET! +- BasicClientHTTP: how to `GET` a payload from a HTTP server and print it to `Serial` +- BasicClientHTTPS: how to `GET` a paylaod from a HTTPS server and print it to `Serial` +- DataClientHTTPS: how to `GET` a payload from a HTTPS server and handle the data using `StreamString` or a buffer +- POSTClientHTTPS: how to `POST` a payload to a HTTPS server and print the response to `Serial` +- POSTJsonClientHTTPS: how to `POST` an ArduinoJson to a HTTPS server and deserialize the response into another ArduinJson without buffers. All data is buffered into the ArduinoJson only +- POSTSizerClientHTTPS: how to determine the size of a payload by printing to a utility `class Sizer` first. Next the resulting length is used to set the Content-Length request header by calling `POST` +- ReuseConnectionHTTPS: how to reuse a connection, including correct handling of the Connection: close response header. This example DOES NOT WORK on the ESP8266 YET! Still to be done: - Support for redirection to a different host -- Fixing the response header handling. I think an Allocator needs to be defined for this - but I could use some help on this one :) ## Request and response headers -Request headers can be set by just defining them before calling the REST method (`GET`, -`POST`, etc): +Request headers can be set by just defining them before calling the REST method (`GET`, `POST`, etc): ```cpp http.requestHeaders["Content-Type"] = "application/json"; http.requestHeaders["Connection"] = "close"; @@ -65,8 +42,7 @@ Response headers can be collected by just defining them before `::status()` is c ```cpp http.responseHeaders["Content-Type"]; ``` -After the `::status()` command `responseHeaders['Content-Type']` is either `NULL` if the server -did not send the response header, or it is populated with the value sent. +After the `::status()` command `responseHeaders['Content-Type']` is either an empty string if the server did not send the response header, or it is populated with the value sent. ## Skeleton program A HTTP REST API call always follows the following skeleton @@ -77,13 +53,10 @@ A HTTP REST API call always follows the following skeleton - `htpp.connect(host, port)` - Call the REST API method, e.g. `http.POST(payload, payloadLength)` - Send the payload, if any, to the server using standard `Print::write()` commands -- Call `http.status()` which reads the response headers from the stream and returns - the HTTP response code (greater than 0) or an error code (less than 0) +- Call `http.status()` which reads the response headers from the stream and returns the HTTP response code (greater than 0) or an error code (less than 0) - Check any response headers, if needed - Read the payload sent by the server, if any, using standard `Stream::read()` commands -- Close the connection calling `http.stop()`, or reuse the connection if the server - did not sent a `Connection: close` response header, nor closed it's side of the - connection +- Close the connection calling `http.stop()`, or reuse the connection if the server did not sent a `Connection: close` response header, nor closed it's side of the connection ## ArduinoJson integration ArduinoJson documents, both version 5 and version 6, can directly be POSTed to a REST server: @@ -96,8 +69,7 @@ ArduinoJson documents, both version 5 and version 6, can directly be POSTed to a serializeJson(requestDocument, http); ... ``` -Also ArduinoJson documents, both version 5 and 6, can directly be deserialized from the -response: +Also ArduinoJson documents, both version 5 and 6, can directly be deserialized from the response: ```cpp ... http.responseHeaders["Content-Type"]; @@ -107,6 +79,6 @@ response: deserializeJson(responseDocument, http); } ``` - +See the examples for how to use ArduinoJson 5. diff --git a/libraries/ClientHTTP/examples/Authorization/Authorization.ino b/libraries/ClientHTTP/examples/Authorization/Authorization.ino index 741934e19dd..5f6d9ec071f 100644 --- a/libraries/ClientHTTP/examples/Authorization/Authorization.ino +++ b/libraries/ClientHTTP/examples/Authorization/Authorization.ino @@ -121,7 +121,7 @@ void setup() { String credentials = base64::encode((const uint8_t *)"guest:guest", sizeof "guest:guest" - 1); String auth = String("Basic ") + credentials; - http.requestHeaders["Authorization"] = (char *)auth.c_str(); + http.requestHeaders["Authorization"] = auth.c_str(); http.requestHeaders["Connection"] = "close"; if (!http.connect("jigsaw.w3.org", 443)) { diff --git a/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino b/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino index dd8aa0e1ff3..aad3d92ff4f 100644 --- a/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino +++ b/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino @@ -168,6 +168,8 @@ void setup() { Serial.println("Payload succesfully send!"); } + Serial.printf("Response header \"Content-Type: %s\"\n", http.responseHeaders["Content-Type"].c_str()); + const size_t responseCapacity = 2*JSON_ARRAY_SIZE(2) + 3*JSON_OBJECT_SIZE(0) + 2*JSON_OBJECT_SIZE(3) + JSON_OBJECT_SIZE(6) + JSON_OBJECT_SIZE(7) + 618; #if ARDUINOJSON_VERSION_MAJOR == 6 DynamicJsonDocument responseDocument(responseCapacity); diff --git a/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino index a8a981c49d8..ff45faedcb9 100644 --- a/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino +++ b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino @@ -162,7 +162,8 @@ void loop() { Serial.printf("Request failed with error code %d\n", status); } - if(http.responseHeaders["Connection"] != NULL && strcasecmp(http.responseHeaders["Connection"], "close") == 0) { + Serial.printf("Response header \"Connection: %s\"\n", http.responseHeaders["Connection"].c_str()); + if(strcasecmp(http.responseHeaders["Connection"].c_str(), "close") == 0) { Serial.println("Closing connection because the server requested to do so"); http.stop(); } else { diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index c2e17848677..47b191c4dc3 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -400,9 +400,11 @@ bool ClientHTTP::writeRequestHeaders(const char * method, const c } for(auto requestHeader: requestHeaders) { // std::pair - _client.printf("%s: %s\r\n", requestHeader.first, requestHeader.second); +// _client.printf("%s: %s\r\n", requestHeader.first, requestHeader.second); + _client.printf("%s: %s\r\n", requestHeader.first.c_str(), requestHeader.second.c_str()); #if defined(ESP32) - log_v("Request header '%s: %s'", requestHeader.first, requestHeader.second); +// log_v("Request header '%s: %s'", requestHeader.first, requestHeader.second); + log_v("Request header '%s: %s'", requestHeader.first.c_str(), requestHeader.second.c_str()); #endif } @@ -605,8 +607,8 @@ ClientHTTP::http_code_t ClientHTTP::status() { ClientHTTP::http_code_t ClientHTTP::readResponseHeaders() { - char headerLine[RESPONSE_HEADER_LINE_SIZE]; + size_t bytesRead = _client.readBytesUntil('\n', headerLine, sizeof headerLine - 1); headerLine[bytesRead] = '\0'; char * statusPtr = strchr(headerLine, ' '); @@ -651,11 +653,12 @@ ClientHTTP::http_code_t ClientHTTP::readResponseHeaders() { log_v("Read response header '%s: %s'", headerLine, valuePtr); #endif - // Set response header TO DO + // Set response header auto search = responseHeaders.find(headerLine); if(search != responseHeaders.end()) { #if defined(ESP32) log_v("Set response header '%s: %s'", search->first, valuePtr); + search->second = valuePtr; #endif // free(responseHeader.second); // !!!!!! NOT SURE IF THESE POINTERS MUST BE FREE-ED // responseHeader.second = strdup(valuePtr); diff --git a/libraries/ClientHTTP/src/ClientHTTP.h b/libraries/ClientHTTP/src/ClientHTTP.h index b2329c6a42c..9f2b238cbc0 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.h +++ b/libraries/ClientHTTP/src/ClientHTTP.h @@ -21,7 +21,6 @@ */ // TO DO -// map char * allocation and free. responseHeaders are not collected yet // redirect to a different host // Possibly add gzip stream @@ -218,15 +217,20 @@ class ClientHTTP: public Client { void printTo(Print &printer); - std::map requestHeaders; +// std::map requestHeaders; + std::map requestHeaders; // Helper struct for case insensitive comparison of keys in the response headers map struct cmp_str { - bool operator()(char const *a, char const *b) const { - return strcasecmp(a, b) < 0; +// bool operator()(char const *a, char const *b) const { +// return strcasecmp(a, b) < 0; +// } + bool operator()(const std::string & a, const std::string & b) const { + return strcasecmp(a.c_str(), b.c_str()) < 0; } }; - std::map responseHeaders; +// std::map responseHeaders; + std::map responseHeaders; typedef enum { HTTP_10, From 362765e2cda8585c903f79818c1cb1c4a68678ec Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Wed, 25 Mar 2020 22:52:34 +0100 Subject: [PATCH 05/16] Removed #include from POSTJsonClientHTTPS.ino example, because ArduinoJson is not part of the standard libraries --- .../POSTJsonClientHTTPS.ino | 21 ++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino b/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino index aad3d92ff4f..c8217b2a7cd 100644 --- a/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino +++ b/libraries/ClientHTTP/examples/POSTJsonClientHTTPS/POSTJsonClientHTTPS.ino @@ -13,6 +13,9 @@ * This example is in the public domain */ +// BECAUSE ArduinoJson IS NO PART OF THE STANDARD ESP LIBRARIES the #include is COMMENTED OUT +// Remove the comment to use this example! + #if defined(ESP8266) #include #endif @@ -21,7 +24,7 @@ #endif #include -#include +//#include #include @@ -71,6 +74,11 @@ void setup() { Serial.begin(115200); Serial.println(); +#if !defined(ARDUINOJSON_VERSION_MAJOR) + Serial.println("Remove comment from #include to test this example! "); + return; +#endif + Serial.printf("Connecting to %s", ssid); WiFi.mode(WIFI_STA); WiFi.begin(ssid, password); @@ -125,8 +133,9 @@ void setup() { // If the Content-Type: application/json is used, like in this example, data should be a serialized json! const char json[] = "{\"sensor\":\"gps\",\"time\":1351824120,\"data\":[48.756080,2.302038]}"; - const size_t requestCapacity = JSON_OBJECT_SIZE(3) + JSON_ARRAY_SIZE(2) + 60; + size_t payloadLength = 0; #if ARDUINOJSON_VERSION_MAJOR == 6 + const size_t requestCapacity = JSON_OBJECT_SIZE(3) + JSON_ARRAY_SIZE(2) + 60; DynamicJsonDocument requestDocument(requestCapacity); DeserializationError error = deserializeJson(requestDocument, json); // Test if parsing succeeds. @@ -135,9 +144,10 @@ void setup() { Serial.println(error.c_str()); return; } - size_t payloadLength = measureJson(requestDocument); + payloadLength = measureJson(requestDocument); #endif #if ARDUINOJSON_VERSION_MAJOR == 5 + const size_t requestCapacity = JSON_OBJECT_SIZE(3) + JSON_ARRAY_SIZE(2) + 60; DynamicJsonBuffer jsonBuffer(requestCapacity); JsonObject& root = jsonBuffer.parseObject(json); // Test if parsing succeeds. @@ -145,7 +155,7 @@ void setup() { Serial.println("parseObject() failed"); return; } - size_t payloadLength = root.measureLength(); + payloadLength = root.measureLength(); #endif if(!http.POST("/post", payloadLength)) { @@ -170,8 +180,8 @@ void setup() { Serial.printf("Response header \"Content-Type: %s\"\n", http.responseHeaders["Content-Type"].c_str()); - const size_t responseCapacity = 2*JSON_ARRAY_SIZE(2) + 3*JSON_OBJECT_SIZE(0) + 2*JSON_OBJECT_SIZE(3) + JSON_OBJECT_SIZE(6) + JSON_OBJECT_SIZE(7) + 618; #if ARDUINOJSON_VERSION_MAJOR == 6 + const size_t responseCapacity = 2*JSON_ARRAY_SIZE(2) + 3*JSON_OBJECT_SIZE(0) + 2*JSON_OBJECT_SIZE(3) + JSON_OBJECT_SIZE(6) + JSON_OBJECT_SIZE(7) + 618; DynamicJsonDocument responseDocument(responseCapacity); DeserializationError error = deserializeJson(responseDocument, http); // Test if parsing succeeds. @@ -184,6 +194,7 @@ void setup() { serializeJsonPretty(responseDocument, Serial); #endif #if ARDUINOJSON_VERSION_MAJOR == 5 + const size_t responseCapacity = 2*JSON_ARRAY_SIZE(2) + 3*JSON_OBJECT_SIZE(0) + 2*JSON_OBJECT_SIZE(3) + JSON_OBJECT_SIZE(6) + JSON_OBJECT_SIZE(7) + 618; DynamicJsonBuffer jsonBuffer(responseCapacity); JsonObject& rootReceived = jsonBuffer.parseObject(http); // Test if parsing succeeds. From 9eea2f136fc52c808ea74d5a5bb90797fc8267ce Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Fri, 27 Mar 2020 08:15:34 +0100 Subject: [PATCH 06/16] Added statusAsync(), the idea is: make the MCU available while waiting for the server response. Not tested yet --- libraries/ClientHTTP/src/ClientHTTP.cpp | 13 +++++++++++++ libraries/ClientHTTP/src/ClientHTTP.h | 3 +++ 2 files changed, 16 insertions(+) diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index 47b191c4dc3..e5bec60f3cb 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -606,6 +606,19 @@ ClientHTTP::http_code_t ClientHTTP::status() { } +ClientHTTP::http_code_t ClientHTTP::statusAsync() { + if(_state != CLIENT_RESPONSE_HEADER) { + return HTTP_ERROR; + } + + if(!available()) { + return ASYNC_RESPONSE_HEADERS; + } + + return status(); +} + + ClientHTTP::http_code_t ClientHTTP::readResponseHeaders() { char headerLine[RESPONSE_HEADER_LINE_SIZE]; diff --git a/libraries/ClientHTTP/src/ClientHTTP.h b/libraries/ClientHTTP/src/ClientHTTP.h index 9f2b238cbc0..2ce7885eb44 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.h +++ b/libraries/ClientHTTP/src/ClientHTTP.h @@ -93,6 +93,7 @@ class ClientHTTP: public Client { // HTTP codes see RFC7231 typedef enum { + ASYNC_RESPONSE_HEADERS = 0, HTTP_ERROR = -1, REDIRECT_LIMIT_REACHED = -2, REDIRECT_CONNECTION_FAILED = -3, @@ -211,6 +212,8 @@ class ClientHTTP: public Client { http_code_t status(); + http_code_t statusAsync(); + size_t leftToRead(); size_t leftToWrite(); From 9f97294c7fb3a81d26a0ea17dea7d62a53a69269 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Fri, 27 Mar 2020 10:23:49 +0100 Subject: [PATCH 07/16] Changed from void to size_t printTo(Print &printer) and return the number of bytes written to the printer --- libraries/ClientHTTP/src/ClientHTTP.cpp | 17 +++++++++++++++-- libraries/ClientHTTP/src/ClientHTTP.h | 2 +- 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index e5bec60f3cb..489ee331729 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -791,7 +791,10 @@ size_t ClientHTTP::leftToWrite() { } -void ClientHTTP::printTo(Print &printer) { +size_t ClientHTTP::printTo(Print &printer) { + size_t bytesWritten = 0; + size_t bytesToWrite = leftToRead(); + while(leftToRead()) { uint8_t printBuffer[PRINT_BUFFER_SIZE]; @@ -799,6 +802,16 @@ void ClientHTTP::printTo(Print &printer) { if(bytesToRead > PRINT_BUFFER_SIZE) bytesToRead = PRINT_BUFFER_SIZE; size_t bytesRead = read(printBuffer, bytesToRead); - printer.write(printBuffer, bytesRead); + + if(bytesRead == 0) { // timeout +#if defined(ESP32) + log_v("Timeout, %u bytes of %u bytes written", bytesWritten, bytesToWrite); +#endif + break; + } + + bytesWritten += printer.write(printBuffer, bytesRead); } + + return bytesWritten; } diff --git a/libraries/ClientHTTP/src/ClientHTTP.h b/libraries/ClientHTTP/src/ClientHTTP.h index 2ce7885eb44..d10c8595cc0 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.h +++ b/libraries/ClientHTTP/src/ClientHTTP.h @@ -218,7 +218,7 @@ class ClientHTTP: public Client { size_t leftToWrite(); - void printTo(Print &printer); + size_t printTo(Print &printer); // std::map requestHeaders; std::map requestHeaders; From 0ded84422e85744ec24a07e997a8011811fcf206 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Fri, 27 Mar 2020 10:34:17 +0100 Subject: [PATCH 08/16] Add missing menu upload speed options for 4 boards to pass checks --- boards.txt | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/boards.txt b/boards.txt index 2a524dfc5fd..ee9b9299b77 100644 --- a/boards.txt +++ b/boards.txt @@ -2025,8 +2025,20 @@ esp32-evb.menu.FlashFreq.40=40MHz esp32-evb.menu.FlashFreq.40.build.flash_freq=40m +esp32-evb.menu.UploadSpeed.921600=921600 +esp32-evb.menu.UploadSpeed.921600.upload.speed=921600 esp32-evb.menu.UploadSpeed.115200=115200 esp32-evb.menu.UploadSpeed.115200.upload.speed=115200 +esp32-evb.menu.UploadSpeed.256000.windows=256000 +esp32-evb.menu.UploadSpeed.256000.upload.speed=256000 +esp32-evb.menu.UploadSpeed.230400.windows.upload.speed=256000 +esp32-evb.menu.UploadSpeed.230400=230400 +esp32-evb.menu.UploadSpeed.230400.upload.speed=230400 +esp32-evb.menu.UploadSpeed.460800.linux=460800 +esp32-evb.menu.UploadSpeed.460800.macosx=460800 +esp32-evb.menu.UploadSpeed.460800.upload.speed=460800 +esp32-evb.menu.UploadSpeed.512000.windows=512000 +esp32-evb.menu.UploadSpeed.512000.upload.speed=512000 esp32-evb.menu.PartitionScheme.default=Default esp32-evb.menu.PartitionScheme.default.build.partitions=default @@ -2073,8 +2085,20 @@ esp32-gateway.menu.FlashFreq.40=40MHz esp32-gateway.menu.FlashFreq.40.build.flash_freq=40m +esp32-gateway.menu.UploadSpeed.921600=921600 +esp32-gateway.menu.UploadSpeed.921600.upload.speed=921600 esp32-gateway.menu.UploadSpeed.115200=115200 esp32-gateway.menu.UploadSpeed.115200.upload.speed=115200 +esp32-gateway.menu.UploadSpeed.256000.windows=256000 +esp32-gateway.menu.UploadSpeed.256000.upload.speed=256000 +esp32-gateway.menu.UploadSpeed.230400.windows.upload.speed=256000 +esp32-gateway.menu.UploadSpeed.230400=230400 +esp32-gateway.menu.UploadSpeed.230400.upload.speed=230400 +esp32-gateway.menu.UploadSpeed.460800.linux=460800 +esp32-gateway.menu.UploadSpeed.460800.macosx=460800 +esp32-gateway.menu.UploadSpeed.460800.upload.speed=460800 +esp32-gateway.menu.UploadSpeed.512000.windows=512000 +esp32-gateway.menu.UploadSpeed.512000.upload.speed=512000 esp32-gateway.menu.PartitionScheme.default=Default esp32-gateway.menu.PartitionScheme.default.build.partitions=default @@ -2115,8 +2139,20 @@ esp32-poe.menu.FlashFreq.40=40MHz esp32-poe.menu.FlashFreq.40.build.flash_freq=40m +esp32-poe.menu.UploadSpeed.921600=921600 +esp32-poe.menu.UploadSpeed.921600.upload.speed=921600 esp32-poe.menu.UploadSpeed.115200=115200 esp32-poe.menu.UploadSpeed.115200.upload.speed=115200 +esp32-poe.menu.UploadSpeed.256000.windows=256000 +esp32-poe.menu.UploadSpeed.256000.upload.speed=256000 +esp32-poe.menu.UploadSpeed.230400.windows.upload.speed=256000 +esp32-poe.menu.UploadSpeed.230400=230400 +esp32-poe.menu.UploadSpeed.230400.upload.speed=230400 +esp32-poe.menu.UploadSpeed.460800.linux=460800 +esp32-poe.menu.UploadSpeed.460800.macosx=460800 +esp32-poe.menu.UploadSpeed.460800.upload.speed=460800 +esp32-poe.menu.UploadSpeed.512000.windows=512000 +esp32-poe.menu.UploadSpeed.512000.upload.speed=512000 esp32-poe.menu.PartitionScheme.default=Default esp32-poe.menu.PartitionScheme.default.build.partitions=default @@ -2157,8 +2193,20 @@ esp32-poe-iso.menu.FlashFreq.40=40MHz esp32-poe-iso.menu.FlashFreq.40.build.flash_freq=40m +esp32-poe-iso.menu.UploadSpeed.921600=921600 +esp32-poe-iso.menu.UploadSpeed.921600.upload.speed=921600 esp32-poe-iso.menu.UploadSpeed.115200=115200 esp32-poe-iso.menu.UploadSpeed.115200.upload.speed=115200 +esp32-poe-iso.menu.UploadSpeed.256000.windows=256000 +esp32-poe-iso.menu.UploadSpeed.256000.upload.speed=256000 +esp32-poe-iso.menu.UploadSpeed.230400.windows.upload.speed=256000 +esp32-poe-iso.menu.UploadSpeed.230400=230400 +esp32-poe-iso.menu.UploadSpeed.230400.upload.speed=230400 +esp32-poe-iso.menu.UploadSpeed.460800.linux=460800 +esp32-poe-iso.menu.UploadSpeed.460800.macosx=460800 +esp32-poe-iso.menu.UploadSpeed.460800.upload.speed=460800 +esp32-poe-iso.menu.UploadSpeed.512000.windows=512000 +esp32-poe-iso.menu.UploadSpeed.512000.upload.speed=512000 esp32-poe-iso.menu.PartitionScheme.default=Default esp32-poe-iso.menu.PartitionScheme.default.build.partitions=default From c5e0f30af6047c7390977c4556da384c8357f0ed Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Fri, 27 Mar 2020 10:42:33 +0100 Subject: [PATCH 09/16] Revert to original boards.txt because this did not solve the check issue --- boards.txt | 48 ------------------------------------------------ 1 file changed, 48 deletions(-) diff --git a/boards.txt b/boards.txt index ee9b9299b77..2a524dfc5fd 100644 --- a/boards.txt +++ b/boards.txt @@ -2025,20 +2025,8 @@ esp32-evb.menu.FlashFreq.40=40MHz esp32-evb.menu.FlashFreq.40.build.flash_freq=40m -esp32-evb.menu.UploadSpeed.921600=921600 -esp32-evb.menu.UploadSpeed.921600.upload.speed=921600 esp32-evb.menu.UploadSpeed.115200=115200 esp32-evb.menu.UploadSpeed.115200.upload.speed=115200 -esp32-evb.menu.UploadSpeed.256000.windows=256000 -esp32-evb.menu.UploadSpeed.256000.upload.speed=256000 -esp32-evb.menu.UploadSpeed.230400.windows.upload.speed=256000 -esp32-evb.menu.UploadSpeed.230400=230400 -esp32-evb.menu.UploadSpeed.230400.upload.speed=230400 -esp32-evb.menu.UploadSpeed.460800.linux=460800 -esp32-evb.menu.UploadSpeed.460800.macosx=460800 -esp32-evb.menu.UploadSpeed.460800.upload.speed=460800 -esp32-evb.menu.UploadSpeed.512000.windows=512000 -esp32-evb.menu.UploadSpeed.512000.upload.speed=512000 esp32-evb.menu.PartitionScheme.default=Default esp32-evb.menu.PartitionScheme.default.build.partitions=default @@ -2085,20 +2073,8 @@ esp32-gateway.menu.FlashFreq.40=40MHz esp32-gateway.menu.FlashFreq.40.build.flash_freq=40m -esp32-gateway.menu.UploadSpeed.921600=921600 -esp32-gateway.menu.UploadSpeed.921600.upload.speed=921600 esp32-gateway.menu.UploadSpeed.115200=115200 esp32-gateway.menu.UploadSpeed.115200.upload.speed=115200 -esp32-gateway.menu.UploadSpeed.256000.windows=256000 -esp32-gateway.menu.UploadSpeed.256000.upload.speed=256000 -esp32-gateway.menu.UploadSpeed.230400.windows.upload.speed=256000 -esp32-gateway.menu.UploadSpeed.230400=230400 -esp32-gateway.menu.UploadSpeed.230400.upload.speed=230400 -esp32-gateway.menu.UploadSpeed.460800.linux=460800 -esp32-gateway.menu.UploadSpeed.460800.macosx=460800 -esp32-gateway.menu.UploadSpeed.460800.upload.speed=460800 -esp32-gateway.menu.UploadSpeed.512000.windows=512000 -esp32-gateway.menu.UploadSpeed.512000.upload.speed=512000 esp32-gateway.menu.PartitionScheme.default=Default esp32-gateway.menu.PartitionScheme.default.build.partitions=default @@ -2139,20 +2115,8 @@ esp32-poe.menu.FlashFreq.40=40MHz esp32-poe.menu.FlashFreq.40.build.flash_freq=40m -esp32-poe.menu.UploadSpeed.921600=921600 -esp32-poe.menu.UploadSpeed.921600.upload.speed=921600 esp32-poe.menu.UploadSpeed.115200=115200 esp32-poe.menu.UploadSpeed.115200.upload.speed=115200 -esp32-poe.menu.UploadSpeed.256000.windows=256000 -esp32-poe.menu.UploadSpeed.256000.upload.speed=256000 -esp32-poe.menu.UploadSpeed.230400.windows.upload.speed=256000 -esp32-poe.menu.UploadSpeed.230400=230400 -esp32-poe.menu.UploadSpeed.230400.upload.speed=230400 -esp32-poe.menu.UploadSpeed.460800.linux=460800 -esp32-poe.menu.UploadSpeed.460800.macosx=460800 -esp32-poe.menu.UploadSpeed.460800.upload.speed=460800 -esp32-poe.menu.UploadSpeed.512000.windows=512000 -esp32-poe.menu.UploadSpeed.512000.upload.speed=512000 esp32-poe.menu.PartitionScheme.default=Default esp32-poe.menu.PartitionScheme.default.build.partitions=default @@ -2193,20 +2157,8 @@ esp32-poe-iso.menu.FlashFreq.40=40MHz esp32-poe-iso.menu.FlashFreq.40.build.flash_freq=40m -esp32-poe-iso.menu.UploadSpeed.921600=921600 -esp32-poe-iso.menu.UploadSpeed.921600.upload.speed=921600 esp32-poe-iso.menu.UploadSpeed.115200=115200 esp32-poe-iso.menu.UploadSpeed.115200.upload.speed=115200 -esp32-poe-iso.menu.UploadSpeed.256000.windows=256000 -esp32-poe-iso.menu.UploadSpeed.256000.upload.speed=256000 -esp32-poe-iso.menu.UploadSpeed.230400.windows.upload.speed=256000 -esp32-poe-iso.menu.UploadSpeed.230400=230400 -esp32-poe-iso.menu.UploadSpeed.230400.upload.speed=230400 -esp32-poe-iso.menu.UploadSpeed.460800.linux=460800 -esp32-poe-iso.menu.UploadSpeed.460800.macosx=460800 -esp32-poe-iso.menu.UploadSpeed.460800.upload.speed=460800 -esp32-poe-iso.menu.UploadSpeed.512000.windows=512000 -esp32-poe-iso.menu.UploadSpeed.512000.upload.speed=512000 esp32-poe-iso.menu.PartitionScheme.default=Default esp32-poe-iso.menu.PartitionScheme.default.build.partitions=default From 681d143a8bac4dad2f2f84735c7bf1962fbb93ba Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Fri, 27 Mar 2020 15:16:58 +0100 Subject: [PATCH 10/16] Add setTimeout() function --- libraries/ClientHTTP/src/ClientHTTP.cpp | 6 ++++++ libraries/ClientHTTP/src/ClientHTTP.h | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index 489ee331729..57aa2c4a379 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -91,6 +91,12 @@ int ClientHTTP::connect(const char * host, uint16_t port) { return _client.connect(host, port); } + +void ClientHTTP::setTimeout(unsigned long timeout_ms) { + _timeout = timeout_ms; +} + + size_t ClientHTTP::write(uint8_t c) { if(_state != CLIENT_REQUEST_HEADER_SENT) { #if defined(ESP32) diff --git a/libraries/ClientHTTP/src/ClientHTTP.h b/libraries/ClientHTTP/src/ClientHTTP.h index d10c8595cc0..240e8567dc3 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.h +++ b/libraries/ClientHTTP/src/ClientHTTP.h @@ -167,6 +167,11 @@ class ClientHTTP: public Client { virtual int connect(const char * host, uint16_t port); + // setTimeout() should be called after connect(), because some Clients update this value during connection + // The value is in microseconds and is passed directly into the _timeout value of class Stream because some + // Client implementations use seconds, which is IMHO incorrect! + virtual void setTimeout(unsigned long timeout_ms = 1000); + virtual size_t write(uint8_t); virtual size_t write(const uint8_t *buf, size_t size); From a8e083cb0936ea697863600180a80f92943d0b48 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Sat, 28 Mar 2020 12:52:50 +0100 Subject: [PATCH 11/16] Changed install-platformio-esp32.sh as advised by Atanisoft to pass checks --- .github/scripts/install-platformio-esp32.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/scripts/install-platformio-esp32.sh b/.github/scripts/install-platformio-esp32.sh index ce6de4a2ef4..a8af4f71358 100755 --- a/.github/scripts/install-platformio-esp32.sh +++ b/.github/scripts/install-platformio-esp32.sh @@ -6,10 +6,10 @@ echo "Installing Python Wheel ..." pip install wheel > /dev/null 2>&1 echo "Installing PlatformIO ..." -pip install -U https://github.com/platformio/platformio/archive/develop.zip > /dev/null 2>&1 +pip install -U https://github.com/platformio/platformio/archive/develop.zip echo "Installing Platform ESP32 ..." -python -m platformio platform install https://github.com/platformio/platform-espressif32.git#feature/stage > /dev/null 2>&1 +python -m platformio platform install espressif32 echo "Replacing the framework version ..." if [[ "$OSTYPE" == "darwin"* ]]; then From fa94bc74ce6ed4633384bab682671b6a3304658d Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Sun, 29 Mar 2020 16:29:17 +0200 Subject: [PATCH 12/16] Restore _state to CLIENT_READY in ::stop(). ::stop() should be called if, on a reused connection, ::status() returns a negative value indicating a failure. The example ReuseConnectionClientHTTPS.ino is adapted accordingly --- .../ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino | 1 + libraries/ClientHTTP/src/ClientHTTP.cpp | 2 ++ 2 files changed, 3 insertions(+) diff --git a/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino index ff45faedcb9..3641b2c560c 100644 --- a/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino +++ b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino @@ -160,6 +160,7 @@ void loop() { Serial.println(); } else { Serial.printf("Request failed with error code %d\n", status); + http.stop(); } Serial.printf("Response header \"Connection: %s\"\n", http.responseHeaders["Connection"].c_str()); diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index 57aa2c4a379..b78903005e5 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -314,6 +314,8 @@ void ClientHTTP::stop() { #endif _client.stop(); + + _state = CLIENT_READY; } From f6b0143876b91760a1cf3f80342889e58c4f3ca0 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Mon, 30 Mar 2020 11:42:58 +0200 Subject: [PATCH 13/16] Remove some unnecessary comments --- libraries/ClientHTTP/src/ClientHTTP.cpp | 6 ------ 1 file changed, 6 deletions(-) diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index b78903005e5..1084e1c9023 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -35,12 +35,6 @@ ClientHTTP::~ClientHTTP() { free(_location); - -// TO DO RESPONSE HEADERS -// for(auto responseHeader: responseHeaders) { -// free(responseHeader.second); // !!!!!! NOT SURE IF THESE POINTERS MUST BE FREE-ED -// } - free(_host); } From 45dfef41c5084dd80ece7090a5cdbdf8d5863095 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Mon, 6 Apr 2020 15:38:43 +0200 Subject: [PATCH 14/16] Add single qoutes around request headers --- libraries/ClientHTTP/src/ClientHTTP.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index 1084e1c9023..c20a139ea08 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -403,10 +403,10 @@ bool ClientHTTP::writeRequestHeaders(const char * method, const c for(auto requestHeader: requestHeaders) { // std::pair // _client.printf("%s: %s\r\n", requestHeader.first, requestHeader.second); - _client.printf("%s: %s\r\n", requestHeader.first.c_str(), requestHeader.second.c_str()); + _client.printf("'%s': '%s'\r\n", requestHeader.first.c_str(), requestHeader.second.c_str()); #if defined(ESP32) // log_v("Request header '%s: %s'", requestHeader.first, requestHeader.second); - log_v("Request header '%s: %s'", requestHeader.first.c_str(), requestHeader.second.c_str()); + log_v("Request header '%s': '%s'", requestHeader.first.c_str(), requestHeader.second.c_str()); #endif } From 19952e2194898e6a53474d245db35c77d41a2700 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Mon, 6 Apr 2020 15:48:01 +0200 Subject: [PATCH 15/16] Remove single qoutes around request headers again, because NodeJs shows two pairs of quotes otherwise --- libraries/ClientHTTP/src/ClientHTTP.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libraries/ClientHTTP/src/ClientHTTP.cpp b/libraries/ClientHTTP/src/ClientHTTP.cpp index c20a139ea08..cf4116ad3c8 100644 --- a/libraries/ClientHTTP/src/ClientHTTP.cpp +++ b/libraries/ClientHTTP/src/ClientHTTP.cpp @@ -403,10 +403,10 @@ bool ClientHTTP::writeRequestHeaders(const char * method, const c for(auto requestHeader: requestHeaders) { // std::pair // _client.printf("%s: %s\r\n", requestHeader.first, requestHeader.second); - _client.printf("'%s': '%s'\r\n", requestHeader.first.c_str(), requestHeader.second.c_str()); + _client.printf("%s: %s\r\n", requestHeader.first.c_str(), requestHeader.second.c_str()); #if defined(ESP32) // log_v("Request header '%s: %s'", requestHeader.first, requestHeader.second); - log_v("Request header '%s': '%s'", requestHeader.first.c_str(), requestHeader.second.c_str()); + log_v("Request header %s: %s", requestHeader.first.c_str(), requestHeader.second.c_str()); #endif } From b720725f16139bb6e30a5d884aefa611fdb94a38 Mon Sep 17 00:00:00 2001 From: Jeroen88 Date: Thu, 9 Apr 2020 17:06:51 +0200 Subject: [PATCH 16/16] Made certificate list static in example ReuseConnectionHTTPS.ino to keep it alive during the full running of the sketch --- libraries/ClientHTTP/README.md | 2 +- .../ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/libraries/ClientHTTP/README.md b/libraries/ClientHTTP/README.md index c45d9d1f28c..83b76f0f3ac 100644 --- a/libraries/ClientHTTP/README.md +++ b/libraries/ClientHTTP/README.md @@ -27,7 +27,7 @@ The following examples are available. All but one are HTTPS / TLS 1.2 examples: - POSTClientHTTPS: how to `POST` a payload to a HTTPS server and print the response to `Serial` - POSTJsonClientHTTPS: how to `POST` an ArduinoJson to a HTTPS server and deserialize the response into another ArduinJson without buffers. All data is buffered into the ArduinoJson only - POSTSizerClientHTTPS: how to determine the size of a payload by printing to a utility `class Sizer` first. Next the resulting length is used to set the Content-Length request header by calling `POST` -- ReuseConnectionHTTPS: how to reuse a connection, including correct handling of the Connection: close response header. This example DOES NOT WORK on the ESP8266 YET! +- ReuseConnectionHTTPS: how to reuse a connection, including correct handling of the Connection: close response header. Still to be done: - Support for redirection to a different host diff --git a/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino index 3641b2c560c..a555ebe4f92 100644 --- a/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino +++ b/libraries/ClientHTTP/examples/ReuseConnectionClientHTTPS/ReuseConnectionClientHTTPS.ino @@ -111,7 +111,7 @@ void setup() { Serial.println(asctime(localtime(&epoch))); #if defined(ESP8266) - BearSSL::X509List cert(rootCACert); + static BearSSL::X509List cert(rootCACert); // Used static to make sure 'cert' is alive during the full running of the sketch client.setTrustAnchors(&cert); #endif #if defined(ESP32)