Mastering ArduinoJson1. 2. 3..

Mastering ArduinoJson 6 Efficient JSON serialization for embedded C++

BENOIT BLANCHON CREATOR OF ARDUINOJSON

SECOND EDITION

Contents

Contents iv

1 Introduction 11.1 About this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.2 Code samples . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.3 What’s new in the second edition . . . . . . . . . . . . . . . . . 3

1.2 Introduction to JSON . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.1 What is JSON? . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.2 What is serialization? . . . . . . . . . . . . . . . . . . . . . . . 51.2.3 What can you do with JSON? . . . . . . . . . . . . . . . . . . 51.2.4 History of JSON . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2.5 Why is JSON so popular? . . . . . . . . . . . . . . . . . . . . . 81.2.6 The JSON syntax . . . . . . . . . . . . . . . . . . . . . . . . . 91.2.7 Binary data in JSON . . . . . . . . . . . . . . . . . . . . . . . 121.2.8 Comments in JSON . . . . . . . . . . . . . . . . . . . . . . . . 13

1.3 Introduction to ArduinoJson . . . . . . . . . . . . . . . . . . . . . . . 141.3.1 What ArduinoJson is . . . . . . . . . . . . . . . . . . . . . . . 141.3.2 What ArduinoJson is not . . . . . . . . . . . . . . . . . . . . . 141.3.3 What makes ArduinoJson different? . . . . . . . . . . . . . . . 151.3.4 Does size really matter? . . . . . . . . . . . . . . . . . . . . . . 171.3.5 What are the alternatives to ArduinoJson? . . . . . . . . . . . . 181.3.6 How to install ArduinoJson . . . . . . . . . . . . . . . . . . . . 201.3.7 The examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2 The missing C++ course 282.1 Why a C++ course? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.2 Stack, heap, and globals . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.2.1 Globals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.2.2 Heap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Contents v

2.2.3 Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.3 Pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.3.1 What is a pointer? . . . . . . . . . . . . . . . . . . . . . . . . 362.3.2 Dereferencing a pointer . . . . . . . . . . . . . . . . . . . . . . 362.3.3 Pointers and arrays . . . . . . . . . . . . . . . . . . . . . . . . 372.3.4 Taking the address of a variable . . . . . . . . . . . . . . . . . 382.3.5 Pointer to class and struct . . . . . . . . . . . . . . . . . . . 382.3.6 Pointer to constant . . . . . . . . . . . . . . . . . . . . . . . . 392.3.7 The null pointer . . . . . . . . . . . . . . . . . . . . . . . . . . 412.3.8 Why use pointers? . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.4 Memory management . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.4.1 malloc() and free() . . . . . . . . . . . . . . . . . . . . . . . . 432.4.2 new and delete . . . . . . . . . . . . . . . . . . . . . . . . . . 432.4.3 Smart pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . 442.4.4 RAII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472.5.1 What is a reference? . . . . . . . . . . . . . . . . . . . . . . . 472.5.2 Differences with pointers . . . . . . . . . . . . . . . . . . . . . 472.5.3 Reference to constant . . . . . . . . . . . . . . . . . . . . . . . 482.5.4 Rules of references . . . . . . . . . . . . . . . . . . . . . . . . 492.5.5 Common problems . . . . . . . . . . . . . . . . . . . . . . . . 492.5.6 Usage for references . . . . . . . . . . . . . . . . . . . . . . . . 50

2.6 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512.6.1 How are the strings stored? . . . . . . . . . . . . . . . . . . . . 512.6.2 String literals in RAM . . . . . . . . . . . . . . . . . . . . . . . 512.6.3 String literals in Flash . . . . . . . . . . . . . . . . . . . . . . . 522.6.4 Pointer to the “globals” section . . . . . . . . . . . . . . . . . . 532.6.5 Mutable string in “globals” . . . . . . . . . . . . . . . . . . . . 542.6.6 A copy in the stack . . . . . . . . . . . . . . . . . . . . . . . . 552.6.7 A copy in the heap . . . . . . . . . . . . . . . . . . . . . . . . 562.6.8 A word about the String class . . . . . . . . . . . . . . . . . . 572.6.9 Pass strings to functions . . . . . . . . . . . . . . . . . . . . . 58

2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3 Deserialize with ArduinoJson 623.1 The example of this chapter . . . . . . . . . . . . . . . . . . . . . . . 633.2 Deserializing an object . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3.2.1 The JSON document . . . . . . . . . . . . . . . . . . . . . . . 643.2.2 Placing the JSON document in memory . . . . . . . . . . . . . 643.2.3 Introducing JsonDocument . . . . . . . . . . . . . . . . . . . . . 65

Contents vi

3.2.4 How to specify the capacity? . . . . . . . . . . . . . . . . . . . 653.2.5 How to determine the capacity? . . . . . . . . . . . . . . . . . 663.2.6 StaticJsonDocument or DynamicJsonDocument? . . . . . . . . . . . 673.2.7 Deserializing the JSON document . . . . . . . . . . . . . . . . 67

3.3 Extracting values from an object . . . . . . . . . . . . . . . . . . . . . 693.3.1 Extracting values . . . . . . . . . . . . . . . . . . . . . . . . . 693.3.2 Explicit casts . . . . . . . . . . . . . . . . . . . . . . . . . . . 693.3.3 When values are missing . . . . . . . . . . . . . . . . . . . . . 703.3.4 Changing the default value . . . . . . . . . . . . . . . . . . . . 71

3.4 Inspecting an unknown object . . . . . . . . . . . . . . . . . . . . . . . 723.4.1 Getting a reference to the object . . . . . . . . . . . . . . . . . 723.4.2 Enumerating the keys . . . . . . . . . . . . . . . . . . . . . . . 733.4.3 Detecting the type of value . . . . . . . . . . . . . . . . . . . . 733.4.4 Variant types and C++ types . . . . . . . . . . . . . . . . . . . 743.4.5 Testing if a key exists in an object . . . . . . . . . . . . . . . . 75

3.5 Deserializing an array . . . . . . . . . . . . . . . . . . . . . . . . . . . 763.5.1 The JSON document . . . . . . . . . . . . . . . . . . . . . . . 763.5.2 Parsing the array . . . . . . . . . . . . . . . . . . . . . . . . . 763.5.3 The ArduinoJson Assistant . . . . . . . . . . . . . . . . . . . . 78

3.6 Extracting values from an array . . . . . . . . . . . . . . . . . . . . . . 793.6.1 Retrieving elements by index . . . . . . . . . . . . . . . . . . . 793.6.2 Alternative syntaxes . . . . . . . . . . . . . . . . . . . . . . . . 793.6.3 When complex values are missing . . . . . . . . . . . . . . . . . 80

3.7 Inspecting an unknown array . . . . . . . . . . . . . . . . . . . . . . . 823.7.1 Getting a reference to the array . . . . . . . . . . . . . . . . . . 823.7.2 Capacity of JsonDocument for an unknown input . . . . . . . . . 823.7.3 Number of elements in an array . . . . . . . . . . . . . . . . . 833.7.4 Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 833.7.5 Detecting the type of an element . . . . . . . . . . . . . . . . . 84

3.8 The zero-copy mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . 863.8.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 863.8.2 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 863.8.3 Input buffer must stay in memory . . . . . . . . . . . . . . . . 88

3.9 Reading from read-only memory . . . . . . . . . . . . . . . . . . . . . 903.9.1 The example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903.9.2 Duplication is required . . . . . . . . . . . . . . . . . . . . . . 903.9.3 Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 913.9.4 Other types of read-only input . . . . . . . . . . . . . . . . . . 92

3.10 Reading from a stream . . . . . . . . . . . . . . . . . . . . . . . . . . 943.10.1 Reading from a file . . . . . . . . . . . . . . . . . . . . . . . . 94

Contents vii

3.10.2 Reading from an HTTP response . . . . . . . . . . . . . . . . . 953.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

4 Serializing with ArduinoJson 1054.1 The example of this chapter . . . . . . . . . . . . . . . . . . . . . . . 1064.2 Creating an object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

4.2.1 The example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1074.2.2 Allocating the JsonDocument . . . . . . . . . . . . . . . . . . . 1074.2.3 Adding members . . . . . . . . . . . . . . . . . . . . . . . . . 1084.2.4 Alternative syntax . . . . . . . . . . . . . . . . . . . . . . . . . 1084.2.5 Creating an empty object . . . . . . . . . . . . . . . . . . . . . 1094.2.6 Removing members . . . . . . . . . . . . . . . . . . . . . . . . 1094.2.7 Replacing members . . . . . . . . . . . . . . . . . . . . . . . . 109

4.3 Creating an array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1114.3.1 The example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1114.3.2 Allocating the JsonDocument . . . . . . . . . . . . . . . . . . . . 1114.3.3 Adding elements . . . . . . . . . . . . . . . . . . . . . . . . . . 1124.3.4 Adding nested objects . . . . . . . . . . . . . . . . . . . . . . . 1124.3.5 Creating an empty array . . . . . . . . . . . . . . . . . . . . . 1134.3.6 Replacing elements . . . . . . . . . . . . . . . . . . . . . . . . 1134.3.7 Removing elements . . . . . . . . . . . . . . . . . . . . . . . . 1144.3.8 Adding null . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1144.3.9 Adding pre-formatted JSON . . . . . . . . . . . . . . . . . . . 114

4.4 Writing to memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1164.4.1 Minified JSON . . . . . . . . . . . . . . . . . . . . . . . . . . . 1164.4.2 Specifying (or not) the size of the output buffer . . . . . . . . . 1164.4.3 Prettified JSON . . . . . . . . . . . . . . . . . . . . . . . . . . 1174.4.4 Measuring the length . . . . . . . . . . . . . . . . . . . . . . . 1184.4.5 Writing to a String . . . . . . . . . . . . . . . . . . . . . . . . 1194.4.6 Casting a JsonVariant to a String . . . . . . . . . . . . . . . . 119

4.5 Writing to a stream . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1214.5.1 What’s an output stream? . . . . . . . . . . . . . . . . . . . . 1214.5.2 Writing to the serial port . . . . . . . . . . . . . . . . . . . . . 1224.5.3 Writing to a file . . . . . . . . . . . . . . . . . . . . . . . . . . 1234.5.4 Writing to a TCP connection . . . . . . . . . . . . . . . . . . . 123

4.6 Duplication of strings . . . . . . . . . . . . . . . . . . . . . . . . . . . 1274.6.1 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1274.6.2 Keys and values . . . . . . . . . . . . . . . . . . . . . . . . . . 1284.6.3 Copy only occurs when adding values . . . . . . . . . . . . . . 1284.6.4 ArduinoJson Assistant’s “extra bytes” . . . . . . . . . . . . . . 129

Contents viii

4.6.5 Why copying Flash strings? . . . . . . . . . . . . . . . . . . . . 1294.6.6 serialized() . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

5 Advanced Techniques 1335.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1345.2 Filtering the input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1355.3 Deserializing in chunks . . . . . . . . . . . . . . . . . . . . . . . . . . 1395.4 JSON streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1445.5 Automatic capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1475.6 Fixing memory leaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1495.7 Using external RAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1515.8 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1545.9 Buffering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1575.10 Custom readers and writers . . . . . . . . . . . . . . . . . . . . . . . . 1605.11 MessagePack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1655.12 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

6 Inside ArduinoJson 1706.1 Why JsonDocument? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

6.1.1 Memory representation . . . . . . . . . . . . . . . . . . . . . . 1716.1.2 Dynamic memory . . . . . . . . . . . . . . . . . . . . . . . . . 1726.1.3 Memory pool . . . . . . . . . . . . . . . . . . . . . . . . . . . 1736.1.4 Strengths and weaknesses . . . . . . . . . . . . . . . . . . . . . 174

6.2 Inside JsonDocument . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1756.2.1 Differences with JsonVariant . . . . . . . . . . . . . . . . . . . 1756.2.2 Fixed capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . 1756.2.3 Implementation of the allocator . . . . . . . . . . . . . . . . . . 1766.2.4 Implementation of JsonDocument . . . . . . . . . . . . . . . . . 177

6.3 Inside StaticJsonDocument . . . . . . . . . . . . . . . . . . . . . . . . . 1796.3.1 Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1796.3.2 Stack memory . . . . . . . . . . . . . . . . . . . . . . . . . . . 1796.3.3 Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1806.3.4 Other usages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1816.3.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 181

6.4 Inside DynamicJsonDocument . . . . . . . . . . . . . . . . . . . . . . . . 1826.4.1 Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1826.4.2 Shrinking a DynamicJsonDocument . . . . . . . . . . . . . . . . . 1826.4.3 Automatic capacity . . . . . . . . . . . . . . . . . . . . . . . . 1836.4.4 Heap memory . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Contents ix

6.4.5 Allocator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1846.4.6 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 1856.4.7 Comparison with StaticJsonDocument . . . . . . . . . . . . . . . 1866.4.8 How to choose? . . . . . . . . . . . . . . . . . . . . . . . . . . 186

6.5 Inside JsonVariant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1876.5.1 Supported types . . . . . . . . . . . . . . . . . . . . . . . . . . 1876.5.2 Reference semantics . . . . . . . . . . . . . . . . . . . . . . . . 1876.5.3 Creating a JsonVariant . . . . . . . . . . . . . . . . . . . . . . 1886.5.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 1896.5.5 Two kinds of null . . . . . . . . . . . . . . . . . . . . . . . . . 1906.5.6 The unsigned long trick . . . . . . . . . . . . . . . . . . . . . . 1916.5.7 Integer overflow . . . . . . . . . . . . . . . . . . . . . . . . . . 1916.5.8 ArduinoJson’s configuration . . . . . . . . . . . . . . . . . . . . 1926.5.9 Iterating through a JsonVariant . . . . . . . . . . . . . . . . . . 1936.5.10 The or operator . . . . . . . . . . . . . . . . . . . . . . . . . . 1946.5.11 The subscript operator . . . . . . . . . . . . . . . . . . . . . . 1956.5.12 Member functions . . . . . . . . . . . . . . . . . . . . . . . . . 1966.5.13 Comparison operators . . . . . . . . . . . . . . . . . . . . . . . 1996.5.14 Const reference . . . . . . . . . . . . . . . . . . . . . . . . . . 200

6.6 Inside JsonObject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2016.6.1 Reference semantics . . . . . . . . . . . . . . . . . . . . . . . . 2016.6.2 Null object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2016.6.3 Create an object . . . . . . . . . . . . . . . . . . . . . . . . . . 2026.6.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 2026.6.5 Subscript operator . . . . . . . . . . . . . . . . . . . . . . . . . 2036.6.6 Member functions . . . . . . . . . . . . . . . . . . . . . . . . . 2046.6.7 Const reference . . . . . . . . . . . . . . . . . . . . . . . . . . 207

6.7 Inside JsonArray . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2096.7.1 Member functions . . . . . . . . . . . . . . . . . . . . . . . . . 2096.7.2 copyArray() . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

6.8 Inside the parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2156.8.1 Invoking the parser . . . . . . . . . . . . . . . . . . . . . . . . 2156.8.2 Two modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2166.8.3 Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2166.8.4 Nesting limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2176.8.5 Quotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2186.8.6 Escape sequences . . . . . . . . . . . . . . . . . . . . . . . . . 2196.8.7 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2206.8.8 NaN and Infinity . . . . . . . . . . . . . . . . . . . . . . . . . 2206.8.9 Stream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Contents x

6.8.10 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2216.9 Inside the serializer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

6.9.1 Invoking the serializer . . . . . . . . . . . . . . . . . . . . . . . 2226.9.2 Measuring the length . . . . . . . . . . . . . . . . . . . . . . . 2236.9.3 Escape sequences . . . . . . . . . . . . . . . . . . . . . . . . . 2246.9.4 Float to string . . . . . . . . . . . . . . . . . . . . . . . . . . . 2246.9.5 NaN and Infinity . . . . . . . . . . . . . . . . . . . . . . . . . 225

6.10 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2266.10.1 The version macro . . . . . . . . . . . . . . . . . . . . . . . . . 2266.10.2 The private namespace . . . . . . . . . . . . . . . . . . . . . . 2266.10.3 The ArduinoJson namespace . . . . . . . . . . . . . . . . . . . 2276.10.4 ArduinoJson.h and ArduinoJson.hpp . . . . . . . . . . . . . . . . 2276.10.5 The single header . . . . . . . . . . . . . . . . . . . . . . . . . 2286.10.6 Code coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . 2286.10.7 Fuzzing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2286.10.8 Portability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2296.10.9 Online compiler . . . . . . . . . . . . . . . . . . . . . . . . . . 2306.10.10 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

6.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

7 Troubleshooting 2337.1 Program crashes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

7.1.1 Undefined Behaviors . . . . . . . . . . . . . . . . . . . . . . . . 2347.1.2 A bug in ArduinoJson? . . . . . . . . . . . . . . . . . . . . . . 2347.1.3 Null string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2357.1.4 Use after free . . . . . . . . . . . . . . . . . . . . . . . . . . . 2357.1.5 Return of stack variable address . . . . . . . . . . . . . . . . . 2377.1.6 Buffer overflow . . . . . . . . . . . . . . . . . . . . . . . . . . 2387.1.7 Stack overflow . . . . . . . . . . . . . . . . . . . . . . . . . . . 2407.1.8 How to diagnose these bugs? . . . . . . . . . . . . . . . . . . . 2407.1.9 How to prevent these bugs? . . . . . . . . . . . . . . . . . . . . 243

7.2 Deserialization issues . . . . . . . . . . . . . . . . . . . . . . . . . . . 2457.2.1 IncompleteInput . . . . . . . . . . . . . . . . . . . . . . . . . . 2457.2.2 InvalidInput . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2467.2.3 NoMemory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2507.2.4 NotSupported . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2517.2.5 TooDeep . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

7.3 Serialization issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2547.3.1 The JSON document is incomplete . . . . . . . . . . . . . . . . 2547.3.2 The JSON document contains garbage . . . . . . . . . . . . . . 254

Contents xi

7.3.3 Too much duplication . . . . . . . . . . . . . . . . . . . . . . . 2567.4 Common error messages . . . . . . . . . . . . . . . . . . . . . . . . . . 258

7.4.1 X was not declared in this scope . . . . . . . . . . . . . . . . . 2587.4.2 Invalid conversion from const char* to char* . . . . . . . . . . 2587.4.3 Invalid conversion from const char* to int . . . . . . . . . . . . 2597.4.4 No match for operator[] . . . . . . . . . . . . . . . . . . . . . 2607.4.5 Ambiguous overload for operator= . . . . . . . . . . . . . . . . 2617.4.6 Call of overloaded function is ambiguous . . . . . . . . . . . . . 2627.4.7 The value is not usable in a constant expression . . . . . . . . . 263

7.5 Asking for help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2647.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

8 Case Studies 2678.1 Configuration in SPIFFS . . . . . . . . . . . . . . . . . . . . . . . . . 268

8.1.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2688.1.2 The JSON document . . . . . . . . . . . . . . . . . . . . . . . 2688.1.3 The configuration class . . . . . . . . . . . . . . . . . . . . . . 2698.1.4 load() and save() members . . . . . . . . . . . . . . . . . . . . 2708.1.5 Saving an ApConfig into a JsonObject . . . . . . . . . . . . . . 2718.1.6 Loading an ApConfig from a JsonObject . . . . . . . . . . . . . 2718.1.7 Copying strings safely . . . . . . . . . . . . . . . . . . . . . . . 2728.1.8 Saving a Config to a JSON object . . . . . . . . . . . . . . . . 2738.1.9 Loading a Config from a JSON object . . . . . . . . . . . . . . 2738.1.10 Saving the configuration to a file . . . . . . . . . . . . . . . . . 2748.1.11 Reading the configuration from a file . . . . . . . . . . . . . . . 2758.1.12 Choosing the JsonDocument . . . . . . . . . . . . . . . . . . . . 2768.1.13 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

8.2 OpenWeatherMap on mkr1000 . . . . . . . . . . . . . . . . . . . . . . 2798.2.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2798.2.2 OpenWeatherMap’s API . . . . . . . . . . . . . . . . . . . . . 2798.2.3 The JSON response . . . . . . . . . . . . . . . . . . . . . . . . 2808.2.4 Reducing the size of the document . . . . . . . . . . . . . . . . 2818.2.5 The filter document . . . . . . . . . . . . . . . . . . . . . . . . 2838.2.6 The code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2848.2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

8.3 Reddit on ESP8266 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2868.3.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2868.3.2 Reddit’s API . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2878.3.3 The response . . . . . . . . . . . . . . . . . . . . . . . . . . . 2888.3.4 The main loop . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

Contents xii

8.3.5 Escaped Unicode characters . . . . . . . . . . . . . . . . . . . . 2908.3.6 Sending the request . . . . . . . . . . . . . . . . . . . . . . . . 2908.3.7 Assembling the puzzle . . . . . . . . . . . . . . . . . . . . . . . 2918.3.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

8.4 JSON-RPC with Kodi . . . . . . . . . . . . . . . . . . . . . . . . . . . 2948.4.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2948.4.2 JSON-RPC Request . . . . . . . . . . . . . . . . . . . . . . . . 2958.4.3 JSON-RPC Response . . . . . . . . . . . . . . . . . . . . . . . 2958.4.4 A JSON-RPC framework . . . . . . . . . . . . . . . . . . . . . 2968.4.5 JsonRpcRequest . . . . . . . . . . . . . . . . . . . . . . . . . . 2978.4.6 JsonRpcResponse . . . . . . . . . . . . . . . . . . . . . . . . . . 2988.4.7 JsonRpcClient . . . . . . . . . . . . . . . . . . . . . . . . . . . 2998.4.8 Sending a notification to Kodi . . . . . . . . . . . . . . . . . . 3008.4.9 Reading properties from Kodi . . . . . . . . . . . . . . . . . . . 3028.4.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

8.5 Recursive analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3068.5.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3068.5.2 Read from the serial port . . . . . . . . . . . . . . . . . . . . . 3068.5.3 Flushing after an error . . . . . . . . . . . . . . . . . . . . . . 3078.5.4 Testing the type of a JsonVariant . . . . . . . . . . . . . . . . 3088.5.5 Printing values . . . . . . . . . . . . . . . . . . . . . . . . . . . 3098.5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

9 Conclusion 312

Index 313

Chapter 4Serializing with ArduinoJson

Any fool can write code that a computer can understand.Good programmers write code that humans can understand.– Martin Fowler, Refactoring: Improving the Design of Existing Code

https://amzn.to/2YqvTrt

Chapter 4 Serializing with ArduinoJson 106

4.1 The example of this chapter

Reading a JSON document is only half of the story; we’llnow see how to write a JSON document with ArduinoJ-son.

In the previous chapter, we played with GitHub’s API. We’lluse a very different example for this chapter: pushing datato Adafruit IO.

Adafruit IO is a cloud storage service for IoT data. Theyhave a free plan with the following restrictions:

• 30 data points per minute

• 30 days of data storage

• 5 feeds

If you need more, it’s just $10 a month. The service is veryeasy to use. All you need is an Adafruit account (yes, youcan use the account from the Adafruit shop).

As we did in the previous chapter, we’ll start with a simpleJSON document and add complexity step by step.

Since Adafruit IO doesn’t impose a secure connection, we can use a less powerfulmicrocontroller than in the previous chapter; we’ll use an Arduino UNO with an EthernetShield.

https://io.adafruit.com


4.2 Creating an object

4.2.1 The example

Here is the JSON object we want to create:

{

"value": 42,

"lat": 48.748010,

"lon": 2.293491

}

It’s a flat object, meaning that it has no nested object or array, and it contains thefollowing members:

1. "value" is an integer that we want to save in Adafruit IO.

2. "lat" is the latitude coordinate where the value was measured.

3. "lon" is the longitude coordinate where the value was measured.

Adafruit IO supports other optional members (like the elevation coordinate and the timeof measurement), but the three members above are sufficient for our example.

4.2.2 Allocating the JsonDocument

As for the deserialization, we start by creating a JsonDocument to hold the memoryrepresentation of the object. The previous chapter introduces the JsonDocument; pleasego back and read ”Introducing JsonDocument” if needed, as we won’t repeat here.

As we saw, a JsonDocument has a fixed capacity that we must specify when we create it.Here, we have one object with no nested values, so the capacity is JSON_OBJECT_SIZE(3).Remember that you can use the ArduinoJson Assistant to compute the required capac-ity.

For our example, we can use a StaticJsonDocument because the document is so smallthat it can easily fit in the stack.

https://arduinojson.org/v6/assistant/


Here is the code:

const int capacity = JSON_OBJECT_SIZE(3);

StaticJsonDocument<capacity> doc;

The JsonDocument is currently empty and JsonDocument::isNull() returns true. If weserialize it now, the output is “null.”

4.2.3 Adding members

An empty JsonDocument automatically becomes an object when we add members to it.We do that with the subscript operator ([]), just like we did in the previous chapter:

doc["value"] = 42;

doc["lat"] = 48.748010;

doc["lon"] = 2.293491;

The memory usage is now JSON_OBJECT_SIZE(3), so the JsonDocument is full. When theJsonDocument is full, you cannot add more members, so don’t forget to increase thecapacity if you need.

4.2.4 Alternative syntax

With the syntax presented above, it’s not possible to tell whether the insertion suc-ceeded. Let’s see another syntax:

doc["value"].set(42);

doc["lat"].set(48.748010);

doc["lon"].set(2.293491);

The compiler generates the same executable as with the previous syntax, except thatyou can tell if the insertion succeeded. Indeed, JsonVariant::set() returns true forsuccess or false if the JsonDocument is full.

To be honest, I never check if insertion succeeds in my programs. The reason is simple:the JSON document is roughly the same for each iteration; if it works once, it alwaysworks. There is no reason to bloat the code for a situation that cannot happen.


4.2.5 Creating an empty object

We just saw that the JsonDocument becomes an object as soon as you insert a member,but what if you don’t have any member to add? What if you want to create an emptyobject?

When you need an empty object, you cannot rely on the implicit conversion any-more. Instead, you must convert the JsonDocument to a JsonObject explicitly withJsonDocument::to<JsonObject>():

// Convert the document to an object

JsonObject obj = doc.to<JsonObject>();

This function clears the JsonDocument, so all existing references become invalid. Then,it creates an empty object at the root of the document and returns a reference to thisobject.

At this point, the JsonDocument is not empty anymore and JsonDocument::isNull()

returns false. If we serialize this document, the output is “{}”.

4.2.6 Removing members

It’s possible to erase a member from an object by calling JsonObject::remove(key).However, for reasons that will become clear in chapter 6, this function doesn’t releasethe memory in the JsonDocument.

The remove() function is a frequent cause of bugs because it creates a memory leak.Indeed, if you add and remove members in a loop, the JsonDocument grows, but memoryis never released.

4.2.7 Replacing members

It’s possible to replace a member in the object, for example:

obj["value"] = 42;

obj["value"] = 43;


Most of the time, replacing a member doesn’t require a new allocation in theJsonDocument. However, it can cause a memory leak if the old value has associatedmemory, for example, if the old value is a string, an array, or an object.

Memory leaksReplacing and removing values produce a memory leak inside theJsonDocument.In practice, this problem only happens in programs that use a JsonDocument

to store the state of the application, which is not the purpose of Arduino-Json. Let’s be clear; the sole purpose of ArduinoJson is to serialize anddeserialize JSON documents.Be careful not to fall into this common anti-pattern and make sure you readthe case studies to see how ArduinoJson should be used.


4.3 Creating an array

4.3.1 The example

Now that we know how to create an object, let’s see how we can create an array. Ournew example will be an array that contains two objects.

[

{

"key": "a1",

"value": 12

},

{

"key": "a2",

"value": 34

}

]

The values 12 and 34 are just placeholder; in reality, we’ll use the result fromanalogRead().

4.3.2 Allocating the JsonDocument

As usual, we start by computing the capacity of the JsonDocument:

• There is one array with two elements: JSON_ARRAY_SIZE(2)

• There are two objects with two members: 2*JSON_OBJECT_SIZE(2)

Here is the code:

const int capacity = JSON_ARRAY_SIZE(2) + 2*JSON_OBJECT_SIZE(2);



4.3.3 Adding elements

In the previous section, we saw that an empty JsonDocument automatically becomes anobject as soon as we insert the first member. Well, this statement was only partiallyright: it becomes an object as soon as we use it as an object.

Indeed, if we treat an empty JsonDocument as an array, it automatically becomes anarray. For example if we call JsonDocument::add():

doc.add(1);

doc.add(2);

After these two lines, our JsonDocument contains [1,2].

Alternatively, we can create the same array like so:

doc[0] = 1;

doc[1] = 2;

However, this second syntax is a little slower because it requires walking the list ofmembers. Don’t use this syntax if you need to add many elements to the array.

Now that this topic is clear, let’s rewind a little because that’s not the JSON array wewant to create; instead of two integers, we want two nested objects.

4.3.4 Adding nested objects

To add the nested objects to the array, we call JsonArray::createNestedObject(). Thisfunction creates a nested object, appends it the array, and returns a reference.

Here is how to create our sample document:

JsonObject obj1 = doc.createNestedObject();

obj1["key"] = "a1";

obj1["value"] = analogRead(A1);

JsonObject obj2 = doc.createNestedObject();

obj2["key"] = "a2";

obj2["value"] = analogRead(A2);


Alternatively, we can create the same document like so:

doc[0]["key"] = "a1";

doc[0]["value"] = analogRead(A1);

doc[1]["key"] = "a2";

doc[1]["value"] = analogRead(A2);

As I already said, this syntax runs slower, so only use it for small arrays.

4.3.5 Creating an empty array

We saw that the JsonDocument becomes an array as soon as we add elements, but thisdoesn’t allow creating an empty array. If we want to create an empty array, we need toconvert the JsonDocument explicitly with JsonDocument::to<JsonArray>():

// Convert the JsonDocument to an array

JsonArray arr = doc.to<JsonArray>();

Now the JsonDocument contains [].

As we already saw, JsonDocument::to<T>() clears the JsonDocument, so it also invalidatesall previously acquired references.

4.3.6 Replacing elements

As for objects, it’s possible to replace elements in arrays using JsonArray::operator[]:

arr[0] = 666;

arr[1] = 667;

Most of the time, replacing the value doesn’t require a new allocation in theJsonDocument. However, if there was memory held by the previous value, for example,a JsonObject, this memory is not released. It’s a limitation of ArduinoJson’s memoryallocator, as we’ll see later in this book.


4.3.7 Removing elements

As for objects, you can remove an element from the array, with JsonArray::remove():

arr.remove(0);

Again, remove() doesn’t release the memory from the JsonDocument, so you should nevercall this function in a loop.

4.3.8 Adding null

To conclude this section, let’s see how we can insert special values in the JSON docu-ment.

The first special value is null, which is a legal token in a JSON. There are several waysto add a null in a JsonDocument; here they are:

// Use a nullptr (requires C++11)

arr.add(nullptr);

// Use a null char-pointer

arr.add((char*)0);

// Use a null JsonArray, JsonObject, or JsonVariant

arr.add(JsonVariant());

4.3.9 Adding pre-formatted JSON

The other special value is a JSON string that is already formatted and that ArduinoJsonshould not treat as a regular string.

You can do that by wrapping the string with a call to serialized():

// adds "[1,2]"

arr.add("[1,2]");


// adds [1,2]

arr.add(serialized("[1,2]"));

The program above produces the following JSON document:

[

"[1,2]",

[1,2]

]

This feature is useful when a part of the document never changes, and you want tooptimize the code. It’s also useful to insert something you cannot generate with thelibrary.


4.4 Writing to memory

We saw how to construct an array. Now, it’s time to serialize it into a JSON document.There are several ways to do that. We’ll start with a JSON document in memory.

We could use a String, but as you know, I prefer avoiding dynamic memory allocation.Instead, we’d use a good old char[]:

// Declare a buffer to hold the result

char output[128];

4.4.1 Minified JSON

To produce a JSON document from a JsonDocument, we simply need to callserializeJson():

// Produce a minified JSON document

serializeJson(doc, output);

Now, the string output contains:

[{"key":"a1","value":12},{"key":"a2","value":34}]

As you see, there are neither space nor line breaks; it’s a “minified” JSON document.

4.4.2 Specifying (or not) the size of the output buffer

If you’re a C programmer, you may have been surprised that I didn’t provide the sizeof the buffer to serializeJson(). Indeed, there is an overload of serializeJson() thattakes a char* and a size:

serializeJson(doc, output, sizeof(output));


However, that’s not the overload we called in the previous snippet. Instead, we calleda template method that infers the size of the buffer from its type (in this case,char[128]).

Of course, this shorter syntax only works because output is an array. If it were a char*

or a variable-length array, we would have had to specify the size.

Variable-length arrayA variable-length array, or VLA, is an array whose size is unknown at compiletime. Here is an example:

void f(int n) {

char buf[n];

// ...

}

C99 and C11 allow VLAs, but not C++. However, some compilers supportVLAs as an extension.This feature is often criticized in C++ circles, but Arduino users seem tolove it. That’s why ArduinoJson supports VLAs in all functions that accepta string.

4.4.3 Prettified JSON

The minified version is what you use to store or transmit a JSON document becausethe size is optimal. However, it’s not very easy to read. Humans prefer “prettified”JSON documents with spaces and line breaks.

To produce a prettified document, you must use serializeJsonPretty() instead ofserializeJson():

// Produce a prettified JSON document

serializeJsonPretty(doc, output);

Here is the content of output:

[

{


"key": "a1",

"value": 12

},

{

"key": "a2",

"value": 34

}

]

Of course, you need to make sure that the output buffer is big enough; otherwise, theJSON document will be incomplete.

4.4.4 Measuring the length

ArduinoJson allows computing the length of the JSON document before producing it.This information is useful for:

1. allocating an output buffer,

2. reserving the size on disk, or

3. setting the Content-Length header.

There are two methods, depending on the type of document you want to produce:

// Compute the length of the minified JSON document

int len1 = measureJson(doc);

// Compute the length of the prettified JSON document

int len2 = measureJsonPretty(doc);

In both cases, the result doesn’t count the null-terminator.

By the way, serializeJson() and serializeJsonPretty() return the number of bytesthat they wrote. The results are the same as measureJson() and serializeJsonPretty(),except if the output buffer is too small.


Avoid prettified documentsWith the example above, the sizes are 73 and 110. In this case, the prettifiedversion is only 50% bigger because the document is simple, but in mostcases, the ratio is largely above 100%.Remember, we’re in an embedded environment: every byte counts, and sodoes every CPU cycle. Always prefer a minified version.

4.4.5 Writing to a String

The functions serializeJson() and serializeJsonPretty() have overloads taking aString:

String output = "JSON = ";

serializeJson(doc, output);

The behavior is slightly different: the JSON document is appended to the String; itdoesn’t replace it. That means the above snippet sets the content of the output variableto:

JSON = [{"key":"a1","value":12},{"key":"a2","value":34}]

This behavior seems inconsistent? That’s because ArduinoJson treats String like astream; more on that later.

4.4.6 Casting a JsonVariant to a String

You should remember from the chapter on deserialization that we must cast JsonVariantto the type we want to read.

It is also possible to cast a JsonVariant to a String. If the JsonVariant contains astring, the return value is a copy of the string. However, if the JsonVariant containssomething else, the returned string is a serialization of the variant.

We could rewrite the previous example like this:

// Cast the JsonDocument to a string

String output = "JSON = " + doc.as<String>();


This trick works with JsonDocument and JsonVariant, but not with JsonArray andJsonObject because they don’t have an as<T>() function.


4.5 Writing to a stream

4.5.1 What’s an output stream?

For now, every JSON document we produced remained in memory, but that’s usuallynot what we want. In many situations, it’s possible to send the JSON document directlyto its destination (whether it’s a file, a serial port, or a network connection) withoutany copy in RAM.

We saw in the previous chapter what an “input stream” is, and we saw that Arduinorepresents this concept with the Stream class. Similarly, there are “output streams,”which are sinks of bytes. We can write to an output stream, but we cannot read. Inthe Arduino land, an output stream is materialized by the Print class.

Here are examples of classes derived from Print:

Library Class Well known instancesCore HardwareSerial Serial, Serial1…

ESP

BluetoothSerial SerialBT

File

WiFiClient

WiFiClientSecure

Ethernet EthernetClient

EthernetUDP

GSM GSMClient

LiquidCrystal LiquidCrystal

SD File

SoftwareSerial SoftwareSerial

WiFi WiFiClient

Wire TwoWire Wire

std::ostream

In the C++ Standard Library, an output stream is represented by thestd::ostream class.ArduinoJson supports both Print and std::ostream.


4.5.2 Writing to the serial port

The most famous implementation of Print is HardwareSerial, which is the class ofSerial. To serialize a JsonDocument to the serial port of your Arduino, just pass Serial

to serializeJson():

// Print a minified JSON document to the serial port

serializeJson(doc, Serial);

// Same with a prettified document

serializeJsonPretty(doc, Serial);

You can see the result in the Arduino Serial Monitor, which is very handy for debug-ging.

There are also other serial port implementations that you can use this way, for example,SoftwareSerial and TwoWire.


4.5.3 Writing to a file

Similarly, we can use a File instance as the target of serializeJson() andserializeJsonPretty(). Here is an example with the SD library:

// Open file for writing

File file = SD.open("adafruit.txt", FILE_WRITE);

// Write a prettified JSON document to the file

serializeJsonPretty(doc, file);

You can find the complete source code for this example in the WriteSdCard folder of thezip file provided with the book.

You can apply the same technique to write a file on an ESP8266, as we’ll see in thecase studies.

4.5.4 Writing to a TCP connection

We’re now reaching our goal of sending our measurements to Adafruit IO.

As I said in the introduction, we’ll suppose that our program runs on an Arduino UNOwith an Ethernet shield. Because the Arduino UNO has only 2KB of RAM, we’ll notuse the heap at all.

Preparing the Adafruit IO account

If you want to run this program, you need an account on Adafruit IO (a free account issufficient). Then, you need to copy your user name and your “AIO key” to the sourcecode.

#define IO_USERNAME "bblanchon"

#define IO_KEY "baf4f21a32f6438eb82f83c3eed3f3b3"

We’ll include the AIO key in an HTTP header, and it will authenticate our program onAdafruit’s server:

X-AIO-Key: baf4f21a32f6438eb82f83c3eed3f3b3


Finally, to run this program, you need to create a “group” named “arduinojson” in yourAdafruit IO account. In this group, you need to create two feeds: “a1” and “a2.”

The request

To send our measured samples to Adafruit IO, we have to send a POST request to http://

io.adafruit.com/api/v2/bblanchon/groups/arduinojson/data, and include the followingJSON document in the body:

{

"location": {

"lat": 48.748010,

"lon": 2.293491

},

"feeds": [

{

"key": "a1",

"value": 42

},

{

"key": "a2",

"value": 43

}

]

}

As you see, it’s a little more complex than our previous sample because the array is notat the root of the document. Instead, the array is nested in an object under the key"feeds".

Let’s review the HTTP request before jumping to the code:

POST /api/v2/bblanchon/groups/arduinojson/data HTTP/1.1

Host: io.adafruit.com

Connection: close

Content-Length: 103

Content-Type: application/json

X-AIO-Key: baf4f21a32f6438eb82f83c3eed3f3b3

http://io.adafruit.com/api/v2/bblanchon/groups/arduinojson/data

http://io.adafruit.com/api/v2/bblanchon/groups/arduinojson/data


{"location":{"lat":48.748010,"lon":2.293491},"feeds":[{"key":"a1",...

The code

OK, time for action! We’ll open a TCP connection to io.adafruit.com using anEthernetClient, and we’ll send the request. As far as ArduinoJson is concerned, thereare very few changes compared to the previous examples because we can pass theEthernetClient as the target of serializeJson(). We’ll call measureJson() to set thevalue of the Content-Length header.

Here is the code:

// Allocate JsonDocument

const int capacity = JSON_ARRAY_SIZE(2) + 4 * JSON_OBJECT_SIZE(2);


// Add the "location" object

JsonObject location = doc.createNestedObject("location");

location["lat"] = 48.748010;

location["lon"] = 2.293491;

// Add the "feeds" array

JsonArray feeds = doc.createNestedArray("feeds");

JsonObject feed1 = feeds.createNestedObject();

feed1["key"] = "a1";

feed1["value"] = analogRead(A1);

JsonObject feed2 = feeds.createNestedObject();

feed2["key"] = "a2";

feed2["value"] = analogRead(A2);

// Connect to the HTTP server

EthernetClient client;

client.connect("io.adafruit.com", 80);

// Send "POST /api/v2/bblanchon/groups/arduinojson/data HTTP/1.1"

client.println("POST /api/v2/" IO_USERNAME

"/groups/arduinojson/data HTTP/1.1");


// Send the HTTP headers

client.println("Host: io.adafruit.com");

client.println("Connection: close");

client.print("Content-Length: ");

client.println(measureJson(doc));

client.println("Content-Type: application/json");

client.println("X-AIO-Key: " IO_KEY);

// Terminate headers with a blank line

client.println();

// Send JSON document in body

serializeJson(doc, client);

You can find the complete source code of this example in the AdafruitIo folder ofthe zip file. This code includes the necessary error checking that I removed from themanuscript for clarity.Below is a picture showing the results on the Adafruit IO dashboard.


4.6 Duplication of strings

Depending on the type, ArduinoJson stores strings ether by pointer or by copy. If thestring is a const char*, it stores a pointer; otherwise, it makes a copy. This featurereduces memory consumption when you use string literals.

String type Storageconst char* pointerchar* copyString copyconst __FlashStringHelper* copy

As usual, the copy lives in the JsonDocument, so you may need to increase its capacitydepending on the type of string you use.

4.6.1 An example

Compare this program:

// Create the array ["value1","value2"]

doc.add("value1");

doc.add("value2");

// Print the memory usage

Serial.println(doc.memoryUsage()); // 16

with the following:

// Create the array ["value1","value2"]

doc.add(String("value1"));

doc.add(String("value2"));

// Print the memory usage

Serial.println(doc.memoryUsage()); // 30


They both produce the same JSON document, but the second one requires much morememory because ArduinoJson copies the strings. If you run these programs on anATmega328, you’ll see 16 for the first one and 30 for the second. On an ESP8266, itwould be 32 and 46.

4.6.2 Keys and values

The duplication rules apply equally to keys and values. In practice, we mostly use stringliterals for keys, so they are rarely duplicated. String values, however, often originatefrom variables and then entail string duplication.

Here is a typical example:

String identifier = getIdentifier();

doc["id"] = identifier; // "id" is stored by pointer

// identifier is copied

Again, the duplication occurs for any type of string except const char*.

4.6.3 Copy only occurs when adding values

In the example above, ArduinoJson copied the String because it needed to add it tothe JsonDocument. On the other hand, if you use a String to extract a value from aJsonDocument, it doesn’t make a copy.

Here is an example:

// The following line produces a copy of "key"

doc[String("key")] = "value";

// The following line produces no copy

const char* value = doc[String("key")];


4.6.4 ArduinoJson Assistant’s “extra bytes”

As we saw in the previous chapter, the Assistant shows the number of bytes requiredto duplicate all the strings of the document.

In practice, this value much higher than necessary because you don’t duplicate all thestrings but only some. So, in the case of serialization, you should estimate the requiredvalue based on your program and use the Assistant’s “extra bytes” only as an upperlimit.

As you probably noticed, the Assistant generates a “serializing program” that assumesall the strings are constant, and therefore, entirely ignores this problem. So rememberto increase the capacity as soon as you change the type of the strings.

4.6.5 Why copying Flash strings?

I understand that it is disappointing that ArduinoJson copies Flash strings into theJsonDocument. Unfortunately, there are several situations where it needs to have thestrings in RAM.

For example, if the user calls JsonVariant::as<char*>(), a pointer to the copy is re-turned:


// The value is originally in Flash memory

obj["hello"] = F("world");

// But the returned value is in RAM (in the JsonDocument)

const char* world = obj["hello"];

It is required for JsonPair too. If the string is a key in an object and the user iteratesthrough the object, the JsonPair contains a pointer to the copy:

// The key is originally in Flash memory

obj[F("hello")] = "world";

for(JsonPair kvp : obj) {

// But the key is actually stored in RAM (in the JsonDocument)

const char* key = kvp.key().c_str();

}

However, retrieving a value using a Flash string as a key doesn’t cause a copy:

// The Flash string is not copied in this case

const char* world = obj[F("hello")];

Avoid Flash strings with ArduinoJsonStoring strings in Flash is a great way to reduce RAM usage, but rememberthat ArduinoJson copies them in the JsonDocument.If you wrap all your strings with F(), you’ll need a much bigger JsonDocument.Moreover, the program will waste much time copying the string; it will bemuch slower than with conventional strings.

I plan to avoid this duplication in a future revision of the library, but it’s not on theroadmap yet.


4.6.6 serialized()

We saw earlier in this chapter that the serialized() function marks a string as a JSONfragment that should not be treated as a regular string value.

serialized() supports all the string types (char*, const char*, String, andconst __FlashStringHelper*) and duplicates them as expected.


4.7 Summary

In this chapter, we saw how to serialize a JSON document with ArduinoJson. Here arethe key points to remember:

• Creating the document:

– To add a member to an object, use the subscript operator ([])

– To append an element to an array, call add()

– The first time you add a member to a JsonDocument, it automatically becomesan object.

– The first time you append an element to a JsonDocument, it automaticallybecomes an array.

– You can explicitly convert a JsonDocument with JsonDocument::to<T>().

– JsonDocument::to<T>() clears the JsonDocument, so it invalidates all previouslyacquired references.

– JsonDocument::to<T>() return a reference to the root array or object.

– To create a nested array or object, call createNestedArray() orcreateNestedObject().

– When you insert a string in a JsonDocument, it makes a copy, except if it’s aconst char*.

• Serializing the document:

– To serialize a JsonDocument, call serializeJson() or serializeJsonPretty().

– To compute the length of the JSON document, call measureJson() ormeasureJsonPretty().

– serializeJson() appends to String, but it overrides the content of a char*.

– You can pass an instance of Print (like Serial, EthernetClient, andWiFiClient) to serializeJson() to avoid a copy in the RAM.

In the next chapter, we’ll see advanced techniques like filtering and logging.

Continue reading...

That was a free chapter from “Mastering ArduinoJson”; the book contains seven chap-ters like this one. Here is what readers say:

This book is 100% worth it. Between solving my immediate problem inminutes, Chapter 2, and the various other issues this book made solvingeasy, it is totally worth it. I build software but I work in managed languagesand for someone just getting started in C++and embedded programming thisbook has been indispensable. — Nathan Burnett

I think the missing C++course and the troubleshooting chapter are worththe money by itself. Very useful for C programming dinosaurs like myself.

— Doug Petican

The short C++section was a great refresher. The practical use of Arduino-Json in small embedded processors was just what I needed for my homeautomation work. Certainly worth having! Thank you for both the bookand the library. — Douglas S. Basberg

For a really reasonable price, not only you’ll learn new skills, but you’ll also be one ofthe few people that contribute to sustainable open-source software. Yes, givingmoney for free software is a political act!

The e-book comes in three formats: PDF, epub and mobi. If you purchase the e-book,you get access to newer versions for free. A carefully edited paperback edition isalso available.

Ready to jump in?Go to arduinojson.org/book and use the coupon code THIRTY to get a 30% discount.

https://arduinojson.org/book/

Mastering ArduinoJson1. 2. 3..

Documents

Transcript of Mastering ArduinoJson1. 2. 3..