LoRa Arduino

LoRa

The radio technology that is used by physical computing for ‘Long Range’ (LoRa), low bandwidth communication. These radios may be programmed to be used in either point-to-point or point-to-gateway (LoRaWAN) mode, configurable with software. They use the unlicensed ISM radio bands, 911MHz in Australia.

https://en.wikipedia.org/wiki/LoRa

LoRaWAN

The radio network and infrastructure used to collect sensor data, via publically accessible gateways.

Arduino

A microprocessor board and programming tools, which use the Atmel (and other) microprocessor. The original aim was to create a programming system which was easy to use for hobbyists and members of the maker community. Arduino microprocessor boards can be used with a LoRa radio to create sensors which can transmit data via LoRaWAN and the internet to be collected and used by other applications. https://www.arduino.cc/

Arduino Shield

This is additional hardware that has been designed to plug directly into an Arduino board and allow it to be easy to use.

Arduino IDE

The Interactive Development Environment (IDE) used to program an Arduino microprocessor board. It is available as a downloadable Java program, and as a web based application. The software is available from the Arduino website.

Arduino Libraries

Additional software packages which can be installed to extend Arduino programs. For example, the LMIC library is used to allow our Arduino programs to use the LoRaWAN radio shield.  

Libraries can be downloaded from within the Arduino IDE, or downloaded and installed manually.

LoRaWAN Sensor Node

A device which collects data and transmits to the LoRaWAN network. It uses a LoRa radio. They need to be registered and authenticated with the LoRaWAN network to transmit correctly.

LoRaWAN Gateway

A device which listens for LoRaWAN radio transmissions from nodes in its area and forwards them to the servers on the LoRaWAN network. They need to be connected to the internet and be registered to operate.

The Things Network

A free website (registration required) which allows access to data collected by LoRaWAN sensors. It is used to register LoRaWAN gateways on the network, and create LoRaWAN applications.

https://www.thethingsnetwork.org/

MQTT

MQ Telemetry Transport. This is the communication software and protocol  used to allow sensor data to be distributed and used once it has been collected by The Things Network.

NodeRed

A web server program, which can be installed on a home computer or server (or Raspberry Pi) which can collect and use sensor data collected. It uses MQTT to access sensor data.

NodeRed is an automation environment and is useful for a lot more than collecting and displaying our IoT data. See the website for more details.

https://nodered.org/

TTNmapper

https://ttnmapper.org/ Mobile application which can be used to map LoRaWAN coverage. 

Sensor/Node Example

The following are some example of the sensors that will be built in the workshop.

Moisture Sensor

A soil moisture sensor which sends measurements via LoRaWAN to a monitoring application created by NodeRed

The data is displayed in a web browser as the following.

The LoRa Radio and LoRaWAN

The LoRa radio module transmits in the Industrial, Scientific and Medical (ISM) unlicensed radio band[1]. In Australia (and New Zealand) LoRa transmits between 915-928 MHz[2]. For uploading data, the band is divided into 64 narrow bandwidth channels of 125 kHz, with an additional 8 overlapping broader channels of 500 kHz.

The data is encoded using a chirp signal. The rate that this frequency changes is defined by a ‘spreading factor’ (SF), and a doubling of the spreading factor approximately doubles the amount of time it take to transmit data[3].

LoRaWAN Channel Allocation for Australia and New Zealand

Notes:
[1] ISM Band – https://en.wikipedia.org/wiki/ISM_band
[2] https://lora-alliance.org/sites/default/files/2018-05/lorawan_regional_parameters_v1.0.2_final_1944_1.pdf
[3] https://medium.com/home-wireless/testing-lora-radios-with-the-limesdr-mini-part-2-37fa481217ff

Hardware

Dragino Arduino Shield

To use LoRa and LoRaWAN we are using the following LoRaWAN Arduino Shield available from the local suppliers. It is supplied to transmit and receive in the 915 MHz. As purchased the jumpers were not set, and needed to be installed as indicated in the picture below.

When developing hardware, it is only possible to use digital input/outputs 3,4 and 5 (3 lines). Digital IO 0 and 1 are used by serial IO and interfere with programmng. The digital lines D11-13 are used to control the LoRa radio.

The available Arduino pins for projects are

  • D3-5 (3 digital lines) – Can be used for Digital IO
  • D0-1 (2 digital lines) – Can be used for Digital IO but interferes with in circuit programming. Needs to be disconnected first.
  • A0-5 – Can be used for Analog Input or Digital IO
  • SCL,SDA – Can be used for Digital IO and Analog In on Arduino Uno.
  • AREF – Used as analog reference voltage
  • GND – Ground

Software

Arduino Library for LoRaWAN (LMIC)

The Arduino LMIC library that we are using is being maintained by Thomas Laurenson and can be found on Github[1]. The original LMIC Library was written by IBM and has been ported for use with the Arduino board.

Once downloaded and installed in the Ardiuino IDE library directory[2], the software needs to be configured to use the Australian frequency plan which is done by editing the file ‘project_config/lmic_project_config.h’. For some reason the Australian configuration is called CFG_au921.

// project-specific definitions
//#define CFG_eu868 1
//#define CFG_us915 1
define CFG_au921 1
//#define CFG_as923 1
// #define LMIC_COUNTRY_CODE LMIC_COUNTRY_CODE_JP       /* for s923-JP */
//#define CFG_in866 1
define CFG_sx1276_radio 1
//#define LMIC_USE_INTERRUPTS

Notes:
[1] Github: https://github.com/thomaslaurenson/arduino-lmic
[2] When unzipping the software, there may be duplicated recursive top level directory names (eg. arduino-lmic/arduino-lmic). Ensure that only one level is copied into Arduino library folder.

Example Arduino Sketch

An example Arduino sketch also available from the GitHub repository, which can be used to connect to the LoRaWAN network. See arduino-lmic/examples/ttn-otaa-dragino-lorashield-au915[1].

When connecting to the LoRaWAN network there are two authentication methods “Authtication by Personalisation” (ABP) and “Over the Air Authentication” (OTAA). While it is a little more complicated, OTAA is preferred as it allows the network more control over the authentication process, but requires two-way communication with the end-node. ABP does not require the end-node to receive data from the network but the network needs to know when the end-node is ever reset[2].

For OTAA, there are three pieces of data that need to be copied into the sketch, and are obtained from The Things Network console[3].

  • Device EUI (DEVEUI) – This is the unique identifier for this device on the network. This can be changed. This needs to be used in the sketch in Least Significant Byte (lsb) order.
  • Application EUI – This is the application identifier, and is unique across the LoRaWAN network. This needs to be used in the sketch in Least Significant Byte (lsb) order.
  • Application Key – This is an initial encryption key shared between the Node and Application. It can be changed. This needs to be used in sketch in Most Significant Byte (msb) order.

The webpage tries to make this as easy as possible by allowing you to adjust the formatting and the copy to clipboard button.

Device configuration parameters in The Things Network

These details are copied into the Arduino sketch in the FILLMEIN locations as shown below.

// This EUI must be in little-endian format, so least-significant-byte 
// first. When copying an EUI from ttnctl output, this means to reverse
// the bytes. For TTN issued EUIs the last bytes should be 0xD5, 0xB3,
// 0x70.
static const u1_t PROGMEM APPEUI[8]= { FILLMEIN };
void os_getArtEui (u1_t* buf) { memcpy_P(buf, APPEUI, 8);}

// This should also be in little endian format, see above.
static const u1_t PROGMEM DEVEUI[8]= { FILLMEIN };
void os_getDevEui (u1_t* buf) { memcpy_P(buf, DEVEUI, 8);

// This key should be in big endian format (or, since it is not really a
// number but a block of memory, endianness does not really apply). In
// practice, a key taken from the TTN console can be copied as-is.
static const u1_t PROGMEM APPKEY[16] = { FILLMEIN };
void os_getDevKey (u1_t* buf) { memcpy_P(buf, APPKEY, 16);}

In the sketch, the following line can also be changed.

static uint8_t mydata[] = "OTAA" 

This is the data that is going to be transmitted over the network. This short string can be edited to something more meaningful. The sketch can then be compiled and installed, and if a LoRaWAN Gateway is within range, transmitted data will start to be collected by The Things Network Application.

Notes:
[1] Github: https://github.com/thomaslaurenson/arduino-lmic/tree/master/examples
[2] The packet counters need to be reset. This is used to stop data ‘replay’ attacks.
[3] Console->Applicaion->Devices

LoRaWAN Data

The transmitted packet data will appear under the Application Data on The Things Network.

Data packets being received from an IoT device via LoRaWAN

The information includes the data transmitted by the IoT device and metadata from the network about the transmission. Expanding the packet entry shows this detail.

In the case above, to assist with debugging a custom Payload Format decoder has been added which converts the bytes to a string which can be seen in the ‘receivedString’ field[1].

// Decode an uplink message from a buffer
// (array) of bytes to an object of fields.
function Decoder(bytes, port) {
// var decoded = {};
// if (port === 1) decoded.led = bytes[0];
// return decoded;

// Decode plain text; for testing only
return {
receivedString: String.fromCharCode.apply(null, bytes)
};
}

The raw transmitted data can be seen in the Payload.

The metadata fields show the frequency used (917.8 MHz), the datarate (SF7BW125 – Spreading Factor 7, Bandwidth 125 kHz) and the gateways which heard the transmission and their details.

Notes:
[1] https://core-electronics.com.au/tutorials/encoding-and-decoding-payloads-on-the-things-network.html

Installation of Node-RED

Node-RED is a web server program which uses Node.js, which can be installed on a home computer, server, Raspberry Pi or in the Cloud. It can be used to collect and process the data collected by The Things Network[1].) We will be using MQTT to get access sensor data.

For supported Operating Systems and installation instructions, see: https://nodered.org/docs/getting-started/

On Linux Operating Systems (Ubuntu, Debian, Endebian etc.), Node-RED can be installed with

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

sudo systemctl enable nodered.service

It also appears to be installable on Windows 10 using the Windows Subsystem for Linux (WSL)[2].

Once installed, it can be accessed via web browser by connecting to port 1880. If it is installed on the local computer, it can be accessed by visiting the URL – http://localhost:1880/

A User Guide with Tutorials are available from the Node-RED website[3]. If you have access to a working Node-RED server then the Tutorials are a good way to get started.

One final note. Flows are stored on the computer under the computers name (hostname). If that is ever changed, then the flows will not be loaded and it will look like your configuration has dissapeared. It is still there, and will reappear if the hostname is changed back. The easiest way to fix this is to export your flows to a file first (json) and then import them afterwards. This option is under the main menu.

Notes:
[1] Node-RED is a general purpose automation framework and can do a lot more than mentioned here
[2] https://schellingerhout.github.io/bash%20on%20ubuntu%20on%20windows/nodered-windows/
[2] https://nodered.org/docs/

Working with Node-RED

Once Node-RED has been installed it can be accessed via port 1880.

Some additional packages are required. These are installed via the Manage Palette menu.

node-red-contrib-persist
node-red-dashboard

Configuration is done by connecting processing nodes with virtual wires to create flows. This allows messages to be processed as they pass through the system.

A Node-RED flow configured to process IoT data.

Data collected by The Things Network is pulled into Node-RED by using the MQTT node. This requires the name of The Things Network application and an application key/password.

Add an MQTT node and add a new mqtt-broker (server)

  • Name: Something memorable eg. Meshed
  • Connection
    • Server: thethings.meshed.com.au
    • Port: 1883
    • SSL/TLS: (Unchecked)
    • Use Legacy MQTT 3.1 support: (Checked)
  • Security
    • Username: enfieldlibrary_iot_trial
    • Password: (application-key copied from TTN website, starts with ‘ttn-account-v2’)

The topic is the MQTT channel to subscribe to where ‘+’ is a wildcard and will match any device name.

enfieldlibrary_iot_trial/devices/+/up

There are a range of channels defined in the The Things Network API[1] covering different types of messages. Eg. up, down.

Node Decription

MQTT CLIENT

This node connects to The Things Network and subscribes to the defined topic. Any data sent by our LoRaWAN devices will be sent out this node.

JSON

This node converts a text payload to JSON format. This is useful for allowing later nodes to access individual parts of the payload.

SWITCH

Diverts messages between different  processing flows. In the above flow, it is being used to send device data to different dashboard labels. This allows us to display the latest data coming from each sensor.

FUNCTION

Code can be written in JavaScript to manipulate the messages passed to the node.

In our example we take the data transmitted by our device and received by The Things Network (eg. Sensor: 0x0267)[2], extract the data (substring) and convert to a meaningful number (moisture). This is passed directly to the next node in the payload of the message (the ‘msg’ variable).

var payload = msg.payload.payload_fields.receivedString
payload = payload.substring(10)
var integer = parseInt(payload,16)
var moisture = integer/1024.0 * 100.0
msg.payload = moisture

// To allow plotting of multiple series
msg.label = "sensor-1"

return msg;

DASHBOARD LABEL

Creates label elements to display on the dashboard. These are updated by messages.

DASHBOARD GRAPH

Displays a graph from data. By default, the data is not permanantly stored and will not persist between flow deployments. The graph data can be stored persistantly by including save and restore nodes.

PERSIST (SAVE DATA / RESTORE DATA)

These nodes are used to ensure that the graphed data is persistent between reboots. Under particular conditions the date is written to a non-volatile location, and is read back into the system when starting up.

Multiple sets of data can be saved, but the save and restore nodes need to match.

Notes:
[1] https://www.thethingsnetwork.org/docs/applications/mqtt/api.html
[2] For the workshop there was a deliberate choice made to use ASCII strings to encode the data. This is inefficient, but allows the data to be easily viewed for training and debugging purposes. Have a look at Cayenne Low Power Payload (Cayenne LPP) for one encoding method to improve this.

Summary

The nodes above can be used in Node-RED to receive, display, graph and store data from LoRaWAN enabled sensors. It is a general purpose system and as such is missing some of the whistles and bells of some other systems.

Improvements can be made by hooking in some additional software packages. In particular, data can be stored in a database specifically designed to hold time series of sensor data (eg. InfluxDB[1]) and a more functional dashboard can be added for better graphing and analysis (eg. Grafana[2]).

Notes:
[1] https://www.influxdata.com/
[2] https://grafana.com/