Why LoRaWAN, and what makes it different from other types of low power consumption, high range wireless protocols like ZigBee, Z-Wave, etc … ?
LoRa is a wireless modulation for long-range, low-power, low-data-rate applications developed by Semtech. The main features of this technology are the big amount of devices that can connect to one network and the relatively big range that can be covered with one LoRa router. One gateway can coordinate around 20’000 nodes in a range of 10–30km. It’s a very flexible protocol and allows the developers build various types of network architectures according to the demand of the client. The general description of the LoRaWAN protocol together with a small tutorial are available in my previous post.
What is the ELK stack, and why use it with LoRaWAN?
In the figure above, you can see a simplified model of what a typical LoRaWAN network looks like.
As you can see, the data from the LoRa endpoints, has to go through several devices before it reaches the back-end application. Nowadays there are a lot of tools that would allow us to gather and manipulate the data. A very good solution is the ELK stack which consists of Elasticsearch, Logstash and Kibana; these three tools allow to gather, store and analyze big amounts of data. More information and details can be found on the official website: https://www.elastic.co/.
Mainly, in this post I will describe how to set up a private LoRa network, and will provide details about each module between the sensor data acquisition and the data analysis in Kibana. I will describe two possible architectures: one which uses a free public cloud server, and another one which involves a private server or PC.
The Sodaq Mbili with the RN2483 are the LoRa end devices, which can read sensor data and send it to the gateway. A tutorial on how to set them up is available in my previous post (link in the introduction) or on the official support page: http://support.sodaq.com/mbili/.
In this example I am using the ic880a-spi LoRa gateway from IMST GmbH, which is one of the cheapest and practical solutions for setting up a private gateway. The documentation can be found at: http://www.wireless-solutions.de/products/long-range-radio/ic880a.
It is recommended to use Raspbian Jessie Lite with no desktop graphical interface. The SPI interface has to be enabled; this can be done in the “raspi-config” under the “Advanced” sub menu.
The gateway can either store the data locally, send it to a private server, or send it to The Things Network (TTN) public network. First I will show how to connect to the free TTN service.
There are two ways, you can either download the basic packet forwarder and install it on the rpi: https://github.com/LoRa-net/packet_forwarder/tree/master/basic_pkt_fwd and configure it manually, or you can simply download the installer provided by TTN at: https://github.com/ttn-zh/ic880a-gateway/tree/spi. Simply run the ./install.sh api and then follow the instructions on the terminal. Make sure that the Rpi is connected to the internet before running the installation.
After the installation the Rpi will reboot, and the gateway service will always run in the background. At this point we created a public LoRaWAN gateway, that can be used by any node in the range; later in this post will explain how to make the network private and secure.
If there is no data on the URL, you can try running the packet_forwarder script manually.
As in the previous steps, make sure to use your own devaddr in the url.
“data_json” – the same data, but in a number format.
Kibana allows to read this data in various ways. For this example I created three simple line charts that would show the sensor data, the signal to noise ration and RSSI over time.
In here you can see the data from a sound sensor in the Trifork office in Eindhoven over an hour, together with the signal to noise ratio and RSSI. The dashboard can be customized in any way possible, and multiple nodes can be added.
The main benefit of using the free TTN cloud server is that you don’t need a personal server with static IP address that has to run 24/7. On the other hand, by using their server, the data becomes public, and can be accessed by anyone. In addition, anyone can use the same devaddr for the nodes and fill your buffer with unnecessary data.
And the run the ./start.sh from the same folder. Now the gateway is sending JSON packages to your server.
As you can see there is no filter applied, which means that all the information will be redirected. The gateway sends json packages about the gateway’s status, periodical acknowledgements and the data received from the nodes. More information can be found in the PROTOCOL.TXT file which can be found in the Rpi’s folder /opt/ttn_gateway/packet_forwarder.
In order to get only the node’s data, you need to apply a filter in logstash, or choose a specific search index in Kibana. In order to visualize the data in kibana, simply choose logstash* as the search index.
The last step is to decode the node data. In the previous setup, the data was decoded by TTN framework, but in this case you have to decode it using a base64 decoder. The encryption keys can be set up in the Arduino code by changing the variables netwskey and appskey. The decryption can be done with logstash using the cipher plugin.
thanks for your tutorial .
i’m working on the same project using IC880A-SPI with RN2483 End-point .
I want receive data on my PC SERVER without TTN. I tested with TTN and it work but i want receive data directly on my server .
But not working , i see the payload received but the data wasn’t included .
In TTN Forum , they said that we have to receive an ACK from server to join network and the data transmitted , you think that with Logstash can do it ?