A DIY low-cost LoRa gateway

C. Pham, LIUPPA laboratory, University of Pau, France.  http://web.univ-pau.fr/~cpham

last update: September 28th, 2016.

Follow this link for building end-devices

Get the step-by-step tutorial on how to build the low-cost LoRa gateway

Introduction

This page describes our low-cost LoRa gateway based on a Raspberry PI. The gateway can receive from any LoRa device and is designed to be fully customizable for a targeted application with post-processing features based on high-level languages such as python. Typical post-processing features are to push the received data on various IoT/cloud platforms. Currently, we provide example for DropBoxTM, FirebaseTM, ThingSpeakTM, SensorCloudTM, GroveStreamsTM, FIWARE.

The work presented here is part of the EU H2020 WAZIUP project (grant agreement number 687607, 2016-2019) which objective is to develop low-cost IoT solutions for deployment in sub-saharian African countries. Various applications are considered: water quality monitoring, cattle rustling, logistics and goods transportation. More details will come soon, but right now you can get the presentation of the developped gateway in .pdf.

There are many advanced and well-integrated LoRa gateways capable of simultaneous reception on several channels and implementing the LoRaWAN specification (see slides). These gateways are based on the SX1301 baseband concentrator. Our LoRa gateway could be qualified as "single connection" as it uses the SX1272, much like an end-device would do. However, in order to increase LoRa transmission robutsness we improve the LoRa transmission with CSMA features (or so-called Listen Before Talk) and add Quality of Service guarantees with regards to radio time limitations. This solution keeps the cost of the gateway low and can satisfy small to medium size deployment scenario for ad-hoc application cases in various private usages, farming, agriculture, infrastructure surveillance, application-specific telemetry systems,... Note that more than 1 gateway can be deployed to serve several channel settings. However, it is probably not adapted, in the current state of development, to large-scale deployment with a large number of end customers from various different organizations with their own and different requirements regarding data management, confidentiality and security.

Download: github (drop me a mail if you use our development so that we could advertise it)

  1. Raspberry folder
    1. the modified SX1272 library (initial version comes from Libelium) with enhanced features: support of SX1276, LBT & CSMA-like, ...
    2. the arduPi library for RPI1 and RPI2
    3. the lora_gateway .cpp code (which compile on both RPI and Arduino)
    4. the post-processing python scripts (some parts need your own customization for cloud access: firebase, ThingSpeak, FIWARE, freeboard,...)
    5. the makefile

  2. Arduino folder
    1. the modified SX1272 library (initial version comes from Libelium) with enhanced features: support of SX1276, LBT & CSMA-like, ...
    2. the lora_gateway .ino code (which compile on both RPI and Arduino) that is also the code for the interactive end-device
    3. the code of a temperature sensor with an Arduino (tested on Uno, Mega, Due, Pro Mini (with some modifications)), performing CS/LBT. Should work out-of-the box with the gateway

  3. a README.md for some installation/compilation instructions

  4. our FAQ, explaining our LoRa framework, and especially why we are LoRaWAN compatible

Download: full SD card zipped image for the Raspberry gateway (based on Raspbian Jessie)

  1. Supports Raspberry 1B+, RPI2 and RPI3.
  2. Includes all the advanced features described in the gw_advanced github
  3. Get the zipped image, unzip it, install it on an 8GB SD card, see this tutorial from www.raspberrypi.org
  4. Plug the SD card into your Raspberry
  5. Connect a radio module (see below)
  6. Power-on the Raspberry
  7. The LoRa gateway starts automatically when RPI is powered on
  8. With an RPI3, the Raspberry will automatically act as a WiFi access point. For RPI 1&2, see instructions on github
  9. By default, incoming data are uploaded to the WAZIUP ThingSpeak demo channel
  10. Works out-of-the-box with the Arduino_LoRa_Simple_temp sketch

Hardware components

The gateway is based on a Raspberry PI. RPI 1B+/2B/3B can be used. The LoRa modules comes from (a) Libelium LoRa radio module, (b) HopeRF RFM92W or HopeRF RFM95W, (c) Modtronix inAir9 or inAir9B, (d) NiceRF LoRa1276. Libelium LoRa and  RFM92W use the Semtech SX1272 chip while RFM95W, inAir9/9B and NiceRF LoRa1276 use the SX1276 which is actually more versatile (can be used also in the 433MHz band).


Figure: supported (tested) radio modules


Figure: Left: Hardware components with a Modtronix inAir9 radio module. Right: RPI GPIO header for RPI 1B (short) and RPI 2B/3B (long)


Figure:GPIO header of the RPI

Using Libelium Lora

For the Libelium LoRa module, we directly connected the LoRa module without the connection bridge developed by Libelium to save the extra cost of the connection bridge, by just connecting the required SPI pins (MISO, MOSI, SPI_CLK and SPI_nSSEL), VCC and GND to the corresponding pins on the RPI (CE0 on the RPI for SPI_nSSEL and 3v3 for VCC). Pin out diagrams for the LoRa module in XBee format is shown below. The LoRa module from Libelium is however quite expensive: around 45.

   
Figure: XBee pin-out diagram for the Libelium LoRa module

We solder the wires to the pin as shown in the figure below. If the RPI is put in a case for outdoor usage, the radio module could just be fixed with the antenna or an SMA extension cable could be used. At the Raspberry side, you can simply plug the right cable end to the corresponding GPIO pin.


Figure: using two 10-pin 2mm female socket to connect the LoRa module that has an XBee format. Right: the module is seen from the back side


Figure: connect the LoRa radio module to the RPI GPIO header

Using HopeRF RFM92W/RFM95W

Now, the HopeRF RFM92W (RFM95W) module is shown below. An adapter is need to have the break-out pins (the RFM92W module is quite small) and most importantly the antenna plug (which is in Female SMA). Note that the latest news from HopeRF indicated that the RFM92W is discontinued because the RFM95W is better. So you probably will have an RFM95W if you buy them now.

       
Figure: the RFM92W module (left), the RFM92W adapter (middle), the RFM92W and the adapter before soldering, comparison with the Libelium LoRa for size (right)

After some delicate soldering, we have the RFM92W module ready to be plugged on our Raspberry like previously: just connect the right cable to the GPIO header or use an additional header for fast insertion/removal of the radio module.

       
Figure: the RFM92W and the adapter, after soldering (left), ready to be plugged on the Raspberry (middle), the adapter pin-out (right)

The RFM92W (or RFM95W) with the adapter costs less than half of the Libelium LoRa (around 15) which makes it quite attractive for our low-cost LoRa gateway. As both use the native SPI communication with the SX1272, the Libelium library can also drive the RFM92W, as explained in next section. See our low-cost gateway based on the RFM92W/95W at the end of the page.

Using Modtronix inAir9

We also tested with the Modtronix inAir9 which is based on the SX1276. This module has 2 advantages: it costs less than half of the Libelium LoRa (around 15) and can come with the header pins already soldered! The left figure shows the radio module and the right figure shows the pin out.

   

To connect the inAir to the Raspberry, proceed as previously: just connect the right cable to the GPIO header or use an additional header for fast insertion/removal of the radio module. Our first tests show that the inAir is reliable. Given its low cost and readiness (the RFM92W/RFM95W need some soldering) it is definitely a good choice. Note that there is an inAir9B with +20dBm transmit power but regulation is quite strict on such transmit power usage.

Important notice for the SX1272 library

Note that the original Libelium library to drive the LoRa module does not use DIO pins as many other libraries do, so there is no need to connect these pins. You can use other development codes using DIO pins by connecting the required pins that are mostly configured as follows DIO0 (RXdone or TX done), DIO1 (RX timeout) and DIO5 (ready). Of course, you have to check first. Some may also use the RST pin to reset the module.

Main architecture

Initially, the gateway was implemented on an Arduino (MEGA/Due) for test purpose and for having a "direct" transparent radio bridge. We enhanced the gateway code but maintained compatibility with Arduino therefore the main features are available on both platforms. On the RPI, we use the arduPI layer provided by Libelium to run both the SX1272 library and the gateway program. The original SX1272 library has been significantly improved to support also the SX1276, to add CSMA-like capability to increase LoRa efficiency and implement the possibility to dynamically ask for an ACK from the receiver side. This library basically drives the SX1272 through the SPI interface (it then works with no modifications with the HopeRF RFM92W and has some modifications for the SX1276). The most important point to mention is that the original library adds 5 bytes for internal usage as shown is the following figure, taken from the Libelium documentation.

dst addr (1 byte)
src addr (1 byte)
sequence number (1 byte)
payload length (1 bytes)
payload data (variable length)
retry counter (1 byte)

The dst addr allows the library to filter packet at the receiver by comparing with the end-device addr that should be set at startup. Therefore, the current maximum number of nodes is 254 (1 is usually used for the gateway and 0 is usually used to indicate broadcast).

Our SX1272/76 modified library uses a modified header where both the packet length and retry field are removed (packet length will be determined at receiver side). A packet type field has been added to ease the decoding process.

dst addr (1 byte)
packet type (1 byte)
src addr (1 byte)
sequence number (1 byte)
payload data (variable length)

The library is simple enough for you to modify it according to your need: more address space, longer sequence number, network/app/device key,...

The packet type structure is as follows, for the moment Ptype = 1 (B0001) for a DATA packet and Ptype = 2 (B0010) for an ACK packet:

Ptype (4 bits)
ack_requested flag (1 bit)
data_encrypted flag (1 bit)
with_appkey flag (1 bit)
is_binary flag (1 bit)

Now, the philosophy of our proposed gateway is to keep the gateway code very simple while having advanced post-processing features implemented by end-user with high-level languages such as python. We provide a post_processing_gw.py script with several cloud features to demonstrate our approach and to serve as a starting point for your own development needs.

Figure: Main architecture of the gateway and post-processing feature

Basically, the gateway prints on UNIX stdout everything that it receives from the end-devices. When receving a packet, the source address, the sequence number, the payload length, the SNR and the RSSI are printed.

The gateway LoRa parameters can be configured remotely with ASCII control sequences. We use prefix '/@' to indicate a remote control packet. The command should be terminated with an '#'. Available commands are:

/@M1#: set LoRa mode 1
/@C12#: use channel 12 (868MHz)
/@PL/H/M/x/X#: set power to Low, High, Max, extreme (PA_BOOST), eXtreme (+20dBm) if available
/@A9#: set node addr to 9

Before the gateway accepts to be configured remotely, the '/@Upin_number#' command must be issued. For instance, if the pin number is 1234 then to unlock the gateway, command '/@U1234#' should be sent to the gateway. To lock again, just issue the same command. There is a limited number of trials before the gateway is completely blocked for remote configuration. You can change all these settings by simply recompiling the gateway program.

The Libelium SX1272 library defines 10 so-called LoRa mode that use various combinations of LoRa parameters: bandwidth, spreading factor, coding rate. Figure below shows these combinations and the time on air associated to them. The payload value includes the 5-bytes header.


Figure: Defined Libelium LoRa mode and their time on air

Simple gateway

After compiling the lora_gateway program, the most simple way to start the gateway is in standalone mode as shown below. The default gateway configuration is to use Libelium LoRa mode 4 (865.2MHz with BW=500kHz, SF=12, CR=4/5) at maximum transmission power (not +20dBm). The preamble length is 8 and the address is 1.


Figure: Gateway running in standalone mode

In the example above, let's assume that sensor 10 sends "T=23". Then the gateway will print:

--- rxlora. dst=1 type=0x10 src=10 seq=0 len=5 SNR=5 RSSIpkt=-54
^p1,16,10,0,5,5,-54
^r500,5,12
^t2016-02-17T19:56:17.121
T=23

The part in red is what the end-device has actually sent. Lines "^p1,16,10,0,5,5,-54", "^r500,5,12" and "^t2016-02-17T19:56:17.121" summarizes information for the received packet in a condensed manner that can be further exploited by the post-processing stage as it will be shown later on.  Note that the packet type is expressed in decimal in the ^p string.

The gateway can actually be launched on another Libelium mode by using the --mode option in the command line:

> sudo ./lora_gateway --mode 2

Adding post-processing

Our approach is to enable information passing between the gateway program (basically running in a transparent manner) and post-processing operation implemented by end-users in a high-level language. We provide a simple post_processing_gw.py python script to demonstrate how advanced and complex post-processing tasks can be realized. Note how the gateway is launched now:

> sudo ./lora_gateway | python ./post_processing_gw.py

The first example for post-processing is for instance getting and exploiting information on the last received packet.

Getting information for the last received packet

As indicated previously, the gateway prints "^p1,16,10,0,5,5,-54" to indicate that the destination is 1 (the gateway), the packet type is 0x10 (DATA packet), source is 10, the sequence number if 0, the data length is 5, the SNR is 5 and the RSSI is -54. Our provided post-processing program looks for special, well-defined prefix sequences. The special sequence '^p' gives information on the last received packet. In the figure below, "^p1,16,10,0,5,5,-54" is issue by the gateway program, but intercepted by post_processing_gw.py therefore it is not shown. Instead, post_processing_gw.py displays:

rcv ctrl pkt info (^p): 1,16,10,0,5,5,-54
(dst=1 type=0x10 src=10 seq=0 len=5 SNR=5 RSSI=-54)

"^r500,5,12" means that the LoRa modulation uses bandwidth of 500kHz, coding rate of 4/5 and spreading factor of 12. "^t2016-02-17T19:56:17.121" indicated the received packet's timestamp in ISO format.

All lines that are not prefixed by some special sequence are displayed unchanged by post_processing_gw.py.


Figure: Getting information for last received packet

Indicating advanced data management

Special prefix sequences can be inserted by the end-devices to be intercepted and acted upon by post_processing_gw.py. We demonstrate the flexibility of this approach by intercepting and interpreting some prefixes: '^$', '\$', '\&', '\!'

We chose the following logic: '^' will indicate a prefix from the gateway program and '\' indicates a prefix inserted by end-devices. Then we chose '$' for Dropbox redirection, '&' for a Firebase redirection and '!' for a ThingSpeak redirection. You can add your own.

Dropbox

In the following figure, we show sensor 10 sending "\$T=23" that indicates redirection to a Dropbox shared file. The prefix '\$' for Dropbox makes post_processing_gw.py to write in "telemetry.log" file. In order to have Dropbox support on the RPI, check this page. This is the solution we chose. There is also an Dropbox-uploader solution but it has not been tested. With Dropbox, you can have access to these files accross the Internet and on all your Dropbox devices, and of course your smartphone.


Figure: Using prefix from end-devices

post_processing_gw.py will display the following information:

--- rxlora. dst=1 type=0x10 src=10 seq=0 len=5 SNR=5 RSSIpkt=-54
rcv ctrl pkt info (^p): 1,16,10,0,5,5,-54
(dst=1 type=0x10 src=10 seq=0 len=5 SNR=5 RSSI=-54)
rcv msg to log (\$) on dropbox: T=23

When the Dropbox connection is setup, here is a sample of the "telemetry.log" file when using '\$' prefix. Note that you can log any information because we currently only use string format when sending from an end-device.

(src=10 seq=0 len=5 SNR=9 RSSI=-54) 2015-11-04T22:43:04.091803> T=23
(src=3 seq=0 len=5 SNR=8 RSSI=-54) 2015-11-04T22:43:04.091947> H=85%
(src=10 seq=1 len=62 SNR=9 RSSI=-54) 2015-11-18T13:34:48.351417> indicates that line should be logged in a file (telemetry.log)
(src=10 seq=2 len=5 SNR=9 RSSI=-54) 2015-11-18T13:34:51.354996> T=23
(src=10 seq=3 len=62 SNR=9 RSSI=-54) 2015-11-18T13:35:17.809479> indicates that line should be logged in a file (telemetry.log)

The gateway program also uses '^$' to identify important messages that should be logged in a Dropbox file. post_processing_gw.py writes these information in "gateway.log". For instance, information at gateway startup is prefixed by '^$' so that these information can be logged. In this way, we can have a trace of important operation performed by the gateway from anywhere and on any of your Drobox device. If you need to have additional information from the gateway to be prefixed by '^$' you have to modify the gateway program and recompile it.

post_processing_gw.py will display the following information for instance:

rcv gw output to log (^$): **********Power ON: state 0
rcv gw output to log (^$): LoRa mode: 4
rcv gw output to log (^$): Setting mode: state 0
rcv gw output to log (^$): Channel CH_10_868: state 0
rcv gw output to log (^$): Power M: state 0
rcv gw output to log (^$): Get Preamble Length: state 0
rcv gw output to log (^$): Preamble Length: 8
rcv gw output to log (^$): LoRa addr 1 : state 0
rcv gw output to log (^$): SX1272/76 configured as LR-BS. Waiting RF input for transparent RF-serial bridge

And here is a sample of "gateway.log" showing 2 reboots of the gateway and 3 unsuccessful tentative to unlock remote configuration feature:

2015-11-06T17:24:24.945577> **********Power ON: state 0
2015-11-06T17:24:24.956489> LoRa mode: 4
2015-11-06T17:24:24.971446> Setting mode: state 0
2015-11-06T17:24:24.987483> Channel CH_10_868: state 0
2015-11-06T17:24:25.014320> Power M: state 0
2015-11-06T17:24:25.022156> Get Preamble Length: state 0
2015-11-06T17:24:25.029817> Preamble Length: 8
2015-11-06T17:24:25.045820> LoRa addr 1 : state 0
2015-11-06T17:24:25.074857> SX1272/76 configured as LR-BS. Waiting RF input for transparent RF-serial bridge
2015-11-06T17:27:33.346322> Parsing command
2015-11-06T17:27:33.370780> /@U1235#
2015-11-06T17:27:58.291488> Parsing command
2015-11-06T17:27:58.314954> /@U1235#
2015-11-06T17:28:08.232510> Parsing command
2015-11-06T17:28:08.247389> /@U1235#
2015-11-06T17:28:08.258191> Bad pin
2015-11-06T17:28:42.741779> Parsing command
2015-11-06T17:28:42.757910> /@U1235#
2015-11-06T17:28:42.774729> Remote config locked
2015-11-13T13:59:14.393888> **********Power ON: state 0
2015-11-13T13:59:14.407369> LoRa mode: 4
2015-11-13T13:59:14.413498> Setting mode: state 0
2015-11-13T13:59:14.432380> Channel CH_10_868: state 0
2015-11-13T13:59:14.459327> Power M: state 0
2015-11-13T13:59:14.466547> Get Preamble Length: state 0
2015-11-13T13:59:14.488392> Preamble Length: 8
2015-11-13T13:59:14.510941> LoRa addr 1 : state 0
2015-11-13T13:59:14.521834> SX1272/76 configured as LR-BS. Waiting RF input for transparent RF-serial bridge
2015-11-13T14:00:12.705857> ACK requested by 10
2015-11-13T14:03:36.455707> ACK requested by 10
2015-11-13T14:05:19.096071> ACK requested by 10

Note: in order to enable Dropbox gateway information logging, you should provide the -L or --loggw option:

> sudo ./lora_gateway | python ./post_processing_gw.py -L 

Firebase

In the following figure, we show how the '\&' prefix triggers a POST on a Firebase database. post_processing_gw.py uses the python-firebase package. Check this page for more information. The result comes from the '\&H=85%' message that indicates redirection to a Firebase database.


Figure: posting on Firebase

post_processing_gw.py will display the following information:

--- rxlora. dst=1 type=0x10 src=6 seq=0 len=8 SNR=3 RSSIpkt=-51
rcv ctrl pkt info (^p): 1,16,6,0,8,3,-51
(dst=1 type=0x10 src=6 seq=0 len=8 SNR=3 RSSI=-51)
rcv msg to log (\&) on firebase: H=85%

Note: you must have a Firebase account with a valid Firebase database identifier. Also, in order to enable Firebase access, you should provide the -f or --firebase option:

> sudo ./lora_gateway | python ./post_processing_gw.py -f

ThingSpeak

In the following figure, we show how the '\!' prefix triggers a POST on a ThingSpeak channel. post_processing_gw.py uses curl to issue POST commands.


Figure: Posting on ThingSpeak

post_processing_gw.py will display the following information:

--- rxlora. dst=1 type=0x10 src=3 seq=13 len=10 SNR=4 RSSIpkt=-60
rcv ctrl pkt info (^p): 1,16,3,13,10,4,-60
(dst=1 type=0x10 src=3 seq=13 len=10 SNR=4 RSSI=-60)
rcv msg to log (\!) on ThingSpeak ( default , 1 ): 19.6
ThingSpeak: will issue curl cmd
curl -s -k -X POST --data field1=19.6&field5=13 https://api.thingspeak.com/update?key=AAAAAAAAAAAAAAAA
ThingSpeak: returned code from server is 1232

Note: you need to have a ThingSpeak channel with a valid write key. This write key will replace the part in red shown above.  The end-device has actually to send '\!write_key#field_index#20.1. write_key is your ThingSpeak channel write key. Then field_index is a number between 1 and 8 for the corresponding field in the channel. If you omit both the write_key and the field_index (sending '\!##20.1' for instance) then a default write_key is indicated in post_processing_gw.py for this case and the field index will be 1. For test purposes, we provide a test channel (default channel) whose write key is SGSH52UGPVAUYG3S. You can therefore test right away your gateway. In order to enable ThingSpeak feature, you should provide the -t or --thingspeak option:

> sudo ./lora_gateway | python ./post_processing_gw.py -t

See the ThingSpeak channel for 2 temperature LoRa devices (sensor3 and sensor10) sending data every 10 minutes to the gateway. We will keep it running to test the gateway. See temperature from one of the sensor below.


FiWare

We also tested support of the FIWARE platform and post_processing_gw.py can simply call FIWARE python script to create/update entities on the platforms. Script have been provided by Easy Global Market, a FIWARE partner. Last temperature data from sensor3 and sensor10 can be seen on freeboard IoT cloud platform.


Figure: using freeboard IoT cloud

Here are screenshoots from a smartphone:

   
Figure: real time data displayed on ThingSpeak and freeboard IoT (using data from FIWARE Orion broker)

Note: you need to have a FIWARE account. In order to enable FIWARE, you should provide the --fiware option:

> sudo ./lora_gateway | python ./post_processing_gw.py --fiware

Other cloud platforms

We also tested with other cloud platforms such as SensorCloud and GroveStreams. As most platforms provides REST API access it is quite easy to add the connection in the post-processing stage.


Figure: using SensorCloud


Figure: using GroveStreams

Running multiple gateways

We actually have 2 gateways, one is tested in operational condition and collects data from sensor 3 and sensor10. It runs on LoRa mode 4 (BW=500kHz, SF=12, CR=5). Another gateway is running on mode 2 (BW=250kHz, SF=12, CR=5) and is used for other tests. With multiple gateways, you can specify a different "index" for log files (gateway.log, telemetry.log for instance) in case these files are written in the same (shared) folder.

> sudo ./lora_gateway | python ./post_processing_gw.py -t -f -L -a 2

Logging all outputs, including post-processing, from the gateway

In order to log all outputs from the gateway, including those from the post-processing stage, we just add another redirection level. We provide a log_gw.py script that simply takes everything from stdout, logs it on Dropbox in a file called post-processing.log and writes again to stdout for display. Example:

 > sudo ./lora_gateway | python ./post_processing_gw.py -t -f -L | python ./log_gw.py

Here is a sample of such a file from our gateway that collects temperature information from 2 LoRa temperature sensors for our ThingSpeak channel. The ThingSpeak channel write key has been replaced with AAAAAAAAAAAAAAAA. Sensor 3 writes on field 1 and sensor 10 on field 2.

2015-11-20T14:28:36.668501> rcv gw output to log (^$): **********Power ON: state 0
2015-11-20T14:28:36.671743> rcv gw output to log (^$): LoRa mode: 4
2015-11-20T14:28:36.698168> rcv gw output to log (^$): Setting mode: state 0
2015-11-20T14:28:36.741636> rcv gw output to log (^$): Channel CH_10_868: state 0
2015-11-20T14:28:36.800036> rcv gw output to log (^$): Power M: state 0
2015-11-20T14:28:36.813887> rcv gw output to log (^$): Get Preamble Length: state 0
2015-11-20T14:28:36.828750> rcv gw output to log (^$): Preamble Length: 8
2015-11-20T14:28:36.846557> rcv gw output to log (^$): LoRa addr 1 : state 0
2015-11-20T14:28:36.858616> rcv gw output to log (^$): SX1272/76 configured as LR-BS. Waiting RF input for transparent RF-serial bridge
2015-11-20T14:30:21.144953> --- rxlora. dst=1 type=0x10 src=3 seq=0 len=11 SNR=4 RSSIpkt=-59
2015-11-20T14:30:21.153355> rcv ctrl pkt info (^p): 1,16,3,0,11,4,-59
2015-11-20T14:30:21.172314> splitted in:  [1, 1, 3, 0, 11, 4, -59]
2015-11-20T14:30:21.189229> (dst=1 type=1 src=3 seq=0 len=11 SNR=4 RSSI=-59)
2015-11-20T14:30:22.206677> rcv msg to log (\!) on ThingSpeak ( default , 1 ): 22.94
2015-11-20T14:30:22.215569> will issue curl cmd
2015-11-20T14:30:22.255881> curl -s -k -X POST --data field1=22.94&field5=0 https://api.thingspeak.com/update?key=AAAAAAAAAAAAAAAA
2015-11-20T14:30:22.271261> returned code from server is 26
2015-11-20T14:30:22.298647>
2015-11-20T14:30:46.475160> --- rxlora. dst=1 type=0x10 src=10 seq=0 len=11 SNR=4 RSSIpkt=-66
2015-11-20T14:30:46.484312> rcv ctrl pkt info (^p): 1,16,10,0,11,4,-66
2015-11-20T14:30:46.501523> splitted in:  [1, 16, 10, 0, 11, 4, -66]
2015-11-20T14:30:46.518002> (dst=1 type=0x10 src=10 seq=0 len=11 SNR=4 RSSI=-66)
2015-11-20T14:30:47.586317> rcv msg to log (\!) on ThingSpeak ( default , 2 ): 24.41
2015-11-20T14:30:47.599120> will issue curl cmd
2015-11-20T14:30:47.612896> curl -s -k -X POST --data field2=24.41&field6=0 https://api.thingspeak.com/update?key=AAAAAAAAAAAAAAAA
2015-11-20T14:30:47.631814> returned code from server is 27
2015-11-20T14:30:47.657144>
2015-11-20T14:41:10.114686> --- rxlora. dst=1 type=0x10 src=3 seq=1 len=11 SNR=5 RSSIpkt=-57
2015-11-20T14:41:10.123342> rcv ctrl pkt info (^p): 1,16,3,1,11,5,-57
2015-11-20T14:41:10.139994> splitted in:  [1, 16, 3, 1, 11, 5, -57]
2015-11-20T14:41:10.157897> (dst=1 type=0x10 src=3 seq=1 len=11 SNR=5 RSSI=-57)
2015-11-20T14:41:11.156688> rcv msg to log (\!) on ThingSpeak ( default , 1 ): 22.46
2015-11-20T14:41:11.165119> will issue curl cmd
2015-11-20T14:41:11.181350> curl -s -k -X POST --data field1=22.46&field5=1 https://api.thingspeak.com/update?key=AAAAAAAAAAAAAAAA
2015-11-20T14:41:11.228312> returned code from server is 28
2015-11-20T14:41:11.237319>
2015-11-20T14:41:36.516028> --- rxlora. dst=1 type=0x10 src=10 seq=1 len=11 SNR=4 RSSIpkt=-68
2015-11-20T14:41:36.525644> rcv ctrl pkt info (^p): 1,16,10,1,11,4,-68
2015-11-20T14:41:36.541918> splitted in:  [1, 16, 10, 1, 11, 4, -68]
2015-11-20T14:41:36.563111> (dst=1 type=0x10 src=10 seq=1 len=11 SNR=4 RSSI=-68)
2015-11-20T14:41:37.544064> rcv msg to log (\!) on ThingSpeak ( default , 2 ): 21.48
2015-11-20T14:41:37.552806> will issue curl cmd
2015-11-20T14:41:37.598217> curl -s -k -X POST --data field2=21.48&field6=1 https://api.thingspeak.com/update?key=AAAAAAAAAAAAAAAA
2015-11-20T14:41:37.605706> returned code from server is 29
2015-11-20T14:41:37.617876>

Once again, if you run multiple gateways, you can issue the following command:

 > sudo ./lora_gateway | python ./post_processing_gw.py -t -f -L -a 2 | python ./log_gw.py -a 2

Tools for testing & build a LoRa gateway for on-the-go tests

We describe here some hints for you want to do some range tests or add&test new functionalities into the post-processing stage.

Use an Arduino-based gateway

As mentioned previously, the lora_gateway code can be compiled and uploaded to an Arduino board (Uno/MEGA/Due). If you connect the Arduino gateway to your laptop all outputs will be sent through the serial port.


You can use any serial tool to get data from the Arduino gateway. We use the very convenient serial monitor of the Arduino IDE that allows us to easily send command to the Arduino gateway through the serial port for configuring various LoRa parameters as described previously (when these commands come from serial line, the gateway does not ask for the unlock pin):

/@M1#: set LoRa mode 1
/@C12#: use channel 12 (868MHz)
/@PL/H/M#: set power to Low, High or Max
/@A9#: set node addr to 9

With the gateway connected to the laptop and output displayed by a serial tool, you will see all received frame as in the RPI gateway standalone mode described previously.


Figure: screenshot of the Arduino IDE with the Arduino gateway outputs

If you connect the Arduino gateway and use a python script to forward data from serial port to stdout (see for instance our SerialToStdout python script available in 38400 or 115200 baud versions from our test-bed page) you can have an Arduino-based gateway with full post-processing features (executed on your laptop) provided that your laptop has Internet connectivity. For field tests, I use my smartsphone 3G/4G connection to get Internet access on the laptop. In this way you can have a LoRa gateway on-the-go for your tests.

> python 38400SerialToStdout.py /dev/ttyUSB0 | python ./post_processing_gw.py -t -f -L | python ./log_gw.py


Figure: Arduino gateway with full post-processing features

Use the Raspberry gateway

You can use the DIY Raspberry gateway connected to your laptop as well. In this case, the laptop is use to provide the power and you would simply launch the Raspberry gateway by opening an ssh session. You can just use a cross-over Ethernet cable to connect the Raspberry gateway to you laptop which is supposed to have Internet connectivity for full post-processing features. If your laptop has Dropbox, your Raspberry can have it as well as indicated earlier. For field tests, I use my smartsphone 3G/4G connection to get Internet access on the laptop, then share this connection with the Raspberry.

End-device for sending LoRa messages

We also developped a simple end-device that will allow you with the Arduino IDE to enter ASCII string for transmission to the gateway (by default, the destination address is 1). It waits continuously for input from serial port, then send the message. With this end-device you can dynamically send prefixed messages to test the various post-processing tasks. You can perform range tests when you request an ACK from the gateway. You can send remote control commands to the gateway (by providing the unlock pin) to change its LoRa parameters (be sure to be able to communicate with it again).

In addition, the end-devices accepts the following command for its configuration. We use prefix '/@' to indicate a command packet. The command should be terminated with an '#'. Available commands are:

/@M1#: set LoRa mode 1
/@C12#: use channel 12 as defined in the SX1272.h file (868MHz band)
/@PL/H/M/x/X#: set power to Low, High, Max, extreme (PA_BOOST), eXtreme (+20dBm)
/@A9#: set node addr to 9
/@ACK#hello w/ack : sends "hello w/ack" and request an ACK
/@ACKON#: enables ACK (for all messages)
/@ACKOFF#: disables ACK
/@CAD#: performs an SIFS CAD, i.e. 3 or 6 CAD depending on the LoRa mode
/@CADON3#: uses 3 CAD when sending data (normally SIFS is 3 or 6 CAD, DIFS=3SIFS)
/@CADOFF#: disables CAD (IFS) when sending data
/@RSSI#: toggles checking of RSSI before transmission and after CAD
/@EIFS#: toggles for extended IFS wait
/@T5000#: send a message at regular time interval of 5000ms. Use /@T0# to disable periodic sending
/@TR5000#: send a message at random time interval between [2000, 5000]ms.
/@Z200#: sets the packet payload size to 200 for periodic sending
/@S50#: sends a 50B user payload packet filled with '#'. The real size is 55B with the Libelium header
/@D56#: set the destination node to be 56, this is permanent, until the next D command
/@D58#hello: send "hello" to node 56, destination addr is only for this message
/@D1#/@M1#: send the command string "/@M1#" to node 1 (i.e. gateway), this would make the gateway to switch to mode 1

For instance, if you want to test the ThingSpeak features with the provided test channel, you can use the end-device to send "\!SGSH52UGPVAUYG3S#1#21.6". If you want to request an ACK from the gateway, you can use "/@ACK#\!SGSH52UGPVAUYG3S#1#21.6".

You can use this interactive end-device to perform range tests with various LoRa modes. For instance you can request an ACK in order to verify that the gateway is correctly receiving your message:

/@ACK#hello w/ack

If the message has been correctly received, you can test again with an other LoRa mode to see if connectivity is still maintained:

/@ACK#/@M2#
/@M2#
/@ACK#hello w/ack

The first command asks the gateway to switch to LoRa mode 2 and request an ACK to be sure that the command has been correctly received. Then the end-device switchs itself to LoRa mode 2 and finally sends the message by requesting an ACK. Don't forget to unlock the gateway in order to be able to remotely configure the gateway. To do this, issue command /@ACK#/@U1234# prior to the /@ACK#/@M2# message.

When requesting an ack from the gateway, the end-device will receive both the SNR of the packet received at the gateway (carried by the ACK) and the SNR of the received ACK. In this way, you can know the quality of the link. The first SNR is 7dB and it is the SNR of the packet received at the gateway, sent by by the gateway in the ACK packet. Then the second SNR (6dB) is the SNR of the received ACK at the end-device.

Rcv serial: /@ACK#hello
^$Parsing command
^$/@ACK#hello
Sending. Length is 5
hello
Payload size is 5
ToA is w/5B Libelium header 281
Packet number 0
wait for ACK

Starting 'getACK'
## ACK received:
Destination: 6
Source: 1
ACK number: 0
ACK length: 2
ACK payload: 0
ACK SNR of rcv pkt at gw: 7
##

--- rxlora ACK. SNR=6 RSSIpkt=-16
LoRa (ACK) Sent in 1586
LoRa Sent w/CAD in 1586
Packet sent, state 0

Locally test&debug post-processing stage

You can write a test text file (test.txt) containing lines similar to :

^p1,16,10,0,5,9,-54
T=23

^p1,16,3,0,5,8,-54
H=85%

^p1,16,10,1,7,9,-54
\$T=23

^p1,16,10,0,4,9,-54
\!SGSH52UGPVAUYG3S#2#13.5

^p1,16,3,0,4,9,-54
\!##27.5

^p1,16,10,2,5,9,-54
hello

^p1,16,3,2,5,9,-54
world

and then issue the following command:

> cat test.txt | python ./post_processing_gw.py -t

Using Network key and Application key to filter out messages to log

When sending data to the gateway, it is possible to add a network key to the SX1272 library's header and check for the key to filter incoming messages.

network key (2 bytes)
dst addr (1 byte)
packet type (1 byte)
src addr (1 byte)
sequence number (1 byte)
payload length (1 bytes)
payload data

Then, at the end-device level an additional application key can be used as prefix to all messages sent by the device. We chose the solution to let the end-device, thus the application owner, to insert before the real application payload the application key (but from the library perspective, it belongs to the payload data. The gateway's behavior therefore remains unchanged and the application key will be passed as regular data to the post-processing stage. You can see the LoRa temperature example provided in download to see how application key is implemented. The current application key is coded on 4 bytes. post_processing_gw.py will then check whether the packet type is 0x12 (that indicates a DATA packet and the with_appkey flag)  to read the application key and find whether the key is a registered application key or not (the list of valid application key is defined in post_processing_gw.py and should match with the application key defined at end-devices).

Note: To enforce application key, --wappkey should be indicated to post_processing_gw.py:

 > sudo ./lora_gateway | python ./post_processing_gw.py -t -f --fiware -L --wappkey | python ./log_gw.py

Here is the post-processing output with application key enabled:

2015-11-30T13:02:39.286530> --- rxlora. dst=1 type=0x12 src=6 seq=136 len=17 SNR=9 RSSIpkt=-56
2015-11-30T13:02:39.291314> rcv ctrl pkt info (^p): 1,18,6,136,17,9,-56
2015-11-30T13:02:39.295361> splitted in:  [1, 18, 6, 136, 17, 9, -56]
2015-11-30T13:02:39.299561> (dst=1 type=0x12 src=6 seq=136 len=17 SNR=9 RSSI=-56)
2015-11-30T13:02:39.304258> got first framing byte
2015-11-30T13:02:39.312637> --> got app key sequence
2015-11-30T13:02:39.316382> app key is: [5, 6, 7, 8]
2015-11-30T13:02:39.320188> in app key list
2015-11-30T13:02:39.333956> valid app key: accept data
2015-11-30T13:02:39.857162> rcv msg to log (\!) on ThingSpeak ( default , 3 ): 20.99
2015-11-30T13:02:39.861032> ThingSpeak: will issue curl cmd
2015-11-30T13:02:39.864619> curl -s -k -X POST --data field3=20.99&field7=136 https://api.thingspeak.com/update?key=AAAAAAAAAAAAAAAA
2015-11-30T13:02:39.868063> ThingSpeak: returned code from server is 2490

If --wappkey is not set, post_processing_gw.py will log all LoRa data prefixed by correct logging prefix (i.e '\$', '\&', '\!'). With --wappkey, all LoRa data logging services will need a valid application key otherwise the data will be discarded as shown below:

2015-11-30T13:02:39.286530> --- rxlora. dst=1 type=0x12 src=6 seq=136 len=17 SNR=9 RSSIpkt=-56
2015-11-30T13:02:39.291314> rcv ctrl pkt info (^p): 1, 18, 6,136,17,9,-56
2015-11-30T13:02:39.295361> splitted in:  [1, 18, 6, 136, 17, 9, -56]
2015-11-30T13:02:39.299561> (dst=1 type=0x12 src=6 seq=136 len=17 SNR=9 RSSI=-56)
2015-11-30T13:02:39.304258> got first framing byte
2015-11-30T13:02:39.312637> --> got app key sequence
2015-11-30T13:02:39.316382> app key is: [9, 10, 11, 12]
2015-11-30T13:02:39.320188> not in app key list
2015-11-30T13:02:39.333956> invalid app key: discard data

Administrating your gateway remotely

Once your gateway has an IP address, you can use ssh to log on it remotely like with any other Unix/Linux systems. You can decide to mount the Dropbox shared folder if you want, or use the local folder. Just remember to create a ~/Dropbox/LoRa-test folder so that log files can be written, either locally, or using Dropbox (once mounted). You can also change this behavior dynamically as all files are opened and closed for each logged output. Then, you can launch the gateway from the ssh terminal with the following command, using the nohup command to detach the process from the ssh terminal:

 > sudo nohup ./lora_gateway | python ./post_processing_gw.py -t -f --fiware -L --wappkey | python ./log_gw.py &

If for some reason you have to update the various scripts (in post_processing_gw.py mostly when adding new cloud connection for instance) then just kill all processes (normally 4 processus when using the complete redirection chain) and launch again the gateway.

Additional links

Other works that we are doing

Other DIY LoRa gateway initiatives

  1. A LoRa gateway based on Raspberry and Multitech mCard from Nestor Ayuso
    https://github.com/mirakonta/lora_gateway/wiki/Part-1:-Choose-the-hardware
  2. A proof-of-concept single channel LoRa gateway with SX1272/1276
    https://github.com/tftelkamp/single_chan_pkt_fwd
  3. An early Raspberry LoRa gateway developped by Dave Akerman for High Altitude Ballooning
    http://www.daveakerman.com/?p=1719

LoRa and python

  1. A python library to drive a LoRa radio module: Raspberry and inAir9
    This can be used to make the script running on the gateway able to send information back to end-devices
    https://github.com/mayeranalytics/pySX127x
  2. A kickstarter project for multi-radio node and gateway: LoPy
    https://www.kickstarter.com/projects/1795343078/lopy-the-lora-wifi-and-bluetooth-iot-development-p?ref=discovery

Other community based IoT LoRa initiatives

  1. TheThingNetwork

LoRa radios

  1. LoRa modulation

Some words about LoRaWAN (gateway and end-device)

The gateway is not LoRaWAN compatible. It is not it's purpose. A real LoRaWAN gateway should be multi-channel to strictly follow the specification. However, we believe LoRaWAN is not necessary for everybody and might even be not adapted for small to medium scale deployment, where tight control on the deployed infrastructure is preferable.

However, some LoRaWAN features can be added if one wants to do so. If you want a quick explaination of LoRaWAN, there is this nice and simple tutorial (and also how to set a LoRaWAN network server) and this one from LinkLabs. Actually, there are some DIY "single-channel" gateways that do have some LoRaWAN features such as:

  1. the aforementioned single channel LoRa gateway by Thomas Telkamp: https://github.com/tftelkamp/single_chan_pkt_fwd. A nice discussion on this topic is available on TheThinkNetwork forum: http://forum.thethingsnetwork.org/t/single-channel-gateway/798

Our gateway can be configured to work on one of the LoRaWAN channel (i.e. 868.1MHz with BW=125kHz, CR=4/5 and SF=7) by using so-called mode 11:

> sudo ./lora_gateway --mode 11

You can add other modes if needed however in its current stage of development it will have very limited interest as the LoRaWAN packet format is not supported. Future development might include an option to support LoRaWAN packet format, at least so-called "unconfirmed data up".

You can launch the gateway with complete settings as follows (identical to the so-called mode 11):

> sudo ./lora_gateway --bw 125 --cr 5 --sf 7 --freq 868.1

Then you can build LoRaWAN end-devices by either using one of the LoRaWAN implementations that exists (LMIC and LoRaMAC-Node) with a simple LoRa module plugged in a board. Again, Thomas Telkamp has a great port of LMIC for Arduino. Or you can use a radio module that already has an implementation of LoRaWAN. In most cases, you will be able to issue commands to the radio module for realizing LoRaWAN operations. Check this page from TheThingNetwork for a nice summary of LoRa/LoRaWAN hardware: http://thethingsnetwork.org/wiki/Hardware/OverviewNodes.

I just want to mention a very light implementation of LoRaWAN that was originally intented for the Ideetron Nexus board (ATmega328P and RFM95W) but that compile straight away on most of Arduino boards with the RFM95W module connected accordingly (it uses DIO0 for RxDone, DIO1 for Timeout and DIO5 for Ready). It is worth looking at it if you want to start understanding a bit the basic of LoRaWAN: https://github.com/Ideetron/RFM95W_Nexus by Gerben den Hartog. The AES implementation is quite lightweight and I've tested the code on both an Arduino MEGA2560 and an Arduino Pro Mini. Of course, only a minimum support of LoRaWAN is provided but it is a nice starting point. Note that M. Westenberg merged the LMIC port of Thomas Telkamp with the lightweight AES library of Gerben den Hartog to for a more lighweight LoRaWAN stack. Here are pictures of these tests:

   
Figure: simple LoRaWAN test

I also want to mention this great and nice project for a simple open-source private LoRaWAN network server by Orne Brocaar.

Some additional pictures


Figure: First version of the Raspberry and the Libelium LoRa radio module connected on the SPI bus



Figure: The gateway receiving packets from 2 LoRa devices. These information are posted on our ThingSpeak channel


Figure: get a box for our gateway


Figure: then drill some holes for the antenna, power and Ethernet cables


Figure: Put the Raspberry inside (here a Raspberry PI 2B), fix it, and connect the LoRa radio to the Raspberry GPIO header pins


Figure: finally, the Raspberry gateway with the antenna, the power and Ethernet cable, ready to collect your data!


Figure: another version of the low-cost gateway, based on an RPI 1B+ and the RFM92W, in a bigger box. Cost is less then 50. Probably the version that will be "produced in mass" for various deployments. Power supply will be provided by a simple 110V/220V to 5V adapter placed in another box in order to make both parts independant.

   
Figure: our LoRa temperature sensor based on an Arduino Pro Mini. Lot's of cable but we will improve it :-)

Enjoy!
C. Pham

Deployments

University Gaston Berger, Saint-Louis, Senegal

The gateway will be used to deploy low-cost IoT solutions in the context of the H2020 WAZIUP project.

Easy Global Market, Nice, France

The gateway will be used to deploy LoRa service for various demonstration purposes

As part of the WAZIUP project, a starter kit with a gateway will be deployed at project's partner's site:

1- Farmerline (Ghana)
2- iSpace (Ghana)
3- CTIC (Senegal)


IIDRE SAS

The gateway will be used to deploy LoRa service for various demonstration purposes

Connecting Nature

The gateway will be used to deploy and test LoRa-based telemetry services for various agriculture applications

Chuck Swiger from West Virginia (US)...

has a ds18b20 temp probe ThinkSpeak channel using our gateway

The Oceanographic Observatory of Banuyls/mer (part of University of Paris 6)

The gateway will be used to deploy and test LoRa-based telemetry services for various environmental surveillance applications

Matthew Way from New Zeland

Develops great LoRa-based pest surveillance system. He is testing our solution as well as his own custom design solutions.