Benchmarking IEEE 802.15.4 low-resource device test-beds for audio traffic: procedure & tools

as part of EAR-IT WP1: Acoustic Test-bed Qualification

C. Pham (LIUPPA laboratory, University of Pau, France & EGM) and P. Cousin (EGM, EAR-IT deputy project manager)

In the context of the FP7 EAR-IT project on acoustic surveillance in smart environments, this page describes and provides links to various tools for benchmarking low-resource device test-beds based on IEEE 802.15.4 connectivity.

last update: July 17th, 2014.

Why doing a benchmark ?

The EAR-IT project as working in various test beds in city (Santander) and with building (in Geneva) has demonstrated that nice applications can be developed using audio ( traffic monitoring, security, energy efficiency, etc). Also using advanced audio codec (i.e. speex, codec2) we have demonstrated that even constrained network using 802.15 wireless network can be used for audio applications as audio streaming (the most constrained case) can be performed with only 2kbps bandwidth which is often available on these networks.

The project has now defined the minimum condition for any test bed to be capable of hosting audio and audio related applications (see EAR-IT deliverable 1.2). Doing this benchmark is easy and will allow your test bed to expand its usage for a broad range of amazing applications and research cases using acoustic.

Objectives of the benchmark

  1. Determine whether a given test bed is capable of providing the minimum requirements for supporting audio traffic

  2. Indicators and target values are given together with supporting documentation

What you need to do

  1. Download the procedures and be ready to perform the tests on your test bed
  2. Either use the developped audio mote or a simple traffic generator with a promiscuous packet sniffer that can also be downloaded
  3. Determine if your test bed is “audio ready” by filled-in data in an excel sheet given where script can generate indicators which can be compared to minimum necessary
  4. Audio source and audio hardware on TelosB can be borrowed to check on a real audio streaming conditions

Documents and EAR-IT deliverables

The proposed benchmark procedure is described in a set of slides "WP1 Acoustic Test-bed Qualification/Benchmarking procedure for other test-beds". Read this document for detailed instructions on the benchmark procedure and the usage of the various tools that have been developped. The general benchmark methodology is also described in an earlier document "WP1 Acoustic Test-bed Qualification/Qualify and Benchmark Test-beds for Acoustics in Deployment of Targeted Applications". Our test-bed and various control software are also described in "WP1 Acoustic Test-bed Qualification/Audio Test-bed Description". Please refer to these document as well as to deliverable "WP1 Acoustic Test-bed Qualification/D1.2 : Miminium requirements for use of acoustic sensors" that describe the developped audio board, the audio constraints and the purposes the test-bed benchmarking procedure. Additionally, there are a number of publications that you might find usefull as well:
  1. C. Pham, P. Cousin, A. Carer, "Real-time On-Demand Multi-Hop Audio Streaming with Low-Resource Sensor Motes", Proceedings of IEEE SenseApp, in conjunction with LCN 2014, Edmonton, Canada, September 2014.
  2. C. Pham and P. Cousin, "Benchmarking low-resource device test-beds for real-time acoustic data", Proceedings of the 9th International Conference on Testbeds and Research Infrastructures for the Development of Networks & Communities (TridentCom'2014) , Guangzhou, China, May 5-7, 2014. Slides .pdf
  3. C. Pham and P. Cousin, "Streaming the Sound of Smart Cities: Experimentations on the SmartSantander test-bed", Proceeding of the 2013 IEEE International Conference on Internet of Things (iThings2013), Beijing, China, August 20-23, 2013. Slides .pdf

Benchmark tools

We developped in collaboration with INRIA CAIRNS and Feichter Electronics a daughter audio board with speex compression capability. The audio board is connected to a low-resource device, allowing real-time audio capture and compression. You actually don't need the audio board for the benchmark, but if you want to test real audio streaming on your test-bed, we can provide you with the audio mote, see the "Contact" section at the end of this page.

1/ Developped audio board and associated software

Figures below show the developped audio board that was initially designed to be connected to an AdvanticSys CM3000 mote (or CM3300 or CM4000), that will be referred to as the TelosB audio mote.

    

The control software for the Telosb audio mote gets the compressed data from the audio board and sends them to the next hop (a BaseStation or a relay node). The BaseStation is another TelosB mote that is connected to a Linux computer to act as a Sink. The BaseStation is not mandatory in the benchmark procedure but we can use it to listen in real-time to the audio stream. The archive for the source code of the TelosB audio and the BaseStation can be obtained here, all source code are under the TinyOS 2.1.2 operating system. Please refer to TinyOS installation instructions for setting up the TinyOS environment.

The TelosB audio mote can be used to benchmark the test-bed for acoustic data. speex handles audio data in FRAME_SIZE. The value of FRAME_SIZE on the audio board is 160 bytes. Then encoding takes FRAME_SIZE bytes and compresses them is a number of encoded bytes. For 8000 bps rate, the encoded packet size is 20 bytes and the periodicity is 20ms. The BaseStation receives packets from the audio board. Each packet has frame delimiters (0xFF 0x55 SN) prior to the number of bytes (0x14=20) per packet in the output stream. SN will store an 8-bit sequence number. An example is shown below:

    0xFF 0x55 0x00 0x14 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
    0xFF 0x55 0x01 0x14 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
    0xFF 0x55 0x02 0x14 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

To build the TelosB audio mote with IEEE 802.15.4 16-bit address of 0x0090:

    AudioBoard_modif> CFLAGS="-DNODE_SHORT_ADDRESS=0x0090 -DDEST_SHORT_ADDRESS=0x0100" make telosb

Then install with:

    AudioBoard_modif> make telosb reinstall bsl,/dev/ttyUSB0

To build and install the BaseStation mote with IEEE 802.15.4 16-bit address of 0x0100:

    BaseStation> CFLAGS=-DNODE_SHORT_ADDRESS=0x0100 make telosb
    BaseStation> make telosb reinstall bsl,/dev/ttyUSB0

Then refer to "WP1 Acoustic Test-bed Qualification/Benchmarking procedure for other test-beds" to see how the TelosB audio mote can be controlled wirelessly with the XBeeSendCmd tool.

2/ XBeeSendCmd

We use an XBee S1 from Digi (802.15.4, not ZigBee) as a radio gateway to send control messages. We provide the XBeeSendCmd tool that uses such gateway to send ASCII command. However, you can use any similar tools (or the X-CTU program provided by Digi) to send pure ASCII command sequence with an IEEE 802.15.4 radio module. Please refer to "WP1 Acoustic Test-bed Qualification/Benchmarking procedure for other test-beds" for a list of ASCII control sequence used to control the TelosB mote.

The source code for XBeeSendCmd is available here:

To compile:

    > g++ -Wno-write-strings -o XBeeSendCmd XBeeSendCmd.c -lrt

Example (trigger audio capture and transmission at the TelosB audio mote):

    > XBeeSendCmd -p /dev/ttyUSB0 -addr 0090 "@/C1#"

3/ Simple speex decoder at the sink

We also provide a simple speex decoder that waits for frame delimiters (0xFF 0x55) and that will decompress the audio stream into raw format to Linux's stdout. The source code for speex_sampledec_wframing is available here:

To compile:

    > gcc -DWITH_PKT_FRAMING -o speex_sampledec_wframing speex_sampledec.c -lspeex -lspeexdsp

See below for an example of usage

4/ 1-hop scenario with TelosB audio mote and BaseStation

Once you have the TelosB audio mote and the BaseSation mote, you can test by triggering the audio capture and listen in real-time to the audio stream. Here are the procedure:

  1. Connect the XBee gateway to a computer (on /dev/ttyUSB0 for instance)
  2. Connect the TelosB BaseStation to a computer (the same here, on /dev/ttyUSB1 for instance)
    1. run a python script that will continuously read the serial port for compressed audio data and that will forward these data to the speex decoder

      > python 115200SerialToStdout.py /dev/ttyUSB1 | ./speex_sampledec_wframing | play --buffer 100 -t raw -r 8000 -s -2 -

  3. Power on the TelosB audio mote
  4. Use XBeeSendCmd
    1. to activate the TelosB audio mote, you may need to send the control command twice

      > XBeeSendCmd -p /dev/ttyUSB0 -addr 0090 "@/C1#"

    2. to stop the TelosB audio mote

      > XBeeSendCmd -p /dev/ttyUSB0 -addr 0090 "@/C0#"

5/ Promiscuous packet sniffer for wireshark

We provide a promiscuous packet sniffer under TinyOS to be connected to the wireshark packet analyser. It's usage for the benchmark of test-bed is described in "WP1 Acoustic Test-bed Qualification/Benchmarking procedure for other test-beds". The promiscuous packet sniffer is based on the TKN154 protocol stack and the TestPromiscuous or packetsniffer example. We improved TestPromiscuous to build a sniffer node. The source code is available here:

To build and install the packet sniffer:

    > CFLAGS="-DSNIFFER_CONF -DPCAP_SERIAL_OUTPUT" make telosb
    > make telosb reinstall bsl,/dev/ttyUSB0

Then there is a simple python program that will continuously read the serial port and send data to wireshark. The mote will capture packets and will send pcap-formatted data to the serial port. More information on pcap format can be found here. The python program is TelosbToStdoutPcap.py

Then you can run the following command with your TelosB mote plugged in your computer on /dev/ttyUSB0:

    > python TelosbToStdoutPcap.py | wireshark -k -i -

you may need to give sudo permission:

    > python TelosbToStdoutPcap.py | sudo wireshark -k -i -

If running on /dev/ttyUSB1, just specifiy it in the command:

    > python TelosbToStdoutPcap.py /dev/ttyUSB1 | wireshark -k -i -

You can see the graphical result below:



You can see various scenarios in this snapshot:
  1. broadcast packets do not need acknowledgment (see frame 1 for instance)
  2. unicast packets need acknowledgment and the ACK is captured when the receiver is active (see frames 5 and 6 for instance)
  3. unicast packet to an non-existing device will generate 1 transmission and 3 retransmissions (the default retransmission count in IEEE 802.15.4, see frame 13-16 for instance)

If you use a recent version of wireshark with 6LowPan/CoAP dissector, you will be able to see RPL messages and CoAP exchanges as in the CoapBlip example of TinyOS.


Limitations:

  1. The timestamps for ACKs are normally incorrect from the SFD, but a turn around is proposed when using wireshark
    1. when a packet is an ACK packet, take the previous timestamp and add 192us (12 symbol=aTurnAroundTime)
    2. additionally adds 354us which is the ACK transmission time at 250kbps (11 bytes)
  2. FCS is not valid so all frames will have bad checksum but it is not important as all captured frames already have good checksum for the radio module to accept them
  3. When the sniffer is started while there are lot's of traffic, it may happen that the script sends a truncated frame read from the serial port resulting in an error such as "frame too long". You can have a more "secure" start by:
    1. press and maintain the reset button on the sniffer mote
    2. start the python script
    3. when wireshark is running, release the reset button. Since the radio module only accept valide frames (FCS is checked) starting the script well before the mote ensures that no truncated information from the serial port will be sent to wireshark.

Acknowledgments:

The original development tool for plugging a mote to wireshark has been provided by Pierre-Yves Lucas from University of Brest. He wrote a simple program to translate XBee API format to pcap format in order to be able to use wireshark with XBee module. We improved this idea by porting it to TelosB and MicaZ and CC2420 radio using TinyOS and TKN154 which is a much more powerful environment.

6/ Packet analysis script and Excel template

As described in "WP1 Acoustic Test-bed Qualification/Benchmarking procedure for other test-beds" here are:

  1. An example of the wireshark capture converted into text format
  2. The pkt-loss-rate awk script
  3. The Excel template

Connecting the audio board to an other sensor mote platform

The audio board communicates with the host sensor mote with an UART line. The communication speed is normally 115200 baud on the TelosB, but this can be changed. We successfully connected the developed audio board to a Libelium WaspMote sensor board at the speed of 38400 baud. Please contact us for more information.


Contact information

The TelosB audio board can be borrowed if you are willing to benchmark your test-bed. Please contact Philippe Cousin (EGM) from EAR-IT project.


C. Pham
University of Pau, France & EGM