Yes, sure! You can buy a Sonoff RF and you are good to go, I guess. But I didn’t and I was not so sure about the no-named RF receiver so I ended thinking about adding my own.

But first things first. The Sonoff is an ESP8266 based smart switch by ITEAD which comes with a custom firmware that communicates with the manufacturer cloud to provide “smart” capabilities like remote switching or scheduling. The cool thing is that it has a line of pins that expose the VCC, GND, RX and TX pins of the ESP8266 and a buttons attached to GPIO0 so very soon it got hacked and there are a number of firmwares already available. I’m not an early adopter and some work has been done and reported by Peter Scargill, Javier or even in instructables.

The ITead Sonoff Smart WiFi Switch after a small hack to use the Avidsen RF remote to toggle it

About two months ago I bought 3 Sonoffs TH, the ones ready for mains switching but without RF receiver. When they arrived I reckon I tried ITead eWeLink App for Android but in less that 5 minutes I decided I had nothing to loose getting rid of it and flashing my own firmware.

Truth is that I’m living my own ESP-fever so I immediately plugged it to an FTDI adapter and tried a simple program. Goodbye eWeLink and guarantee voided (does it have any?).  When I started figuring out what use could I give them I thought I could embbed them behind the switches in the wall but they are a dime too big and they use simple SPST relays, so no commuting (that’s a fail).

So next thing were lamps and other plugged appliances, starting with a warm lamp in the living room. But this particular lamp was already plugged to a RF switch and switched on/off from a remote. I realized I did not want to loose that. Having to unlock the phone to switch on the light sounded a bit weird. Why didn’t I bought the Sonoff RF instead?

Well, because I was not sure about the RF module it uses. How to handle it from custom code? Will my remote work with it (probably not) or will I have to buy theirs?

Time for a little hacking!

First step was to redo all the work I had already done when I started with these remote. It worked fine with Arduino but not with the Sonoff until I realize the 433MHz receiver I was using required 5V to operate. Mmm… No worries, even thou the ESP8266 works at 3V3 I remembered that in the schema for the Sonoff the AC/DC transformer feeds 5V to an LDO, so I could get the 5V from there.

Front view of the Sonoff board with pin descriptions

Back view of the Sonoff board with pin descriptions

Second problem was where to plug it. The answer was the line of holes where the RF module is in the Sonoff RF. Only two of those pins are tied to the ESP8266, the closest to the relay are GND and 3V3. Could I be so lucky? NO. The receiver I have (the standard RFLink you can find at ebay and such) has 4 pins: GND, DATA, DATA and VCC. Besides, the module just did not fit so close to the relay so the first pin would go in the second hole (the 3V3 hole)… OK, time for the X-Acto knife, the iron and the dremel.

The RF receiver. I removed the black plastic around the pins to solder it closer to the board, otherwise it won’t fit in the case.

First I cut the traces leading to the 3V3 pin in the RF row of holes (that’s the second hole counting from the relay). There are surface traces on both sides of the board, so I had to cut both and then bridge them together again. Then I soldered the module in the 2nd, 3rd, 4th and 5th holes. I brought GND from the 1st to the 2nd hole, I then connected the DATA pin to the GPIO14 in the perpendicular row of pins through a voltage divider to ground to lower the 5V from the module to 3V3. A 10K and 20K 0805 SMD resistors do the job, but it was really hard to solder them between the pins… Next I soldered the VCC pin of the RF module to a spot in the board I checked was connected to the 5V rail.

Check the cut just by the LDO, there is another one in the same spot in the other side of the board. But cutting those traces removes power from the ESP8266 so I had to wire it again (it’s the white wire in the right). The other white wire powers the RF module with 5V. You can also see the GND bridge between pins 1 and 2 in the top row and the voltage divider from the DATA line to the GPIO14 and GND pins.

I also had to dremel a bit the case to allow some room for the module

Done. A bit of a mess but it worked!

“My” Sonoff RF

But the maximum distance from the remote to the module was something in the 20cm… until I soldered a 17.3cm wire to the antena hole in the RF module (that 1/4 of a 433MHz wavelength). Now I can be 10 meters away, even with a wall in between and I can still switch it on. And before you say anything: yes, range should be better for a 433MHz module but it’s fine, after all I can still switch it on/off from Australia via the Internet.

ESPurna Firmware

There are some fine control firmwares out there for the ESP8266.  Some of them are generic and let you manage the pins one by one, some of them are more specific for “smart switches” based on the Espressif chip. Most of them have MQTT and Web configuration. Some are small enough so you can flash them Over-the-Air to the 1Mbyte Sonoff. None of them (at least I haven’t found any) have RF capabilities.

So I decided to build my own. After all that’s fun. I’ve named it ESPurna (“espurna” means “spark” in catalan) and it features:

• WebServer for configuration and simple relay toggle
• Flashing firmware Over-The-Air (OTA)
• Up to 3 configurable WIFI networks
• MQTT support with configurable host and topic
• Manual switch ON/OFF with button
• Support for custom RF module (check blog post)
• Visual status of the connection via the LED

ESPurna status page

ESPurna administration page

As always, if you are interested in the firmware it is in a espurna repository in bitbucket.

“Wiring” it.

Final step is to “wire” it all together. The Sonoff with the ESPurna firmware connects to my home WiFi and subscribes to a certain MQTT topic. Upon reception it switches on or off the relay depending on the message payload (0 for OFF, 1 for ON, 2 for TOGGLE). Whenever the relay state is changed it also publishes the new state to the same topic (there is code to avoid recursion). So if you change it manually pressing the button or with the RF remote you still have a record of that.

In my home server I have Node-RED running. A few months ago I switched from a bunch of python scripts to Node-RED for message persistence (in MySQL), notifications and to push data to some online services like Xively or TheThings.io.

With MQTT and Node-RED working adding support for Blynk was a piece of cake. In 10 minutes I had a simple dashboard in my phone from where I could switch the lights on and off. In fact, I’ve spent way more time moving around controls in the dashboard canvas trying to make it look cool (I failed).

My blynk configuration in Node-RED. So easy!

You know something is well designed when things fit in like magic.

Wow. I feel like I should explain many more things about this project, but there will be a follow up so stay tuned

Since I discovered the Sonoff I’ve been thinking about embedding it inside a switch. I started looking for old power meters, timers,… I had at home but the Sonoff is a bit too long. Why didn’t they design a square board? I event bought a bulky Kemo STG15 case with socket.

Next I decided to design my own board. It is meant to be the “official” hardware for the ESPurna project so it’s called ESPurna too. It’s opensource hardware and available at the ESPurna project repository at Bitbucket. I have some boards already for the first iteration (version 0.1). They are mostly OK but I’m already working on a revision.

But then ITead’s released their S20 Smart Socket. It’s the Sonoff in a wall socket enclosure. Almost 100% what I wanted. And at 11.70€ it’s hard to beat. There are other wifi smart sockets available, mainly Orvibo and BroadLink (an SP2 Centros should be arriving home anyday now) but ITead’s is cheaper and you can easily re-flash it. Just solder a 4 pins header, connect it to your FTDI programmer, hold the S20 button, connect the programmer to your computer and flash. Done.

OK, not so fast. Why would I do that? Why would I change the stock firmware?

The answer for me is a mixed up of philosophy and practicity. But you are right. Let’s go step by step.

Hardware

First let’s take a look at the device. The wall socket is nice looking and pretty small (at least compared to the STG15!!). In the socket you can read “16/250 ~” which I don’t know what it means since the relay inside is the same the Sonoff TH has and it’s rated at 10A and the label in the back says 2kW max, but maybe there will be a 16A version like the new Sonoff TH 16A. The ground terminals (connected from input to output) are not tightly fit and they make a metallic sound when you shake the device (OK, why should you shake it?). The ON/OFF button symbol is upside down (ehem). But the overall feeling is really good.

To take a look inside the enclosure you have to remove a single screw on the back of the device, but there is a “tore invalid” label covering it (hey, blame Google translate), so don’t do it.

S20 front view, yes, the ON/OFF button icon is upside down…

S20 back with the “tore invalid” label

But if you do, this is what you will see:

A detail of the upside down button from the inside. It looks like a communication problem between the guy that was working on the enclosure and the guy that designed the button itself…

S20 guts. The ground clip has been removed. You can see the programming header: GND, RX, TX and VCC, everything a hacker needs (almost).

On the back of the board you can see the short live tracks, the ESP8266EX, SPI flash memory and the AMS1117-3V3.

The guts of the S20 are very much like those of the Sonoff albeit with a different layout. Connections to mains are in the upper side, that provides a better isolation between the safe part and the unsafe part, whilst on the Sonoff you have hot tracks running all along the board. Like in the Sonoff it has live lines duplicated on both sides of the board and an extra of tin on the bottom. There are air gaps between the mains lines and the low voltage side, but they had to leave a bridge, probably to provide mechanical strength to the board.

On the bottom of the board you have several components including the ESP8266EX, a Winbond 25Q80DVSIG 8Mbit SPI Flash memory and an ASM1117-3V3 that provides regulated 3.3 volts to the ESP8266 and the flash chips from the 5V output of the AC/DC power supply. Whilst 1Mbyte on flash ROM is enough for most applications, even with OTA functionality (check my ESPurna firmware) some might find it a bit too short. If that’s your case there are cheap and easy ways to upgrade the memory chip to a 4Mbytes one.

There is a button attached to GPIO0 and two LEDs, a green one connected to GPIO13 like in the Sonoff and a blue one to the GPIO12, like the relay, so whenever the relay is closed the LED will lit blue. Close by the blue LED there is an unpopulated header perfectly labelled with VCC, RX, TX and GND lines. That’s everything you need to flash the ESP8266 onboard a custom firmware.

Software

ITead’s home automation products are managed through the eWeLink app for Android and iPhone. The goal of the app is to become a hub for different wifi home automation devices: switches, lamps, power strips, fans,… and the operation should be straight forward: set the device in pairing mode (AP mode), configure wifi credentials and control it.

I swear I have tried hard to use it. Several times. Even accepting weird permissions I would have not accepted to any other app (why should it ask for my phone number or my location?). But I never ever managed to make it work. I have not been able to pass the pairing step. Not even after giving the app permissions over my personal life. Impossible. I quitted. If anyone has a clue on how to do it, please let me know. Permissions and connection problems are common between the comments on Android app market. No one has commented yet on iTunes. Weird.

This is the furthest I got

But even thou the app had worked fine I wouldn’t use it. I’m not saying you shouldn’t use it. My case might not be yours. Why would I flash my own firmware? Basically to really own the device.

We are in the very early days of the IoT and we are not even sure of what it means. Companies are trying to capitalize the concept providing their users all-in-one solutions including the device, the app and the cloud (whatever that is). But why should anyone know when you set your livingroom lights in romantic mode? And what happens if the company shuts off? Greenwave by TCP or Revolv cases are well know. Google has a reputed history of closed services (not IoT related, true).

And what about interoperatibility? You can easily end up having to use an app to switch your reading lamp and a different one for your ceiling light! Even those that provide open cloud solutions have weak days: Phillips recently released a trial balloon about locking out 3rd party hardware from Hue brigde. Elliot Williams at hackday put it simply: “The 900-pound gorilla in the corner of the Internet of Things (IoT) hype that everyone is trying to ignore is interoperability”.

That’s why i would always prefer open APIs and protocols over proprietary ones. I’m not even talking about open/free source software or hardware but protocols. You don’t own a device and the information it manages if you cannot talk to it directly, without middlemen.

The great thing about the Sonoff or the S20 Smart Socket is that you can easily flash your own firmware on them. So now I can control it with through my MQTT local broker, add schedules, behaviours,… through my local Node-RED instance or rely on 3rd party cloud services like Blynk or IFTTT if I want to, because if they ever shut off, it’s no big deal.

Conclusions

From the end-user point of view the device is a great deal: cheap, nice, wifi-enabled. But the mobile app is a disaster and I would expect people returning their devices because of this.

From the hacker point of view it’s a even greater deal. You can easily flash a custom firmware and make it talk to your local services. It could have more flash memory. It could have more GPIOs exposed in the header to add sensors like a simple LDR to switch ON/OFF depending on the ambient light or a PIR to detect that someone’s near. Or maybe it could have a jack out like the new Sonoff TH 10A/16A.

I will recommend the S20 Smart Socket to anyone with the skills to customize it. For less than 12€ it’s an amazing device you can own.

Have you ever forgotten your wet clothing inside the washer for a whole day? I have. Even for two days. They smell. You have to wash them again and you know you might end up forgetting about them again!

Actually that is happening to me since me moved to an old house in a town north of Barcelona. Instead of having the washer in the kitchen, like we used to, now we have it in the cellar, in a place I don’t normally pass by to notice the laundry is done.

So I started thinking about monitoring the washer to get notifications when the laundry is done. And since I was at the same time playing with ITead’s Sonoffs, which has an AC/DC transformer and a powerful controller with wifi, it looked like a nice project to put together.

EXTREME CAUTION: live mains are very dangerous. Don’t work with them unless you know what you are doing and what the consequences might be.

Wiring the ADC pin

The Sonoff does not bring out the ADC pin on then ESP8266EX. Some Sonoff TH versions do bring out GPIO14 and latest Sonoff TH 10A/16A probably do the same through a jack interface. But that’s not the case for TOUT aka ADC pin.

No problem, follow me:  grab the iron with a narrow tip, a short thin cable and a magnifier, because you are going to need it. If you are not really good at soldering (and I’m not) you can still do it, but you will have to check and double check for bridges between the ADC pin and its neighbours. The ADC pin is the 6th in the left side of the chip (with the dot on the top left corner).

First solder a 3 pin header on the unpopulated holes for the RF module. If you use the first 3 counting from the relay you will already have GND and 3V3 on the first two, so you just need to connect the ADC pin in the Espressif chip to the third header. Take your time, do not apply heat for more than a second and leave it cool before doing it again. Check with your continuity tester after and prepare a simple gig to test your hack: a 10k potentiometer with a 20k resistor to build a voltage divider will do.

Update:

Don’t work with the board connected to mains! Again: Don’t work with the board connected to mains! After testing for shorts and continuity hot glue everything to prevent the cable from comming loose and create a potentially fatal short between mains.

That was tricky! Bringing out the ADC GPIO in the ES8266EX to a 3 pin header using the unpopulated RF header of the Sonoff…

Sensor board

Now that I had the ADC pin I prepared a simple stripboard with a current sensor and a simple voltage bias circuit. The current sensor I use is a chep (and not very accurate) Talema ASM-010. It outputs a small voltage proportional to the current flowing through the hole. According to the charts in the datasheet the ratio is about 300A per 1V. That’s a very poor value, and even thou I could pass the cable twice, reducing the ratio to 150A per V the YHDC SCT-013-030, a very common non invasive current, has a ratio of 30A per 1V, 5 times more accurate. But I just need to know whether there is current flowing or not.

The board has two resistors (nominal 10k and 56k) to build a voltage divider to bias the output voltage by 0.5V so the ADC will always read positive values, and a 10uF capacitor to soften signal.

The sensor board with one of the live cables doing two turns inside the coil. There are also two resistors to bias the voltage and a capacitor.

The Sonoff and the sensor board inside the enclosure.

Adapting the EmonLib

The EmonLib is the unofficial standard library to monitor current on an Arduino and alikes. But it doesn’t support ESP8266 based boards yet. Current issues are:

• It only supports 10 bit ADCs (ESP8266 ADC is 10bits too, but it only has one analog pin, so chances are you will use an external ADC chip in some projects, like de 16bits ADS1115)
• Voltage reference calculation doesn’t take into account non-atmel chips (the ESP8266 ADC is 1V referenced)
• Analog reading is hardcoded in the library, again, what if you are using and external chip with I2C communication?

The usual approach is to include parts of the library code into your code and monitor analog pin 0. You could also use and external ADS1115 like Tisham Daar does.

This is not the first project I use current monitoring on a ESP8266, so I decided to wrap all the functionality I wanted in a proper library. And so EmonLiteESP was born. As the name says it’s a lite version of the EmonLib, since it only supports current monitoring at the time (only apparent power). But truth is that naming it after the EmonLib could be misleading since it is not API compatible (and maybe I’m violating some trade mark, didn’t check that).

The library main features are:

• Customizable ADC bit count
• Customizable ADC voltage reference
• Read ADC values via callbacks

And its easy to use:

#include "EmonLiteESP.h"

EmonLiteESP power;

unsigned int adcBits = 10; // ADC bit resoltion (normally 10)
double referenceVoltage = 1.0; // 1.0V for a bare ESP8266 chip
double currentRatio = 30; // 30A 1V for a YHDC SCT-013-030
double mainsVoltage = 230.0 // EU standard
unsigned int samples = 1000; // 1000 samples each time we measure current

unsigned int currentCallback() {
   return analogRead(0);
}

void setup() {
   Serial.begin(115200);
   power.initCurrent(currentCallback, adcBits, referenceVoltage, currentRatio);
}

void loop() {
  double current = power.getCurrent(samples);
  Serial.print(F("Power now: "));
  Serial.print(int(current * mainsVoltage));
  Serial.println(F("W"));
  delay(5000);
}

Now that…

Some time ago, at a previous job, our CTO was very concerned about unnecessary code. He had a list of anti-patterns to identify bad practices when developing a new feature. The YANGNI (You Are Not Gonna Need It, meaning just code what you are going to use now) was his favourite one. Another favourite was the YAQUE (spanish for “Now that…”). Now that we are touching this code, why don’t we add this cool feature?

So breaking the YAQUE rule I added a DHT22 sensor because I could, and because we have moisture related problems in the cellar. The sensor is connected to the GPIO14 pin already available in the Sonoff board. I soldered a 10K SMD0805 resistor between the left-most 2 pins of the DHT22 (VCC and DATA) and drilled a hole in the case to place the sensor outside the box.

Another improvement was to add a case mounted momentary button on the enclosure, soldered to the onboard button, so I can manually switch the relay ON and OFF.

A wired Sonoff board with connections to the current sensor board, to the DHT22 temperature and humidity sensor and to the case mounted button.

The final project box

I have included all these features in my last ESPurna firmware version. Please check it out at espurna bitbucket repository.

Analyzing the data

So the goal of this project was to get a notification when my laundry is done. You don’t really need an accurate power measurement for that, only monitoring if there is current flowing or not and keeping track on how long has the washer been “quiet”.

First step is to see how the washer power profile looks like. It will probably depend on the washing program you are using but for the one we normally use for day-by-day laundry it looks like this:

So you can see the washer has been working for 27 minutes (from 11:27 to 11:54) with several quiet windows of no more than 2 minutes each. Please note this measurements have been taken every minute averaging 6 readings every 10 seconds. For this program it might be enough to check whether the washer has been quiet for the last, say, five minutes after seeing activity and then emit the notification.

Node-RED driven notifications

Final step is to set a notification when the laundry is done. I’ve been using Node-RED for some time now and I like having everything in one place in a flexible and structured way: notifications, database persistence, cloud service connections (xively, blynk, thethings.io, thigspeak…), schedulers,…

Now that I have an MQTT topic with the washer power consumption is pretty easy to write a javascript function that monitors it and sends a notification when the washer has been idle for five minutes after some significant activity. To implement that I built a 5 positions ring that stores whether there has been activity or not for the last 5 minutes or last 5 messages. It sets a flag when at least 3 of those messages contain non-zero values (that’s a significat activity). After that, when the buffer empties (5 minutes without activity) it sends the notification and clears the flag.

I’m sure the code will be much more clear than my explanation:

// Configuration
const WASHER_BUFFER = 5;

// Get current state
var washer_flag = flow.get('washer_flag') || 0;
var washer_sum = flow.get('washer_sum') || 0;
var washer_store = flow.get('washer_store') || Array(WASHER_BUFFER);
var washer_pointer = flow.get('washer_pointer') || 0;

// Update state
var current = parseInt(msg.payload) < 100 ? 0 : 1;
washer_sum = washer_sum - washer_store[washer_pointer] + current;
if (washer_sum < 0) washer_sum = 0;
washer_store[washer_pointer] = current;
washer_pointer = (washer_pointer + 1) % WASHER_BUFFER;
if (washer_sum > 2) washer_flag = 1;

node.status({
    fill: (washer_sum === 0) ? "red" : "green",
    shape: "dot",
    text: "Washer machine state: " + (washer_sum === 0 ? "idle" : "working")
});

// Notifications
if ((washer_flag === 1) && (washer_sum === 0)) {
    washer_flag = 0;
    msg.topic = "Washer";
    msg.payload = "Your laundry is done!!";
    msg.trigger = {
        'topic': '/trigger/washer/done',
        'payload': 1
    };
} else {
    msg = null;
}

// Store state
flow.set('washer_flag', washer_flag);
flow.set('washer_sum', washer_sum);
flow.set('washer_store', washer_store);
flow.set('washer_pointer', washer_pointer);

return msg;

The message is then passed to a subflow that sends it to Pushover service and also publishes it to the local MQTT broker.

And voilà: no more forgotten laundries… I hope.

Some 3 years ago I started building my own wireless sensor network at home. The technology I used at the moment has proven to be the right choice, mostly because it is flexible and modular.

MQTT is the keystone of the network. The publisher-subscriber pattern gives the flexibility to work on small, replaceable, simple components that can be attached or detached from the network at any moment. Over this time is has gone through some changes, like switching from a series of python daemons to Node-RED to manage persistence, notifications and reporting to several “cloud” services.

But MQTT talks TCP, which means you need some kind of translators for other “languages”. The picture below is from one of my firsts posts about my Home Monitoring System, and it shows some components I had working at the time.

All those gears in the image are those translators, sometimes called drivers, sometimes bridges, sometimes gateways. Most of them have been replaced by Node-RED nodes. But not all of them. This is the story of one of those gateways.

Moving from XBee to Monteino

I want you to focus on the “Xbee MQTT Client” in the previous image. XBees are fairly expensive, hard to configure but also pretty powerful and full featured. I could deploy battery powered end devices that run for months from a coin cell like my door sensor or backed by a solar panel like in my weatherstation.

A year ago I started playing with LowPowerLab’s Monteinos as a replacement for the XBee network. A Monteino is an ATMega328 with the Arduino bootloader and HopeRF RFM69 radio on board, everything running at 16MHz and 3.3V. It is a truly low power device and the guy behind LowPowerLab, Felix Rusu, along with the Monteino community have put together an awesome set of libraries to use them. RFM69 use the ISM 868MHz band (here in Europe) with a longer range than the 2.4GHz XBees (although the later have mesh capabilities).

But Monteinos, just like XBees, do not speak TCP. They speak their own language over a FSK modulated radio signal at 868MHz. So you need a gateway to translate messages back and forth the two networks. That’s it, an RFM69 to MQTT bridge.

My first approach a year ago was to copy the XBee gateway idea I was using at the time, basically an XBee in coordinator mode listening to packets from the nodes and forwarding them over serial port to a computer running a python daemon with the python-xbee library to decode API frames and map them to MQTT topics using the Mosquitto python library (before it was donated to the Eclipse Paho project). So I wrote a simple program adapted from Felix Rusu’s gateway.ino example running in a Monteino with an FTDI adapter (yes, there is a MonteinoUSB but I don’t have it) that passes messages over serial port to the computer that translates them to MQTT messages.

Standalone gateway

A Monteino with an FTDI adapter, passing data over serial in a custom protocol to a python script to translate them to MQTT… and back. I was not satisfied with the solution and I never finished the migration from XBee to Monteino.

But now I’ve done another step in what I feel like is a good solution. A standalone ESP8266-based gateway with an RFM69CW radio module on board. I’m not going to say this is the final solution, still have some doubts about it but I like it because it’s a microcontroller based solution, that does just that, bridging two different networks, without help from any other component.

RFM69GW board v0.1

RFM69GW v0.3 – top view: fixed ESP12 footprint error, improved silkscreen labels and bigger holes

RFM69GW v0.3 – bottom view

For the first version I had to take some decisions:

• Through hole components except for the radio modules, the AMS1117 and the buttons. I wanted it to be easy to solder.
• The board form factor was that of the Sick of Beige DP6037 case by SeeedStudio.
• Also I used RFM69CW footprint because it’s pin-compatible with the RFM12b, so in theory (I have not test it), you could make it work with old Monteinos or JeeNodes.
• Non FTDI-like programming header. This was a hard decision but I had not very much free room and decided to bring out a GPIO instead, in case I wanted to attach some digital one-wire sensor.

Version 0.1 of the board has some errors, but still it’s usable. There was a fail in the ESP12 footprint I used, GPIO4 and GPIO5 were swapped. As a consequence DIO0 in the RFM69 module is tied to GPIO5 instead, nothing that could not be fixed changing a value in code. Also somehow I drew M2 holes in the board instead of M3 and the programming header is too close to the bottom-right hole so I have problems with common standoffs. And finally, there are problems with some of the silkscreen labels being too small (the button labels for instance) or missing (like the component values). This error has no impact on the functionality but I made it also on two other designs I sent to fab at the same time (ouch!).

Features

Testing the gateway. The monteino in the picture has Rusu’s example gateway firmware, so I could check that nodes were sending well formed messages.

The project firmware packs some interesting features, IMHO. Let me summarize them first:

• Up to 3 configurable WIFI networks
• Over-The-Air firmware update (OTA)
• MQTT support, off course
• Dynamically map nodes/keys and MQTT topics
• WebServer for configuration using the great PureCSS framework
• Persistent configuration across reboots using Embedis
• Optional default topic for non mapped nodes/keys.
• Configurable IP and Heartbeat topics
• Visual status of the received messages via the ESP12 LED

Topic mapping

As I explained in another post about MQTT topic naming I think the gateway should be the only responsible for translating messages from one network to another, and this means it has to have the logic to publish messages to the right MQTT topic, for instance. So I wanted to be able to dynamically define a map between nodes, variables and topics.

Web configuration page for the node-to-mqtt map

The gateway expects to receive messages with the key, the value and optionally the packetID (from the sender point of view) separated by colons (i.e. “BAT:2890:34″). This format is not the best in terms of size, but it’s compatible with Rusu’s MotionMote, for instance, and I had already used it in my XBee network. You can then map the combination of nodeID (available in the header of the message) and key to an MQTT topic.

You can also define a default topic that will be used when no match has been found in the map. The default topic can contain {nodeid} and {key} as placeholders to create custom on-the-fly MQTT topics.

Persistence

The other great feature is the configuration module. Just one word: Embedis. You have to use this library by PatternAgents. It’s basically a key-value database for microcontrollers that supports different platforms (including Arduino for AVRs and ESP8266) and different storage like SPI or I2C memories, EEPROM or emulated EEPROM like in the ESP8266. It’s easy to use, robust and powerful, and comes with console management with customizable commands as bonus feature.

Web configuration portal

Another aspect of previous projects I wanted to improve is the web configuration portal. I wanted to give PureCSS by Yahoo a try and it worked great, with jQuery as client language and ArduinoJson and Embedis in the backend. The layout is simple but looks great both on my laptop and on my phone and it’s much more usable than my previous efforts.

The JustWifi library supports up to 3 different networks. If it fails to connect to one of them it will try the next one.

Wifi & Radio management

Also, I have worked in encapsulating wifi and rmf69 functionality in two classes. The first one, JustWifi, is inspired by WifiManager but it just handles wifi connection (hence the name) ripping off all the webserver and DNS stuff. The second one, RFM69Manager, encapsulates the setup and receiving code and outputs a custom packet to the provided callback with all the useful information for a message (nodeID, key, value, packetID and rssi). It also wraps the send method to format a compatible message (“key:value:packetID”).

RFM69 and ESP8266

The moment I read this post in the LowPowerLabs forum I decided to do this project. Felix’s library for the RFM69 was compatible with the ESP8266 almost without modification! But the comments there were a bit confusing. So this is how I made it work:

1. Modify the SPIFlash.cpp file to check SPCR & SPSR before using. My solution is to wrap them between #if clauses to avoid compile errors if not defined, which happens in ESP8266. I’ve forked Felix’s library. Here you have the diff for the changes I made.
2. Initialize the RFM69_ATC object (or the RFM69, whichever you use) passing the SS and interrupt pins.

RFM69_ATC radio = RFM69_ATC(SPI_CS, IRQ_PIN, false, IRQ_NUM);

I use this code in my RFM69Manager library. With these 2 changes it just works. No more changes needed.

This is what the console output looks like. Messages are from a node that’s continuously sending  the same payload. The node was powered after the gateway came to life, that’s why the first packetID is 1. You can also see how the RSSI value goes down. This is due to the Auto Transmission Control in the RFM69_ATC library that’s set to a minimum of -70. The transmitter will keep on lowering the output power to match this RSSI value, saving power and reducing radio pollution. Key “BAT” for node ID 10 is mapped to “/home/frontdoor/battery” topic (see screen capture above).

[RADIO] Listening at 868 Mhz...
[RADIO] RFM69_ATC Enabled (Auto Transmission Control)
[WIFI] Connecting to last successful network: daoiz
[WIFI] Connected to daoiz with IP 192.168.1.111
[MQTT] Connecting to broker at 192.168.1.100 anonymously: connected!
[MQTT] Sending /raw/rfm69gw/ip 192.168.1.111
[MESSAGE] nodeID:10 packetID:1 name:BAT value: 2310 rssi:-31
[MQTT] Sending /home/frontdoor/battery 2310
[MESSAGE] nodeID:10 packetID:2 name:BAT value: 2310 rssi:-31
[MQTT] Sending /home/frontdoor/battery 2310
[MESSAGE] nodeID:10 packetID:3 name:BAT value: 2310 rssi:-32
[MQTT] Sending /home/frontdoor/battery 2310
[MESSAGE] nodeID:10 packetID:4 name:BAT value: 2310 rssi:-35
[MQTT] Sending /home/frontdoor/battery 2310
[MESSAGE] nodeID:10 packetID:5 name:BAT value: 2310 rssi:-37
[MQTT] Sending /home/frontdoor/battery 2310

Similar projects

Funny enough, while waiting for the boards to arrive from a chinese factory, a couple of very similar projects have come to public attention, the Ebulobo by James Coxon and the Espism by Johan Kanflo.

Both boards are smaller than mine, basically because they place each module on one side of the PCB. They have both chosen to use an USB-A socket to power the board resembling an USB stick, while I use a 2.1mm barrel jack. The Ebulobo board has an SMA connector to plug an antenna, but the Espism uses a pigtail wire as antenna. My board has both options.

James’ Ebulobo uses a header to program it, very much like I do, but without power line (the board has to be plugged to a powered USB port) and instead of having a FLASH button on-board to pull GPIO0 down on boot, it brings out the GPIO0 in the header and you have to use a jumper to tie it to ground. Johan uses a custom programming connector (the Esprog) to flash the firmware on the Espism. This way he reduces board size but the payload is… that you need a custom programming connector.

Both projects have open sourced their schematics, board and code, so they are great projects to learn from. I like their form factor, not sure about the USB connector, not even if there was an FTDI chip and circuitry to program it like you would do to a NodeMCU. OTA is a better option in any case. If the reason is the ubiquity of a power supply connector a microUSB socket would be a better option.

Improvements

I’m just starting to deploy devices using this gateway. I guess I will find things to improve on the way. Right now, the main improvement I have in mind is to support sending messages from MQTT to a remote actuator with an RFM69 radio. Right now the gateway is only one-direction.

The problems I have identified in the board layout are fixed with version 0.3 in the repository, but since the board is fully usable I don’t plan to send it to fab. Yet.

I’m happy with the firmware, it’s a huge improvement from what I had been doing lately and I plan to migrate other projects like my Espurna Smart Socket firmware to use PureCSS and Embedis.

Again, any comment will be welcome.
Tinker and have fun!

My MQTT network at home moves up and down a lot of messages: sensor values, triggers, notifications, device statuses,… I use Node-RED to forward the important ones to PushOver and some others to a Blynk application. But I also happen to have an LED display at home and that means FUN.

LED displays are cool. Your team’s score, your number in the IRS queue, the estimated arrival time for your next commute,… Now that TVs are replacing LED displays (like the later did with the electromechanical ones) they have acquire an almost vintage-status.

This LED display I own even has a name: The Rentalito. The Rentalito is an old friend, one of those projects you revisit because LED displays are cool… Originally it was an Arduino Uno with an Ethernet Shield in a fancy cardboard case. Then it went WiFi using a WiFly module. And then a SparkCore replaced the Arduino. Now… well, ESP8266 is driving my life.

Let me introduce you the latest iteration of the Rentalito, the MQTT LED matrix display.

The board

Nothing special about the board. A power jack, a regulator, the ESP12 module and the shrouded header for the display flat cable. I brought out VCC, GND, RX and TX to be able to program the ESP8266 the first time, from there on OTA is my preferred way: its a lot faster, specially when uploading the file system. A couple of buttons allow me to reset the controller and pull down GPIO0 to enter flash mode. They are mostly intended for development.

The bare minimum components

Like in the previous version I’ve added an IR receiver and a DHT22 temperature and humidity sensor. The DHT22 is completely optional, the IR receiver is meant to be able to set the device in sleep mode, so it would only display prioritary messages.

Soldering SMD parts

Most of the parts are SMD. I wanted to gain some practice soldering them and I wanted to keep the board under 50x50mm (cheaper to fab) while soldering parts only on one side of it.

To solder the SMD0805 resistors and capacitors I first added a small blob of solder to one of the pads, then placed the part in place and applied a fin iron tip for a very short time (a lot less than a second) while pressing with the tweezers to place it flat, then heat the other end and apply a small drop of tin. It’s very doable for 0805 parts. And quite fast. I’m really happy with the result although it’s obvious that it’s been hand soldered. Only the electrolythic capacitors gave me some problems. For the bigger one (the one close to the regulator) the pad was completely hidden under the part. Next time I will give my heat gun a try.

SMD parts use a lot less board space, and at 0805 they are still hand solderable

The board was on the same batch as the RFM69 Gateway I wrote about a few weeks ago, and it has some of the same problems. Some texts are too small and unreadable, not a big deal but I needed the schematic to know the names and values of each part. Also the shrouded connector is too close to one of the holes so the screw head does not fit in… Luckily, none of these problems are critical.

The display

The display is actually a pair of 3216 Bicolor Red & Green LED 5mm Dot Matrix Display by Sure Electronics with 4 Holtek HT1362C datasheet drivers each. In total 64×16 or 1Kpixel. These come in a sturdy 241x147mm black PCB with connectors on both ends so you can daisy chain more boards, although the manufacturer recommends to connect “up to two boards” only.

A detail of the daisy chained connectors between the two boards. You can also read the pinout.

There are shrouded connectors for data and power and screw terminals for power. According to the datasheet each board can use up to 1.37A when everything ON at 100% duty cicle. So if I want all the LEDs in orange (red+green) I should go for a 5V 3A power supply. In real life it very seldom overpasses the amp with the two boards connected.

The Holtek HT1632C is the driver responsible for the LEDs. Every chip can handle 256 individual LEDs. Since the display has 1024 2-colour pixels there are 8 of these chips

The library to manage the driver is based on my own sparkcore-ht1632, which I ported from wildstray’s ht1632c library for Arduino. This library is equivalent to wildstray’s except that it only supports 32×16 geometries (the only one I had for testing) and does not have blinking effect. On the other side the library allows for two scroll buffers, so you can have texts scrolling in the same or different directions at the same time.

Not the final version, but pretty close

Code folder structure

I’m using the Arduino Core for ESP8266 and PlatformIO as  build manager. In case you have never used PlatformIO I strongly recommend it.

This is what the project code folder looks like at the moment:

.
├── data 
│   ├── images 
│   |   └── ... 
│   └── index.html.gz 
├── gulpfile.js 
├── html 
│   ├── checkboxes.css 
│   ├── checkboxes.js 
│   ├── custom.css 
│   ├── custom.js 
│   ├── grids-responsive-min.css 
│   ├── images 
│   |   └── ... 
│   ├── index.html 
│   ├── jquery-1.12.3.min.js 
│   ├── pure-min.css 
│   └── side-menu.css 
├── lib 
│   ├── ht1632c 
│   |   └── ... 
│   ├── JustWifi 
│   |   └── ... 
│   └── readme.txt 
├── package.json 
├── pio_hooks.py 
├── platformio.ini 
└── src 
    ├── debug.h 
    ├── defaults.h 
    ├── dht.ino 
    ├── display.ino 
    ├── main.ino 
    ├── mqtt.ino 
    ├── ota.ino 
    ├── settings.ino 
    ├── version.h 
    ├── webserver.ino 
    ├── websockets.ino 
    └── wifi.ino

The platformio.ini file defines the framework, board, and other build settings. You can define as many “environments” as you want. For instance: I use NodeMCU board to test because it’s easy to carry with me so I have an environment for it, another one to flash rentalito via an FTDI programmer and yet another one to flash it over-the-air.

The “src” folder contains the firmware files (C++ code). Lately I’m splitting my code in different files (wifi, webserver, mqtt, ota,…), each for one functionality with extension “.ino”. This way I benefit from the preprocessing of ino files that prepend the declarations of all the defined functions, so I can mostly forget about dependencies between them. Splitting the files gives me a way to handle the code easily, which is a major benefit because the project has more than 1000 lines of code excluding libraries.

The “html” folder is the web interface development folder. I preprocess its contents before uploading them to the target board with Gulp. That’s what the package.json (Gulp dependencies), gulpfile.json (processing definition) and pio_hooks.py (PlatformIO hook to automatically preprocess the files before uploading the file system) are there for. The preprocessing populates the “data” folder. The contents of the “data” folder are then flashed to the board. Preprocessing and compressing them makes the webserver in the ESP8266 more reliable and improves performance a lot. Check my post about optimizing files for SPIFFS with Gulp to learn more about it.

PlatformIO library manager handles most the library dependencies:

• Adafruit’s DHT Sensor library
• Benoit Blanchon’s impressive ArduinoJson library
• The already classic MQTT PubSubClient library by Nick O’Leary
• Markus Sattler WebSockets library
• The awesome Embedis library by David Turnbull, Tom Moxon and The PatternAgents

Only the ht1632c library and my custom WiFi manager (JustWifi) come bundled with the project under the “lib” folder.

The web interface

One of the requirements of the project was that the mapping between MQTT topics and the message displayed on the LED matrix must be configurable and not hardcoded. With the possibility to embed a webserver in the ESP8266 this has become much easier.

A little of jQuery and WebSockets and here you have an interface to add, edit and delete mappings. For each mapping you can define:

• The MQTT topic the Rentalito will be listening to
• The name of the value, it will be displayed in the first row of the matrix
• The value, I can use placeholder to format it
• A filter, so it will only match when the value matches the filter
• A priority switch that defines weather the message will be displayed right away or queued
• An expiration time in seconds

The normal use case is when you define a name (like “Temperature”) and a value using a the “{value}” placeholder (like “{value}ºC”). With the filter field empty and priority set to “no” the message will be queued and displayed until the expiration time passes. A new MQTT message on the same topic resets the expiration counter.

The pic below shows the use case of an alert: the empty value means the name will be displayed in big capital letters at full screen, the filter makes the rule match only when the MQTT topic value is “1” and the priority set to “yes” means it will displayed as soon as it is received. So a big “WASHER!!” message will be displayed as soon as the laundry is done. It will also be queued and displayed for the next half an hour.

Configuring the Rentalito to display an alter when my laundry is done!

 Case (so to speak)

The previous version of the Rentalito was already a sandwich of methacrylate , very much like my Wordclock project. This time I decided to go for a smaller version and hide the board behind the display and to use polystyrene instead of methacrylate. Conclusions: bad idea.

The sandwich: the display between two laser cut polystyrene boards and some spacers

The polystyrene melts with the laser and tends to rejoin after. The cut is not sharp and shows small cracks, the surface is not crystal clear because it has suffered from the heat of the laser. This effect is more visible where the plastic was in contact with the metallic bed of the laser cut machine.

The border is irregular and shows 1mm cracks. The surface also shows the effects of the laser heat.

Here the border irregularities are more evident, also some marks from the metallic bed of laser cut

Will go back to methacrylate, even thou it’s more expensive.

Conclusions

The ESP8266-based Rentalito is the most powerful of all the iterations, also the most stable. i’m still working on the firmware and it will surely benefit from improvements I’ll do in other projects. The “enclosure” is certainly a point to improve.

I’m very happy with my new display that notifies me when my laundry is done 

This project code (firmware and web interface), PCB schematics and designs and enclosure models are available as free open source software & hardware on the Rentalito ESP8266 Bitbucket repository.

The post MQTT LED Matrix Display appeared first on Tinkerman.

Some weeks ago a tweet by Manolis Nikiforakis (@niki511) with the #ESP8266 hashtag drew my attention. Manolis had just received a “smart lamp” branded by Ai-Thinker, the AiLight. Yes, the same Ai-Thinker that has sold millions of ESP8266 based modules. Chances were it had an ESP8266 microcontroller inside. Too good not to buy one and take a look at the inside.

Manolis shared the link where he bought his at Ebay for a bit more than USD 10 plus shipping. Unfortunately it’s out of stock there and there are amazingly few other places where you can buy it. I only found the same product with prices around USD 15-18 at Ebay as ESP8266 Smart Home Wifi RGB LED Light [Ebay] or Noduino Open Light LED Smart Bulb [Ebay] also in a cheaper pack of two light bulbs [Ebay]. You can also find it at Aliexpress sold as “DIY Wifi LED Bulb E27 5W AC110-240V lampada LED Dimmable Bulb Lamp Remote Control Led Spot Light for iPhone Android Phones” [Aliexpress]. Don’t you love those seo-ugly names?

I actually bought two because you never know. And they arrived last Thursday. It took me less that 1 minute to open one of the boxes, pop out the cap and take a look at the inside just to see what I already knew. Time to play

Some pics to start

I’m not good or fond of “unboxings” but since it’s a hard to find product I might add here some pics of the boxing and the light bulb.

Each light bulb comes in a light brown box (149x105x67mm if you ask) labelled “WiFi Light” on both faces and with a light bulb pic and a mysterious empty square (?). Looks like a place to mark the variation (they sell them in silver and gold finish), but the sticker with the color is actually on the other side of the box…

There is a label on the bottom of the box with the legend “M1636_SVWIFI” and the usual “Made in China”. I guess SV means “silver” so the model would be “M1636” (I actually found some of the references above looking for this model number).

Inside the box there is a simple two fold paper with Chinese instructions to download the app to control the device on your cellphone. Two things to note: first the funny guy pictured in the paper. I’m not much into Chinese culture but he looks like a 60yo nerd. And then the baseline “If you are not an engineer”. What does it mean? Maybe they meant it’s a product for everybody, you don’t need to be an engineer. Or maybe it’s a “use the provided app unless you plan to hack it”?

And finally the light bulb it’s a copy of the Philips Hue [Amazon] without the logo, with WiFi instead of Zigbee and at a fraction of the price.

Taking a closer look

The bulb cap is not glued, it just fits in. Just pull with a bit of strength and it will pop out. After removing it you have full access to the PCB.

The PCB it’s actually 2 PCBs. The outer one has 8 high power white LEDs, 6 red LEDs, and 4 blue and 4 green LEDs. The inner PCB holds the logic circuit with an ESP8266EX microcontroller and a Winbond 25Q80BVSIG 1MByte flash memory chip. These are the usuals suspects.  There is also a SOT23-6 package labelled ADJG and a 20 pin IC labelled MY9291QD.

There are also 5 pads labelled 3V3, GND, RX, TX and IO0. Too good to be true. They were probably used at the factory to flash the stock firmware on the controller using a gig with pogo pins. Anyway it looks easy to solder some thin wires and upload a custom firmware.

The only doubt left is how to control the LEDs. Googling the MY9291 chip throws one perfect match: the My-Semi MY9291 LED driver. Bingo. But the datasheet lacks any information about how to use it…

Digging on the Internet

Aside from the product in Ebay or Aliexpress there is little information about these light bulbs in the Internet. But with the information I already had I dug a little bit on the Internet and found two interesting pages.

FCC aplication

On one hand I there is the FCC application by Ai-Thinker dated November 9th, 2016. In the application, aside from some interesting documentation like the cellphone app “Tuya Smart” user manual, there are plenty of images from the inside of the product.

Picture from the FCC application

As you can see in the pictures below the PCB is glued and behind it there is a small AC/DC transformer. The bulb itself is 116mm long and 62mm diameter.

Picture from the FCC application

Picture from the FCC application

Picture from the FCC application

Noduino OpenLight

On the other hand, going down the path of the MY9291 reference you quickly found the Noduino OpenLight wiki page at JacksLab, by Jack Tan, alias comcat, the guy behind the Noduino project. In that page you have lots of information about the light bulb including sample code and the schematics of (part of) the circuitry by Jack.

Apparently Jack Tan or the Noduino team is behind this product somehow. Anyone knows what relation is there between Noduino and Ai-Thinker? In the Noduino OpenLight page there are pictures of a virtually identical light bulb in the shape but also with the same PCB. But not only that.

You have an precise description of the LED PCB and reference to the MY9291 LED driver by Taiwan Mingyang (hence MY-Semi). It also confirms that the AC/DC transformer outputs 12V at 1A max. And describes two different step-down ICs. The MP1470 and the MP2359, both by MPS. The MP1470 has the ADJ mark.

But most interesting, the page includes a section on how to compile your own firmware for the OpenLight based on the Noduino SDK.

A library and a little bit of code

I checked out the Noduino SDK and there it was: a C driver for the MY9291. That’s everything I needed. By the way, I noticed the Noduino SDK includes my HLW8012 library (but not the most up-to-date version).

The driver is licensed GPLv3 and copyrighted to Maike Labs, Maike Labs is icamgo and icamgo is Noduino. You can use the driver as is but I took some time to encapsulate it in a library with some helper methods to manage the color and state of the LEDs. I have tested it with the ESP8285 in the AiLight but should work on ATmega328 or other AVR chips.

The MY9291 library for Arduino ESP8266 is released as free open software and can be checked out at my my9291 repository on Github.

Let me show you a simple example on how to set red LED on at 100% duty cycle (MY9291_COMMAND_DEFAULT defines 8-bit color depth so 255 means 100%):

#include <my9291.h>

#define MY9291_DI_PIN   13
#define MY9291_DCKI_PIN 15

my9291 _my9291 = my9291(
    MY9291_DI_PIN, 
    MY9291_DCKI_PIN, 
    MY9291_COMMAND_DEFAULT
);

void setup() {
    _my9291.setColor((my9291_color_t) { 255, 0, 0, 0 });
    _my9291.setState(true);
}

void loop() {}

Flashing it

It might look hard to solder those small pads in the beginning but it’s actually really simple. Get a 5 wire cable and remove 1-2mm of insulation from the wires, tin them a bit. Apply a hot wire on the pads and leave a small drop of tin on them too. Then just touch each wire with a pad and heat them together for a fraction of a second.

Connect the wires to you FTDI (or alike) board. TX to RX and RX to TX. Connect GND and IO0 to ground and finally 3V3. Once you plug your programmer to the computer the board will boot into flash mode. While you are flashing it you can remove the IO0 connection. Upon reboot it will enter normal mode and you should see the debug messages in the screen.

ESPurn’d

And of course it has been ESPurn’d. Version 1.6.7 of the ESPurna firmware supports the Ai-Thinker AiLight / Noduino OpenLight. You can turn on/off the light bulb and select the color from the web interface or you can do both things via MQTT from Node-RED for instance.

The post AiLight – A hackable RGBW light bulb appeared first on Tinkerman.

After a busy month I decided to spend some energy on doing hardware instead of software and the result was the ESPurna board I posted about just yesterday. The goal was to have a device based on the ESP8266 I could fit into my house wall gangs, with an SPDT relay to work with multi-way switches and power monitoring using the same IC the Sonoff POW uses: the HLW8012.

As a side project today I’ve been searching on the box of the TODO projects and I have rescued a KEMO STG15 [Ebay] plug housing with socket. These sockets are somewhat expensive and really bulky but the good thing is that there is quite some room to fit some electronics inside, but not a Sonoff board, too big.

But, what about an ESPurna board?

ESPurna inside

The ESPurna-H board is released under the Creative Commons Attribution-ShareAlike 3.0 Unported License (CC-BY-SA 3.0) and follows the terms of the OSHW (Open-source hardware) Statement of Principles 1.0. It can be checked out at my ESPurna board repository on Github.

The socket has some placeholders to screw a board inside, but the form factor does not match that of the ESPurna board. They draw a rectangle of 52x34mm while the ESPurna board has holes disposed in a square of 44x44mm. Then I thought about hot-gluing the board somehow. Problem is that the HLK-PM01 gets hot when operating and didn’t feel hot gluing was a good idea.

Then I started thinking about the ventilation dents on the sides of the socket. From the inside they look like rack slots with 6mm step between them. Maybe I could rack the board there? The ESPurna board is 50mm width. I needed around 60mm to rack it in both sides of the socket so I cut an adaptor from a piece of methacrylate of 60x50mm with holes for the ESPurna board.

The 2mm width plastic sheet fits tight in the vents. There is still room to let the air flow and even to add a case mounted push button and a notification LED. Finally I did add some hot glue on these two to prevent the cables to come loose inside the box.

Gallery

The ESPurna board, fully populated with the methacrylate adator behind.

Connections to the push button and LED. I hot-glued them to prevent the cables to come loose inside the case.

The adaptor fits nicely inside the KEMO STG15 between the side vents. You can also see the hot glued contacts for the push button and the LED.

All connections made. Ground to ground (yellow-green), neutrals connected to both N terminals in the ESPurna board (blue) and the power line to the NO terminal of the relay (brown).

Final result

The final result looks quite nice. Not as slim as the Itead Studio S20 but this one has power monitoring capabilities and flashed with the ESPurna firmware it supports MQTT and Alexa amongst other goodies.

Now it’s time to play with it. I’ve been monitoring my house power consumption for a few years already, now I have the tool to do it for each appliance and manage them remotely from my cellphone or from Node-RED based on different inputs.

The post ESPurna smart socket appeared first on Tinkerman.

A few months ago I wrote about the Sonoff SC sensor hub by Itead Studio. It’s a device with a Sharp GP2Y1010AU0F [Aliexpress] dust sensor, a DHT11 humidity and temperature sensor, an LDR as light sensor and a mic. The sensors are driven by an ATMega328P microcontroller but there is also an ESP8266 on board for WiFi communication, a pretty standard set up when you have several sensors and the ESP8266 GPIOs are just not enough.

On the first post I already did a small mod to replace the DHT11 humidity and temperature sensor with a more accurate and pin compatible DHT22. Since then, several readers have contributed with code and ideas. My progress implementing and testing them is slow, so I though about writing a first post about some modifications I (and others) have done to the device.

So this is a work in progress post. At the moment, all the code for these modifications is in the dev branch of the repository.

This custom Sonoff SC firmware is released as free open software and can be checked out at my SonoffSC repository on Github.

Flashing the ATMega328p via bootloader

First I’d like to say is that, although I used a AVR programmer for the ATMega328 in my last post this is not required. You can use an FTDI USB to TTL [Ebay, also Aliexpress] board to flash the AVR too since it comes with the Arduino bootloader.

Just remember to connect TX to TX and RX to RX since the pins in the board are labeled from the programmer point of view. Connect RST to the DTR line and the power lines and you are good to go.

Adding a microwave radar as presence sensor

Now, this is a suggestion in the issues section of the repository. Since the usual PIR sensors are quite bulky my first idea was to use one of those nice Panasonic EKMB PIR [Ebay]. They are small, very sensitive (with versions with up to 12m detection) and really expensive. They are ultra low power devices meant for low power sensors, but that’s not the case here.

So I moved to a microwave sensor I first spotted at Scargill tech blog. This is a 5.8GHz microwave sensor [Ebay, also on Aliexpress] that costs around 1.5 USD, has a configurable range of up to 8 meters and a detection pulse that’s 30s by default. The device can “sense” movement even through a thin wall so the SonoffSC enclosure is no big deal. But after a few tests I realized it was capturing people on the street outside my house so I decided to replace the resistor in R9_1 with a 100k one. Now the range is about 3-4 meters, safely covering the main walking paths in my livingroom. I also added a noise filter in the power line using a capacitor and a resistor as per Pete’s advice.

The microwave sensor is connected to GPIO13 (SCK) in the ISP header. A bit of solder to the VCC, GND and SCK lines and hot glue again to prevent it from going loose.

The placement of the sensor is still under testing. Right now I have it on the base of the SonoffSC, hot glued to it with the cables running to the oposite side of the board. Even thou the placement is neat, I have found that sometimes it misses a moving target, still testing if it’s due to the microwave beam or the code.

Since GPIO13 has no interrupts on the ATMega328P the code checks in the main loop for the pin state. If it changes it immediately reports the change to the ESP8266 via serial.

Adding an RGB LED ring – again

Actually this one was already done thanks to the contributions by Blair Thompson (check his video here). But I decided to go for a slighly different path. The ring is a 24 WS2812 LED ring [Ebay, also on Aliexpress] placed in the middle compartment of the enclosure. The VCC, GND and DI lines are brought out and connected to the ISP header like with the microwave sensor using GPIO 12 (MISO) now. Just like Blair’s approach.

WS2812FX

The difference with Blair’s code is in the library I’m using. On top of  the Adafruit NeoPixel library I’m using the WS2812FX library by Harm Aldick (kitesurfer1404). The library defines an indexed list of effects for LED stripes and so. Very convenient to be called from a remote MQTT based RPC.

Another incompatible change is that by default (although this might change in the final version) it performs a blue fade in when movement is detected and fades out when no movement. This behaviour is hardcoded in the current dev branch but I might:

1. Allow the user to configure the behaviour via the web interface
2. Use an external service (Node-RED) to write the business inteligence and just send data and receive instructions via MQTT

I like the second option better, moving the intelligence outside the device looks reasonable and more flexible. But I’m aware most users might want to have the behaviour embedded…

Blair’s code is still in the master branch, version 0.3.0. Once the current dev branch is merged to master it will be tagged as version 1.0.0.

Adding a mini-fan to improve dust sensor readings

One of the issues several people have detected is that the dust sensor has very little range. This is probably due to it being inside an enclosure with little air vent. There was a little debate about it in the repo and I decided to try a small SEPA MF15B-05 5V fan I found on Aliexpress. It’s a 15x15mm 0.06A fan.

The fan is driven by the LED signal in the board (GPIO7). I decided to hot glue it on top of the dust sensor so it forces the air to flow from the outside of the enclosure down. Whenever a new dust reading is scheduled or requested, the fan starts up for 5 seconds before the reading is actually performed. I have also changed the code to use ug/m3 instead of mg/m3.

I have been testing it for the last 2 days. Still not enough data but it looks like the daily average value is twice the value it used to be and the standard deviation is also greater. I still have to test with the fan running for a longer time (or even all the time) before reading the sensor data. The good thing is that the fan is noise-less (as far as I can hear).

Changes in the code

The code has also gone through a major review. Basically importing features from the ESPurna project. Aside from the changes to support the new functionalities in this post, other main changes include:

• Improvements in the SerialLink library (now it allows long integers as argument, good for RGB values)
• Debug via UDP (using Serial is not possible since it’s the communication channel between both microcontrollers)
• Moved debug strings to program memory
• Embed the webserver pages and resources into the firmware (faster, more free room for code): will write about it soon

There are already some other minor changes, improvements, bug fixes. I want to include more issues from the repo before merging this to master. Some of these improvements include new sensors. At some point you will have to choose from the different options since there are just so many GPIOs.

Flashing

Remember you will have to flash both microcontrollers again. But this time it won’t be necessary to flash the SPIFFS partition on the ESP8266. You can flash both using an FTDI USB to TTL programmer like the one I linked before. While flashing remove the communication jumpers. Also remember that the pins are labeled from the programmer point of view.

Since there are changes in the memory layout, erase the ESP8266 flash memory prior to upload the new code. The new layout is the 1m64 (1 Mbyte with 64Kbytes for SPIFFS). If you are using PlatformIO these settings are defined in the project file. Otherwise you will have to select the right option from the Arduino menu if you want OTA to work.

That’s it. As I have already said this is a work in progress so there might be changes and new additions to the project. Check the repo from time to time. I’m aware there are missing details here and there so don’t hesitate in asking. And thanks for reading me

The post Itead Studio Sonoff SC Revisited appeared first on Tinkerman.

When you are hacking with IoT devices at home you get to face the challenge of accessing remotely to them, that is from outside your home network. I’m not saying your home network is a safe place, beware. But that thing outside, you know, “the Internet”, it’s so scary… Unfortunately, most IoT devices are just not ready for the jungle. Neither the commercial ones, nor the hacked ones you might have. I wouldn’t dare to open a port in my router to anything inside unless it’s encrypted. So what should we do?

Options

There exist different apps that let you control your devices from anywhere you might be. Think of Blynk for instance. You can use SSL with the blynk-library for ESP8266 or Arduino. You can also do MQTT over SSL. But either one eats so much memory you can do very little else. ESP8266 is barely capable of handling one SSL connection, don’t ask it to also perform as a webserver.

The other popular solution is to create a secured entry point to your network and the most common configuration for that is a reverse proxy using Nginx with SSL. This is what I have at home and since I’m doing some changes to it (soon on another post) I have decided to write down here all the steps required to configure a Raspberry Pi as a reverse proxy to access you0 IoT devices (or any other service you might have at home) from the outside in a secure way.

I’m using a Raspberry Pi 3 at home which hosts several services (Node-RED, InfluxDB, Grafana, Mosquitto,…). It uses Raspbian 8 based on Debian Jessie. I guess you could use older version of the RPi but this one is really fast and I have had no problems so far.

Nginx is a web server, just like Apache. I have been using it as a web server for the last 6 years maybe because it’s so much light weight and faster than Apache and you can pretty much anything you want with it. But we will not be setting up a webserver here, but a reverse proxy. Nginx stands out as a reverse proxy precisely because it’s so light weight it adds very little overhead to the communication.

What is a reverse proxy, you might say? A reverse proxy is a service that sits typically on the edge of your network, just behind your router, and retrieves resources from inside the network. While doing that it might do some other tasks like translating adresses, encrypting/decrypting the communication or hidding resources, to name a few. Here we are interested in the encrypting/decrypting part. Our reverse proxy will encrypt the communication going outside our network and decrypt it when it comes back to us. So even thou our devices do not “speak” SSL all the communications going outside our network throught the proxy will be encrypted on behalf of them.

Install Nginx

Ok, hands on. I’m assuming you have a machine with a Debian Jessie OS (be it a Raspbian 8 or any other). If you have some other OS there might be some differences on the commands below but the overall picture should be the same.

First thing to do is to install Nginx. It couldn’t be easier:

sudo apt-get update
sudo apt-get install nginx

Done. You now should have a Nginx server running on your machine. By default it exposes the contents of “/var/www/html/” on port 80 (HTTP, not secured). If you write the URL of the machine on your browser you should see a welcome message from Nginx. Nice.

Install Let’s Encrypt agent

Now let’s install the SSL certificates. We don’t want to use port 80 but 443 (HTTPS) but before disabling it, let’s see how to create and install the certificates. You can create them yourself or use the awesome service Let’s Encrypt. You can easily create and deploy Let’s Encrypt certificates using EFF Certbot. Certbot comes as a command line tool that automates everything.

if you want to install Certbot on a Debian Jessie you will need to install it from the backports repository. Backports are packages that are being “ported back” from newer versions of the OS. The latests stable Debian is 9.1.

echo "deb http://ftp.debian.org/debian jessie-backports main contrib" | sudo tee /etc/apt/sources.list.d/backports.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B48AD6246925553
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7638D0442B90D010
sudo apt-get update
sudo apt-get install certbot -t jessie-backports

Get SSL certificates for your proxy

Now we just have to tell Certbot to request and deploy a new certificate from Let’s Encrypt. First Certbot will create a key pair and start the communication with Let’s Encrypt providing the public key. Let’s Encrypt must validate that we are administrators of the domain we are requesting the certificate for. So they will provide a challenge: a certain file has to be created under that domain. Certbot will create that file for us, sign the nonce with the private key and tell Let’s Encrypt that we are ready for the challenge. If everything goes OK now Let’s Encrypt will register that our public key is authorized to manage the certificates for our domain. You can read a (probably better) explanation about the process in the how it works page in letsencrypt.org.

So basically Certbot takes care of the whole process. Only, we need to ensure that Let’s Encrypt will be able to read the challenge file from our server under that certain domain. Now, I have dynamic IP at home so I use DynDNS.org services to access my server at home (usually only for SSH communications). Dynamic DNS or even subdomains are not supported by most certificate authorities, but Let’s Encrypt does support them. It may happen that browsers complain about the certificate or you might face an error saying there have been too many requests for a certain domain (like homeip.org, managed by dyndns.org) when requesting the certificate.

To solve this second issue I map a subdomain in one of my public (true) domains to the dynamic domain as a CNAME. So imagine you have a “geek.homeip.net” dynamic domain always pointing at your home IP. And you are also the proud owner of “fibonacci.com” (valued at 55k€ right now). All you have to do is to create a CNAME record in your main domain pointing your dynamic domain:

geek.fibonacci.com.        CNAME  geek.homeip.net.

Of course if you map a domain to your dynamic one you might as well solve the first problem above. For the moment I’m living with it…

You will then issue a request to get the certificates for geek.fibonacci.com. On your home router you will also have to open a port to your proxy server. Opening a port simply means mapping the communications coming to a certain port to a machine inside your network. You will need to look for “NAT” on your router configuration and tell the router to redirect all communications on port 80 (HTTP) to your proxy server. You will need to do the same for port 443 (HTTPS).

Now you are ready to instruct Certbot to request the certificate:

sudo certbot certonly --webroot -w /var/www/html -d geek.fibonacci.com

If you are getting errors double check that you can reach your Raspberry Pi from outside your network. I normally place a file with some random content on “/var/www/html/hello.txt” and then I visit “http://geek.fibonacci.com/hello.txt” from outside my home network, disabling WiFi on my mobile phone, for instance. If there were no problems you can check where the certificate and key have been stored on the system like this:

xose@acrux:~ $ sudo certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Found the following certs:
Certificate Name: geek.fibonacci.com
Domains: geek.fibonacci.com
Expiry Date: 2017-12-01 09:38:00+00:00 (VALID: 89 days)
Certificate Path: /etc/letsencrypt/live/geek.fibonacci.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/geek.fibonacci.com/privkey.pem
-------------------------------------------------------------------------------

Configure the reverse proxy

Now that we have the certificate let’s configure Nginx to work as an SSL reverse proxy. I usually create a “/etc/nginx/sites-available/proxy” file with these contents:

server {

    # general server parameters
    listen                      443;
    server_name                 geek.fibonacci.com;
    access_log                  /var/log/nginx/geek.fibonacci.com.access.log;       

    # SSL configuration
    ssl                         on;
    ssl_certificate             /etc/letsencrypt/live/geek.fibonacci.com/fullchain.pem;
    ssl_certificate_key         /etc/letsencrypt/live/geek.fibonacci.com/privkey.pem;
    ssl_session_cache           builtin:1000  shared:SSL:10m;
    ssl_protocols               TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers                 HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
    ssl_prefer_server_ciphers   on;
    
    location /livinglamp/ {

      # header pass through configuration
      proxy_set_header        Host $host;
      proxy_set_header        X-Real-IP $remote_addr;
      proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header        X-Forwarded-Proto $scheme;

      # extra debug headers
      add_header              X-Forwarded-For $proxy_add_x_forwarded_for;

      # actual proxying configuration
      proxy_ssl_session_reuse on;
      proxy_pass              http://livinglamp.local/;
      proxy_redirect          default;
      #proxy_redirect          ^/(.*)$ /livinglamp/$1;
      proxy_read_timeout      90;

    }


}

There as some things here to note. First we specify the server name (geek.fibonacci.com) and we listen on port 443 (HTTPS). Remember you will have to open that port in your router pointing to your reverse proxy machine. Then we configure the certificates, the routes are the same we got previously when running “sudo certbot certificates”. Then we can define as many locations as devices or services we want to map under that domain. Here I just have one (/livinglamp/) mapped to an Itead S20 I have on my living room flashed with ESPurna. The local address of the device is used here (livinglamp.local). As you can guess, we are saying that all traffic comming to “https://geek.fibonacci.com/livinglamp/” should be redirected to “http://livinglamp.local/”. We are translating a public URL to a private one and changing from HTTPS to HTTP.

The tricky part here is the proxy_redirect option. It defines the “translation back” and will depend on how your device fetches other local resources like images, for instance. Also this configuration does not deal with query strings, the part of the URL that comes after the ‘?’.

Autorenew certificates

Once you have this working you can go back to your home router settings and delete or disable the map you did on port 80 (HTTP). The only ports you should open are those meant for encrypted communications (HTTPS, SSL, MQTT of SSL,…).

No. Don’t do that. You will need that port to autorenew your SSL certificates. Let’s Encrypt certificates have a validity of 90 days so you will be reneweing them often or forget and the whole set up  will fail. That’s why autorenewing is cool.

Leave that connection open but beware that anything under “/var/www/html” will be publicly visible. We will be using a crontab to try to renew them weekly, if they are not close to the expirity date it will just end silently. We will have to do it from the root crontab:

sudo crontab -e

The file will probably be populated with some explanation and maybe some crontab rules already. We will add these two lines at the bottom:

@weekly /usr/bin/certbot renew --quiet --no-self-upgrade -w /var/www/html/
@daily  service nginx reload

Basically we are asking the crontab to execute the autorenew procedure once a week, and since Nginx won’t reload the new certificates we are also reloading it once a day.

That’s all. I hope this helps you having a more secure experience accessing your IoT devices from the Internet. Of course any comments or ammends you might have, just use the comments section below.

The post Secure remote access to your IoT devices appeared first on Tinkerman.

In my last post I tried to explain how to access your IoT devices at home from the Internet in a secure way using a reverse proxy. Truth is that I had it running since maybe 6 months ago without giving it too much use until recently. Reason? My Nexus 5 had been having serious problems (battery not charging, screen broken, earpiece not working,…) and I decided to replace it with a new mobile phone and the new one has Google Assistant built in.

So one of the first things I have tried is to make Google Assistant toggle my smart switches flashed with ESPurna. Nad as it turns out it’s not hard to do but -at the moment- you have to relay on a cloud service like IFTTT (IF This Then That). This is a key difference with Amazon Alexa services you have to be aware. It has some benefits but also some drawbacks.

Amazon Alexa vs. Google Assistant

The “vs.” might sound a bit like a fight but I’ll be focusing on few aspects on the way the two work when managing IoT devices like smart switches.

First let me name things correctly. Amazon Alexa is the service behind devices like Amazon Echo [Ebay] or Amazon Dot [Ebay]. The same way Google Assistant is the intelligence behind Google Home [Ebay]. On Android or iOS you have, for instance, Reverb, which is an app that uses Amazon Alexa to answer your commands. You can use it with your smart home devices that support Alexa too. Same thing for Google Assistant or Allo, both apps use Google Assistant servers. So I’m not focusing on the devices here but the service underneath, although some differences between the two have to do with the devices themselves.

Amazon Alexa

On one hand the standard way to use Amazon Alexa is using skills, either the ones you can create or third party ones available in the Amazon Alexa web page. Here you have skills to control your Philips Hue [Ebay] or your Belkin Wemo switches [Ebay], amongst others.

The case of the Wemo switches is well known since the protocol was reverse engineered. Basically, your Amazon Alexa powered device is listening for a key work to wake up (“Alexa”or “Echo”), then it records your command and sends it to Amazon servers to decode it. The command, now converted in something your device can understand, comes back to it and it executes the action. For Wemo switches it an be a discovery process (Belkin uses a standard uPNP process) or an event, a custom TCP connection requesting the switch to turn on or off.

Since we know the protocol we can emulate being a Wemo switch and accept commands from an Amazon Echo or Dot, that’s how my fauxmoESP library for ESP8266 works. Unfortunately, not all Amazon Alexa powered devices support the Wemo protocol the same way Echo or Dot does, and that means that if you are using a AlexaPi on a Raspberry Pi or Yonomi on your mobile phone you are out of luck. Reverb is a brilliant exception, since it looks like it communicates with your Amazon devices and commands them to execute the actions.

Anyway, for this specific use case (emulating being an Alexa compatible smart switch), only the translation from your voice command to machine language is performed outside your network. The communication with Alexa Voice Service (AVS) is done using HTTPS so, as long as you trust Amazon you are good and secure.

Google Assistant

As for Google Assistant (GA), everything is a service, as you might have guessed. Google is partnering with different providers to include options for their services on Google Assistant. One of such partners is IFTTT (IF This Then That) which means you can create custom behaviours responding to your voice commands. That’s pretty cool and IFTTT has already a ton of different options. One of this options is WebHooks, which provides an API to perform HTTP requests.

So the standard way to interact with your IoT devices using Google Assistant implies that:

1. You speak your command
2. Google voice services transcript it and search the different providers for one that can handle it, including IFTTT
3. IFTTT grabs the command and tells GA “yeah, I can do that”
4. IFTTT translates it again based on an applet you have defined
5. The action in your applet instructs IFTTT to tell WebHooks to do an HTTP request
6. WebHooks performs the request to an address that points to your device
7. Your device executes the action

Some very important things are missing in the previous explanation. On point 6 you have to have a way to redirect that request to your device but that’s exactly what I was explaining on my previous post about secure access your network. And on point 7 your device has to understand the command. Well, ESPurna does have an HTTP API!!

The main differences with the way Amazon Alexa works are obvious here. You have 3 different cloud services involved here: Google Assistant itself, IFTTT and WebHooks. Again it’s a matter of trusting them (!!). And the final command comes from outside your network, always.

HTTP API for your DIY IoT device

Stop writing and let’s do something to make this work. First thing is to be able to send an HTTP request to your device from outside your network. If you have read my previous post about secure accessing your home network you already know the basics. Go ahead and install a reverse proxy with HTTPS support. That’s the first thing to do.

Second requirement is to have an device with an HTTP API. Some commercially available do. If you are a maker and you use ESP8266 on your home devices (and if you are reading this blog chances are you are) you know some open source firmwares for ESP8266-based smart switches/sensors do have an HTTP API. And ESPurna does have. Of course, you can also build your own firmware or use an REST library like aRest.io.

For instance. I can control my living room light using my browser by browsing to “http://livinglamp.local/api/relay/0?apikey=0123456789ABCDEF&value=2” (faked API key :). That URL will toggle the light (value=2 means “toggle”, 0 for “off” and 1 for “on”).

From the command line using cURL I would do a PUT request instead since it’s more ortodox (ESPurna supports GET when setting values out of convenience):

curl -X PUT http://livinglamp.local/api/relay/0 --data "apikey=0123456789ABCDEF&value=2"

But ESPurna uses HTTP, not HTTPS. Inside your home network it’s fine (mostly) but that’s not an option from the outside. Here’s where the reverse proxy comes to play. The configuration in my previous post maps this device inside my home to a certain domain and URL inside the domain and only allow HTTPS communication, so I can do the same test from a machine outside my network, using also cURL:

curl -X PUT https://geek.fibonacci.com/livinglamp/api/relay/0 --data "apikey=0123456789ABCDEF&value=2"

Here “geek.fibonacci.com” is the example subdomain I’m using. As you can see the protocol and URI has changed from “http://livinglamp.local/” to “https://geek.fibonacci.com/livinglamp/”, no more changes are needed. And it does toggle my ESPurna powered lamp!

Creating an IFTTT applet

Now let’s configure the applet in IFTTT. You will have to create an account if you have not created one before. Then go to “My Applets” and click on “New Applet”. An applet is a combination of a trigger and an action. Both are provided by “services”. Our trigger will be provided by Google Assistant. Click on the “this” word and look for “google assistant” on the search box.

Each provider could provide one or more services, here we need the “Say a phrase with a text ingredient”. We will use the text ingredient to specify wether we want to switch it on, off or toggle the switch. As you can see in the screenshot below you can configure several (up to 4) different ways to say the command and the dollar sign will be replaced by the ingredient.

Once you have it configured click next and choose an action. In this case the only one provided by WebHooks, the “Make a web request”. We will just fill the fields with the same option as in the cURL command before replacing the value field in the body with “{{TextField}}”, to pass the ingredient that Google Assistant has identified.

Finally it will ask you to name the applet and once activated you are ready to go. In my case I click and hold the home button in my mobile phone to wake Google Assistant and speak “Turn on living lamp” and enjoy.

NOTE: As you might have noticed the “ingredient” will be a word. ESPurna 1.9.3 (still not realeased) supports “ON”, “OFF” and “TOGGLE” to change the status of the relay. If using a previous version or your device only supports numbers you might want to test the “Say a phrase with a number” trigger service instead. In this case you might want to be able to say “Set living room status to 1”, for instance. Less human-friendly but still usable.

All an all this approach is more flexible than that from Amazon, even thou there are more services involved. Using IFTTT alone, without Google Assistant, can provide an easy way to do schedules or actions depending on, for instance, an email or a tweet. That’s something you can also do with Node-RED for instance but IFTTT provides an easier way to do it…

As always, comments welcome!

The post Using Google Assistant to control your ESP8266 devices appeared first on Tinkerman.

Itead Studio has been releasing interesting gadgets for the Home Automation community based on a low price tag and extreme hackability. You can google “sonoff” (the main brand for Itead Studio home automation devices) to get more than a million hits, including official pages, reviews and lots of hacks. The ubiquitous ESP8266 (or its sibling ESP8285) is the core of all those devices, using WiFi instead of the traditional RF messages, replacing a remote with mobile apps or voice commands. But also, using custom firmwares like ESPurna, technologies and solutions like MQTT, Node-RED or Home Assistant. But one of the latests devices from the chinese firm tries to bridge the gap between those two technologies: the Sonoff RF Bridge 433.

Itead Studio was kind enough to send me an engineer sample before holidays but I have had very few time to work on it. I should have written this post a month ago. Anyway you can buy one from the Itead Studio webshop or in the usual online markets looking for Sonoff RF Bridge [Aliexpress].

Out of the box

If you had read me before you might know that I have critisized the eWeLink app. It’s not that I didn’t like it. I just couldn’t make it work. With every new Sonoff device I gave it a chance and the outcome was always the same. Until now. Last month I tried it with the RF Bridge and the 4CH Pro and it worked like charm. Don’t know what was wrong. Maybe an uncompatibility with some other service or app in my phone. At some point the problem has been addressed and I could finally test it.

It works, you can discover the device, force pair mode for any of the 4 available buttons and use them. it also work with Alexa and eWeLink custom skill. Nice. But I just don’t want to use it. I will not go (I have never gone) much into the app, the user experience, the use cases,… because this is about hacking it with he aim of really own the device and have the freedom to choose what 3rd party services you will use with it, if any at all.

The device

Outside

The Sonoff RF Bridge 433 is a small black box with a microUSB port for power, a recessed reset button and three LEDs for notifications (power, custom and radio). The button is connected to GPIO0 on the ESP8266 so it doubles as flash mode button. In the original firmware it sets the device into “discoverable” mode, so you can use the eWeLink app to discover it.

MicroUSB and reset button.

The three LEDs in the front of the device are used in the original firmware to notify radio usage (red), wifi connection (blue) and power (green)

Short section, right. There is really not much more about it. A sticker with a QR linking to the eWeLink user guide.

Inside

Opening the box is quite easy with the help of a couple of plastic opening tools. Inside the box there is a single PCB with all the components.

A single PCB holds all the components

On one of the corners you can spot the PCB WiFi antenna and the ESP8285 (an ESP8266 with a 1Mbyte flash memory embedded) along with the required header to flash it (3V3, RX, TX and GND). You also have access to GPIO2 (labelled SDA) on the same header.

In the center of the board you can see the EFM8BB1. An 8-bit microcontroller by SiLabs that manages the radio communication. This is the same microcontroller IteadStudio has used for the Slampher and the Sonoff RF. Moving the real-time encoding and decoding out of the ESP8285 has benefits since those are very time-picky functionalities that could interfere with the WiFi communications.

The EFM8BB1 microcontroller is responsible for encoding and decoding radio messages

But the EFM8BB1 is not alone. There is a SYN470R ASK superhet receiver decoding the messages. You might have also noticed two quarter-wavelength wire antennae, one close to the receiver and the opposite one for transmitting.

The SYN470R ASK decoder and the receiving wire antenna

You can also spot a buzzer for pairing notifications and the ASM1117 regulator.

Finally, a SPDT switch that controlls the communications between the ESP8285 and the EFM8BB1. The communication is done via serial (RX and TX) and that could be a problem when trying to flash a new firmware. Moving the switch to the OFF position you effectively disconnect the communication between both microcontrollers.

The back of the board is empty, all the components are on the front

Enough about the PCB. You can check the schematics for the Sonoff RF Bridge 433 yourself at the IteadStudio wiki.

ESPurna

Flashing it

As we have seen the board has all the connections to allow anyone to flash a custom firmware on the ESP8285 (and on the EFM8BB1, but I will focus on the Espressif chip). All you have to do is:

• Connect the ESP8286 header to your USB2UART programmer (RX to TX, TX to RX and GND to GND)
• Place the switch in the OFF position
• Press and hold the reset button
• And power the device either via the microUSB port or via the 3V3 pin on the same programming header

At that moment the ESP8285 will boot into flash mode, ready to get a new firmware. If you are using PlatformIO (recommended), just checkouot the ESPurna repository, browse or open the Atom IDE on the code folder and build and flash the itead-sonoff-rfbridge environment. From the console it would be something like:

pio run -e itead-sonoff-rfbridge -t upload

Done. If you are using Arduino IDE instead check out the instructions from the Arduino IDE page in the ESPurna wiki to configure the environment and flash the board.

Since 1.9.0 ESPurna fully supports Sonoff RF Bridge 433 but the approach the firmware uses is quite different from that on the official app.ç

The ESPurna firmware is released as free open software and can be checked out at my Espurna repository on Bitbucket.

Public API

It turns out that the API between the ESP8285 and the EFM8BB1 is public under the serious name of RF Universal Transceive Module Serial Protocol. The document is spare, has one or two typos and misses some basic information, but it’s good enough to interface the EDM8BB1 from you own ESP8285 firmware.

The key points in the API is that:

• You can set the device in “listen mode” which basically beeps the buzzer to provide some user interaction
• Any code the SYN470R decodes the EFM8BB1 will report it to the ESP8285
• The ESP8285 can send raw codes to the EFM8BB1 and it will forward them

So we have a good enough API to work with. The ESP8285 handles the user interaction, stores codes (there is room for more than a 100 different codes in the emulated EEPROM) and commands the 8-bit microcontroller.

From the ESPurna web UI you can manage the stored codes. By default you have 6 ON/OFF pairs but you can change that in the configuration files, anyway it way more than the 4 codes the official app can handle. Once “learnt”, you can use them from the main status page, like you would with any other switch.

Once you have the codes assigned to a certain “virtual” switch you can manage the from MQTT easily by switching the false “relay” on and off:

mosquitto_pub -t /test/rfbridge/relay/0/set -m 1

or using the REST API so you can do that from outside your home too. Check my post about secure access to your IoT devices from the intenet, also with Google Home.

MQTT power

But the firmware also exposes direct access to the Receive Key and Transmit Key Value entry points. That means you can sniff RF messages from Node-RED, for instance, and send them by simply copy paste the codes. Here you have a screenshot of a fully functional interface for the Sonoff RF Bridge 433 in Node-RED. That’s all it takes.

You can sniff and store as many codes as you want or use other apps or services to send them via MQTT. The Sonoff RF Bridge 433 as a true, transparent, MQTT-433MHz-MQTT bridge.

Will it work with any RF device?

I have a couple of old RF switches sets at home. I have used them in the past and time has come to give them a second life. These have slightly different implementations based on the Manchester encoding but since the code does not matter about the message structure itself it shouldn’t matter, right?

Well, as it turns out one of the remotes (branded Noru) works flawlessly but with the other one (Avidsen) the RF Bridge struggles to understand the messages and often requires several tries before they got learnt. So what’s the difference?

My only guess at the moment is the timing. The code structure is simple:

• Sync time (2 bytes)
• Low level time (2 bytes)
• High level time (2 bytes)
• Data (3 bytes)

So this is what a Noru message looks like: “24E0 0140 0384 D00358”. That means: 9440us sync time, 320us low time and 900us high time. Data is “D00358”, this identifies the button in the remote. The timing is not necessarily strict, but you can see the high time is 3 times longer than the low time and the sync time is 10 times longer than the high time. And the minimum pulse length is around 320us and that’s about 3KHz. The SYN470R datasheet talks about a max frequency of 5KHz when in sweeping mode. The pulses in the Avidsen look like being faster than that… but more investigation is needed here.

RF relays

But apart from the Avidesen remotes all the other devices I have tested have been a success. All the RF enabled products by IteadStudio, of course but I have also bought some extremely cheap RF relays to play with.

The ones I bought are 1 channel 433MHz enabled relays [Aliexpress]. You can buy them alone, with a remote, with an enclosure and they come also in multiple channels [Aliexpress]. They work best in interlock mode, with two different commands to switch them on and off. This way ESPurna will transmit each code several times to ensure the relay has been switched, otherwise it will only transmit it once.

Anyway it’s a really cheap solution for home automation and start controlling your assets with MQTT, Alexa, Google Home or a bunch of other solutions.

Overall this is an enabling device. If you are still using 433MHz devices at home, or if you decided to stop using them because, you know, you never find the remote when you need it, now it may be time to make them talk MQTT (somewhat).

The post Hacking the Sonoff RF Bridge 433 appeared first on Tinkerman.

Six months ago I was reviewing the AiThinker AiLight, a great looking light bulb with an embedded ESP8266EX microcontroller, driven by a MY9291 LED driver. Just before summer IteadStudio released it’s Sonoff B1 [Itead.cc] light bulb, heavily inspired (probably same manufacturer) by the AiLight, at least on the design.

Now that IteadStudio has become popular between the home automation community you can also find the Sonoff B1 on global marketplaces like Ebay or Aliexpress for around 13€.

A closer look at the B1 uncovers some important differences. But before going deeper into the details let me first say that this post will probably look more like a review, at least more than I use to write. And second, yes: ESPurna supports the Sonoff B1

An unboxing?

Not quite so. I leave that to other people with better skills on the video editing world. Let me just tell you than the “box” is somewhat different from what I expected. You might recall the AiLight box: a simple beige drawer-like box with a “WiFi Light” text and a simple icon. No colors, pictures, specifications,… nothing.

Instead, the Sonoff B1 I received from IteadStudio comes in a colorful box, with the usual pictures and data you can find in retail products.

Inside the box the light bulb is comfy housed in a polyethylene foam, along with a quality control certification and a small “getting started” manual in English and Chinese.

A heat sink?

Don’t think so. The first thing I noticed when I opened the box was that the bulb was very similar to the AiLight, the second the only visual difference. It certainly looks like a big heat sink. I almost fear touching it while connected. But how much heat can you generate if the light is rated 6W? The bulb body houses a basic AC/DC power supply (90-250VAC to 12VDC) and is accessible unscrewing the metal frame (the heat-sink part from the smooth part with the “sonoff” logo).

The AiLight is also 6W and you can safely touch it, even when it has been at full power for a lot of time. The Sonoff B1 shouldn’t be different. So I’m lean towards thinking it’s an aesthetic decision. Unless there are some beefy power LEDs inside.

Power LEDs?

Not all of them. Anyway I think this is the aspect where the B1 clearly differentiates from the AiLight. The later has 8 cold white power LEDs, as well as 6 red, 4 green and 4 blue power LEDs. The Sonoff B1 also has 8 cold white ones. But then it features 8 warm white power LEDs and 3 5050 RGB LEDs!

I don’t have a luximeter but the difference when fully white between the two is hard to spot. But the warm white color really makes the difference in favor of the Sonoff bulb. On the other hand, the 3 5050 SMD LEDs are clearly not enough. Even more: since the RGB LEDs are closer to the center of the round PCB, just around the WiFi antenna, the shadow of the antenna is very noticeable if you are using a colored light.

Hard to tell which one is brighter for the naked eye…

The pic does not justice the difference. The right on is the AiLight with the white power LEDs at full duty. The left on is the Sonoff B1 using the warm white power LEDs (you can see the yellowish color in the wall). The cold white LEDs are brighter but, depending on the room, the warm white LEDs could be more suitable.

Both bulbs again, now with the red channel at full duty. No need for words.

3 5050 RGB LEDs, 3 shadows of the antenna

A view without the cap, red LEDs are at 100% duty cycle, white LEDs are only at 10%…

I think the Sonoff B1 could be a better choice when used to illuminate with a warm white light your living room or your bedroom than the AiLight. If you need a colorful illumination, discotheque moods or a nice cold white for your kitchen, use the AiLight. Another possible (and interesting) use for Sonoff B1 would be as a notification light using traffic light color code, for instance. Clearly visible but not disturbing colors.

The controller?

Not the same. It is actually an ESP8285. In practice, you can talk to it like if it was an ESP2866 with a 1Mb embedded flash using DOUT flash mode. So that’s my recommended configuration.

The ESP8285 and required components with the 5050 RGB LEDs

As you can see in the pictures, the PCB is actually 2 PCB, one for the power LEDs and the other one for the microcontroller, some components and the 5050 on the front, a buck converter (12VDC to 3.3VDC for the ESP8285) and the LED driver on the back. The two PCBs are soldered together and glued to the underneath support.

In the AiLight the LED driver is a MY9291 [datasheet, PDF] by My-Semi. The Sonoff B1 uses another My-Semi driver, the MY9231 [datasheet, PDF]. The MY9291 is a 4 channels LED driver but the MY9231 is just 3 channels… so how is it possible to do RGB plus white and warm? Well actually these ICs are daisy chainable, so there are two MY9231 controllers in the Sonoff B1, the first one controlling the white power LEDs and the second the 5050 RGB LEDs.

I did not want to remove the glue under the PCB. But you can glimpse one My-Semi controller through the bottom hole.

ESPurna?

The ESPurna firmware is released as free open software and can be checked out at my Espurna repository on Bitbucket.

Sure! You can flash the Sonoff B1 following the same procedure of the AiLight. There are 6 pads on the PCB labelled 3V3, RX, TX, GND, GPIO0 and SDA. You will need to wire the first 5 (tin you cable, apply a small drop on the pad and then heat them together). Connect RX to TX, TX to RX, GND to GND, GPIO0 to GND and finally 3V3 to the 3V3 power source of your programmer. It will then enter into flash mode (GPIO0 is grounded). You can either flash the bin file from the ESPurna downloads section or build your own image (check the ESPurna wiki for docs).

Wired flashing of the Sonoff B1

Since ESPurna version 1.9.0 you define and control any number of dimming channels, you can also define the first three to be RGB channels. If you do, the web UI will show you a colorpicker to select the color.

You can also control it via MQTT. It supports CSS notation, comma separated or color temperature, as well as brightness and status, of course.

// 100% red
mosquitto_pub -t /home/study/light/color/set -m "#FF0000";

// 100% warm white
mosquitto_pub -t /home/study/light/color/set -m "0,0,0,0,255";

// 300 mired color temperature
mosquitto_pub -t /home/study/light/color/set -m "M300";

// 4000 kelvin color temperature
mosquitto_pub -t /home/study/light/color/set -m "K4000";

Of course you can also use Home Assistant MQTT Light component. The configuration would look like this:

light:
  - platform: mqtt
    name: 'AI Light TEST'
    state_topic: '/home/study/light/relay/0'
    command_topic: '/home/study/light/relay/0/set'
    payload_on: 1
    payload_off: 0
    rgb_state_topic: '/home/study/light/color'
    rgb_command_topic: '/home/study/light/color/set'
    rgb: true
    optimistic: false
    color_temp: true
    color_temp_command_topic: '/home/study/light/mired/set'
    brightness: true
    brightness_command_topic: '/home/study/light/brightness/set'
    brightness_state_topic: '/home/study/light/brightness'
    white_value: true
    white_value_command_topic: '/home/study/light/channel/3/set'
    white_value_state_topic: '/home/study/light/channel/3'

Either way, flashing a custom firmware like ESPurna on a 13€ Sonoff B1 [Ebay] device allows you to first fully control your device (no connections outside your home network if you don’t want to) and second, make it interoperate with other services like Home Assistant, Domoticz, Node-RED or any other MQTT o REST capable services.

After all, I’m talking about Technological Sovereignty.

The post Sonoff B1, lights and shades appeared first on Tinkerman.

Some time ago I worked on a home project to get a notification when my washing machine had done its job based on monitoring its power consumption. There was a good reason for that, the machine was outside the house and I had already forgotten about the laundry several times. And when that happens your only option is to wash it again, because it really smells musty…

Monitoring your appliances

Use ESPurna

OK, there are different ways to get the info about power consumption. But since we want to be able to process the data ourselves most commercial products won’t be suitable unless we modify it.

Alternatively, those that use radio communication to send data from the meter to the base station might be suitable for a man-in-the-middle hack. For instance, if you own an Efergy power meter you must know you can sniff the data it sends using a simple RTL-SDR dongle.

But for most cases, your best chance is to get your hands on a commercial product with an ESP8266 chip in it and change the firmware to suit your needs. You can write your own or use an existing firmware like ESPurna, that already supports a bunch of power metering smart switches.

What info do you need?

The idea is to report (via MQTT) power data from each individual appliance very minute. You can then use Node-RED along with InfluxDB and Grafana (or Graphite) to receive, persist and graph your data like in the image below.

There is a lot of useful information you can get just by graphing this data: device power footprint, device health, power consumption habits, duty cycles (for the fridge, for instance), max minute power (a key factor in your electricity bill) and, of course, foresee your next bill.

The info you will be getting has two main limitations: sensor precision (and error) and data granularity. Your sensors (clamps, power monitoring chips,…) will have a precision of maybe 5, 10 or 20W. And, even thou you are maybe reading them every few seconds you are probably averaging to get minute readings.

With this info, you will not be able to “see” an LED light turning on and off and you will probably get a baseline of a few tens of watts that you won’t know where they exactly come from. But for some “big” appliances like your washer machine, your dryer, the oven, the dishwasher and such, you will get a pretty accurate footprint since their duty cycles usually last for 30 minutes, 60 minutes or even more.

One thing you can do is to monitor their power consumption to know when they are done (my washer machine example before). But since you have some significant data you can think on adding a bit more info to that “your laundry is done” message.

Analysing the data with Node-RED

The simplest analysis you can do on the power data is to know if the appliance is using any power at all. A value over 0W would then trigger a counter that will be aggregating energy (power times time) until the power goes back to 0W.

Problem is that some appliances might have duty cycles with periods of inactivity and you might also face the problem of having spurious or non-zero baseline power values (those damned LED pilots!). So you might want to set the power threshold to something different than 0 and maybe also set a “grace time” before triggering the done event.

I’ve been doing this already using a function node in Node-RED for some time now. But now that I wanted to add more info and connect more appliances to the flow I decided to put all the logic into a proper Node-RED node.

node-red-contrib-power-monitor

The node-red-contrib-power-monitor node is released under the Apache 2.0 license and can be installed from the Node-RED Manage Palette, using npm or checked out from my node-red-contrib-power-monitor repository on GitHub.

The node (node-red-contrib-power-monitor) accepts a real number as an input payload (in Watts) and does the analysis of the info based on 3 settings: the power threshold, a start grace period and a stop grace period.

The node installation is very simple and it does not have any external dependency. You can install it directly from the Manage Palete in you Node-RED UI or using npm from your node-red folder (typically ~/.node-red):

npm i node-red-contrib-power-monitor

Using it

The node configuration allows you to make it match the appliance power footprint with simple settings:

• Name: Name of the appliance. Will be attached to the output object.
• Power threshold: Value (in watts) to tell whether the appliance is running or not, an ideal value would be 0 (0W if not running).
• Start after: Number of messages with readings over the threshold to trigger a start event.
• Stop after: Number of messages with readings below the threshold to trigger a stop event.

Examples

My washer machine below shows a clear and easy to analyze pattern. The power baseline is noiseless (I use a Sonoff POW R2) and the footprint does not show any 0W valley in between. Setting the default values for the node configuration (0W threshold and 1 for start and stop after counts) would work just fine.

The dryer is more tricky, but still easy. As you can see the base line is not that clear (I’m using a Sonoff POW and it reads noise of up to 5W). But the activity power value is very obvious and over 500W constantly. Once it’s done it enters a maintenance cycle to prevent wrinkles that lasts for an hour but the drying cycle is already finished. I’m using a threshold of 100W and leave the start and stop after counts to 1.

Finally, the dishwasher shows a more complex pattern with 3 different heating periods with low power activity in the middle raging from around 8W to more than 50W. In the end, there is a standby period with readings of around 3W. I’m using a custom board based on the HLW8012 power monitoring chip here, the same as in the POW but the baseline is pretty clean and reads 0W. I use a 5W threshold here but combined with a value for the start and stop counts of 5 to prevent false positives.

Output

Each node will provide real-time info about the running time and energy consumption so far for that given cycle.

The output is split into two, a connector for the start event and another one for the stop event. The start event is just that, a JSON payload with an event type “start” and the name of the appliance. The stop event adds a couple of very useful values: the total time the appliance has been running in seconds and the total energy in kWh.

You might want to pre-process this info before feeding it to a template node to build a message. In the example above the “pre-process” function node calculates the time in minutes and the cost, assuming a flat rate, but it since you know the current time and the total running time you can make a fair estimate in case you have some kind of time discrimination.

Pre-process code:

msg.payload.time = Math.round(msg.payload.time / 60);
msg.payload.cost = Math.round(msg.payload.energy * 16.66) / 100;
return msg;

Message template (mustache syntax):

The {{ payload.name }} has finished.
Total time: {{ payload.time }} minutes.
Total energy: {{ payload.energy }} kWh.
Cost: {{ payload.cost }} €

Nice notifications using Telegram

Now that we have all the info it’s time to set up nice notifications. A while ago Germán Martín published a screenshot of his phone receiving a notification from his @iotfridgesaver project. He was also using ESPuna and Node-RED and the results where superb. I just had to copy him

Gracias a @NodeRED, #esp8266 y #espurna mi lavadora me avisa por #Telegram cuando termina y me dice cuánto ha costado la energía de ese lavado. @xoseperez @iotfridgesaver pic.twitter.com/2YimV5vO7C

— Germán Martín (@gmag12) December 25, 2017

Creating a Telegram bot and start sending messages from Node-RED is really easy. You can find a lot of resources on the Internet so I won’t go into the details. Just a small script of what you have to do:

• Open a chat with @BotFather he-she-it will guide you through the process of creating a bot.
• Install the node-red-contrib-telegrambot node in your Node-RED
• Configure the token of the node to that provided by the @BotFather
• You will also need a chatId where the bot has to send the messages. The easiest way is to use a bot like @my_id_bot. If you open a chat with it it will tell you your chatId so your bot will send you direct messages. If you invite @my_id_bot to a group it will tell you the group ID, very useful if you want to share notifications with someone else.

The node-red-contrib-telegrambot accepts a JSON payload with the info to send. You can use this template to build it from the previous payload:

{
    "chatId": 224857347,
    "type": "message",
    "content": "{{ payload }}"
}

You can hardcode the chatId in the node or pass it as a parameter like in the example above. The result will look like this:

Cool?

The post Useful notifications from your home appliances using Node-RED appeared first on Tinkerman.

This time I’d like to write a quick post about a small project we’ve been working on at the The Things Network community in Barcelona (@ttncat). We were worried about the monitoring of the gateways we have deployed, both as a community but also as individuals. Some of our partners have also deployed their own gateways and they are part of the community.

So how do we get (almost) real-time notifications of incidences in the local TTN network? We want to be able to track them as soon as possible and open the proper ticket in our management system to solve them if the solution is in our hands or notify the entity responsible for the gateway.

An undocumented API

There are several ways to get info from the TTN network. To be honest it is somewhat untidy at the moment. Of course, you can use the console but that’s not an option if you want to automate processes. MQTT entry points are fine for data retrieval but there are no (known) MQTT entry points for gateway metadata. Instead, we have to fall back to gRPC or an HTTP API.

There is a simple (documentation missing) API entry point you can use to query all gateways in a given zone, a country, or even the whole world!

# query all gateways 50km around Barcelona
https://www.thethingsnetwork.org/gateway-data/location?latitude=41.3870734&longitude=2.1700660&distance=50000

# query all gateways in Spain
https://www.thethingsnetwork.org/gateway-data/country/es

# query all gateways in the world!
https://www.thethingsnetwork.org/gateway-data/

The output from these queries is a JSON object with all the gateways for that specific scope. Like this:

{
  eui-b827ebfffea93d27: {
    id: "eui-b827ebfffea93d27",
    description: "xp-gw01-rpi3-ic880a",
    owner: "xoseperez",
    owners: [
      "xoseperez"
    ],
    location: {
      latitude: 41.60128807,
      longitude: 2.62262466,
      altitude: 10
    },
    country_code: "es",
    attributes: {
      brand: "Raspberry Pi DIY",
      frequency_plan: "EU_863_870",
      model: "IMST",
      placement: "indoor"
    },
    last_seen: "2018-10-29T14:29:33Z"
  },
  eui-240ac4fffe13da78: {
    id: "eui-240ac4fffe13da78",
    description: "xp-gw04-esp32-rak833",
    owner: "xoseperez",
    owners: [
      "xoseperez"
    ],
    location: {
      latitude: 41.6013278,
      longitude: 2.62261325,
      altitude: 0
    },
    country_code: "es",
    attributes: {
      frequency_plan: "EU_863_870",
      placement: "indoor"
    },
    last_seen: "2018-10-23T11:13:57Z"
  },
  ...
}

You could easily parse the contents looking for the gateways you are interested in and getting the “last_seen” attribute to check if they are online.

Or you can use yet another undocumented API:

$ wget -q -O - http://noc.thethingsnetwork.org:8085/api/v2/gateways/eui-b827ebfffea93d27 | jq --unbuffered 
{
  "timestamp": "2018-10-29T14:47:33.052648468Z",
  "uplink": "10270",
  "downlink": "3329",
  "location": {
    "latitude": 41.60129,
    "longitude": 2.6226184,
    "altitude": 10
  },
  "frequency_plan": "EU_863_870",
  "platform": "IMST + Rpi",
  "gps": {
    "latitude": 41.60129,
    "longitude": 2.6226184,
    "altitude": 10
  },
  "time": "1540824453052648468",
  "rx_ok": 10270,
  "tx_in": 3329
}

As you can see this API entry point lets you target a specific gateway (by its EUI) and provides interesting information about uplink and downlink packets. On the other side, you do not get the info about the owner or the description. This is what I meant about “untidy”, there is no single place where you could get all the info.

Node-RED implementation

As a first approach let’s monitor a series of known gateways (the ones in our community) using the second API above. Doing it from Node-RED is very simple as we only have to trigger (inject) a message every X minutes to perform an HTTP request and parse the output. Then we can store it in an InfluxDB database or use an RBE node (report-by-exception) to check status changes.

This is an example of the flow to check several gateways (both private, community managed and third-party owned) around the Barcelona area:

The reporting is done using Telegram with a simple “The eui-0000… gateway status has changed to offline“. You can check and download a template flow based on the above one from the Node-RED flow library here:  TheThingsNetwork gateway monitor with Telegram alarms.

Community monitoring

It’s not that we want to be the big brother of the community, but sometimes it can be useful to know what’s going on around you. At The Things Network Catalunya, we are trying to be visible and accessible to anyone willing to know about TTN or how to start using the network. We want to join forces with anyone interested in a bottom-up IoT network. It is important to know who’s rowing in the same direction.

The flow below checks new gateway activations at 4 levels: 50km around Barcelona, the approximate territory of Catalunya, Spain and the whole world. The data for total and online gateways for each scope is stored into an InfluxDB database.

And also, the list of gateways for a given region (Catalunya in our case) is compared after every request to the previous list (stored in disk), trying to find new activations. Each new activation is then reported to a private group channel in Telegram along with the location, EUI, and description of the gateway. Optionally it uses the  Open Caged Data geocoding API based on OpenStreetMaps to reverse geocode the location. It’s our very own ttn_gateways service.

You can check and download a template flow based on the above one from the Node-RED flow library here:  The Things Network – New community gateways.

And the final result in Telegram looks like this:

It’s funny to notice that Telegram uses different map providers depending on the platform. The image above is the app for Android and the image belowe is the web interface using Yandex.

Using Grafana to visualize gateway status

A final step is to use Grafana to visualize the status of the gateways along time, and also since we are already grabbing information of total and online gateways per zone, showing a few graphics of that too.

The post Monitor your TTN gateways with Node-RED appeared first on Tinkerman.