We all know it: After a long day of work you chilled out on your bean bag and fell asleep early. You gotta get up and into your bed upstairs. So usually light goes on, you go upstairs, into bed. And there you have it: You’re not sleepy anymore.
Partially this is caused by the light you turned on. If that light is bright enough and has the right color it will wake you up no matter what.
To fight this companies like Apple introduced things like “NightShift” into iPhones, iPads and Macs.
“Night Shift uses your computer’s clock and geolocation to determine when it’s sunset in your location. It then automatically shifts the colors in your display to the warmer end of the spectrum.”
Simple, eh?. Now why does your house not do that to prevent you being ripped out of sleepy state while tiptoeing upstairs?
Right! This is where the smart house will be smart.
Nowadays we’ve got all those funky LED bulbs that can be dimmed and even their colours set. Why none of those market offerings come with that simple feature is beyond me:
After sunset, when turned on, default dim to something warmer and not so bright in general.
I did implement and it’s called appropriately the “U-Boot light”. Whenever we roam around the upper floor at night time, the light that follows our steps (it’s smart enough to do that) will not go full-blast but light up dim with redish color to prevent wake-up-calls.
The smart part being that it will take into account:
– movement in the house
– sunset and dawn depending on the current geographic location of the house (more on that later, no it does not fly! (yet))
– it’ll turn on and off the light according to the path you’re walking using the various sensors around anyways
This is Leela. She is a 7 year old lilac white British short hair cat that lives with us. Leela had a sister who used to live with us as well but she developed a heart condition and passed away last year. Witnessing how quickly such conditions develop and evaluate we thought that we can do something to monitor Leelas health a bit to just have some sort of pre-alert if something is changing.
Kid in a Candystore
As this Internet of Things is becoming a real thing these days I found myself in a candy store when I’ve encountered that there are a couple of really really cheap options to get a small PCB with input/output connectors into my house WiFi network.
One of the main actors of this story is the so called ESP8266. A very small and affordable system-on-a-chip that allows you to run small code portions and connect itself to a wireless network. Even better it comes with several inputs that can be used to do all sorts of wonderful things.
And so it happened that we needed to know the weight of our cat. She seemed to get a bit chubby over time and having a point of reference weight would help to get her back in shape. If you every tried to weigh a cat you know that it’s much easier said than done.
The alternative was quickly brought up: Build a WiFi-connected scale to weigh her litter box every time she is using it. And since I’ve recently bought an evaluation ESP8266 I just had to figure out how to build a scale. Looking around the house I’ve found a broken human scale (electronics fried). Maybe it could be salvaged as a part donor?
A day later I’ve done all the reading on that there is a thing called “load-cell”. Those load cells can be bought in different shapes and sizes and – when connected to a small ADC they deliver – well – a weight value.
I cracked the human scale open and tried to see what was broken. It luckily turned out to have completely fried electronics but the load-cells where good to go.
Look at this load cell:
That brought down the part list of this project to:
an ESP8266 – an Adafruit Huzzah in my case
a HX711 ADC board to amplify and prepare the signal from the load-cells
a human scale with just enough space in the original case to fit the new electronics into and connect everything.
The HX711 board was the only thing I had to order hardware wise – delivered the next day and it was a matter of soldering things together and throwing in a small Arduino IDE sketch.
My soldering and wiring skills are really sub-par. But it worked from the get-go. I was able to set-up a small Arduino sketch and get measurements from the load-cells that seemed reasonable.
Now the hardware was all done – almost too easy. The software would be the important part now. In order to create something flexible I needed to make an important decision: How would the scale tell the world about it’s findings?
Two basic options: PULL or PUSH?
Pull would mean that the ESP8266 would offer a webservice or at least web-server that exposes the measurements in one way or the other. It would mean that a client needs to poll for a new number in regular intervals.
Push would mean that the ESP8266 would connect to a server somewhere and whenever there’s a meaningful measurement done it would send that out to the server. With this option there would be another decision of which technology to use to push the data out.
Now a bit of history: At that time I was just about to re-implement the whole house home automation system I was using for the last 6 years with some more modern/interoperable technologies. For that project I’ve made the decision to have all events (actors and sensors) as well as some additional information being channeled into MQTT topics.
“MQTT1 (formerly MQ Telemetry Transport) is an ISO standard (ISO/IEC PRF 20922) publish-subscribe-based “lightweight” messaging protocol for use on top of the TCP/IP protocol. It is designed for connections with remote locations where a “small code footprint” is required or the network bandwidth is limited. The publish-subscribe messaging pattern requires a message broker. Thebroker is responsible for distributing messages to interested clients based on the topic of a message. Andy Stanford-Clark and Arlen Nipper of Cirrus Link Solutions authored the first version of the protocol in 1999.”
Something build for oil-pipelines can’t be wrong for your house – can it?
So MQTT uses the notation of a “topic” to sub-address different entities within it’s network. Think of a topic as just a simple address like “house/litterbox/weight”. And with that topic MQTT allows you to set a value as well.
The alternative to MQTT would have been things like WebSockets to push events out to clients. The decision for the home-automation was done towards MQTT and so far it seems to have been the right call. More and more products and projects available are also focussing on using MQTT as their main message transport.
For the home automation I had already set-up a demo MQTT broker in the house – and so naturally the first call for the litterbox project was to utilize that.
The folks of Adafruit provide the MQTT library with their hardware and within minutes the scale started to send it’s measurements into the “house/litterbox/weight” topic of the house MQTT broker.
Some tweaking and hacking later the litterbox was put together and the actual litterbox set on-top.
Since Adafruit offers platform to also send MQTT messages towards and create neat little dashboards I have set-up a little demo dashboard that shows a selection of data being pushed from the house MQTT broker to the Adafruit.io MQTT broker.
These are the raw values which are sent into the weight topic:
So the implementation done and used now is very simple. On start-up the ESP8622 initialises and resets the weight to 0. It’ll then do frequent weight measurements at the rate it’s configured in the source code. Those weight measurements are being monitored for certain criteria: If there’s a sudden increase it is assumed that “the cat entered the litterbox”. The weight is then monitored and averaged over time. When there’s a sudden drop of weight below a threshold that last “high” measurement is taken as the actual cat weight and sent out to a /weight topic on MQTT. The regular measurements are sent separately to also a configurable MQTT topic.
And off course with a bit of logic this would be the calculated weight topic:
Of course it is not enough to just send data into MQTT topics and be done with it. Of course you want things like logging and data storage. Eventually we also wanted to get some sort of notification when states change or a measurement was taken.
MQTT, the cloud and self-hosted
Since MQTT is enabling a lot of scenarios to implement such actions I am going to touch just the two we are using for our house.
We wanted to get a push notification to our phones whenever a weight measurement was taken – essentially whenever the cat has done something in the litterbox. The easiest solution: Set-Up a recipe on If This Than That (IFTTT) and use PushOver to send out push notifications to whatever device we want.
To log and monitor in some sort of a dashboard the easiest solution seemed to be Adafruits offer. Of course hosted inside our house a combination of InfluxDB to store, Telegraf to gather and insert into InfluxDB and Chronograf to render nice graphs was the best choice.
Since most of the above can be done in the cloud (as of: outside the house with MQTT being the channel out) or inside the house with everything self-hosted. Some additional articles will cover these topics on this blog later.
There’s lots of opportunity to add more logic but as far as our experiments and requirements go we are happy with the results so far – we now regularly get a weight and the added information of how often the cat is using her litterbox. Especially for some medical conditions this is quite interesting and important information to have.
“The internet of things” is a buzzword used more and more. It means that things around you are connected to the (inter)network and therefore can talk to each other and, when combined, offer fantastic new opportunities.
So NodeRed is a NodeJS based toolset that allows you to create so called “flows” (see picture above). Those flows determine what reacts and happens when things happen. Fantastic, told you!
I’ve added Alarming to hacs a while ago and I’ve now extended the built-in SMS gateway providers with the german telekom services called “Global SMS API”.
This API is offered through the Telekom own portal called developer garden and is as easy to use as it can possibly be. You only need to set-up the account with developergarden and after less than 5 minutes you can send and receive SMS and do a lot more. They got APIs for nearly everything you possible want to do … fancy some “talk to your house”-action? Would be easy to integrate into h.a.c.s. using their Speech2Text APIs.
They have a short video showing how to set it all up:
So I’ve added the SMS-send capabilities to the hacs internal alarming system with it’s own JSON configuration file looking like this:
And this simple piece of configuration leads to SMS getting sent out as soon as – in this example – a window opens:
Before the Telekom Global SMS API I’ve used a different provider (SMS77) but since the delivery times of this provider varied like crazy (everything from 30 seconds to 5 minutes) and the provider had a lot of downtimes my thought was to give the market leader a try.
Usually the actors that allow you to switch power on/off and who measure power usage use the 434Mhz or 868Mhz wireless bands to communicate with their base station. Now the german manufacturer AVM came up with a solution that allows you to switch on/off (with an actual button on the device itself and wireless!) and to measure the power consumption of the devices connected to it.
The unspectacular it looks the spectacular are the features:
switch up to 2300 watts / 10 ampere
use different predefined settings to switch on/off or even use Google Calendar to tell it when to switch
measure the energy consumption of connected devices
For around 50 Euro it’s quite an investment but maybe I’ll give it a shot – especially the measurement functionality sounds great. Since I do not have one yet I don’t know anything about how to access it through third party software (h.a.c.s.?)
So first a small video to get an idea what I am implementing right now:
I plan to add a lot more – like for swiping gestures. So this will be – just like h.a.c.s – a continuous project. Since I switched to OS X entirely at home I use the great Coda2 to write and debug the code. It helps a lot to have two browser set-up because for some reason I still not feel that well with the WebKit Web Inspector.
Another great feature of Coda2 is the AirPreview – which means it will preview your current page in the editor on an iOS device running DietCoda – oh how I love those automations.
I was asked if it would be possible to get the ELV MAX! Cube interfacing functionality outside of h.a.c.s. – maybe as a library. Sure! That is possible. And to speed up things I give you the ELV MAX! Cube C# Library called: MAXSharp
It’s a plain and simple library without much dependencies – in fact there’s only some threading and the FastSerializer. Since I am using this library with h.a.c.s. as well I did not remove the serializer implementation.
There’s a small demo program included which is called MAXSharpExample. The library itself contains the abstractions necessary to get information from the ELV MAX! Cube. It does not contain functionality to control the cube – if you want to add, feel free it’s all open sourced and I would love to see pull requests!
The architecture is based upon polling – I know events would make a cleaner view but for various reasons I am using queues in h.a.c.s. and therefore MAXSharp does as well. The example application spins up the ELV MAX interfacing / handling thread and as soon as you’re connected you can access all house related information and get diff-events from the cube.
It’s been some weeks since I wrote a status update on the ELV MAX! cube protocol reverse engineering and integration into my own home automation project called h.a.c.s..
So first of all I want to give a short overview over what has been achieved so far:
I wrote a C# library, highly influenced by a PHP implementation from the domotica forum, which allows you to continuesly get status information from the ELV MAX! cube with current (1.3.6) firmware. It is tested so far with a fairly big set-up for the ELV MAX! cube (see below)
I was able to integrate that library into my own home automation project called h.a.c.s. – There the ELV MAX! cube is just another device, alongside a EzControl XS1 and a SolarLog 500. The cube is monitored using my library and diff-sets as well as status information are stored automatically with the h.a.c.s. built-in mechanisms. In fact you can access for example the window shutter contact information just like you would with any other door contact in the EzControl XS1.
You can use events coming from the ELV MAX! cube to create new events – how about switching off/on devices when opening/closing windows?
Every bit of information from all integrated sensor monitoring and actor handling devices come together in h.a.c.s.
I started the reverse engineering with just one shutter contact and one thermostat. After all my test were successful I went for the big package and ordered some more sensors. This is how the setup is currently configured:
I’ve learned a lot of interesting things about the ELV MAX! cube hardware and software. One is that you need to be ready for surprised. The documentation of the cube tells you the following:
Did you spot the funny fact? 50 devices – we’re well below that limit. 10 rooms – holy big mansion batman! We’re well over that. How is that possible? Well take it as a fact – you can create more than 10 rooms. And that is very handy. I’ve created 13 rooms and there are probably more to come because those shutter contacts are quite cheap and can be used for various other home automation sensory games. The tool to set-up and pair those sensors just came up with a notice that said “Oh well, you want to create more than 10 rooms? If you’re sure that you want that we allow you to, but hey, don’t blame us!”. Cool move ELV! – As of now I haven’t found any downside of having more than 10 rooms.
All my efforts started with firmware version 1.3.5. This firmware seemed to have some severe memory leaks – because just by retrieving the current configuration information every 10 seconds the device would stop communicating after more then 48 hours. Only a reboot could revive it – sometimes amnesia set in which led to a house roundtrip for me.
With some changes in the library (like keeping the connection open as long as possible) and a new firmware version 1.3.6. the cube was way more cooperative and hasn’t crashed for about 1 month now (with 10 seconds update times).
So what does my library do? It is designed to run in it’s own thread. When it’s started it opens a connection to the cube and retrieves the current status and configuration information. Those informations are stored in an object called “House”. This house consists of multiple rooms – and those rooms are filled with window shutter contacts and thermostats. All information related to those different intances are stored along with them. The integration into h.a.c.s. allows the library to generate sensor and actor events (like when a temperature changes, a window opens/closes) which are passed back to h.a.c.s. and handled in the big event loop there.
With all that ELV MAX! cube data I wanted to plug a quite nice tool that I am using in the iPhone and the iPad. It’s called “Moni4home” and it allows you to control the EzControl XS1 directly. Because it’s only accessing the EzControl XS1 I used h.a.c.s. to “inject” additional sensor data into the standard EzControl XS1 data. So basically data flow is like this: iPad app accesses h.a.c.s. which acts as a proxy. h.a.c.s. retrieves the EzControl XS1 sensor and actor data and injects additional virtual sensors like those from the ELV MAX! cube. h.a.c.s. then sends that beefed up data to the iPad app. Voilá!
After the successful integration of the ELV MAX! cube I’ve started to work on the next bit of networking home automation equipment in my house – a solar panel data logger called “Solar-Log 500”. This device monitors two solar power inverters and stores the sensory data.
“Funny” story first: this device has the same problem like the ELV MAX! cube. When you start to poll it every 10 seconds (or less) it just stops operating after about 20 hours. Bear in mind: In case of the Solar-Log I just http-get a page that looks like this in the browser:
And by doing so every 10 seconds the device stops working. I am using the current firmware – so one workaround for that issue is to planable reboot the Solar-Log at a time when there is no sun and therefore nothing to log or monitor.
Beside that it’s a fairly easy process: Get that information, log it. Done.
So there you have it – h.a.c.s. interfacing with three different devices and roughly 100 sensors and actors over 434mhz/868mhz, wireless and wired network. There’s still more to come!
A lot of people seem to dive into home automation these days. Apparently Andreas is also at the point of starting his own home automation project. Good to know that he also is using the EzControl XS1 and in the future maybe even the ELV MAX! cube. Party on Andreas!
I had a couple of hours to tinker with my ELV MAX! Cube and there is some progress with the protocol reverse engineering.
Of course there is the domotica forum helping out with some information the guys over there have found but in addition to their very helpful findings I want it to be integrated into h.a.c.s. – and along with it I maybe want to have a way to find eventual protocol changes quick and easy in the future.
So yesterday I partied on the ‘first contact’ – today I am a bit deeper into the protocol itself:
Here are some explanations to the picture:
When a tcp connection to the cube is opened you can immediately read from it – the cube is throwing information at you. There’s always a character at the beginning of each line which marks the type and beginning of the message.
There seem to be these types of messages in the first package of information:
H – Header maybe?
it contains the serial number of your cube, the RF address, the firmware version and several other things like time information
M – Metadata?
this seems to be some kind of global metadata list, containing the rooms with their IDs (it’s the %) in the screenshot). Furthermore it contains the serial numbers and names of the devices in that room – at the moment there’s just a window-state-sensor in that first room called “Fensterkontakt 1”
C – Configuration?
since there are multiple C messages these seem to contain detailed configuration data specific to a device in the MAX! network. Each device seems to be addressed by a RF Address and it’s serial number.
the first C message in the screenshot is associated to the cube itself
the second C message is associated to the window-state-sensor – you can clearly see in there the room id “%)” and the serial of the window-state-sensor.
L – live status?
this message seems to contain room status information. In our case there is only the room with id “%)”. When the window-state-sensor changes state the last byte changes value – interesting, eh?
On the coding side I’ve got several things set-up in my little debug tool. I’ve wrapped those message types into various classes to handle them more easily later on in h.a.c.s.. Furthermore I used a little decompiler-wisdom to extract some more information from the included ELV MAX! cube software.
Thanks to german UrhG paragraph § 69e (german copyright law) I am allowed to decompile the included software in order to achieve interoperability (and only that). That’s exactly what I would like to achieve: Interoperability. And for the record: besides that I also filed a support request to ELV in which I ask them if I could get access to a presumably existing documentation of that protocol.
While waiting on that documentation I am using JD-GUI as a decompiler user interface for java – since the software of the cube is written in java.
There are many interesting things in there but it’s a slow process to get ahold of all the things necessary. There are already some very nice things showing up. Like when you want to know if there’s a cube (or more) in the network you just need to send a multicast ip packet containing a characteristic signature and all the cubes in your network will try to connect back to you with some basic information – nice, isn’t it? Or what about that AES Encryption/Decryption that seems to be built into the cube? Yes that’s right! It seems to be possible to send commands to either encrypt or decrypt according to the AES. Thoughtfully these commands are marked with ‘e’ and ‘d’. Or that if you send “l:” as a command with CR+LF at the end you get a device listing with all stats… and so on.
Some open question to EQ-3/ELV for the end of this article:
Why this strange protocol? Why all the work on both sides? Just because an HTTP server implementation with a RESTful service would have been that more difficult?
Base64 encoded data? The 90s called, they want their 8th bit back.
why that complex local webserver approach when you could have done everything in a java app anyways?
That’s it for today, I just pushed a feature to the Git repository which allows you to run whatever command you like on your cube with the debugging tool:
For several years now I am building my own home automation tools by putting together existing hardware and self-written software. As the central software core of my home automation system I use h.a.c.s. – “home automation control server” which I put up as open source software on GitHub.
Throughout the years I was able to embedd a lot of daily tasks and measurements in one place which can be accessed by a simple web page. It currently looks like this:
You can find some articles on this blog about h.a.c.s. if you want to know more about it.
As of today I can control and measure the states of switches, windows, doors, temperature and humidity and power consumption. Scenarios like “when this door opens, switch on that light” are easy things to do with h.a.c.s.
Now “Winter’s coming!”. And therefore I want to take control of the heating of each and every room in the house. I want to set a goal for a temperature and I want the heating to fire up or cool down with that goal. And of course I want to monitor manual changes of each and every radiator in the house.
Last week then I stumbled upon a piece of kit called “ELV MAX! Cube”. It’s a white cube (as the name implies) which offers a USB port from which it is powered and an RJ-45 ethernet port which connects the cube to the home network.
The cube itself does not draw much power and it can be powered by the routers USB port easily. It allows you to connect some peripherals using 868 mhz rf. Those peripherals can be: window state sensors (closed/open) and thermostats to control the radiators (and a switch but, well… hopefully not necessary).
It comes with it’s own user interface – a java application that connects to the device and allows you to configure it. Quite nice – it runs on Windows and Mac. You can use a cloud service to control the device over the internet, but I have no intention in trying that out right now.
My plan is to extend h.a.c.s. to get information from the cube and handle them and in the end even control the cube by setting temperatures and controlling the outcome of those changes.